Hub
Pricing About
WorkflowWorkflow

KNIME and Hive - load multiple Parquet files at once via external table

KnimeHiveExternalTableLoad
+3
mlauber71 profile image
Draft Latest edits on 
Aug 19, 2024 4:04 PM
Drag & drop
Like
Download workflow
Workflow preview
This workflow demonstrates how to import several Parquet files at once without iteration using an external HIVE table.

The initial structure wil be derived from a sample of one of the files. The rule are very basic: String, Double and Int. You might add rules for BIGINT if you need them
You could use a column as partition.

Please Download the complete folder at: https://hub.knime.com/mlauber71/spaces/Public/latest/kn_example_bigdata_hive_parquet_loader/

External resources

  • KNIME Big Data Extensions User Guide
  • KNIME and Big Data and Spark
  • Hive - how to get from DB-Connectors to Hive (or Impala) tables - KNIME 4.5+
  • School of Hive - with KNIME's local Big Data environment (SQL for Big Data)
Loading deploymentsLoading ad hoc jobs

Used extensions & nodes

Created with KNIME Analytics Platform version 4.7.8 Note: Not all extensions may be displayed.
  • Go to item
    KNIME Base nodesTrusted extension

    KNIME AG, Zurich, Switzerland

    Version 4.7.8

    knime
  • Go to item
    KNIME DatabaseTrusted extension

    KNIME AG, Zurich, Switzerland

    Version 4.7.4

    knime
  • Go to item
    KNIME ExpressionsTrusted extension

    KNIME AG, Zurich, Switzerland

    Version 4.7.3

    knime
  • Go to item
    KNIME Extension for Big Data File FormatsTrusted extension

    KNIME AG, Zurich, Switzerland

    Version 4.7.2

    knime
  • Go to item
    KNIME Extension for Local Big Data EnvironmentsTrusted extension

    KNIME AG, Zurich, Switzerland

    Version 4.7.7

    knime
  • Go to item
    KNIME JavasnippetTrusted extension

    KNIME AG, Zurich, Switzerland

    Version 4.7.0

    knime
  • Go to item
    KNIME Quick FormsTrusted extension

    KNIME AG, Zurich, Switzerland

    Version 4.7.4

    knime

Legal

By using or downloading the workflow, you agree to our terms and conditions.

KNIME
Open for Innovation

KNIME AG
Talacker 50
8001 Zurich, Switzerland
  • Software
  • Getting started
  • Documentation
  • Courses + Certification
  • Solutions
  • KNIME Hub
  • KNIME Forum
  • Blog
  • Events
  • Partner
  • Developers
  • KNIME Home
  • Careers
  • Contact us
Download KNIME Analytics Platform Read more about KNIME Business Hub
© 2025 KNIME AG. All rights reserved.
  • Trademarks
  • Imprint
  • Privacy
  • Terms & Conditions
  • Data Processing Agreement
  • Credits