Hub
Pricing About
WorkflowWorkflow

Create a Big Data Hive/Parquet table with a partition based on an existing KNIME table and add more partitions later

HivePartitionBig dataBigData
+1
mlauber71 profile image
Draft Latest edits on 
Aug 19, 2024 3:57 PM
Drag & drop
Like
Download workflow
Workflow preview
Create a Big Data Hive/Parquet table with a partition based on an existing KNIME table and add more partitions later

You can create a Hive table with Parquet format with the DB Table Creator node with additional options specifying the PARQUET format and a PARTITION. You will leave the partition column out when creating the table from the example and later use the column as partition when you insert the KNIME table into your newly created Hiv table. You can later add more partitions just by uploading it thru "DB Loader" - the partitions will automatically be created or an existing partition will be appended.

=> please download the whole workflow group "kn_example_bigdata_hive_partitions"




External resources

  • Hive - upload data in several ORC files to HDFS and bring them together as an EXTERNAL table
  • KNIME and Hive - load multiple Parquet files at once via external table
  • KN workflow Group: Create a Big Data Hive/Parquet table with a partition based on an existing KNIME table and add more partitions later
  • META: A meta collection of KNIME and databases (SQL, Big Data/Hive/Impala and Spark/PySpark)
  • HUB: Create a Big Data Hive/Parquet table with a partition based on an existing KNIME table and add more partitions later
  • KNIME Big Data Extensions User Guide
  • School of Hive - with KNIME's local Big Data environment (SQL for Big Data)
Loading deploymentsLoading ad hoc jobs

Used extensions & nodes

Created with KNIME Analytics Platform version 4.7.8
  • Go to item
    KNIME Base nodesTrusted extension

    KNIME AG, Zurich, Switzerland

    Version 4.7.8

    knime
  • Go to item
    KNIME DatabaseTrusted extension

    KNIME AG, Zurich, Switzerland

    Version 4.7.4

    knime
  • Go to item
    KNIME ExpressionsTrusted extension

    KNIME AG, Zurich, Switzerland

    Version 4.7.3

    knime
  • Go to item
    KNIME Extension for Big Data File FormatsTrusted extension

    KNIME AG, Zurich, Switzerland

    Version 4.7.2

    knime
  • Go to item
    KNIME Extension for Local Big Data EnvironmentsTrusted extension

    KNIME AG, Zurich, Switzerland

    Version 4.7.7

    knime
  • Go to item
    KNIME JavasnippetTrusted extension

    KNIME AG, Zurich, Switzerland

    Version 4.7.0

    knime
  • Go to item
    KNIME Quick FormsTrusted extension

    KNIME AG, Zurich, Switzerland

    Version 4.7.4

    knime

Legal

By using or downloading the workflow, you agree to our terms and conditions.

KNIME
Open for Innovation

KNIME AG
Talacker 50
8001 Zurich, Switzerland
  • Software
  • Getting started
  • Documentation
  • Courses + Certification
  • Solutions
  • KNIME Hub
  • KNIME Forum
  • Blog
  • Events
  • Partner
  • Developers
  • KNIME Home
  • Careers
  • Contact us
Download KNIME Analytics Platform Read more about KNIME Business Hub
© 2025 KNIME AG. All rights reserved.
  • Trademarks
  • Imprint
  • Privacy
  • Terms & Conditions
  • Data Processing Agreement
  • Credits