Hub
Pricing About
  • Software
  • Blog
  • Forum
  • Events
  • Documentation
  • About KNIME
  • KNIME Community Hub
  • mlauber71
  • Spaces
  • Public
  • kn_example_bigdata_hive_partitions
  • m_001_hive_partitions
WorkflowWorkflow

Create a Big Data Hive/Parquet table with a partition based on an existing KNIME table and add more partitions later

Hive Partition Big data Big Data
+1
mlauber71 profile image

Last edited: 

Drag & drop
Like
Download workflow
Copy short link
Workflow preview
Create a Big Data Hive/Parquet table with a partition based on an existing KNIME table and add more partitions later You can create a Hive table with Parquet format with the DB Table Creator node with additional options specifying the PARQUET format and a PARTITION. You will leave the partition column out when creating the table from the example and later use the column as partition when you insert the KNIME table into your newly created Hiv table. You can later add more partitions just by uploading it thru "DB Loader" - the partitions will automatically be created or an existing partition will be appended. => please download the whole workflow group "kn_example_bigdata_hive_partitions"

External resources

  • Hive - upload data in several ORC files to HDFS and bring them together as an EXTERNAL table
  • KNIME and Hive - load multiple Parquet files at once via external table
  • KN workflow Group: Create a Big Data Hive/Parquet table with a partition based on an existing KNIME table and add more partitions later
  • META: A meta collection of KNIME and databases (SQL, Big Data/Hive/Impala and Spark/PySpark)
  • HUB: Create a Big Data Hive/Parquet table with a partition based on an existing KNIME table and add more partitions later
  • KNIME Big Data Extensions User Guide
  • School of Hive - with KNIME's local Big Data environment (SQL for Big Data)

Used extensions & nodes

Created with KNIME Analytics Platform version 4.7.0
  • Go to item
    KNIME Base nodes Trusted extension

    KNIME AG, Zurich, Switzerland

    Version 4.7.0

    knime
  • Go to item
    KNIME Database Trusted extension

    KNIME AG, Zurich, Switzerland

    Version 4.7.0

    knime
  • Go to item
    KNIME Expressions Trusted extension

    KNIME AG, Zurich, Switzerland

    Version 4.7.0

    knime
  • Go to item
    KNIME Extension for Big Data File Formats Trusted extension

    KNIME AG, Zurich, Switzerland

    Version 4.7.0

    knime
  • Go to item
    KNIME Extension for Local Big Data Environments Trusted extension

    KNIME AG, Zurich, Switzerland

    Version 4.7.0

    knime
  • Go to item
    KNIME Javasnippet Trusted extension

    KNIME AG, Zurich, Switzerland

    Version 4.7.0

    knime
  1. Go to item
  2. Go to item
  3. Go to item
  4. Go to item
  5. Go to item
  6. Go to item
Loading deployments
Loading ad hoc executions

Legal

By using or downloading the workflow, you agree to our terms and conditions.

Discussion
Discussions are currently not available, please try again later.

KNIME
Open for Innovation

KNIME AG
Talacker 50
8001 Zurich, Switzerland
  • Software
  • Getting started
  • Documentation
  • E-Learning course
  • Solutions
  • KNIME Hub
  • KNIME Forum
  • Blog
  • Events
  • Partner
  • Developers
  • KNIME Home
  • KNIME Open Source Story
  • Careers
  • Contact us
Download KNIME Analytics Platform Read more on KNIME Business Hub
© 2023 KNIME AG. All rights reserved.
  • Trademarks
  • Imprint
  • Privacy
  • Terms & Conditions
  • Credits