Hub
  • Software
  • Blog
  • Forum
  • Events
  • Documentation
  • About KNIME
  • KNIME Hub
  • Nodes
  • DB Loader
NodeNode / Sink

DB Loader

DB Read/Write Streamable
Drag & drop
Like
Copy short link

This node uses database specific bulk loading functionality that only some databases (e.g. Hive, MySQL, PostgreSQL and H2) support to load large amounts of data into an existing database table.
Notice:
Most databases do not perform data checks when loading the data into the table which might lead to a corrupt data table . The node does some preliminary checks such as checking that the column order and column names are compatible. However it does not check the column type compatibility. So before using this node please make sure that the column types of the KNIME columns and the database columns are compatible.

Depending on the database an intermediate file format is often used for efficiency which might be required to upload the file to a server. If a file needs to be uploaded, any of the protocols supported by the file handling nodes can be used, e.g. for Apache Hive HDFS or webHDFS. After the loading of the data into a table, the uploaded file gets deleted if it is no longer needed by the database.

If there is no need to upload or store the file for any reason, a remote file input connection prevents execution.

Some databases such as MySQL and PostgreSQL support file based and memory based uploading which require different rights in the database. If this is the case and if you do not have the rights to execute the file based loading of the data try the memory based method instead. If supported the different modes can be changed in the Loader mode section of the "Options" tab which is otherwise hidden.

Depending on the connected database the dialog settings may change. For example, MySQL and PostgreSQL use a CSV file for the data transfer. In order to change how the CSV file is created go to the "Advanced" tab.

This node can access a variety of different file systems. More information about file handling in KNIME can be found in the official File Handling Guide.

Node details

Input ports
  1. Type: Table
    Data to load into the database
    A KNIME data table containing the data rows to load into the database.
  2. Type: DB Session
    DB Connection
    DB Connection to the database.
Output ports
  1. Type: DB Session
    DB Connection
    Input DB Connection that can be changed to DB Data referencing the selected database table.
File System Connection (Dynamic Inport)
The file system connection.
  1. Type: File System

Extension

The DB Loader node is part of this extension:

  1. Go to item

Related workflows & nodes

  1. Go to item
    03.0_Setup_Local_Big_Data_Environment
    Education Data engineering Data engineer
    +3
    This workflow sets up a local big data environment for the next exercise. It creates a lo…
    knime > Education > Courses > L4-DE Best Practices for Data Engineering > solutions > Session_3_ELT_on_Big_Data > 03.0_Setup_Local_Big_Data_Environment
  2. Go to item
    03.0_Setup_Local_Big_Data_Environment
    Education Data engineering Data engineer
    +3
    This workflow sets up a local big data environment for the next exercise. It creates a lo…
    knime > Education > Courses > L4-DE Best Practices for Data Engineering > exercises > Session_3_ELT_on_Big_Data > 03.0_Setup_Local_Big_Data_Environment
  3. Go to item
    02 Getting Started with Big Data - Solution
    Big data Create local big data environment Hive
    Solution to an L4-BD SELF-PACED COURSE exercise: - Create a local big data environment - …
    knime > Education > Self-Paced Courses > L4-BD Introduction to Big Data with KNIME Analytics Platform > Solutions > 02 Getting Started with Big Data - Solution
  4. Go to item
    03 Manipulating Big Data
    Big data Spark Hive
    L4-BD SELF-PACED COURSE exercise: - Manipulate data on Hive with the DB nodes - Perform E…
    knime > Education > Self-Paced Courses > L4-BD Introduction to Big Data with KNIME Analytics Platform > Exercises > 03 Manipulating Big Data
  5. Go to item
    02 Getting Started with Big Data
    Big data Create local big data environment Hive
    EXCERCISE of the L4-BD Introduction to Big Data with KNIME Analytics Platform course: - C…
    emilio_s > Public > l4db exercises > 02 Getting Started with Big Data
  6. Go to item
    02 Getting Started with Big Data - Solution
    Big data Create local big data environment Hive
    Solution to an L4-BD SELF-PACED COURSE exercise: - Create a local big data environment - …
    tqthanh168 > Public > Self-Paced Courses > L4-BD Introduction to Big Data with KNIME Analytics Platform > Solutions > 02 Getting Started with Big Data - Solution
  7. Go to item
    03 Manipulating Big Data
    Big data Spark Hive
    L4-BD SELF-PACED COURSE exercise: - Manipulate data on Hive with the DB nodes - Perform E…
    tqthanh168 > Public > Self-Paced Courses > L4-BD Introduction to Big Data with KNIME Analytics Platform > Exercises > 03 Manipulating Big Data
  8. Go to item
    03 Manipulating Big Data
    Big data Spark Hive
    L4-BD SELF-PACED COURSE exercise: - Manipulate data on Hive with the DB nodes - Perform E…
    halverdog > Public > L4-BD Introduction to Big Data with KNIME Analytics Platform > Exercises > 03 Manipulating Big Data
  9. Go to item
    02 Getting Started with Big Data - Solution
    Big data Create local big data environment Hive
    Solution to an L4-BD SELF-PACED COURSE exercise: - Create a local big data environment - …
    halverdog > Public > L4-BD Introduction to Big Data with KNIME Analytics Platform > Solutions > 02 Getting Started with Big Data - Solution
  10. Go to item
    03 Manipulating Big Data
    Big data Spark Hive
    L4-BD SELF-PACED COURSE exercise: - Manipulate data on Hive with the DB nodes - Perform E…
    manuel1972 > Public > Self-Paced Courses > L4-BD Introduction to Big Data with KNIME Analytics Platform > Exercises > 03 Manipulating Big Data

No known nodes available

KNIME
Open for Innovation

KNIME AG
Hardturmstrasse 66
8005 Zurich, Switzerland
  • Software
  • Getting started
  • Documentation
  • E-Learning course
  • Solutions
  • KNIME Hub
  • KNIME Forum
  • Blog
  • Events
  • Partner
  • Developers
  • KNIME Home
  • KNIME Open Source Story
  • Careers
  • Contact us
Download KNIME Analytics Platform Read more on KNIME Server
© 2022 KNIME AG. All rights reserved.
  • Trademarks
  • Imprint
  • Privacy
  • Terms & Conditions
  • Credits