Import CSV - This workflow will act as a meta workflow trying to answer questions about importing data from CSV and Text files
One of the most frequently asked questions is the one how to import data from CSV and Text files under various circumsatnces. Often these files are messy with unbalanced quotes and column separators that would also appear in the data. Best advise would be to avoid CSV at all and resort to formats the would preserve the column types and formats like Parquet, ORC, ARFF or even H2 or SQLite - but often this is not possible
The (new) CSV Reader should be the first stop
https://hub.knime.com/knime/extensions/org.knime.features.base/latest/org.knime.base.node.io.filehandling.csv.reader.CSVTableReaderNodeFactory
=> please also note that you can configure a lot of settings like "Support changing file schemas" if your CSV structure does change or force data types
https://hub.knime.com/knime/extensions/org.knime.features.base/latest/org.knime.base.node.io.filehandling.csv.reader.CSVTableReaderNodeFactory#relatedworkflowsnodes
use R to read and write custom CSV files and read them back with different nodes
https://hub.knime.com/mlauber71/spaces/Public/latest/kn_example_r_csv_read_write
=> also on the right side a collection of several nodes to import CSV and TXT files
Force all columns to be read as strings (use KNIME nodes only)
https://hub.knime.com/mlauber71/spaces/Public/latest/kn_example_csv_import_force_all_string_pure_knime
Demonstrates how to import several CSV files with the new CSV reader from sub folders at once
https://hub.knime.com/mlauber71/spaces/Public/latest/kn_example_csv_import_multiple_csv_types
-------------
Often if everything else fails R's "Readr" package might still be able to help you
use R library(readr) to read (messy) CSV file into KNIME
https://hub.knime.com/mlauber71/spaces/Public/latest/kn_example_r_read_single_csv_file
=> also export files to ARFF, Parquet and SQLite (with and without R) to preserve column types
import 'dirty' CSV files with the help of R's readr
https://hub.knime.com/mlauber71/spaces/Public/latest/kn_example_r_import_csv_r_readr
Force R’s readr to import all columns of a messy CSV file as strings - allow changing table structures
https://hub.knime.com/mlauber71/spaces/Public/latest/kn_example_r_import_csv_messy_loop_force_all_string
Import CSV with non-printable UNICODE characters
https://forum.knime.com/t/csv-writer-problem/28064/9?u=mlauber71
https://hub.knime.com/mlauber71/spaces/Public/latest/forum/kn_forum_28064_r_import_csv_r_readr_strange_charaters
-------------
CSV's can be used to append data to an existing file without having to reload it again
Download data from a database in chunks and append them to an CSV file
https://hub.knime.com/mlauber71/spaces/Public/latest/kn_example_write_csv_loop_chunks/m_001_write_csv_loop_chunks
-------------
You can also move CSV files around
copy files to two target folders using local URI strings - the basic version
https://hub.knime.com/mlauber71/spaces/Public/latest/kn_example_move_csv_files_basic
copy files to two target folders using local URI strings
https://hub.knime.com/mlauber71/spaces/Public/latest/kn_example_move_csv_files
-------------
And you often combine CSV files with other formats like MS Access, Excel and Big Data systems
Export huge MS sql table to csv without fetching all records together
https://hub.knime.com/mlauber71/spaces/Public/latest/forum/kn_example_huge_db_to_csv/m_001_write_csv_41
CSV files from subfolders to xlsx sheets
https://hub.knime.com/mlauber71/spaces/Public/latest/forum/kn_example_csv_to_excel/m_001_import
KNIME and Hive - load multiple CSV files at once via external table
https://hub.knime.com/mlauber71/spaces/Public/latest/kn_example_bigdata_hive_csv_loader/m_001_import_hive_csv
External resources
- KNIME and Hive - load multiple CSV files at once via external table
- CSV files from subfolders to xlsx sheets
- Export huge MS sql table to csv without fetching all records together
- copy files to two target folders using local URI strings
- copy files to two target folders using local URI strings - the basic version
- Download data from a database in chunks and append them to an CSV file
- Import CSV with non-printable UNICODE characters
- Force R’s readr to import all columns of a messy CSV file as strings - allow changing table structures
- import 'dirty' CSV files with the help of R's readr
- use R library(readr) to read (messy) CSV file into KNIME
- Demonstrates how to import several CSV files with the new CSV reader from sub folders at once
- Force all columns to be read as strings (use KNIME nodes only)
- use R to read and write custom CSV files and read them back with different nodes
- The (new) CSV Reader should be the first stop
Used extensions & nodes
All required extensions are part of the default installation of KNIME Analytics Platform version 4.4.1
No known nodes available
Legal
By using or downloading the workflow, you agree to our terms and conditions.