This workflow demonstrates several methods to import one or many CSV file into Hive
Demonstrated are direct Uploads where you create a Hive table with KNIME nodes. Or you copy your file to an /upload/ folder and point an external table to them. If they all have the same structure they will be read by Hive. You can then use this external file to further handel your files.
If the fies are very large you might have to use partitions. In the KNIME installemnt of the Hive drivers there is a problem with the headers of the CSV files. It is also demonstrated how to get rid of them.
Please familiarize yourself with the concepts of big data and partitions in order to use this. And please note. KNIME's local big data environment is just there to demonstrate the usage. It might work with your large files but it is called Big Data for areason ....
https://hub.knime.com/mlauber71/spaces/Public/latest/kn_example_hive_school_of?u=mlauber71
Demonstrated are direct Uploads where you create a Hive table with KNIME nodes. Or you copy your file to an /upload/ folder and point an external table to them. If they all have the same structure they will be read by Hive. You can then use this external file to further handel your files.
If the fies are very large you might have to use partitions. In the KNIME installemnt of the Hive drivers there is a problem with the headers of the CSV files. It is also demonstrated how to get rid of them.
Please familiarize yourself with the concepts of big data and partitions in order to use this. And please note. KNIME's local big data environment is just there to demonstrate the usage. It might work with your large files but it is called Big Data for areason ....
https://hub.knime.com/mlauber71/spaces/Public/latest/kn_example_hive_school_of?u=mlauber71