Hub
Pricing About
  • Software
  • Blog
  • Forum
  • Events
  • Documentation
  • About KNIME
  • KNIME Community Hub
  • khofenbitzer
  • Spaces
  • Public
  • 01_Amazon_S3_Remote_File_Example
WorkflowWorkflow

Amazon S3 Remote File Example

Amazon S3 Cloud Remote Streaming URL
K

Last edited: 

Drag & drop
Like
Download workflow
Copy short link
Workflow preview
In order to connect to the service, Amazon S3 credentials are required. Folders created on root level are created as buckets on S3. The virtual file system functionality is fully supported by the file handling nodes. In the first step a bucket and a directory inside that bucket is created. After that a file is uploaded to the directory inside the bucket (path: /examplebucket***/exampledirectory). Finally all the files and folders inside the example bucket are listed. The result is a list of URIs which can be used to download files/directories. In the second step we download the iris.csv which we uploaded in the step before. This saves the file to disk; this file can then be read using a csv reader. Since it might be unnecessary to cache the file to disk, the Amazon S3 File Picker provides the functionality to create URLs from which files can be read directly. This works with all file readeres that support reading files from a URL. These URLs have an expiration date which can be set in the dialog. Stream reading also becomes useful in conjunction with the KNIME Streaming Executor as the (possibly large) data is read from the remote side and is immediately processed without being downloaded to the local machine. After having seen how we upload and download files and how we use the File Picker to read files directly we can delete our example bucket from S3.

Used extensions & nodes

Created with KNIME Analytics Platform version 4.1.1 Note: Not all extensions may be displayed.
  • Go to item
    KNIME Amazon Cloud Connectors Trusted extension

    KNIME AG, Zurich, Switzerland

    Version 4.1.1

    knime
  • Go to item
    KNIME Core Trusted extension

    KNIME AG, Zurich, Switzerland

    Versions 4.1.0, 4.1.1

    knime
  1. Go to item
  2. Go to item
  3. Go to item
  4. Go to item
  5. Go to item
  6. Go to item
Loading deployments
Loading ad hoc executions

Legal

By using or downloading the workflow, you agree to our terms and conditions.

Discussion
Discussions are currently not available, please try again later.

KNIME
Open for Innovation

KNIME AG
Talacker 50
8001 Zurich, Switzerland
  • Software
  • Getting started
  • Documentation
  • E-Learning course
  • Solutions
  • KNIME Hub
  • KNIME Forum
  • Blog
  • Events
  • Partner
  • Developers
  • KNIME Home
  • KNIME Open Source Story
  • Careers
  • Contact us
Download KNIME Analytics Platform Read more on KNIME Business Hub
© 2023 KNIME AG. All rights reserved.
  • Trademarks
  • Imprint
  • Privacy
  • Terms & Conditions
  • Credits