Hub
  • Software
  • Blog
  • Forum
  • Events
  • Documentation
  • About KNIME
  • KNIME Hub
  • Nodes
  • Create Databricks Environment
NodeNode / Source

Create Databricks Environment

Tools & Services Apache Spark
Drag & drop
Like
Copy short link

Creates a Databricks Environment connected to an existsing Databricks cluster. See AWS or Azure Databricks documentation for more information.

Note: To avoid an accidental cluster startup, this node creates a dummy DB and Spark port if loaded in executed state from a stored workflow. Reset and execute the node to start the cluster and create a Spark execution context.

Cluster access control : KNIME uploads additional libraries to the cluster. This requires manage cluster-level permissions if your cluster is secured with access control. See the Databricks documentation on how to set up the permission.

Node details

Output ports
  1. Type: DB Session
    DB Connection
    JDBC connection, that can be connected to the KNIME database nodes.
  2. Type: File System
    DBFS Connection
    DBFS connection, that can be connected to the Spark nodes to read/write files.
  3. Type: Spark Context
    Spark Context
    Spark context, that can be connected to all Spark nodes.

Extension

The Create Databricks Environment node is part of this extension:

  1. Go to item

Related workflows & nodes

  1. Go to item
    Working with Databricks
    Big data Databricks Spark
    +2
    This workflow demonstrates the usage of the Create Databricks Environment node which allo…
    knime > Examples > 10_Big_Data > 01_Big_Data_Connectors > 03_DatabricksExample
  2. Go to item
    Working with Utility Nodes
    File handling Zip Unzip
    +4
    Download compressed file, extract it, read extracted file and finaly delete extracted fil…
    knime > Examples > 01_Data_Access > 01_Common_Type_Files > 11_Working_with_Utility_Nodes
  3. Go to item
    Incremental Data Processing with Parquet
    Parquet Incremental loading NYC taxi dataset
    +3
    In this workflow, we will use the NYC taxi dataset to show case a continous preprocessing…
    knime > Examples > 01_Data_Access > 01_Common_Type_Files > 12_Incremental_processing_Parquet_file
  4. Go to item
    Google BigQuery meets Databricks
    Google BigQuery DB query Cloud
    +7
    This workflow connects to the Austin Bikeshare dataset, hosted among the Google BigQuery …
    knime > Examples > 10_Big_Data > 01_Big_Data_Connectors > 07_Will_They_Blend_BigQuery_Databricks
  5. Go to item
    Data Transfer between Clouds
    File handling Google Sharepoint
    +1
    This workflow demonstrates the utilization of the new file system connection nodes within…
    knime > Examples > 01_Data_Access > 06_ZIP_and_Remote_Files > 09_Data_Transfer_between_Clouds

No known nodes available

KNIME
Open for Innovation

KNIME AG
Hardturmstrasse 66
8005 Zurich, Switzerland
  • Software
  • Getting started
  • Documentation
  • E-Learning course
  • Solutions
  • KNIME Hub
  • KNIME Forum
  • Blog
  • Events
  • Partner
  • Developers
  • KNIME Home
  • KNIME Open Source Story
  • Careers
  • Contact us
Download KNIME Analytics Platform Read more on KNIME Server
© 2022 KNIME AG. All rights reserved.
  • Trademarks
  • Imprint
  • Privacy
  • Terms & Conditions
  • Credits