Hub
  • Software
  • Blog
  • Forum
  • Events
  • Documentation
  • About KNIME
  • KNIME Hub
  • knime
  • Spaces
  • Education
  • Courses
  • L4-BD Introduction to Big Data with KNIME Analytics Platform
  • 3_Spark
  • 4_Examples
  • 09_Big_Data_Irish_Meter_on_Spark_only
WorkflowWorkflow

Local big data Irish meter

Hive Spark Spark PCA Spark Pivot Local Big Data Environment
+5

Last edited: 

Drag & drop
Like
Download workflow
Copy short link
Workflow preview
This workflow uses a portion of the Irish Energy Meter dataset, and presents a simple analysis based on the whitepaper "Big Data, Smart Energy, and Predictive Analytics". It is intended to highlight KNIME's Big Data and Spark functionality. The workflow creates a Local Big Data Environment, loads the meter dataset to Hive, and then transfers it into Spark. It uses a series of Spark SQL nodes to create datetime fields, and then uses Spark nodes to aggregate energy usage over these datetime fields. In the wrapped metanode, it performs PCA and k-means using Spark nodes, and does some simple visualizations of the clustered data. Finally, it writes the clustered data out to both Hive and Parquet formats.

Used extensions & nodes

Created with KNIME Analytics Platform version 4.4.0
  • Go to item
    KNIME Base nodes Trusted extension

    KNIME AG, Zurich, Switzerland

    Version 4.4.0

  • Go to item
    KNIME Database Trusted extension

    KNIME AG, Zurich, Switzerland

    Version 4.4.0

  • Go to item
    KNIME Extension for Apache Spark Trusted extension

    KNIME AG, Zurich, Switzerland

    Version 4.4.0

  • Go to item
    KNIME Extension for Local Big Data Environments Trusted extension

    KNIME AG, Zurich, Switzerland

    Version 4.4.0

  • Go to item
    KNIME JavaScript Views Trusted extension

    KNIME AG, Zurich, Switzerland

    Version 4.4.0

  1. Go to item
  2. Go to item
  3. Go to item
  4. Go to item
  5. Go to item
  6. Go to item

Legal

By using or downloading the workflow, you agree to our terms and conditions.

Discussion
Discussions are currently not available, please try again later.

KNIME
Open for Innovation

KNIME AG
Hardturmstrasse 66
8005 Zurich, Switzerland
  • Software
  • Getting started
  • Documentation
  • E-Learning course
  • Solutions
  • KNIME Hub
  • KNIME Forum
  • Blog
  • Events
  • Partner
  • Developers
  • KNIME Home
  • KNIME Open Source Story
  • Careers
  • Contact us
Download KNIME Analytics Platform Read more on KNIME Server
© 2022 KNIME AG. All rights reserved.
  • Trademarks
  • Imprint
  • Privacy
  • Terms & Conditions
  • Credits