Hub
Pricing About
WorkflowWorkflow

Hive/Big Data - a simple "self-healing" (automated) ETL or analytics system on a big data cluster

HiveBig dataEtlReportAutomation
mlauber71 profile image
Draft Latest edits on 
Jan 10, 2022 9:35 AM
Drag & drop
Like
Download workflow
Workflow preview
Hive/Big Data - a simple "self-healing" (automated) ETL or analytics system on a big data cluster The scenario: you have a table on a big data system with daily data ("default.db_main_table") partitioned by d_date and you want to do some reporting (the number of lines per day) stored in a new table "default.db_analytics" The system is a big data system and you want to do this with partitioned Hive tables The main workflow should be able to be run several times per day and then do the report if it has not been done yet. If a day is missing it will do the job again until it is finished. You could do this by hand or schedule that on the KNIME server.

External resources

  • School of Hive - with KNIME's local Big Data environment (SQL for Big Data)
Loading deploymentsLoading ad hoc jobs

Used extensions & nodes

Created with KNIME Analytics Platform version 4.5.1
  • Go to item
    KNIME Base nodesTrusted extension

    KNIME AG, Zurich, Switzerland

    Version 4.5.1

    knime
  • Go to item
    KNIME DatabaseTrusted extension

    KNIME AG, Zurich, Switzerland

    Version 4.5.1

    knime
  • Go to item
    KNIME Data GenerationTrusted extension

    KNIME AG, Zurich, Switzerland

    Version 4.5.0

    knime
  • Go to item
    KNIME Extension for Local Big Data EnvironmentsTrusted extension

    KNIME AG, Zurich, Switzerland

    Version 4.5.1

    knime
  • Go to item
    KNIME JavasnippetTrusted extension

    KNIME AG, Zurich, Switzerland

    Version 4.5.0

    knime
  • Go to item
    KNIME Math Expression (JEP)Trusted extension

    KNIME AG, Zurich, Switzerland

    Version 4.5.0

    knime
  • Go to item
    KNIME Quick FormsTrusted extension

    KNIME AG, Zurich, Switzerland

    Version 4.5.0

    knime

Legal

By using or downloading the workflow, you agree to our terms and conditions.

KNIME
Open for Innovation

KNIME AG
Talacker 50
8001 Zurich, Switzerland
  • Software
  • Getting started
  • Documentation
  • Courses + Certification
  • Solutions
  • KNIME Hub
  • KNIME Forum
  • Blog
  • Events
  • Partner
  • Developers
  • KNIME Home
  • Careers
  • Contact us
Download KNIME Analytics Platform Read more about KNIME Business Hub
© 2025 KNIME AG. All rights reserved.
  • Trademarks
  • Imprint
  • Privacy
  • Terms & Conditions
  • Data Processing Agreement
  • Credits