Hub

88 results

Filter
Filter by tag
Data engineer
Best practices Data engineering Education ELT Spark Big data Orchestration Workflow dependencies Workflow service
  1. Workflow
    Orchestration Testflows
    Best practices Data engineer Data engineering
    +5
    In this workflow, we call all the tetflows, import their statuses and metadata and create a simple pdf report.
    lada > Public > Testing Framework > testflows > report > 03_Orchestration_Testflows
    0
    lada
  2. Workflow
    03.4_Writing_from_Spark_solution
    Education Data engineering Data engineer
    +3
    The company tracks the usage of the website and stores the information about each session. - Various data are collected, e.g., se…
    chemgirl36 > Public Space > L4-DE Best Practices for Data Engineering > solutions > Session_3_ELT_on_Big_Data > 03.4_Writing_from_Spark
    0
    chemgirl36
  3. Workflow
    03.0_Setup_Local_Big_Data_Environment
    Education Data engineering Data engineer
    +3
    This workflow sets up a local big data environment for the next exercise. It creates a local big data environment and loads the u…
    chemgirl36 > Public Space > L4-DE Best Practices for Data Engineering > solutions > Session_3_ELT_on_Big_Data > 03.0_Setup_Local_Big_Data_Environment
    0
    chemgirl36
  4. Workflow
    03.3_Aggregation_on_Spark_exercise
    Education Data engineering Data engineer
    +3
    The company tracks the usage of the website and stores the information about each session. - Various data are collected, e.g., se…
    chemgirl36 > Public Space > L4-DE Best Practices for Data Engineering > exercises > Session_3_ELT_on_Big_Data > 03.3_Aggregation_on_Spark
    0
    chemgirl36
  5. Workflow
    02.1_Data_Anonymization_exercise
    Best practices Data engineer Data engineering
    +1
    The company has two branches: one in the United States and another one in Europe. New US customer data comes from a web service. …
    chemgirl36 > Public Space > L4-DE Best Practices for Data Engineering > exercises > Session_2_ETL_Processing_II > 02.1_Data_Anonymization
    0
    chemgirl36
  6. Workflow
    04.3 Orchestration_exercise
    Best practices Data engineer Data engineering
    +4
    Once the workflow the ETL on Customers data is executed successfully, i.e., customer data are accessed and anonymized, and the da…
    chemgirl36 > Public Space > L4-DE Best Practices for Data Engineering > exercises > Session_4_Orchestration > 04.3_Orchestration
    0
    chemgirl36
  7. Workflow
    04.1_ETL_Customers_exercise
    Best practices Data engineer Data engineering
    +1
    The company has two branches: one in the United States and another one in Europe. New US customer data comes from a web service. …
    chemgirl36 > Public Space > L4-DE Best Practices for Data Engineering > exercises > Session_4_Orchestration > 04.1_ETL_Customers
    0
    chemgirl36
  8. Workflow
    04.2_ELT_Usage
    Education Data engineering Data engineer
    +3
    The company tracks the usage of the website and stores the information about each session. - Various data are collected, e.g., se…
    chemgirl36 > Public Space > L4-DE Best Practices for Data Engineering > exercises > Session_4_Orchestration > 04.2_ELT_Usage
    0
    chemgirl36
  9. Workflow
    04.0_Reset_DB&Big_Data_Environment
    Education Data engineering Data engineer
    +3
    This workflow resets the database by overwriting the customers and statistics tables and sets up a local big data environment and…
    chemgirl36 > Public Space > L4-DE Best Practices for Data Engineering > exercises > Session_4_Orchestration > 04.0_Reset_DB&Big_Data_Environment
    0
    chemgirl36
  10. Workflow
    01.1_Extract_WebService_data_solution
    Best practices Data engineer Data engineering
    +1
    The company has two branches: one in the United States and another one in Europe. New US customer data comes from a web service. …
    chemgirl36 > Public Space > L4-DE Best Practices for Data Engineering > solutions > Session_1_ETL_Processing_I > 01.1_Extract_WebService_data
    0
    chemgirl36
  11. Workflow
    01.2_Extract_S3_data&Blend_solution
    Best practices Data engineer Data engineering
    +1
    The company has two branches: one in the United States and another one in Europe. New US customer data comes from a web service. …
    chemgirl36 > Public Space > L4-DE Best Practices for Data Engineering > solutions > Session_1_ETL_Processing_I > 01.2_Extract_S3_data&Blend
    0
    chemgirl36
  12. Workflow
    01.1_Extract_WebService_data_exercise
    Best practices Data engineer Data engineering
    +1
    The company has two branches: one in the United States and another one in Europe. New US customer data comes from a web service. …
    chemgirl36 > Public Space > L4-DE Best Practices for Data Engineering > exercises > Session_1_ETL_Processing_I > 01.1_Extract_WebService_data
    0
    chemgirl36
  13. Workflow
    02.2 Database update_solution
    Best practices Data engineer Data engineering
    +1
    The company has two branches: one in the United States and another one in Europe. New US customer data comes from a web service. …
    chemgirl36 > Public Space > L4-DE Best Practices for Data Engineering > solutions > Session_2_ETL_Processing_II > 02.2_Database_update
    0
    chemgirl36
  14. Workflow
    Parse Contract Values Testflow
    Best practices Data engineer Data engineering
    +7
    For the company, it is important that contract values are always parsed correctly, even after the changes in the workflow. Theref…
    lada > Public > Testing Framework > testflows > report > 01_Parse_Contract_Values_Testflow
    0
    lada
  15. Workflow
    Best Practices for ETL on Customer Data
    Best practices Data engineer Data engineering
    +4
    This workflow demonstrates how to apply best practices to a simple ETL (Extract, Transform, Load) process on customer data. The c…
    lada > Public > Examples > Best_Practices_for_ETL_on_Customer_Data
    0
    lada
  16. Workflow
    03.1_In-database&Spark_processing_exercise
    Education Data engineering Data engineer
    +3
    The company tracks the usage of the website and stores the information about each session. - Various data are collected, e.g., se…
    chemgirl36 > Public Space > L4-DE Best Practices for Data Engineering > exercises > Session_3_ELT_on_Big_Data > 03.1_In-database&Spark_processing
    0
    chemgirl36
  17. Workflow
    03.2_Missing_value_imputation_on_Spark_exercise
    Education Data engineering Data engineer
    +3
    The company tracks the usage of the website and stores the information about each session. - Various data are collected, e.g., se…
    chemgirl36 > Public Space > L4-DE Best Practices for Data Engineering > exercises > Session_3_ELT_on_Big_Data > 03.2_Missing_value_imputation_on_Spark
    0
    chemgirl36
  18. Workflow
    00.1_Extensions_setup
    Education Data engineering Data engineer
    +1
    Open this workflow and install suggested extensions. Restart KNIME Analytics Platform. After the extensions are installed, you ca…
    hayasaka > KNIME Fall Summit Training 2022 > L4-DE Best Practices for Data Engineering > 00.1_Extensions_setup
    0
    hayasaka
  19. Workflow
    00.2_Setup_PostgreSQL_Database
    Education Data engineering Data engineer
    +1
    This workflow writes tables "customers", "contracts", "statistics" to PostgreSQL database. These tables in the database will be u…
    hayasaka > KNIME Fall Summit Training 2022 > L4-DE Best Practices for Data Engineering > 00.2_Setup_PostgreSQL_Database
    0
    hayasaka
  20. Workflow
    01.1_Extract_WebService_data_exercise
    Best practices Data engineer Data engineering
    +1
    The company has two branches: one in the United States and another one in Europe. New US customer data comes from a web service. …
    hayasaka > KNIME Fall Summit Training 2022 > L4-DE Best Practices for Data Engineering > exercises > Session_1_ETL_Processing_I > 01.1_Extract_WebService_data
    0
    hayasaka