Exercise 01 Development Data Pipeline

Learning objective: In this exercise you'll practice capturing the data transformation workflow segment by means of Integrated Deployment extension and then writing the captured workflow segment to a specific location

Workflow description: This workflow showcases the process of reading orders data from restaurants and performing set of transformations to clean it and write the processed data to the database. Find below the three major tasks performed in this workflow

A sample of Orders data is read using the excel reader node, taking a sample of data allows us to experiment rapidly

Further the data is cleaned to remove duplicate rows and irrelevant columns, and then it is summarized using the shared component

At last, the transformed data is written to development database to allow us to check the consistency of the processed data