04.3 Orchestration_solution

Once the workflow the ETL on Customers data is executed successfully, i.e., customer data are accessed and anonymized, and the database table "customers" is updated, the next process - ELT on Usage data - should be triggered. In the next workflow, the usage data are accessed, transformed on Spark, and aggregated into customer "statistics" table. If the first workflow didn't execute successfully, the second one shouldn't be started.

External resources

• L4-DE Course Slides