Large-Scale Automation / Public Sector
From Pentaho to Airflow: 150–180 automated workflows migrated
I led the technical work to migrate 150–180 ETL workflows from Pentaho to Apache Airflow, with a reported improvement of approximately 80% in performance and maintainability.
- My role
- Technical migration lead · NTT DATA
- Workflows migrated
- 150–180
- Last updated
- 01Pentaho Data Integration
- 02150–180 ETL workflows
- 03Apache Airflow
The context
The Cabildo de Tenerife data platform used Pentaho Data Integration for its ETL processes. The project involved modernising 150–180 workflows and moving them to Apache Airflow.
My contribution
At NTT DATA Europe & LATAM, I led the technical side of the migration, updating the existing platform and moving workflow orchestration to Apache Airflow. The project also included Power BI reporting.
The result
150–180 ETL workflows were migrated from Pentaho Data Integration to Apache Airflow, with a reported improvement of approximately 80% in performance and maintainability, and a more scalable foundation for operating the processes.
Value to the project
The project brought the workflows together in Apache Airflow and made them easier to maintain. The figures describe this specific migration.