Project overview
About the Role This role involves migrating legacy ETL pipelines in Datastage to a modern stack, including SnapLogic, Python, Spark, and Dataflow. The focus will be on building data pipelines and orchestration, with a strong emphasis on data warehousing using Google BigQuery. The ideal candidate will be a proactive communicator and problem-solver, capable of making suggestions and asking questions. Key Responsibilities Analyze existing ETL pipelines and jobs in Datastage and migrate them to a modern stack. Develop new DAGs from scratch using Airflow or Cloud Composer for orchestration. Build data ingestion and ETL pipelines from scratch using SnapLogic, Python, SQL, Dataflow, and Spark. Support data modeling and understand data warehousing fundamentals. Ensure smooth data movement from source to warehouse, semantic or reporting layer, models, and reporting/BI. Qualifications/Requirements Experience in Datastage for migrating legacy ETL pipelines. Experience in SnapLogic is strongly preferred for data pipelines and integrations. Proficiency in Airflow or Cloud Composer for orchestration and DAG development. Experience in data warehousing, specifically with Google BigQuery . Understanding of analytics and data movement processes. Proactive communication skills and a problem-solving mindset. Experience in data modeling is beneficial but not required. Familiarity with technologies such as Kafka, Java, Apache Beam, and Alteryx is a plus.