Remote Senior Data Engineer – Databricks / Azure Data Platform

We are looking for an experienced Senior Data Engineer to support the build-out and further development of a modern data platform in the insurance and reinsurance environment.
In this role, you will help design, build and maintain a scalable data platform based on Databricks on Azure. The platform is intended to centralize fragmented data sources, establish data quality checks, organize data into meaningful business domains and provide reliable data layers for reporting, applications, analytics and future AI use cases.
The platform supports key reinsurance processes, including underwriting, pricing, actuarial, reserving and finance. Data is ingested from various source systems, primarily SAP.

Build, maintain and optimize data pipelines in Databricks

Design and implement scalable Lakehouse and Medallion Architecture concepts

Structure data into meaningful business domains

Develop and maintain Bronze, Silver and Gold data layers

Create Gold layers to support business use cases, especially applications, reports and analytics

Integrate and process data from multiple source systems, primarily SAP

Implement data quality checks, monitoring and validation mechanisms

Support data modelling, database design, storage concepts and performance optimization

Contribute to governance, security and access management concepts

Support or build CI/CD pipelines for data engineering workflows

Support reporting requirements, most likely with Power BI and/or Databricks

Prepare the data platform for future AI and advanced analytics use cases

Enable easy onboarding of business users onto the data platform

Work closely with infrastructure, architecture, business, reporting and data teams

Several years of experience as a Senior Data Engineer or in a comparable role

Strong experience in building and maintaining data pipelines with Databricks

Solid knowledge of Lakehouse Architecture and Medallion Architecture

Good experience with cloud-based data platforms, ideally on Azure

Strong understanding of data quality, data governance, security and access management

Experience with data modelling, databases, storage technologies and performance optimization

Experience with CI/CD concepts in a data engineering environment

Good knowledge of the Databricks ecosystem and related components

Experience with SAP data sources is a strong advantage

Experience in insurance, reinsurance, finance or actuarial environments is beneficial

Knowledge of Power BI and/or Databricks reporting is an advantage

Ability to structure complex data landscapes and translate them into scalable solutions

Senior, independent and structured working style

Strong communication skills with both technical and business stakeholders

Originally posted on Himalayas