Microsoft Fabric Lakehouse Integration

Microsoft Fabric / PySpark Data Engineer Needed

We need an experienced Microsoft Fabric and PySpark freelancer to work with our backend team on modernizing a high-volume SQL-based data processing platform.

Currently, large raw datasets are loaded into SQL Server and processed through multiple stored procedures for data quality, validation, transformations, and business rules. We want to move the heavy data-processing components to Fabric Lakehouse + Delta + Spark/PySpark, while keeping appropriate workloads in SQL/Fabric Warehouse.

You should be able to:

Build Fabric Lakehouse/OneLake architecture
Develop Spark/PySpark transformations
Work with Delta tables
Build/modify Fabric or ADF pipelines
Convert suitable SQL stored-procedure workloads to Spark
Implement incremental/batch processing
Optimize processing of very large datasets
Integrate curated data with Fabric Warehouse/SQL
Work closely with our existing backend developers

Must have: Microsoft Fabric, PySpark/Spark, Delta Lake, SQL, Fabric Warehouse, and ADF/Fabric Data Factory experience.

This is a hands-on development role, not an architecture-only engagement.