Looking for a job

Mitesh Patel

@miteshpatel4

Message

Engineering Lead at Persistent Systems who modernized an Azure Data Mart and improved ETL performance 10x.

India

Message

At Persistent Systems, I designed metadata-driven Azure Data Factory pipelines and led an end-to-end Data Mart modernization using ADLS Gen2, PySpark, and Azure Synapse Analytics.

I migrated workloads from ADLS Gen1 to Gen2 and modernized legacy U-SQL processing to PySpark. I also implemented SCD Type 2 processing to support historical records and point-in-time reporting.

By optimizing Spark-based ETL workloads with PySpark and Azure Synapse Spark Pools, I achieved a demonstrated 10x improvement in processing performance. I integrated transformed datasets with Synapse Dedicated SQL Pool for reporting and analytics.

Earlier, I developed Azure data pipelines at Media Agility and SQL-based ETL processes at Zensar Technologies and Spectrum Consultants India. I hold five Microsoft certifications, including Azure Data Engineer Associate and Fabric Data Engineer Associate.

Experience

Work history, roles, and key accomplishments

Current

Engineering Lead

Current

Persistent Systems

Jan 2023 - Present (3 years 10 months)

Designed and implemented parameterized and metadata-driven data pipelines using Azure Data Factory. Led an end-to-end DataMart modernization initiative using ADLS Gen2, Azure Data Factory, PySpark, and Azure Synapse Analytics, and optimized Spark-based ETL workloads achieving a 10x improvement in performance.

Azure Data Factory ADLS Gen2 Azure Synapse PySpark SQL SCD2 Data Migration Data Reconciliation Performance Optimization Azure DevOps CI CD Agile Delivery Team Leadership

Associate Consultant

Media Agility

Dec 2021 - Dec 2022 (1 year)

Designed and developed cloud-based data pipelines using Azure Data Factory for enterprise data ingestion and transformation. Developed SQL-based data transformation and validation processes to support reporting and downstream analytics.

Azure Data Factory SQL Data Pipelines data transformation Data Validation Production Support

Sr. Software Engineer

Zensar Technologies

Nov 2019 - Nov 2021 (2 years)

Developed and optimized SQL and T-SQL-based ETL processes supporting enterprise data integration and reporting requirements. Designed complex stored procedures, SQL transformations, and data validation processes for large-scale relational datasets.

SQL T SQL ETL Stored Procedures Data Validation Query Optimization

SQL Developer

Spectrum Consultants India

Nov 2017 - Oct 2019 (1 year 11 months)

Developed complex SQL queries, stored procedures, views, and data transformation logic to support business reporting and data processing requirements. Designed and optimized SQL-based ETL processes for reliable extraction, transformation, and loading of enterprise data.

SQL Stored Procedures Views ETL data transformation