Pradeep Gorle

I’m an Azure Data Engineer with 4.4 years of IT experience, focused on designing and developing scalable ETL/ELT pipelines on Microsoft Azure. I specialize in data ingestion, transformation, orchestration, performance optimization, and processing large-scale datasets with a strong reliability mindset.

At Infosys Limited, I architected end-to-end Azure data pipelines using Azure Data Factory and Azure Databricks—processing 100M+ records daily with 99.9% pipeline reliability. I engineered Medallion Architecture (Bronze/Silver/Gold) on Delta Lake, reducing analytics query time by 35% and improving data accessibility across 8 business units.

I’ve also driven faster time-to-insight by re-engineering workflows from batch (hours) to near real-time using incremental load patterns and change data capture (CDC). On the engineering side, I boosted PySpark job performance by 30–40% through partition tuning, broadcast joins, caching strategies, and Databricks cluster optimization.

To make data truly usable, I designed enterprise-grade Star Schema models with fact and dimension tables, enabling self-service reporting for 200+ Power BI dashboard users. I build robust ingestion with REST API pipelines for 10+ external data sources (including end-to-end data validation to reduce defects by 60%), and I enforce governance and security using Unity Catalog, Azure Key Vault, and RBAC—so teams can trust the data they act on.