yogesh selvakumar

I’m a results-driven Azure Data Engineer with 3+ years of experience architecting enterprise-grade data platforms on Azure. I build high-throughput ETL/ELT pipelines using Azure Data Factory and Databricks, turning raw data into analytics-ready datasets with optimized PySpark and SQL transformations.

In my current role at Capgemini (Apr 2023 – Present), I designed source-specific ingestion pipelines in ADF for JSON, CSV, Snowflake, and SQL Server, orchestrated through master pipelines. I land raw data into ADLS in Parquet format and integrate with Azure Databricks using Unity Catalog for governed, centralized access.

I’ve implemented Medallion Architecture (Bronze, Silver, Gold) with Delta Lake and Delta Tables, including ACID transactions, schema evolution, and time travel. By tuning Spark jobs with partitioning, caching, broadcast joins, and cluster right-sizing, I reduced Spark job runtimes by 45–60%, while also delivering SCD Type 1/Type 2 strategies and analytics-ready star schema tables.