At Tata Consultancy Services (TCS), I built production ETL/ELT pipelines processing 5M+ records daily with Python, PySpark, SQL, Airflow, and AWS S3. I also developed data validation and reconciliation frameworks that reduced data defects by 70% and supported enterprise data migration.
I optimized Spark workloads using partitioning and storage optimization techniques, and set up CI/CD pipelines for automated testing and deployment of data pipeline code. At InSolare Energy, I developed an interactive Power BI dashboard for sales performance and business KPIs.
In my projects, I built an ETL pipeline for 1M+ Uber trip records loading into BigQuery and designed a real-time Kafka market-data pipeline using AWS services. I also implemented a Databricks lakehouse using Medallion Architecture and communicated Power BI insights to 50+ business stakeholders.
