At EY, I optimized AI-converted SAS/Spark code with Databricks best practices, reducing processing time from about five hours to under 10–20 minutes for migrated workloads. I also developed Databricks Jobs and SDP pipelines to automate migration workflows and ETL orchestration.
I contributed to a Python-based data engineering framework at EY, including a reusable library that dynamically generates Airflow DAGs from YAML configurations. Its Excel-to-YAML workflow reduced development time by 60%.
For a leading UK insurance company, I developed a Customer Master Data Management data lake that integrated multiple product lines into a unified customer view. I built pipelines using Google Cloud services to extract data from AWS S3, Oracle databases, and FintechOs API.
At Tata Consultancy Services, I architected a data pipeline for a UK telecommunications company processing 500M+ records per hour. Earlier, I developed a star-schema datamart and a low-code ETL framework for an online travel platform, and worked on knowledge graph development using cloud vision, NLP, auto NLP, and Vertex AI.
