At Decision Culture, I architected an event-driven pipeline orchestration framework on Google Cloud Platform using FastAPI, Pub/Sub, Cloud Functions, and Airflow. It reduced manual campaign pipeline triggers by 100%, and I designed reusable DAGs for parallel campaign execution.
I built incremental refresh strategies for property datasets in BigQuery and refactored Jupyter Notebook workflows into Airflow DAGs using Cloud Composer. I also created Looker Studio dashboards for pipeline runtimes and SLA adherence, and helped move quality control processes into Dataplex.
At Precisely Software and Data India, I managed production data pipelines on AWS and automated more than 95% of previously manual tasks with Step Functions, Lambda, and Glue. I reduced AWS costs by 15% through resource and job optimizations; earlier, I worked with PySpark and Databricks at Exertia Consultancy and Hadoop, Hive, and HBase at Tata Consultancy Services.
