Rushiraj Shinde

I’m a Senior Data Engineer with 4+ years of experience building large-scale data pipelines, specializing in GCP, Airflow, SQL, and Python. I focus on performance optimization, scalable architecture, and reliable delivery through metadata-driven frameworks.

At Infosys Limited, I modernized an enterprise Medicaid data warehouse on Google Cloud by architecting MIDAS, a unified metadata-driven ETL audit and control framework (BigQuery + Airflow). I developed end-to-end incremental ELT pipelines for multiple domains processing 100M+ records daily, including idempotent load patterns, automated cycle classification, and metadata-configurable retry behavior.

I also built a metadata-driven Data Quality Rule Engine with pre-load and post-load checks, plus multi-tier audit observability across BATCH_AUDIT, MEASUREMENT_AUDIT, and MD_DQ_LOG for full run lineage and traceable DQ results. Previously at Quantiphi, I led migrations from SAP HANA and Teradata to BigQuery, converted BTEQ/HANA logic into BigQuery procedures/views, and reduced ETL cycle runtime by optimizing scheduling dependencies with Airflow Datasets.