Nishant Dwivedi

At Amex through Cognizant, I've migrated more than 50 analytical datasets from Apache Hive to Google BigQuery across three business units, reducing manual release effort by about 60% through GitHub Actions CI/CD pipelines.

I've designed and deployed 10+ end-to-end ELT pipelines with Apache Airflow and Google Cloud Composer, automating ingestion, transformation, scheduling, and monitoring while reducing manual intervention by about 70%.

I re-engineered PySpark SQL transformations into optimized BigQuery SQL, stored procedures, and UDFs, contributing to an approximately 40% reduction in query execution time. I also build data-quality validation and reconciliation processes, using source-to-target checks for row counts, aggregates, and referential integrity.

My work spans Python, PySpark, SQL, GCP, Apache Spark, dbt, Snowflake, SSIS, and Power BI reporting. I started at Cognizant building SSIS-based batch ETL and sales reporting pipelines, and continue to focus on dependable, performant cloud data systems.