Open to opportunities

AAYUSH PANDEY

@aayushpandey3

Message

Senior Data Engineer specializing in scalable cloud-native data pipelines.

United States

Message

What I'm looking for

I seek senior data engineering roles building secure, scalable cloud-native platforms and real-time pipelines, integrating ML, enforcing governance, mentoring teams, and working in collaborative Agile environments that value observability and compliance.

I am a Senior Data Engineer with 7+ years designing and implementing secure, production-grade data solutions across AWS, Azure and hybrid cloud environments. I build both batch and real-time pipelines and focus on scalable, testable data architectures.

I have delivered event-driven architectures using Apache Spark (PySpark, Scala), Kafka, Kinesis, and serverless components, and I implement modular ETL with DBT, Airflow, AWS Glue and Azure Data Factory. My work has reduced pipeline latency by over 40%, automated forecasting that cut manual investigations by 30%, and improved pipeline uptime and security posture through IaC and observability.

I champion data governance, compliance (HIPAA, SOX, FINRA), and observability while mentoring engineers and collaborating in Agile teams to align data platforms with business outcomes. I seek to scale ML-enabled analytics and robust data platforms that empower business users and maintain strong security controls.

Experience

Work history, roles, and key accomplishments

Current

Senior Data Engineer

Current

Cigna

Nov 2022 - Present (2 years 9 months)

Designed and deployed scalable ETL and real-time data pipelines on AWS, reducing system latency by over 40% and achieving 99.9% pipeline uptime; integrated ML scoring with MLflow and Lambda to reduce manual investigation efforts by 30% while enforcing HIPAA and SOX-compliant access controls.

AWS Glue Pyspark Kafka Kinesis MLFlow DBT Terraform Docker Redshift Cloudwatch

Data Engineer

Albertsons

Jun 2020 - Oct 2022 (2 years 4 months)

Built and optimized Azure-based ETL and streaming pipelines using Azure Data Factory, Databricks (PySpark/Scala), and Kafka, reducing manual reporting by 60% and enabling real-time customer analytics; migrated select pipelines to BigQuery to support multi-cloud benchmarking.

Azure Data Factory Databricks Pyspark Scala Azure Synapse Kafka DBT Airflow Power BI Looker

Data Engineer

Prudential Financial

Jun 2018 - May 2020 (1 year 11 months)

Developed and maintained enterprise ETL pipelines with Informatica, SSIS and Hadoop to support regulatory reporting and actuarial analytics, implementing Star/Snowflake schemas and automated Control-M schedules to improve nightly SLA adherence.