Open to opportunities

sivabhargav p

@sivabhargavp

Message

Senior data engineer specializing in cloud ETL/ELT, orchestration, and data warehouse platforms.

United States

Message

What I'm looking for

I want to lead end-to-end data engineering—ETL/ELT with DBT, orchestration with Dagster/Airflow, and production microservices—on cloud platforms. I’m looking for teams that value data quality, observability, security, and continuous delivery.

I’m an experienced Data Engineer with 10+ years designing, developing, and maintaining robust ETL/ELT pipelines, data warehouses, and data lakes for large-scale analytics and reporting.

In my recent roles as a Sr. Big Data Engineer and Sr Data Engineer, I built enterprise-grade transformations with DBT—using incremental materialization, version-controlled SQL, lineage tracking, and automated testing—so downstream BI and reporting stay accurate and compliant.

I orchestrate complex workflows with Dagster (and Airflow where needed), using sensor-based triggers, retries, idempotent patterns, and strong observability. I’ve also deployed production microservices using Azure Service Fabric and Azure Kubernetes Service, emphasizing health monitoring, rolling upgrades, and zero-downtime deployments.

My work consistently prioritizes reliability, data quality, and security controls—implemented through schema tests and assertions in DBT, CI/CD automation (Jenkins/Git), and secrets management with Azure Key Vault—so teams can move faster with confidence.

Experience

Work history, roles, and key accomplishments

Current

Senior Big Data Engineer

Current

American Express

Jan 2024 - Present (2 years 4 months)

Designed and maintained DBT-based ETL pipelines transforming financial transaction data into dimensional models with incremental materialization and lineage for SOX compliance. Orchestrated Dagster workflows, monitored SLAs with Prometheus/Grafana, and deployed ETL microservices on Azure Service Fabric and Kubernetes using Terraform and Key Vault security controls.

DBT SQL Dagster Azure Service Fabric Azure Kubernetes Service Terraform Prometheus Grafana Azure Key Vault

Senior Data Engineer

Ebay

Mar 2021 - Dec 2023 (2 years 9 months)

Built DBT and SQL ETL pipelines in Azure Synapse Analytics for retail sales forecasting and inventory optimization, enforcing PCI-DSS and GDPR requirements. Orchestrated transformations with Dagster, deployed microservices on Azure Service Fabric, and implemented CI/CD via GitHub Actions with observability using ELK Stack and New Relic.

DBT SQL Dagster Azure Synapse Azure Service Fabric Azure Kubernetes Service Terraform ELK Stack New Relic

Data Engineer

Southwest Airlines

May 2018 - Feb 2021 (2 years 9 months)

Developed DBT ETL pipelines and incremental loading in Azure Synapse Analytics and Azure SQL Database for airline operational reporting, supporting IATA and SOC 2 compliance. Orchestrated jobs using Dagster and Apache Airflow, deployed microservices on Azure Service Fabric and Kubernetes, and implemented monitoring and data quality controls with Splunk and Application Insights.

DBT SQL Azure Synapse Dagster Apache Airflow Azure Service Fabric Azure Kubernetes Service Azure Data Factory Terraform Splunk

Data Engineer / Migration Specialist

Cisco

Jan 2017 - Apr 2018 (1 year 3 months)

Migrated on-prem Hadoop (Hive/HBase) pipelines to AWS using S3, EMR, and Redshift to improve scalability and performance. Implemented Spark and Kafka/Kinesis batch and real-time ingestion with Airflow orchestration, added data quality checks with Great Expectations, and automated infrastructure provisioning with Terraform.

Amazon S3 Amazon EMR Amazon Redshift Apache Spark Kafka Amazon Kinesis Apache Airflow Terraform Great Expectations Tableau

ETL Developer

Change Healthcare

Jul 2015 - Oct 2016 (1 year 3 months)

Developed ETL pipelines using Informatica PowerCenter to extract from Oracle and SQL Server and load data into Amazon S3 and Amazon RDS, then into Redshift for reporting. Built batch processing with Hadoop (Hive/MapReduce) and Spark on EMR, using Sqoop for transfers, and scheduled/monitored jobs with Informatica scheduler, cron, and CloudWatch.