Open to opportunities

Brian Ward

@brianward1

Message

Senior data engineer building scalable cloud ETL/ELT and real-time analytics.

United States

Message

What I'm looking for

I’m looking to lead cloud data platform and ELT/analytics initiatives—modernizing pipelines, improving SLA reliability, and driving measurable cost/performance wins with strong engineering discipline and mentoring.

I’m a Senior Data Engineer with 10+ years designing scalable cloud-native data platforms, building enterprise ETL/ELT pipelines, and enabling real-time analytics across healthcare, streaming media, and education. I focus on distributed systems, data warehousing, platform modernization, and analytics engineering—consistently delivering high-volume solutions that improve reliability and cost efficiency.

At Highmark Health, I spearheaded modernization into Snowflake and Databricks pipelines processing 15TB/day of healthcare and claims data, launching the Care360 Data Lake Initiative with AWS, Spark, Kafka, and Airflow for near real-time intelligence. I’ve also built ELT frameworks with dbt/Snowflake/Airflow to reach 99.9% SLA compliance, reduced execution times by 45%, and cut annual cloud infrastructure costs by more than $400K through warehouse optimization. Earlier at CVS Health and Netflix, I engineered scalable ELT/streaming pipelines, strengthened observability with Datadog/CloudWatch, and supported large-scale analytics and operational intelligence.

Experience

Work history, roles, and key accomplishments

Current

Senior Data Engineer

Current

Highmark Health

May 2022 - Present (4 years 1 month)

Spearheaded development of an enterprise healthcare insights platform, modernizing legacy ETL into scalable Snowflake and Databricks pipelines processing 15TB of claims data daily. Delivered near real-time analytics with AWS/Spark/Kafka/Airflow and improved pipeline reliability to 99.9% SLA compliance while reducing execution times 45% and cloud costs by $400K+.

AWS Snowflake Databricks Apache Spark Kafka Apache Airflow DBT Python Terraform CI CD

Data Engineer

CVS Health

Mar 2021 - Apr 2022 (1 year 1 month)

Engineered scalable ELT pipelines using Snowflake, Airflow, and Python to support enterprise pharmacy analytics and customer reporting. Automated AWS ingestion frameworks, reducing manual processing effort by 60% while improving pipeline stability and monitoring.

Snowflake Apache Airflow Python Kafka Apache Spark AWS SQL CI CD

Data Engineer

Netflix

Aug 2018 - Feb 2021 (2 years 6 months)

Contributed to a streaming analytics optimization platform by building distributed Spark and Kafka pipelines processing billions of events daily. Optimized Snowflake and Redshift warehouse workloads to improve large-scale query performance by 40%, and implemented observability with Datadog and CloudWatch.

Apache Spark Kafka AWS Python Apache Airflow Snowflake Amazon Redshift Datadog Cloudwatch Distributed Systems

Big Data Engineer

Pearson

Feb 2016 - Aug 2018 (2 years 6 months)

Built Hadoop and Spark-based big data solutions for digital learning analytics and student performance reporting. Developed ETL workflows with Hive, Sqoop, Python, and SQL at scale, and migrated on-prem Hadoop workloads to AWS to improve scalability, performance, and operational efficiency.