Skip to main content
EC
Looking for a job

Eric Cao

@ericcao

Senior Data Engineer at Databricks who cut telemetry query latency from 42 to 14 minutes across a multi-cloud lakehouse.

United States
Message

At Databricks, I lead the architecture and roadmap for a multi-cloud telemetry lakehouse that brings together more than 30 operational sources across AWS, Azure, and GCP. It processes approximately 3.2 TB daily for product, engineering, finance, support, and data science teams.

I designed batch and near-real-time pipelines that reduced median source-to-query latency from 42 to 14 minutes. I also built governed Delta Lake medallion layers and dimensional models that support more than 60 datasets and 40+ AI/BI dashboards.

At Change Healthcare, I built pipelines and data models for claims, clinical interoperability, payment analytics, and population-health use cases. I worked with healthcare standards including X12 837/835, HL7, and FHIR, while adding reconciliation and data-quality controls that reduced recurring production issues by approximately 35%.

At PayPal, I developed Java and Scala services for the Gimel Data API, enhanced Kafka connectors, and extended PCatalog metadata capabilities. Across these roles, I’ve worked on reliable, governed data platforms, from production operations and observability to secure access and data discovery.

Experience

Work history, roles, and key accomplishments

Databricks logoDA
Current

Senior Data Engineer

Aug 2022 - Present (4 years 2 months)

Led architecture and roadmap of a multi-cloud telemetry lakehouse, processing ~3.2 TB daily. Designed batch and near-real-time pipelines, built Delta Lake medallion layers, and implemented governance and observability, reducing compute spend by 35% and p95 query latency by 50%.

PayPal logoPA

Software Engineer

May 2018 - Sep 2020 (2 years 4 months)

Developed backend services and streaming connectors for PayPal's enterprise data platform, contributing to the Gimel data-access layer and PCatalog services. Enhanced Kafka connectors and built Elasticsearch-backed metadata indexing.

Education

Degrees, certifications, and relevant coursework

Florida International University logoFU

Florida International University

Master's Degree, Computer Science

2018 - 2020

Master's Degree in Computer Science from Florida International University, completed from 2018 to 2020.

University of Miami logoUM

University of Miami

Bachelor's Degree, Computer Science

2014 - 2018

Bachelor's Degree in Computer Science from the University of Miami, completed from 2014 to 2018.

Interested in hiring Eric?

You can contact Eric and 90k+ other talented remote workers on Himalayas.

Message Eric

People also viewed

View all talent

Get matched with your dream remote job

Sign up now and join over 250,000+ remote workers who receive personalized job alerts, curated job matches, and more for free!

Sign up
Himalayas profile for an example user named Frankie Sullivan