Open to opportunities

Yancheng Liu

@yanchengliu

Message

Senior data engineer delivering scalable real-time analytics and ML pipelines.

United States

Message

What I'm looking for

I seek senior engineering roles building scalable data platforms and ML pipelines, within collaborative teams that value observability, measurable impact, and growth.

I am a Senior Data Engineer with 8+ years designing and delivering end-to-end data solutions for real-time analytics, business intelligence, and AI applications across AWS, Azure, and GCP.

At Airbnb I built the Host Profile Data Service using Scala, Kafka, and Flink, integrated host activity into Snowflake pipelines that powered Host Passport and improved data freshness by about 23%.

I designed unified schemas and Databricks + Spark pipelines that reduced query latency by nearly 20%, developed ML feature pipelines with Python, TensorFlow, and Vertex AI, and automated 200+ Airflow workflows with Terraform to maintain 99% reliability and full observability.

I collaborate closely with PMs, data scientists, and UX teams to align data models with KPIs, and I have a proven track record modernizing metric monitoring, optimizing Delta Lake and Spark jobs, and delivering high-impact data platforms in e-commerce, logistics, and content domains.

Experience

Work history, roles, and key accomplishments

Current

Senior Data Engineer

Current

Airbnb

Jan 2022 - Present (4 years 5 months)

Built Host Profile Data Service using Scala, Kafka, and Flink and integrated host activity into Snowflake pipelines powering Host Passport, improving data freshness by ~23%. Designed Atlas schema and Databricks+Spark pipelines that reduced query latency by nearly 20% and simplified business reporting.

Scala Kafka Flink Snowflake Databricks Spark DBT Airflow Terraform

Senior Data Engineer

Instagram

Jan 2017 - Dec 2019 (2 years 11 months)

Built client-side logging pipeline for Instagram Stories and end-to-end data solutions integrating content integrity models with ad delivery metrics; developed exposure-based A/B testing metrics enabling fine-grained attribution.

Data Ingestion A B Testing Databricks Experimentation Data Modeling

Senior MTS/Developer, Analytics

Athenahealth

Jan 2016 - Dec 2017 (1 year 11 months)

Built large-scale data ingestion to a cloud MPP warehouse reducing query latency ~10x and developed a HIPAA-compliant transformation service enabling secure self-serve analytics.

Data Ingestion ETL HIPAA Cloud SQL

Education

Degrees, certifications, and relevant coursework

Georgia Institute of Technology

Master of Science, Computer Science

2013 - 2015

Completed a Master of Science in Computer Science with coursework and projects focused on large-scale data ingestion, cloud-based analytics, and HIPAA-compliant data transformation services.