Skip to main content
XC
Looking for a job

Xiqian Chen

@xiqianchen

Senior AI Data Engineer at TikTok building real-time Kafka and Flink pipelines that deliver 10B+ daily events for For You Page ranking.

United States
Message

At TikTok, I lead the real-time Kafka and Flink ingestion workstream for For You Page ranking, landing 10B+ daily events in Delta Lake with sub-second freshness. This work contributed to a +0.6% watch-time lift in online A/B tests.

I designed Databricks lakehouse workflows that replaced full recomputes with incremental merges, bringing Silver and Gold table refreshes down from hours to minutes. I also developed multimodal embedding pipelines for 500M+ items and rolled out six model upgrades with zero retrieval downtime.

At TikTok, I delivered an online/offline feature store using Delta Lake, Redis, and Cassandra, reducing training/serving feature drift by 40%. I also optimized Flink and Kafka infrastructure, cutting costs by 25% and eliminating recurring checkpoint-timeout pages during peak traffic.

At DataLab USA, I architected DLINK and CLINK consumer identity resolution platforms using Apache Spark, Databricks, and SQL Server. CDC-based ingestion unified 25+ sources covering 200M+ consumers, while explainable matching supported FCRA-governed use cases.

Experience

Work history, roles, and key accomplishments

TikTok logoTI
Current

Senior Data Engineer

Mar 2022 - Present (4 years 7 months)

Designed Medallion lakehouse ELT workflows on Databricks with PySpark, Auto Loader, and Delta Lake Change Data Feed, replacing full recomputes with incremental merges. Developed batch and streaming multimodal embedding pipelines in PySpark for 500M+ items, and led real-time Kafka and Flink ingestion for the For You Page.

DU

Senior Developer (Data Engineering)

DataLab USA

Feb 2015 - Dec 2021 (6 years 10 months)

Modeled star-schema data marts with Type 2 slowly changing dimensions on Azure SQL Data Warehouse and Azure Synapse Analytics. Architected consumer identity resolution platforms on Apache Spark, Databricks, and SQL Server, using CDC-based incremental ingestion to unify 25+ demographic and behavioral sources.

Education

Degrees, certifications, and relevant coursework

University of Maryland logoUM

University of Maryland

Master of Science, Mathematical Statistics

Master of Science in Mathematical Statistics from the University of Maryland, completed in 2016.

University of Maryland logoUM

University of Maryland

Bachelor of Science, Statistics and Economics

Bachelor of Science in Statistics and Economics from the University of Maryland, completed in 2014.

Interested in hiring Xiqian?

You can contact Xiqian and 90k+ other talented remote workers on Himalayas.

Message Xiqian

People also viewed

View all talent

Get matched with your dream remote job

Sign up now and join over 250,000+ remote workers who receive personalized job alerts, curated job matches, and more for free!

Sign up
Himalayas profile for an example user named Frankie Sullivan