I’m seeking a Senior or Staff Data Engineering role focused on scalable data platforms, real-time pipelines, and AI infrastructure. I enjoy hands-on work with Databricks, Spark, Kafka, and AWS, while owning architecture, mentoring engineers, and delivering business impact.
John Peng
@johnpeng
Staff Data Engineer, 11+ years of experience.
What I'm looking for
Staff Data Engineer, 11+ years, specializing in real-time data platforms at retail and marketplace scale - 360K transactions/sec across 2,000+
T arget stores, 11PB+ and 10,000+ daily ETL jobs at Airbnb. Build streaming and lakehouse infrastructure on Kafka, Flink, Spark,
Databricks/Delta Lake, and Kubernetes, delivered as a self-service platform with T erraform/IaC, CI/CD, Unity Catalog governance, and SLO-
driven reliability. Lead architecture across Data Engineering, ML, SRE, and Supply Chain orgs with a platform-as-a-product mindset:
performance, resiliency, developer experience, and cost.
Experience
Work history, roles, and key accomplishments
Set technical direction for an AWS-based enterprise data platform powering Drive Up, Shipt, inventory availability, order fulfillment, and store operations across 2,000+ stores. Owned the architecture and rollout of the Databricks lakehouse on AWS, built the platform as an internal product, standardized infrastructure delivery with Terraform and CI/CD, and instituted observability, reliability, an
Designed and delivered high-throughput batch and real-time data pipelines for Target's Drive Up/Shipt fulfillment ecosystem. Engineered event-driven ingestion and CDC pipelines, built and optimized large-scale Spark/PySpark pipelines, developed real-time inventory processing workflows, and implemented automated data-quality and reconciliation frameworks.
Built and optimized petabyte-scale data platforms at Airbnb using Python, SQL, Spark, Hive, Kafka, and Airflow, scaling infrastructure to 11PB+, supporting 10,000+ daily ETL jobs, and reducing infrastructure costs by 70% while improving query performance and data quality.
Developed SQL-based reporting and Tableau dashboards to analyze merchant acquisition, store performance, conversion, payment trends, and customer retention. Built automated reporting pipelines and conducted ad hoc KPI analysis to support data-driven decisions across product, marketing, and merchant success teams.
Education
Degrees, certifications, and relevant coursework
University of the District of Columbia
Bachelor's Degree, Electrical and Electronics Engineering
2011 - 2015
Earned a Bachelor's degree in Electrical and Electronics Engineering from the University of the District of Columbia, completed in 2015.
Tech stack
Software and tools used professionally
Amazon Redshift
Apache Spark
Presto
AWS Glue
Apache Flink
Tableau
AWS IAM
Amazon CloudWatch
Amazon S3
Kubernetes
GitHub Actions
PySpark
Debezium
dbt
MySQL
PostgreSQL
Hadoop
RocksDB
Shopify
Gmail
Databricks
Terraform
Python
Java
Scala
MLflow
Kafka
Amazon Kinesis
AWS Lambda
Kafka Streams
OAuth2
Docker
Airflow
Root Cause
Amazon EMR
SQL
Amazon EBS
Apache Iceberg
LlamaIndex
Pinecone
Delta Lake
Great Expectations
Bash
pgvector
Agentic
LangGraph
OpenLineage
Unity Catalog
Apache Kafka
Availability
Location
Authorized to work in
Salary expectations
Social media
Skills
Interested in hiring John?
You can contact John and 90k+ other talented remote workers on Himalayas.
Message JohnGet matched with your dream remote job
Sign up now and join over 250,000+ remote workers who receive personalized job alerts, curated job matches, and more for free!
