Lei Shi

LS

Looking for a job

Lei Shi

@leishi

Lead Data Engineer at Mercaso who built AWS data platforms supporting 100+ database-table and third-party API pipelines.

At Mercaso, I own the architecture and core implementation of an AWS data platform while technically leading a five-person team. I built a platform supporting 100+ database-table and third-party API pipelines, with automated retries, Slack alerts, and PagerDuty escalation.

I also designed a storage and warehouse modernization framework using dbt, Airflow, EMR Serverless, and Apache Iceberg for approximately 200 models. At NEXT Trucking, I developed an offline BigQuery warehouse and replaced manual Kafka Connect deployments with source-controlled lifecycle management.

Experience

Work history, roles, and key accomplishments

ME

Current

Senior Data Engineer / Data Team Lead

Current

Mercaso

Jun 2024 - Present (2 years 4 months)

Owns architecture and core platform code for an AWS data platform and offline warehouse while leading a five-person team. Implemented Terraform infrastructure, Airflow orchestration, and CI/CD, and built a platform supporting 100+ pipelines with automated retries and alerts.

Airflow Spark DBT Apache Iceberg Kafka AWS Terraform Python SQL CI CD EMR S3 Glue Athena Amazon EKS IAM Docker Kubernetes Helm CircleCi

NT

Data Engineer

NEXT Trucking

Sep 2022 - May 2024 (1 year 8 months)

Developed an offline BigQuery warehouse and migrated scheduling from Cloud Composer to Airflow on AWS EKS. Designed source-controlled lifecycle management for approximately 200 Kafka Connect connectors with drift detection and change tracking.

BigQuery Airflow Amazon EKS Kafka Connect API Cloud Composer SQL Python Configuration Management CI CD

MT

Big Data Development Engineer

Meilin Data Technology

Apr 2021 - Sep 2022 (1 year 5 months)

Built Spring Boot and Java interfaces for Hive metadata, HDFS, HBase, and Spark/Flink submission on YARN and Kubernetes. Implemented a YARN client and Application Master for standalone Java and DataX workloads.

Java Spring Boot Hive HDFS HBase Spark Apache Flink Yarn Kubernetes Hadoop CDH Huawei FusionInsight Kerberos DataX

PS

Data Mining Engineer

Ping An Property Insurance, Shaanxi

Apr 2020 - Feb 2021 (10 months)

Developed warehouse tables and dashboards and built a customer-classification model for precision marketing.

Data Warehouse SQL Dashboarding Machine Learning Customer Segmentation

US

Data Mining Engineer Intern

University of Massachusetts Medical School

Nov 2018 - May 2019 (6 months)

Built a deep-learning system to generate text answers from medical image and question pairs.

Deep Learning Python Medical Imaging NLP

QC

Data Analyst

Qitong Investment Corporation

Jan 2015 - May 2017 (2 years 4 months)

Performed data analysis tasks supporting investment decisions.

Data Analysis SQL Excel

AG

Account Manager

Agricultural Bank of China, Guangdong

Jul 2013 - Sep 2014 (1 year 2 months)

Managed client accounts and banking services.

Banking Client Relations Account Management

Education

Degrees, certifications, and relevant coursework

WI

Worcester Polytechnic Institute

Master of Science, Data Science

M.S. in Data Science from Worcester Polytechnic Institute, Massachusetts, USA, completed in 2019.

NU

Nankai University

Bachelor of Arts, Economics

B.A. in Economics from Nankai University, Tianjin, China, completed in 2013.

Tech stack

Software and tools used professionally

Availability

Looking for a job

Location

China

Authorized to work in

Salary expectations

30k+ USD

Social media

Job categories

Data Engineer Big Data Developer Data Mining Engineer

Skills

Data Engineering Data Platform Architecture Data Warehouse ETL ELT Batch Processing Stream Processing Workflow Orchestration Airflow Apache Spark DBT Apache Iceberg Kafka Kafka Connect API Apache Flink Apache Hive

Interested in hiring Lei?

You can contact Lei and 90k+ other talented remote workers on Himalayas.

People also viewed

View all talent

Get matched with your dream remote job

Sign up now and join over 250,000+ remote workers who receive personalized job alerts, curated job matches, and more for free!