Looking for a job

Arpan Samariya

@arpansamariya

Data Engineer at Zepto building real-time pipelines that handle 16M RPM for user experimentation and campaign audiences.

India

Message

At Zepto, I architect Spark Streaming frameworks with alerting, monitoring, and retries, including pipelines handling 16M RPM for user experimentation and campaign audiences with 800+ attributes per user.

I also moved workloads to Iceberg tables, built CDC ingestion and data quality frameworks, and developed an API-driven job creation framework that cut deployment time from one day to 10 minutes.

At HealthifyMe, I created a CDC-enabled Hudi data lake and supported the migration of 500+ OLTP tables to AWS DMS. I also managed a 4-node Redshift cluster, achieving 30% cost savings, and earned the Databricks Certified Data Engineer Professional certification.

Experience

Work history, roles, and key accomplishments

Current

Data Engineer III

Current

Zepto

Aug 2024 - Present (2 years 2 months)

Architected real-time analytics for 1 billion daily events at 500k RPM within 2-minute SLA using ClickHouse. Pioneered Iceberg adoption and delivered Spark Streaming pipelines handling 16M RPM throughput.

Kafka PySpark Databricks CDC Clickhouse Spark Streaming Python Data Quality Terraform Cloudwatch Grafana SQL Data Structures IAM Connectors Data Lake Lakehouse ETL AWS PostgreSQL Data Modeling Warehouse Alerting Infra

SDE - Data Engineer I

HealthifyMe

Apr 2022 - Jul 2024 (2 years 3 months)

Created Hudi managed data lake with change data capture for MySQL RDS using DMS. Directed migration of 500+ OLTP tables to AWS DMS and managed a 4-node Redshift cluster achieving 30% cost savings.

Hudi AWS DMS Redshift Terraform Cloudwatch Slack Tableau Metabase Databricks EMR Glue Kinesis SQL Data Modeling ETL Data Quality

Education

Degrees, certifications, and relevant coursework

National Institute of Technology, Hamirpur

Bachelor of Technology, Computer Science and Engineering

2016 - 2020

Grade: 8.58

Activities and societies: Vice President, ISTE Students' Chapter; Finance Head, TEDxNITH; Member of Organizing, Speaker Curation and Finance Team in first ever TEDx talk at NITH.

Pursued a Bachelor of Technology in Computer Science and Engineering, graduating with a grade of 8.58.