Arunesh Tiwari

I’m a Data Engineer and core contributor on large-scale data pipeline migrations, focused on reliability, data integrity, and minimal downtime. At Bobble.ai, I helped migrate data pipelines from GCP to AWS, leading the data engineering team through high-throughput workloads.

I refactor and optimize pipeline performance during migrations—improving processing efficiency, reducing infrastructure costs (approximately INR 1,00,000/month), and enabling secure transfers between Google BigQuery and AWS Redshift/Athena. I also design and deploy PySpark data and ML pipelines on Dataproc and EMR, using validation filters, UDFs, and performance tuning (repartitioning, broadcast joins, persistent storage) to handle millions of records daily.

I build ETL to process large-scale user data and transaction signals (including 1.5B+ daily session event records and millions of user transaction records), and I lead UAT across data and optimization projects to ensure production readiness. I also strengthen pipeline operations through automated monitoring/alerting, consolidate fragmented datasets across platforms for faster query performance, mentor interns, and run cloud cost-optimization initiatives.