Open to opportunities

Kaushik Tiwari

@kaushiktiwari

Message

I am a data engineer specializing in AWS, PySpark, and data platform optimization.

India

Message

What I'm looking for

I seek mid-level data engineering roles where I can build scalable AWS-based data platforms, lead migrations and automation, optimize costs, and improve data quality within collaborative, data-driven teams.

I am a data engineer with hands-on experience building and operating large-scale data platforms using AWS, Snowflake, PySpark and Apache Iceberg. I focus on designing reliable ETL pipelines, migrations, and automation that improve performance and reduce costs.

I've built an AI-powered chatbot for metadata and lineage discovery that cut lookup time by 70%, migrated 100+ tables to Apache Iceberg to triple query throughput, and minimized Athena workloads to lower processing costs by 65%. I also led a 15TB Redshift-to-Snowflake migration that improved query performance by 35% and developed a data quality and validation framework achieving 95% accuracy.

I prioritize automation, observability, and cost optimization—creating self-serve pipelines and real-time monitoring to reduce manual effort and support load. I enjoy collaborating cross-functionally to deliver scalable, production-ready data solutions that empower analytics and ML teams.

Experience

Work history, roles, and key accomplishments

Current

Software Development Engineer II

Current

Baazi Games

Mar 2025 - Present (5 months)

Built an AI-powered chatbot using AWS SageMaker, Python and Cohere to automate query and table-lineage discovery, reducing metadata lookup time by 70%. Migrated 100+ tables to Apache Iceberg with PySpark and Airflow, tripling query throughput, saving 100+ engineer hours monthly, and implemented Glue monitoring to cut support requests by 40% while lowering storage and processing costs by 65%.

Python AWS Sagemaker Cohere Apache Iceberg Pyspark Airflow Athena DynamoDB

Associate - Platform Services

ZS Associates

Mar 2024 - Mar 2025 (1 year)

Engineered scalable AWS-based data pipelines and PySpark jobs for a US healthcare client, improving processing time by 30% and reducing cloud costs 42% by migrating to AWS Graviton. Standardized reporting with Azkaban, cutting manual effort by 50% and designing data models for 10+ Tableau dashboards.

Python Pyspark Azkaban Data Modeling Tableau AWS Lambda

Software Engineer

To The New

Sep 2022 - Mar 2024 (1 year 6 months)

Led a 15TB Redshift-to-Snowflake migration and refactored ETL to improve query performance by 35%; developed a data quality and validation framework achieving 95% accuracy and built self-serve pipelines processing 800M+ rows/month.

Snowflake Redshift Python Jenkins SQL Data Quality ETL

Intern

Maersk Global Service Centers

Jun 2021 - Dec 2021 (6 months)

Completed an internship at Maersk Global Service Centers (June–December 2021), gaining practical data engineering experience and supporting team projects.

Python SQL Git Shell Scripting Data Modeling Linux