At Motive, I led the design and implementation of a scalable ETL pipeline using Airbyte, Dagster, dbt, and Snowflake. I reduced data processing times by 35% through query optimization and pipeline improvements.
I also worked on Motive’s data platform with Apache Spark, Airflow, Kubernetes, and Snowflake, improving data processing speeds by 40%. For Athena Insight, I enabled ad-hoc queries over S3 data and organized the data lake to improve query performance and cost efficiency.
At Walmart, I led work on ETL and machine learning pipelines for sales prediction, where the team achieved a 15% accuracy improvement. I’ve also created a course on Big Data With PySpark.
