At Ernst & Young, I engineer near-real-time data pipelines with Kafka, Spark Streaming, and Delta Lake to deliver audit and compliance insights within minutes. I also optimized PySpark transformations across Databricks and Snowflake, reducing processing failures by 34%.
I build batch and streaming Spark jobs in Python and Scala, and orchestrate pipeline dependencies with Apache Airflow. My work includes Hadoop pipelines using HDFS, Hive, and Sqoop, as well as serverless data pipelines with AWS Lambda, API Gateway, and Glue.
At Ryder System, I built real-time pipelines with AWS Kinesis and Lambda for operational dashboards. I also optimized SSIS and SQL-based ETL processes for DB2 and PostgreSQL, reducing processing times by 35%, and collaborated on Power BI dashboards for fleet performance.
