At Accenture, I designed, built, and maintained Databricks ETL pipelines for DineDash, an end-to-end food-delivery analytics project. The pipelines process structured dimension tables and nested JSON order data through a Bronze-to-Silver-to-Gold Medallion architecture.
I established Silver-layer validation for nulls, data types, and business rules, and built Bronze tables with schema enforcement and lineage tracking. I also parsed nested order-line items with PySpark and used Spark SQL to support downstream reporting.
I optimized pipeline performance and storage layout, and collaborated with engineering stakeholders to translate data requirements into pipeline designs. I documented data models, schemas, and processes for traceability and troubleshooting.
For my EEG signal project, I processed and validated data using MNE and pyEDFlib, then used hypothesis testing and feature validation to evaluate entropy features. I benchmarked five classifiers with cross-validation; the best configuration achieved 97.39% accuracy and a 0.97 F1 score.
