At Abacus Insights, I architect multi-layered ELT pipelines for healthcare data, delivering domain-specific Foundational Marts for analytics and executive reporting.
I design and maintain Apache Spark and Databricks pipelines that process 500M+ records per run, with Delta Lake ACID guarantees. I also built data quality frameworks and HL7/FHIR validation checks that reduced downstream data defects by 62%.
At Abacus Insights, I redesigned PySpark transformation jobs, increasing processing throughput by 3× and eliminating daily job failures that had caused downstream downtime. I partnered with Product, BA, QA, and Engineering on UAT for four releases, achieving zero compliance violations and on-time delivery across milestones.
Previously, as a Software Engineer at Digital Convergence Technologies (DCT), I supported a migration from AWS S3 and Athena to Databricks, re-engineering Athena SQL queries into Spark-based ETL pipelines while preserving business logic and data continuity. I also analyzed e-commerce datasets to surface customer behavior patterns and funnel drop-offs.
