At Uplight, I designed and built an orchestration layer for long-running Spark workflows, with explicit job lifecycle management and local or Kubernetes execution. I also migrated critical Measurement & Verification workflows into the new framework.
I reworked heavy workflows processing month-scale device telemetry and cut a core Spark workflow’s runtime by ~50% by fixing duplicate reads and transformations. I also modernized deployments from Jenkins-driven Ansible playbooks to Helm charts deployed through Argo CD, reducing feature-branch deployment time to about 3.5 minutes.
At Shipwell Logistics, I led the design and implementation of a backend order consolidation service that integrated with a legacy platform and ran machine-learning models asynchronously. I also moved low-priority location data from MongoDB to Parquet files in S3/Athena, reducing storage from 14TB to 73GB, and built RabbitMQ services for shipment ETA updates. Earlier at Capital One, I built data ingestion and machine-learning evaluation systems, including a pipeline processing more than 500,000 call-center transcripts per day.
