At Rearc, I migrated 1,000+ Java Spark jobs to PySpark and Spark SQL on Databricks, replacing job-specific code with reusable, config-driven transformation templates. I also automated builds, tests, and deployments with GitLab CI/CD, Databricks Asset Bundles, and pytest.
I built a real-time CDC pipeline from Cloud Spanner Change Streams through Dataflow to Pub/Sub, with ordering keys for chronological delivery. I also built self-service tooling for a GCP lakehouse migration, generating Kubernetes and GCP resource specs from validated Java configs.
Previously, I maintained Gravy Analytics' ETL platform processing tens of billions of geolocation data points per day. At SecurityScorecard, I built pipelines that scored 12M+ domains daily and productionized machine learning models for cybersecurity ratings; I hold the Google Cloud Professional Data Engineer certification.
