At bp, I maintain AWS data pipelines and optimized production Glue jobs, reducing workers from 12 to 4 and runtime from 9 hours to 2 per region. I also redesigned sequential pipeline execution to run in parallel and contribute to migrating legacy pipelines to Databricks.
At Positive Technologies, I built Spark Structured Streaming pipelines for real-time vulnerability detection and helped identify vulnerabilities before vendor advisories were published. Across my work, I’ve also completed a legacy SSIS-to-Airflow migration and developed data and ML workflows for products including a credit decision system and an Amazon price prediction project.
