At bp, I maintain and improve AWS data pipelines built with Glue, Lambda, and Step Functions. I optimized production Glue jobs, reducing workers from 12 to 4 and runtime from 9 hours to 2 hours per region, and redesigned sequential processing to run in parallel.
At DataArt, I completed the migration of a monolithic legacy SSIS package to modular Apache Airflow DAGs for billing and CRM reporting. I also built the warehouse with the team and developed its ETL pipelines and SQL transformations.
At Positive Technologies, I developed streaming vulnerability detection pipelines with Spark Structured Streaming and Kafka, integrating ML models into the platform. I also optimized NVD ingestion to run three times faster, cutting execution from 12 hours to 4.
