At EmiratesNBD, I design and build scalable PySpark ETL pipelines for batch processing across UAE and KSA entities. I also develop real-time ingestion with Kafka Streams and Flink, integrating APIs and other sources into data lakes.
I optimize streaming workloads for under-200ms latency and work with Azure Databricks, Oozie, and data quality governance aligned with Collibra standards. I deploy and monitor applications using Grafana, Kibana, and OpenShift.
At DBS Bank, I developed more than 30 PySpark and Spark SQL jobs and optimized over 20 Spark and ETL jobs. I also built a mock data generation framework that reduced reliance on production data, and received the Data Excellence Champion Award for solutions that reduced system downtime by 30%.
India