At Deloitte Touche Tohmatsu India LLP, I design and implement real-time streaming pipelines with Pub/Sub and Cloud Dataflow, handling late-arriving data with windowing, watermarking, and triggers.
I optimize Dataflow performance and costs through worker autoscaling, parallelism tuning, and combiner optimizations. For batch loads, I use dead-letter queues and retry patterns to isolate and reprocess malformed records.
I build BigQuery transformation workflows and PySpark processing frameworks, including jobs on Dataproc that write to BigQuery. I also tune query performance with partitioning, clustering, and materialized views.
Earlier, at YuMe India Private Limited, I developed Hadoop data processing and batch pipeline solutions. At TATA Consultancy Services, I worked on server builds, MySQL performance tuning, and shell scripts for monitoring and backups.
