At Go Digital Technology Consulting LLP, I work on production data engineering projects including iGrid Cloud Data and Forecast Engine. I reduced annual AWS infrastructure costs by about $29,000 and cut a derived data pipeline’s load time from 4.5 hours to 1.5 hours.
For iGrid Cloud Data, I configured, developed, and deployed Kafka pipelines covering more than 100 topics and 200 JDBC source and AWS sink connectors. I also built incremental ELT in PySpark and a pipeline that extracted critical columns from nested JSON without data loss.
On Forecast Engine, I engineered a FastAPI endpoint querying AWS Athena for real-time outage status and duration across solar sites, and integrated Netcool alerting with ServiceNow for automated ticket creation. My work also includes Terraform-based AWS infrastructure, data orchestration, and schema validation across more than 300 tables.
