Irfan Ghat
@irfanghat
Senior software engineer building Generative AI and LLM systems on large-scale data platforms.
KenyaWhat I'm looking for
I’m a Senior Software Engineer specializing in Generative AI systems, LLM applications, and large-scale data infrastructure. I build and evaluate AI agents, RAG systems, and distributed data platforms, with a strong focus on performance optimization and reliability.
In the Apache Spark ecosystem, I architected and open-sourced the first native C++ client for Spark Connect, implementing a gRPC transport layer with Apache Arrow-based serialization and achieving API parity with Python for high-throughput C++ workloads. I also contribute to the Spark Connect Rust connector by expanding test coverage and improving cross-language interoperability, and I author technical documentation for the .NET connector.
At Griffin Global Technologies, I led MLflow lifecycle integration into AI agent workflows for experiment tracking and multi-environment testing across thousands of operators. I designed and deployed an enterprise search and support workflow using Databricks, Vertex AI, LangChain, and AWS, while building scalable retrieval and automation systems that reduced investigation time from hours to minutes and enabled near real-time reporting.
Experience
Work history, roles, and key accomplishments
Open-Source Contributor
Apache Spark Ecosystem
Jun 2024 - Present (2 years)
Architected and open-sourced the first native C++ client for Spark Connect, adding a gRPC transport layer and Apache Arrow-based serialization with API parity to Python. Improved Spark Connect Rust connector reliability and interoperability, and authored .NET connector and Kafka Delta Ingest technical documentation.
Spark Connect C++ Contributor
Apache Spark Ecosystem
Jun 2024 - Present (2 years)
Architected and open-sourced the first native C++ Spark Connect client, adding a gRPC transport layer and Apache Arrow-based serialization for high-throughput C++ workloads. Improved the Spark Connect Rust connector with new APIs, expanded test coverage, and authored documentation for the .NET connector and Kafka Delta Ingest.
Software Engineer
Griffin Global Technologies
Nov 2023 - Present (2 years 7 months)
Integrated MLflow lifecycle management into AI agent workflows, enabling experiment tracking and multi-environment testing across 3,000 operators. Designed and deployed scalable AI agents for enterprise search and automated support workflows, reducing investigation time from hours to minutes via QA retrieval, and delivered low-latency dashboards on multi-billion row Delta Lake datasets.
Software Engineer
Griffin Global Technologies, Doeren Mayhew
Nov 2023 - Present (2 years 7 months)
Integrated MLflow lifecycle management into AI agent workflows to enable experiment tracking and multi-environment testing across 3,000 operators. Built scalable Databricks/Vertex AI/LangChain/AWS AI agents and QA retrieval systems that reduced investigation time from hours to minutes, plus automated pipelines and low-latency dashboards for multi-billion-row Delta Lake datasets.
Software and Audio Engineer
Rogue Radio
Mar 2021 - Apr 2023 (2 years 1 month)
Architected a low-latency audio streaming system using JavaScript and open-source technologies, resolving stability issues to ensure reliable 24/7 web broadcasting. Improved real-time browser-based monitoring and control of studio hardware.
Education
Degrees, certifications, and relevant coursework
Irfan hasn't added their education
Don't worry, there are 90k+ talented remote workers on Himalayas
Tech stack
Software and tools used professionally
Availability
Location
KenyaAuthorized to work in
Portfolio
irfanghat.github.io/portfolioSocial media
Job categories
Skills
Interested in hiring Irfan?
You can contact Irfan and 90k+ other talented remote workers on Himalayas.
Message IrfanFind your dream job
Sign up now and join over 250,000+ remote workers who receive personalized job alerts, curated job matches, and more for free!
