Looking for a job

pankaj kumar

@pankajkumar37

Message

Data Engineer at Beyond Insights building Databricks pipelines and AI-powered client tools.

India

Message

At Beyond Insights, I’m building end-to-end data pipelines on Databricks with PySpark and Delta Lake for ingestion, transformation, and analytics. I also architected Ask BiBi, an AI system with modular adapters and granular client permissions.

At the National Technology Centre for Ports, Waterways & Coasts — IIT Madras, I developed 3D digital twins of Indian ports for operational planning and simulation. My projects include a GNN-based legal summarisation pipeline and an anti-money-laundering system that uses transaction graphs to flag suspicious patterns.

Experience

Work history, roles, and key accomplishments

Current

Data Engineer Intern

Current

Beyond Insights

Jan 2026 - Present (9 months)

Building and optimising end-to-end data pipelines on Databricks using PySpark and Delta Lake for large-scale ingestion, transformation, and analytics workloads. Developed an automated frontend regression testing framework performing URL-vs-URL visual and functional comparisons across deployments, reducing manual QA effort significantly.

Databricks PySpark Delta Lake Frontend Automation SQL Python

Data Engineer Intern

National Technology Centre for Ports, Waterways & Coasts — I

Jan 2023 - Jan 2026 (3 years)

Developed high-fidelity 3D digital twin models of major Indian ports, integrating geospatial data and infrastructure geometry for operational planning and simulation. Engineered a domain-specific chatbot for Sagarmanthan with fine-tuned LLMs on maritime and port-operations corpora, incorporating RAG for document retrieval, SQL-to-text for structured data queries, and TTS/STT for voice interaction.

Digital Twin 3D Modeling Geospatial Data LLM Fine tuning RAG SQL To Text TTS STT Python