Looking for a job

Pankaj Goswami

@pankajgoswami2

Principal Data Engineer at Theorence Labs, modernizing AWS Databricks Lakehouses and consolidating data across business units.

India

Message

At Theorence Labs, I lead AWS Databricks Lakehouse modernization, consolidating a hybrid Spark estate into a Delta Lake platform governed by Unity Catalog. I also designed reusable orchestration frameworks and shared master data tables for multiple business units.

At Amazon Advertising, I architected a petabyte-scale financial Lakehouse and led a federated Trino query engine that delivered 2.75x faster queries and about 30% compute cost savings. I also developed data platforms for analytics and GenAI, including a multi-agent AWS Bedrock platform that reduced ad-hoc engineering tickets by 40%.

Experience

Work history, roles, and key accomplishments

Current

Principal Engineer (Lead Data Architect)

Current

Theorence Labs Pvt Ltd

Apr 2026 - Present (7 months)

Owns end-to-end target architecture, platform engineering, and delivery governance for enterprise cloud data modernization, leading a 6-engineer platform team. Led migration to AWS Databricks Lakehouse and architected multi-source Medallion orchestration.

AWS Delta Lake Unity Catalog Lakeflow Jobs Spark PySpark PostgreSQL RDS Medallion Architecture Data Governance

Senior Data Engineer (TechLead)

Amazon Development Centre India Pvt Ltd

Apr 2019 - Mar 2026 (6 years 11 months)

Architected petabyte-scale financial Lakehouse and led Twitch revenue unification, Zero-ETL federated query engine, headless BI semantic layer, sub-second serving layer, multi-agent GenAI platform, and data quality defense layer. Drove FinOps and federated engineering initiatives.

Lead Architect

Wipro Technologies Limited

Mar 2012 - Mar 2019 (7 years)

Led enterprise analytics delivery for Fortune 500 clients, administered a 200+ node HDP 2.6 Hadoop cluster, and re-architected a managed print services application onto Hadoop with Spark, Talend, Oozie, and Falcon pipelines.

Hadoop HDP Spark Scala Talend Oozie Falcon Hive HQL Power BI SAP Business Objects Star Schemas Dimensional Modeling (Star And Snowflake Schemas)

Module Lead

Tata Consultancy Services

Feb 2010 - Mar 2012 (2 years 1 month)

Led three developers through a legacy-to-SAP BI migration for AmerisourceBergen, designed BusinessObjects Universes and Xcelsius dashboards, and tuned SAP BI performance.

SAP BI Business Objects Xcelsius SAP Data Cubes BEx Queries JDK

Software Engineer

Tech Mahindra Limited

Oct 2006 - Feb 2010 (3 years 4 months)

Built Oracle PL/SQL data marts, ETL workflows, and BusinessObjects / Crystal Reports analytics for ACBS/Fidelity Investment Services, covering loan origination, syndication, and trading portfolios.

Oracle Pl SQL ETL Business Objects Crystal Reports

Education

Degrees, certifications, and relevant coursework

Databricks

Databricks Certified Data Engineer Associate, Data Engineering

Earned the Databricks Certified Data Engineer Associate certification in 2026.

RKDF Institute of Science and Technology

Bachelor of Engineering, Electronics and Communication

Bachelor of Engineering in Electronics and Communication from RKDF Institute of Science and Technology, affiliated with Rajiv Gandhi Technical University, Bhopal.