Himanshu Gupta
@himanshugupta2
Azure-certified Lead Data Engineer with 9+ years of experience.
What I'm looking for
I am an Azure-certified Lead Data Engineer with over 9 years of experience in designing and implementing scalable data solutions across cloud platforms, including Azure and GCP. My expertise lies in ETL development using Informatica and Azure Data Factory, real-time data processing, and building secure, high-performance data pipelines. I have a proven track record of leading cross-functional teams in Agile environments, delivering end-to-end data solutions that align with business objectives.
In my current role at A.P. Moller Maersk, I lead a Master Data Management project focused on Customer Master Data, where I designed and implemented Data Quality metrics. My efforts have enabled actionable insights through dashboards and automated alerts for data anomalies. I have developed scalable solutions for data lakes and warehouse systems, collaborating with stakeholders to gather requirements and deliver data integration pipelines from diverse sources. My commitment to best practices in data security and governance has established robust data quality standards and lineage tracking.
Throughout my career, I have achieved significant milestones, such as reducing pipeline SLA from 3 days to 4 hours through ETL automation and saving ~$5000/month via Spark job optimization. I am passionate about mentoring junior engineers and continuously improving data integration best practices, ensuring high data transparency and quality in all projects I undertake.
Experience
Work history, roles, and key accomplishments
Senior Data Engineer
A.P. Moller Maersk
Apr 2022 - Present (3 years 3 months)
Led a Master Data Management (MDM) project for Customer Master Data, designing and implementing Data Quality (DQ) metrics. Developed scalable solutions for data lakes and warehouse systems using Azure Data Factory, Databricks, Pyspark, ADLS Gen2, and Azure SQL.
Senior Data Engineer
Accenture Applied Intelligence
Sep 2019 - Present (5 years 10 months)
Designed a financial data product for Walmart using Spark and MongoDB, aligning with enterprise data lake architecture. Performed ETL transformations, integrated data from multiple systems, and ensured high data transparency and quality.
Big Data Developer
Xebia
Aug 2018 - Present (6 years 11 months)
Migrated on-premise banking data to Google Big Query and delivered Azure-based real estate analytics for EMAAR. Implemented data pipelines using Spark (Scala), Snowflake, and Azure services.
Data Engineer
Cognizant Technology Solutions
Nov 2015 - Present (9 years 8 months)
Transformed and migrated ETL processes from Informatica to Hadoop using Spark and Hive. Ensured data integrity and validation through automated quality checks.
Education
Degrees, certifications, and relevant coursework
Galgotias University
Bachelor of Technology, Computer Science
Completed a Bachelor of Technology degree with a specialization in Computer Science. Gained foundational knowledge in various aspects of computer science.
Tech stack
Software and tools used professionally
Availability
Location
Authorized to work in
Job categories
Interested in hiring Himanshu?
You can contact Himanshu and 90k+ other talented remote workers on Himalayas.
Message HimanshuFind your dream job
Sign up now and join over 100,000 remote workers who receive personalized job alerts, curated job matches, and more for free!
