Open to opportunities

Siddharth Singh

@siddharthsingh4

Message

Senior Data Platform Engineer with expertise in data architecture.

India

Message

What I'm looking for

I am looking for a role that fosters innovation and collaboration, where I can leverage my data engineering skills to drive impactful projects and contribute to a forward-thinking team.

As a Senior Data Platform Engineer, I have a proven track record of architecting and implementing robust data solutions that drive efficiency and scalability. Currently, I am working at ZoomInfo, where I have successfully developed an enterprise event streaming platform using Confluent Apache Kafka, processing over 100 billion events annually. My work has enabled 12 teams to onboard more than 40 events seamlessly, showcasing my ability to lead complex projects and deliver impactful results.

Previously, I served as a Data Platform Engineer II at Alteryx, where I developed data pipelines that processed over 22 million records daily. My efforts in migrating Airflow deployment to Google Kubernetes Engine resulted in a 45% cost reduction, demonstrating my commitment to optimizing resources. I take pride in mentoring junior engineers and leading initiatives that enhance data integrity and operational efficiency.

Experience

Work history, roles, and key accomplishments

Current

Senior Data Platform Engineer

Current

ZoomInfo

Sep 2023 - Present (1 year 10 months)

Architected an enterprise event streaming platform using Confluent Apache Kafka, processing over 100 billion events annually and enabling 12 teams to onboard 40+ events. Upgraded the AWS mWA Airflow platform across four environments, migrating 100+ DAGs with zero downtime and implementing Airflow RBAC integration with AWS SSO. Engineered a monitoring system that leveraged data lineage propagation

Python Apache Kafka Airflow AWS Snowflake DBT Kubernetes Docker SQL

Data Platform Engineer II

Alteryx

Sep 2021 - Present (3 years 10 months)

Developed data pipelines in Airflow using Python, BigQuery, and GCS on GKE, processing over 22 million records daily. Migrated Airflow deployment to Google Kubernetes Engine, adding auto-scale capabilities and cutting costs by 45%. Deployed LinkedIn DataHub on GKE, which improved data asset discovery across the organization, and enhanced pipeline failure response time by integrating Prometheus and

Python Airflow BigQuery Google Kubernetes Engine Prometheus Grafana Docker Terraform

Data Engineer I

Aaysanalytics

Jan 2021 - Present (4 years 6 months)

Engineered data pipelines with Databricks, data lake, and ADF to support supply chain analytics data, handling a daily load of over 10 million records. Ensured data integrity and accuracy by implementing audit trails across all stages, resulting in a 100% reduction in erroneous reporting.

Databricks Azure Data Factory Data Lake SQL Data Integrity

Junior Data Engineer

Tata Consultancy Services

Sep 2019 - Present (5 years 10 months)

Migrated Microsoft IaaS to PaaS Infrastructure, improving SLA by 8 minutes and cutting costs by 13%. Developed and managed Batch ETL flow, processing 8 million records daily, utilizing Azure Databricks (PySpark), Azure Data Factory, and Microsoft modern data platform.

Microsoft Azure ETL Databricks Pyspark Azure Data Factory IaaS PaaS Data Platform

Education

Degrees, certifications, and relevant coursework

Rustamji Institute of Technology

B.Tech, Information Technology

Completed a Bachelor of Technology degree with a specialization in Information Technology. Gained foundational knowledge and practical skills in various aspects of IT.