Open to opportunities

Rohil Singh

@rohilsingh

Message

Data Engineer skilled in building robust data pipelines and solutions.

India

Message

What I'm looking for

I seek a role that fosters innovation and collaboration, with opportunities for growth.

I am a dedicated Data Engineer with extensive experience in designing and implementing data pipelines and ETL processes. Currently, I work at Shyftlabs, where I have successfully set up a new ELT system using AWS Managed Airflow and dbt, automating data workflows and enhancing data quality. My role involves creating Airflow DAGs for bidirectional data pipelines and developing robust error-tracking systems, ensuring seamless data integration and transformation.

Previously, I laid the foundation for a Data Engineering team at Glints, where I created ETL connectors and optimized existing data pipelines using Airflow and Kafka. My efforts in implementing a data quality validation tool significantly improved data integrity across various projects. With a strong background in database development and cloud technologies, I am passionate about leveraging data to drive business insights and efficiencies.

Experience

Work history, roles, and key accomplishments

Current

Data Engineer 2

Current

Shyftlabs

Sep 2023 - Present (1 year 10 months)

Designed and developed robust data pipelines using AWS Managed Airflow (MWAA) and dbt, automating S3-Snowflake bidirectional data flows and controlling dbt job schedules. Implemented an error-tracking system for file load errors and integrated Airflow with AWS Systems Manager and Secrets Manager for variable and connection synchronization. Migrated legacy data pipelines to Airflow and dbt, enhanci

AWS MWAA DBT Snowflake Airflow S3 Secrets Manager Java Kafka Cloudwatch Terraform Python AWS Lambda

Data Engineer

Glints

Oct 2021 - Present (3 years 9 months)

Established the foundation for a new Data Engineering team, independently managing existing tasks, stakeholder meetings, and new feature implementations for four months before onboarding new team members. Developed ETL connectors for various sources using Python and GoLang, and created Airflow DAGs for data ingestion into PostgreSQL, utilizing dbt for data transformations. Implemented a data quali

Python Golang Airflow DBT Kafka Debezium Great Expectations Docker Gitlab Selenium BeautifulSoup Elasticsearch PostgreSQL Metabase Rancher Jira

Senior Database Developer

Media.net

Jul 2017 - Present (8 years)

Implemented a new, faster reporting interface using Python with Airflow and MSSQL, replacing a slow existing system and reducing average daily resource consumption by 75 minutes while doubling features and improving request execution speed. Created Airflow DAGs for data transfer from third-party dashboards, APIs, and AWS S3 to MSSQL, and for ETL pipelines from MSSQL, AWS S3, MongoDB, and DynamoDB

Python Airflow MSSQL Spark Hive Kafka Debezium Grafana Prometheus Snowflake SSIS PowerShell SQL MySQL MongoDB DynamoDB AWS S3 GKE

Education

Degrees, certifications, and relevant coursework

International Institute of Information Technology, Bangalore

P.G.D. in Data Science, Data Science

Grade: 3.52/4 CGPA

Activities and societies: Final capstone project on preparing a Market-Mix-Model for a fictional E-commerce company.

Completed a Post Graduate Diploma in Data Science. The final capstone project involved preparing a Market-Mix-Model for a fictional E-commerce company.

University of Mumbai

B.E. in Computer Science, Computer Science

Grade: 8.57/10 CGPA

Activities and societies: Final year project on web application to manage leave & payroll of employees implemented at TCET, Mumbai.

Obtained a Bachelor of Engineering in Computer Science. The final year project focused on developing a web application for managing employee leave and payroll, which was implemented at TCET, Mumbai.