Looking for a job

Arnab Chowdhury

@arnabchowdhury1

Message

Results-driven Lead Data Engineer with 8+ years of experience.

India

Message

What I'm looking for

I am looking for a challenging role in a dynamic environment that fosters innovation and growth, where I can leverage my data engineering skills to drive impactful solutions.

I am a results-driven Lead Data Engineer with over 8 years of experience in designing scalable ETL pipelines and optimizing cloud-based data warehouses. My expertise lies in Python, Java, and Scala, and I have a proven track record of automating workflows to reduce costs by up to 70%. I excel in translating complex business needs into technical solutions across various industries, including finance, logistics, and SaaS.

In my current role at Tavant, I have automated ETL pipelines using Python and Snowflake, significantly reducing manual intervention and storage costs. My experience also includes developing AWS Glue workflows and enhancing database query performance. I am passionate about leveraging my skills in Agile methodologies and CI/CD pipelines to drive efficiency and innovation in data management.

Experience

Work history, roles, and key accomplishments

Current

Lead Data Engineer

Current

Tavant

Feb 2023 - Present (2 years 4 months)

Automated ETL pipelines using Python and Snowflake, reducing manual intervention by 90%. Developed AWS Glue workflows to load data into Snowflake, cutting storage costs by 70%. Created Information Marts for comprehensive reports on waste, ecology, environment, and safety.

Snowflake AWS Glue AWS S3 AWS Lambda Python Alteryx Docker

Senior Data Engineer

Tavant

Feb 2022 - Feb 2023 (1 year)

Designed and implemented ETL workflows using Python and Snowflake, orchestrating jobs with Airflow. Improved transaction reconciliation efficiency through targeted query enhancements. Developed jobs to deliver information to customers via SFTP and AWS S3, ensuring secure data distribution.

Snowflake Airflow Python GitHub Docker AWS

Senior Data Engineer

Tavant

Sep 2021 - Feb 2022 (5 months)

Designed and implemented end-to-end ETL workflows using Apache Spark, optimizing data extraction, transformation, and loading processes. Identified and implemented enhancements to improve system performance and efficiency. Utilized HiveQL within Hue and crafted complex queries using Presto to efficiently retrieve data for report generation.

Apache Spark Scala GitHub Presto Python

Software Development Specialist

NTT Data

Feb 2021 - Sep 2021 (7 months)

Streamlined data retrieval from multiple sources, generating daily and monthly reports on historical trades and disputes. Enhanced performance by upgrading object-based storage systems and migrating codebases to Java 8. Designed RESTful solutions with cross-team alignment and interfaced with business teams to address their needs effectively.

SVN Spring Spring Boot OBS Java Shell Scripting SQL Hibernate

System Engineer

TCS

Nov 2016 - Feb 2021 (4 years 3 months)

Built Spring Boot APIs with Java and JUnit tests, ensuring robust and reliable functionality. Collaborated with Business Analysts to refine project requirements and developed comprehensive design documentation with diagrams and data mappings. Executed Proof of Concepts (POCs) using Redis caching, centralized logging/error handling, and connection pooling.

TFS Spring Spring Boot SQL Java Hibernate Junit Redis

Education

Degrees, certifications, and relevant coursework

Techno India, Salt Lake, Kolkata

Bachelor of Technology, Computer Science and Engineering

Grade: 7.88

Completed a Bachelor of Technology in Computer Science and Engineering. Gained foundational knowledge in various aspects of computer science, including programming and system design.

Doon Public School, Dhanbad

High School Diploma, Science, Computer Science

Grade: 71%

Completed secondary education with a focus on Science and Computer Science. Developed a strong academic foundation in core scientific principles.

De Nobili School, Dhanbad

High School Diploma, English, Science, Computer Science

Grade: 91%

Completed secondary education with a focus on English, Science, and Computer Science. Achieved a high academic standing, demonstrating proficiency across multiple subjects.