Open to opportunities

ARKAPROVA SAHA

@arkaprovasaha

Message

Experienced Cloud Data Engineer and Data Analyst with 19 years.

India

Message

What I'm looking for

I am looking for a role that fosters innovation, encourages collaboration, and offers opportunities for professional growth in data engineering and analytics.

I am a seasoned Cloud Data Engineer and Data Analyst with over 19 years of experience in software development and big data processing. My expertise lies in building scalable data pipelines and analytics solutions, particularly using dbt Cloud with Databricks SQL, Snowflake, and BigQuery. At Dr. DM Tech Private Limited, I have successfully led teams to develop innovative solutions that enhance data processing and analytics efficiency.

Throughout my career, I have implemented end-to-end healthcare e-commerce analytics solutions, significantly reducing processing times and improving data availability. My work includes crafting over 50 modular dbt models that transform raw data into actionable insights, which have greatly enhanced decision-making for marketing and operations teams. I am passionate about optimizing data governance and have enforced data accuracy and security across numerous Delta Lake tables, ensuring that organizations can trust their data for critical business decisions.

Experience

Work history, roles, and key accomplishments

Current

Senior Data Engineer

Current

Dr DM Tech Private Limited

Oct 2023 - Present (1 year 7 months)

Implemented an end-to-end healthcare e-commerce analytics solution using Databricks SQL and dbt Cloud, reducing processing time by 40% and improving data availability. Built over 50 modular dbt models to transform raw e-commerce data into actionable insights, enhancing decision-making efficiency for marketing and operations teams. Deployed 25+ interactive dashboards in Power BI and Metabase, impro

SQL DBT Databricks Python Java Trino Metabase Power BI Streamlit AWS

Senior Data Engineer

Dr DM Tech Private Limited

Jan 2023 - Present (2 years 4 months)

Designed optimized ELT pipelines in Snowflake AI Data Cloud Platform, improving query performance by 30% and decreasing data retrieval costs by 15% per month. Engineered complex transformation models in dbt to power 20+ analytical dashboards and executed dbt tests on 15+ critical tables, improving data quality and reliability. Enabled replication and synchronization of 7M records from Oracle to Sn

Snowflake Snowsight Python Airbyte DBT AWS S3

Senior Data Engineer

Dr DM Tech Private Limited

Aug 2022 - Present (2 years 9 months)

Managed data pipelines on Google Cloud Platform for a healthcare reporting system, designing pipelines to upload 30+ GB of data from on-premise Oracle to GCS using NiFi. Processed 20M records from GCS to BigQuery using Apache Beam in Google Dataflow. Set up a scalable Airflow DAG to run 30+ daily ETL tasks, reducing processing errors by 20%.

Google Cloud Platform Apache Beam Google Cloud Storage BigQuery Dataflow Composer Cloud Scheduler Docker Apache NiFi Apache Airflow

Senior Data Engineer

Dr DM Tech Private Limited

Dec 2017 - Present (7 years 5 months)

Streamlined Auto Loader and Delta Live Tables (DLT) to improve pipeline execution time by 40% on Databricks, uploading 50+ GB of data from Oracle to Delta Lake. Programmed a full-stack Java Spring Boot API application used by over 400,000 patients and incorporated Razorpay to manage 3500+ user transactions daily. Supported AWS infrastructure including EC2, IAM, Route 53, Load Balancers, and RDS, a

Databricks Spark AWS Python Delta Lake Angular Java Oracle Flutter MongoDB Spring Boot

Consultant

Cognizant Technology Solutions

Apr 2010 - Present (15 years 1 month)

Engineered end-to-end Sqoop flows to handle 2TB of data for offloading RDBMS data to Hadoop and developed 10+ user-defined functions for custom Hive capabilities. Integrated Kerberos for Hadoop, enhancing security by 90%, and innovated a high-performance data platform in Cloudera (Hortonworks) at 60% lower cost. Offloaded real-time data to Amazon S3 and EMR via Apache Kafka and integrated Mule ESB

Flumes Hadoop Hive HBase Kafka Spark Flume Sqoop Scala Oracle Cloudera Hortonworks

Programmer

National Informatics Centre

Mar 2006 - Present (19 years 2 months)

Implemented Java server-side components using Servlets and EJB, reducing response time by 30% in Kolkata Port, Ministry of Ports Shipping and Waterways, Govt. of India. Conducted unit testing and fixed over 100 bugs.

Java Servlet JSF Platinum ER Win