Looking for a job

mitali band

@mitaliband

Message

Data Engineer at IBM building a PySpark framework for a ~1 PB Db2-to-Iceberg migration.

India

Message

At IBM, I’m contributing to a ~1 PB migration from IBM Db2 to Apache Iceberg for a banking data platform. I designed and developed a PySpark and Shell-based framework for the migration pipeline, including metadata-driven processing, recovery mechanisms, and source-to-target reconciliation.

At Tata Consultancy Services, I designed production ETL pipelines for the Reserve Bank of India and worked on a Snowflake data architecture for TIAA. My work included SCD Type 2 processing, Airflow orchestration, and data validation across source and target systems.

Experience

Work history, roles, and key accomplishments

Current

Data Engineer

Current

IBM

Sep 2025 - Present (1 year 2 months)

Working on a large-scale data migration project involving ~1 PB of data from IBM Db2 to Apache Iceberg tables on HDFS/Cloudera. Designed and developed a scalable PySpark and Shell-based migration framework covering the complete pipeline.

PySpark Shell Scripting Apache Iceberg HDFS Cloudera DB2 Data Migration ETL Data Validation Data Reconciliation

Data Engineer

Tata Consultancy Services

Oct 2019 - Sep 2025 (5 years 11 months)

Designed and implemented production ETL pipelines using PySpark, SQL, Hive, Sqoop, and Talend for large-scale data ingestion, transformation, and validation. Developed SCD Type 2 processing and optimized queries for performance.