Open to opportunities

Anna Iqbal

@annaiqbal

Message

Senior data engineer architecting multi-cloud lakehouses and low-latency streaming backbones.

United States

Message

What I'm looking for

I’m looking for a team where I can architect multi-cloud lakehouses and streaming pipelines, enforce data quality and governance, and drive FinOps-aware production delivery—while leading engineering practices around reliability and testing.

I’ve built and deployed end-to-end Medallion Lakehouse platforms at Dataquest using Databricks, Delta Engine, Unity Catalog, Delta Live Tables, and PySpark—migrating 600+ TB of legacy data into structured Delta Lake and Apache Iceberg tables.

I run production pipelines across Apache Airflow and Azure Data Factory, orchestrating 150+ daily dynamic DAGs for financial and transactional workloads with a 99.95% operational SLA. I’ve also re-engineered slow SQL and Spark jobs on AWS EMR and Google Cloud Dataflow to cut cloud compute spend by 28%.

For streaming, I’ve engineered Kafka-based event pipelines with exactly-once semantics (plus Azure Event Hubs and Confluent Cloud) and transformed operational logs into analytics-ready layers in Snowflake and Amazon Redshift with DBT, Jinja, incremental materializations, and custom macros.

I’m hands-on with governance and reliability—Great Expectations and dbt tests, RBAC and PII masking via Microsoft Purview and Apache Atlas, plus IaC with Terraform and AWS CloudFormation. I also lead code review and testing standards for a team of four, and I’ve applied FinOps to save $140,000 annually through automated compute spin-downs and storage tiering.

Experience

Work history, roles, and key accomplishments

Current

Lead Data Engineer & Data Architect

Current

Dataquest

May 2023 - Present (3 years 3 months)

Architected and deployed Medallion Lakehouse Architecture using Databricks, Delta Engine, and PySpark, migrating 600+ TB of legacy storage into Delta Lake and Apache Iceberg. Designed enterprise pipelines in Apache Airflow and Azure Data Factory, orchestrating 150+ daily DAGs with 99.95% SLA.

Databricks Unity Catalog Delta Live Tables PySpark Apache Iceberg Airflow Azure Data Factory Google Cloud Dataflow Great Expectations DBT Terraform GitHub Actions Docker Microsoft Purview Snowflake FinOps

Senior Data Engineer

Orderly Health

Sep 2019 - Apr 2023 (3 years 7 months)

Constructed real-time event-driven streaming pipelines using Apache Kafka, Confluent Cloud, and Azure Event Hubs, processing 20,000+ messages per second with exactly-once semantics. Developed scalable data transformations in Snowflake and Amazon Redshift using DBT and Jinja templating.

Kafka Confluent Cloud Azure Event Hubs Snowflake Amazon Redshift DBT Jinja Apache Flink Salesforce API Azure Cosmos DB MongoDB PostgreSQL MySQL Microsoft SQL Server Kubernetes Jenkins Pinecone Milvus Weaviate AWS Glue

Data Engineer

Lendbuzz

Nov 2016 - Aug 2019 (2 years 9 months)

Built and scaled operational data ingestion pipelines using Python, SQL, and AWS Lambda. Migrated on-premise scripts to Apache Airflow, reducing pipeline failure points by 40%. Authored analytical queries and stored procedures across Oracle, IBM DB2, and MariaDB.