Open to opportunities

Zulfiqar Ali Zaidi

@zulfiqaralizaidi

Senior data engineer and solution architect delivering compliant lakehouse, streaming, and AI pipelines at enterprise scale.

United States

Message

What I'm looking for

I’m looking for a team building enterprise-scale, compliant data platforms—where I can own lakehouse/streaming architecture, accelerate delivery with DataOps CI/CD, and apply RAG/MLOps responsibly (HIPAA/HITRUST). Open to remote (US).

I’m a Senior Data Engineer and Solution Architect with 12+ years designing enterprise-scale data platforms, real-time streaming pipelines, and HIPAA/HITRUST-compliant lakehouse systems across healthcare, financial services, retail, and SaaS. I’ve delivered $2M+ in annual cloud savings, processed 2.3M+ patient records daily, and led zero-downtime petabyte-scale migrations—translating complex architecture decisions into measurable business outcomes.

I build modern data foundations end-to-end: Spark/Databricks and Delta Lake for scalable lakehouse workloads, Kafka/Debezium/Flink for exactly-once real-time processing, and dbt/DataOps automation to improve reliability. Recently, I’ve also designed LangChain RAG pipelines and multi-agent AI workflows for clinical data products, while leading data mesh and federated governance rollouts that reduced domain onboarding from 6 weeks to 48 hours and improved delivery velocity by 40% through hands-on mentorship.

Experience

Work history, roles, and key accomplishments

Current

Data & Solution Architect

Current

ApTask

Jan 2023 - Present (3 years 5 months)

Designed and delivered LangChain RAG pipelines and multi-agent AI workflows for clinical data products, reducing clinical query response time by 55%. Built an internal self-serve data platform adopted by 8 engineering teams and deployed an Apache Flink CEP engine on AWS EMR processing 500K+ events/hour with sub-3-second latency and 44% fewer false-positive alerts.

Lang Chain RAG Systems Hugging Face Faiss Pinecone Apache Flink Apache Kafka Multi Cloud Architecture

Senior Data Engineer

Truveta, Inc

May 2020 - Dec 2022 (2 years 7 months)

Architected a HIPAA/HITRUST-compliant 12-petabyte Databricks healthcare lakehouse ingesting 2.3M+ patient records daily, enforcing PHI/PII governance with Unity Catalog. Built Kafka + Debezium CDC pipelines extended with Apache Flink for 500K+ events at sub-5-second latency, led zero-downtime 12PB migrations, and cut model deployment time by 60% while improving clinical NLP accuracy by 30%.

Databricks Unity Catalog Apache Kafka Debezium Apache Flink MLFlow AWS Sagemaker DBT Great Expectations

Cloud Data Engineer

Banner Health

Jan 2017 - Apr 2020 (3 years 3 months)

Architected a real-time risk analytics platform processing 15M+ daily financial transactions using Kafka-based streaming with Debezium CDC across AWS and Azure, with end-to-end TLS and VPC-isolated flows. Implemented SOX-compliant audit logging and CPRA/GDPR data lifecycle controls via Collibra MDM, reducing regulatory reporting time by 65%, and modernized legacy ETL workflows into an Airflow + db

Apache Kafka Debezium AWS Azure Collibra MDM Airflow DBT Fivetran Airbyte Terraform

ETL & Data Warehouse Eng

PBG

Jan 2014 - Dec 2016 (2 years 11 months)

Designed a centralized customer analytics data warehouse using Data Vault 2.0 on Snowflake, consolidating data from 12+ SaaS sources and reducing analytical reporting latency by 60%. Built real-time pipelines for 2M+ e-commerce events daily using AWS Lambda and Kinesis Firehose, optimized SQL workloads to reduce database CPU by 35% and improve query response time by 50%, and automated environment

Snowflake AWS Lambda AWS Firehose Terraform PostgreSQL MySQL Apache Airflow