Purushotham User

I’m a Senior Data Engineer with 7+ years of experience designing and maintaining ETL/ELT pipelines, data ingestion, integration, data modeling, and enterprise data warehousing on the Azure platform.

I build large-scale workflows with Azure Data Factory, Synapse Analytics, Microsoft Fabric, Data Lake, and Databricks—delivering reliable ingestion, migration, and analytics at scale. I also develop real-time streaming solutions with Azure Event Hubs, Kafka, and Spark Streaming, using Change Data Capture to keep latency low.

On the transformation and modeling side, I use SQL, Python, PySpark, and DBT (with Spark SQL, Pandas, NumPy, and SciPy) to prepare datasets for reporting, analytics, and machine learning. I design and optimize warehouses in Snowflake and Azure Synapse Analytics using partitioning, clustering, and indexing to improve query performance and scalability.

I’m equally focused on governance and production excellence: I apply Azure Active Directory, RBAC, Key Vault encryption, and compliance practices for HIPAA, GDPR, and PCI-DSS. In my recent role at Med-Metrix, I improved data availability by 35%, achieved 40% faster delivery with 25% lower computing expenses, and reduced job failures by 30% by optimizing pipelines, Spark jobs, and Airflow DAGs. I also automate orchestration and CI/CD with Airflow, NiFi, Azure DevOps, Terraform, Docker, and Kubernetes—and I mentor teammates through code reviews and knowledge sharing.