Pouya Rezaeidivkolaei

Led the design and implementation of a comprehensive internal data platform incorporating Data Quality validation Scala Spark application, Metadata Cataloging for governance, and Job Monitoring Systems. Managed a team of data engineers delivering production-grade pipelines and tooling to enable ingestion, transformation, validation, and secure delivery of large-scale datasets.

Led development of data pipeline software for Truven's DataBridge, integrating clinical and financial data with predictive models. Built machine learning and NLP models for outcome prediction using logistic regression and JavaSpark, wrapped in PySpark.

Developed credit risk and loss prediction models using PySpark and AWS EMR, with extensive data cleansing, augmentation, and imputation. Built forecasting models and deployed analytics workflows across distributed environments using Databricks and Hadoop.

Created machine learning models to predict crop performance, deployed on Domino Data Lab. Automated pipeline configurations with Docker and managed genomic prediction workflows using Spark and SQL Server.

Built distributed data processing systems using PySpark, Kafka, and Elasticsearch in a microservices architecture. This role focused on developing robust and scalable solutions for real-time data ingestion and analysis.

Completed a Master of Science in Computer Science. Thesis focused on "Detection, Classification, and Location Identification of Traffic Congestion from Twitter Stream Analysis."