rishabh gupta
@rishabhgupta21
Data Engineer at adidas who cut Databricks pipeline runtime by 60% while processing 600M+ records.
IndiaWhat I'm looking for
At adidas, I engineered Databricks PySpark Delta pipelines that processed 600M+ records and reduced runtime from 10 hours to 4. I also developed Python data-quality and reconciliation pipelines to detect discrepancies and improve reliability.
I built a LightGBM model scoring 2.6M article-market-season combinations, achieving 0.87 AUC and 0.96 PR-AUC. I deployed Databricks Genie to production, improving merchandising data exploration efficiency by 15%.
My work spans data platforms, cloud infrastructure, and production reliability. At adidas, I’ve supported AWS-hosted applications, observability, Kafka streaming, and infrastructure automation, while leading an eight-engineer team through technical design and delivery.
Experience
Work history, roles, and key accomplishments
Provided SRE/DevOps support for AWS-hosted applications, managed L2/L3 production support, and led an eight-engineer team. Engineered Databricks PySpark Delta pipelines processing 600M+ records, reducing runtime by 60%, and built automated data-quality pipelines.
Led Windows Server and Azure infrastructure migration, implemented deployment automation, and designed NGINX reverse-proxy configurations. Developed ETL workflows using Python and SQL for data ingestion and transformation.
Applied SQL Server SSIS ETL pipelines and SSRS reporting solutions for enterprise data-processing workloads. Optimized SQL queries and stored procedures to boost processing performance by 40% and executed IBM Tivoli Workload Scheduler batch workflows.
Supported enterprise applications through backend development, deployment coordination, and ETL/ELT development. Conducted SQL analysis, query optimization, and production troubleshooting to maintain system reliability.
Education
Degrees, certifications, and relevant coursework
Dr. A.P.J. Abdul Kalam Technical University
B.Tech., Computer Science & Engineering
2012 - 2016
Pursued a Bachelor of Technology in Computer Science and Engineering, focusing on core computer science and engineering principles.
Tech stack
Software and tools used professionally
Splunk
Azure Synapse
Apache Spark
AWS IAM
Amazon CloudWatch
Amazon S3
Kubernetes
Helm
Amazon EKS
Jenkins
GitHub Actions
GitLab CI
PySpark
dbt
PostgreSQL
Microsoft SQL Server
Node.js
Databricks
Terraform
AWS CloudFormation
Azure DevOps
Jira
JFrog Artifactory
JavaScript
Python
PowerShell
Azure Machine Learning
Kafka
Flask
FastAPI
Grafana
Prometheus
Amazon DocumentDB
Linux
Datadog
Confluence
OpenSearch
Ansible
AWS Lambda
Amazon RDS
Amazon Aurora
TypeScript
Git
Docker
Amazon VPC
NGINX
Google BigQuery
SQL
ServiceNow
AWS KMS
LightGBM
Delta Lake
Argo CD
Bash
Unity Catalog
Kustomize
Instana
Retrieval-Augmented Generation (RAG)
AWS
Availability
Location
IndiaAuthorized to work in
Salary expectations
Job categories
Interested in hiring rishabh?
You can contact rishabh and 90k+ other talented remote workers on Himalayas.
Message rishabhGet matched with your dream remote job
Sign up now and join over 250,000+ remote workers who receive personalized job alerts, curated job matches, and more for free!
