I'm open to remote roles where I can build and own scalable Azure, Databricks, and Spark data platforms, lead engineering teams, partner with stakeholders, and apply GenAI through chatbots, agents, and AI-assisted development.
Looking for a job
Shrey User
@shreymarwaha
Senior Data Engineer | Azure - Databricks - PySpark | 8 yrs | Payments & CPG
IndiaWhat I'm looking for
At Fractal Analytics, I build and lead large-scale Azure, Databricks, and Spark data solutions for global CPG and payments clients. My pipelines power daily Power BI reporting across 10+ EU markets and process 5–10 TB of payments data across risk, fraud, customer, and market domains.
I've delivered 13+ end-to-end BI solutions and led a legacy Hive-to-Spark migration that reduced pipeline runtime by 66% and manual effort by 50%. I also built a RAG-based chatbot over a project knowledge base, helping drive an estimated 20% productivity gain for the team.
I stay hands-on with PySpark, Airflow, SQL, Azure Data Factory, Synapse, Databricks, and Hadoop while leading engineering teams of up to six people. I work directly with business stakeholders, translate country-specific requirements into pipeline logic, and guide architecture, code reviews, sprint planning, and delivery strategy.
Earlier, I automated Python and SQL analytics pipelines at Escalent and managed production Hadoop clusters at Infosys. I'm Microsoft-certified in Azure and Fabric data engineering, Claude Certified, and actively applying GenAI through chatbots, agents, and AI-assisted engineering practices.
Experience
Work history, roles, and key accomplishments
Current
Building and leading DE & BI delivery for two enterprise clients.
CPG Client - Azure, Synapse, Databricks | Sep 2025 – Present
· End-to-end pipelines for 10+ EU markets (ADLS + ADF + Synapse) powering 8 daily-refresh Power BI reports for the client's sales team.
· Built a PySpark data quality framework in Databricks - quality, integrity & historical checks with automated stakeholder email reports
– Built a complete ETL pipeline in PySpark + Airflow for acquiring bank portfolio analytics – handling requirement gathering, Spark/Airflow
backend, and data modelling for powering a Tableau report
– Authored and streamlined technical + process documentation on Confluence covering development patterns, deployment guides,
onboarding and project management SOPs – covering end-to-end team operations.
– Automated daily data ingestion and cleaning via Python ETL pipelines – eliminated multiple days of recurring manual effort per cycle.
– Built SQL and Python pipelines for customer behaviour analysis, cutting insight generation and reporting time by 60%.
IN
Worked as a Big Data Administrator in the Data and Analytics Department, with a Fortune 500 client.
- Responsible for managing the Hadoop Ecosystem (total 60 on-premise clusters, 7000+ nodes)
- Led a sub-team of 3 senior engineers within 4 months of joining; drove Ansible automation training for the complete 10-member team
Interned as a backend engineer, working with technologies such JAVA, AWT, Servers, JSP and SQL
- Involved in development of automation scripts to reduce the large-scale manual workloads
- Developed an end-to-end platform (both frontend and backend) for the team to log & collate information into a SQL database for app deployment to Google and IOS app stores.
Education
Degrees, certifications, and relevant coursework
Thapar University
Bachelor of Engineering, Electronics and Communication Engineering
2013 - 2017
Grade: 6.81
GS
Gyan Bharati School
Science with Economics
1999 - 2013
Availability
Looking for a job
Location
IndiaAuthorized to work in
Salary expectations
60k-100k USD
Social media
Job categories
Skills
AzureDatabricksPySparkSQLT SQLPythonAirflowHadoopAzure Data FactoryAzure SynapseAzure Data LakeMicrosoft FabricDelta LakeUnity CatalogLakehouseETLELTData Pipeline ArchitectureData QualityData ModelingPower BITableauLLM APIsPrompt EngineeringAgentic SystemsRAGClaude CodeAnsibleVBAGitHub Co PilotChatGptGitHubAzure PipelinesAgentic AIFactoryData EngineeringApache SparkAzure DatabricksHiveBusiness IntelligenceAmbariClouderaJavaJSPProject ManagementAgile Project ManagementTeam LeadershipRequirements AnalysisTechnical DocumentationGenerative AIiOSBridges
Interested in hiring Shrey?
You can contact Shrey and 90k+ other talented remote workers on Himalayas.
Message ShreyGet matched with your dream remote job
Sign up now and join over 250,000+ remote workers who receive personalized job alerts, curated job matches, and more for free!
