India onlyKey Responsibilities
Experience - 9+yrs
- Data Platforms & Pipeline Engineering
- Lead end-to-end development of scalable data pipelines and orchestration frameworks usingAzure Data Factory (ADF),Azure Synapse Analytics,Azure Databricks, andMicrosoft Fabric.
- Build robustreal-time and batch data pipelines, including integration with streaming sources (e.g., Event Hubs, Kafka) and structured streaming engines.
- Design and implementStructured Streamingapplications in Spark for near-real-time processing of streaming data.
- Develop and maintainETL/ELT pipelinesand transformations leveraging Spark, PySpark, SQL, and fabric orchestration capabilities.
Microsoft Fabric & Azure Ecosystem
- Architect and implement data solutions usingMicrosoft Fabric, including OneLake, Dataflows, warehouses, and Fabric capacity planning to support enterprise analytics.
- Collaborate on data governance, cataloging, and asset organization usingUnity Catalogwithin Databricks and Fabric environments.
- Manage Microsoft Fabric capacity and resource utilization to optimize performance and cost efficiency for analytics workloads.
Databricks Engineering & Dashboards
- Design, deploy, and optimizeDatabricks dashboardsand reporting artifacts for business stakeholders.
- Apply best practices for data modelling, caching, file sizing, andperformance tuning of Spark and Delta Lake jobs(e.g., Z-ORDER, broadcast joins, adaptive query execution).
- Oversee governance, access controls, metadata management, and lineage using Unity Catalog.
Leadership & Collaboration
- Lead and mentor a team of data engineers, fostering best practices in development, operations, documentation, and quality.
- Work with cross-functional teams (architecture, BI, data science, DevOps) to translate business requirements into scalable data solutions.
- Partner with stakeholders to define data strategy, standards, and architectural roadmaps.
Quality, Governance & Compliance
- Establish and enforce standards for data quality, testing, monitoring, operational observability, and governance.
- Implement secure, compliant data access and lineage frameworks across cloud data platforms.
DevOps & CI/CD
- Implement CI/CD pipelines, infrastructure-as-code for data platform artifacts, and automated testing frameworks for data jobs and workflows.
Required Skills & Qualifications
- 10+ yearsof hands-on experience in data engineering on Azure with deep expertise in ADF, Synapse, Databricks, and Microsoft Fabric.
- Proven experience withreal-time data processing, streaming architectures, andSpark Structured Streaming.
- Strong proficiency inAzure Data Factory, Spark (PySpark), SQL, Azure Synapse Analytics, Databricks Runtime, and cloud storage.
- Solid knowledge ofUnity Catalogfor data governance, security, and access management.
- Experience designing and managingDatabricks Dashboards, performance optimization, cost controls, and data platform resource tuning.
- Expertise in building scalable, fault-tolerant, and high-throughput batch & streaming data solutions.
- Excellent leadership, cross-team collaboration, and communication skills.
About Rackspace Technology
We are the multicloud solutions experts. We combine our expertise with the world’s leading technologies — across applications, data and security — to deliver end-to-end solutions. We have a proven record of advising customers based on their business challenges, designing solutions that scale, building and managing those solutions, and optimizing returns into the future. Named a best place to work, year after year according to Fortune, Forbes and Glassdoor, we attract and develop world-class talent. Join us on our mission to embrace technology, empower customers and deliver the future.
More on Rackspace Technology
Though we’re all different, Rackers thrive through our connection to a central goal: to be a valued member of a winning team on an inspiring mission. We bring our whole selves to work every day. And we embrace the notion that unique perspectives fuel innovation and enable us to best serve our customers and communities around the globe. We welcome you to apply today and want you to know that we are committed to offering equal employment opportunity without regard to age, color, disability, gender reassignment or identity or expression, genetic information, marital or civil partner status, pregnancy or maternity status, military or veteran status, nationality, ethnic or national origin, race, religion or belief, sexual orientation, or any legally protected characteristic. If you have a disability or special need that requires accommodation, please let us know.
United States
AU, CA + 4 more
CA, DE + 3 more
Vietnam only
Mexico only
Egypt only