Data Engineer, UK

We’re on a mission to provide equitable access to economic opportunity, for everyone.

We close critical skill gaps in the workforce through a new kind of apprenticeship that combines work and learning. We begin by recognizing high-potential individuals both inside and outside of a company's current workforce and then we create applied, guided and equitable learning programs, with measurable impact. Because we believe the world needs a better way to match its potential.

We work with over 1,500 leading companies including the likes of Microsoft, Citi and Just Eat to help solve their business-critical problems, and we’ve trained over 16,000 professional apprentices in the tech and data skills of the future. This is made possible by our global team who are driven to achieve a mission that matters, together.

Join Multiverse and help us set a new course for work.

The role:

As a Data Engineer, you will be responsible for building and maintaining the data integrations and processing pipelines that fuel Multiverse’s analytics and data science capabilities. You will consolidate, organise and optimise data flows from/to multiple systems that serve every part of the business.

The role will be within the Data & Insight (D&I) team, reporting to the Director of Data Engineering. Within D&I, you will work closely with Analytics Engineers and our DevOps team, as well as Data Product Managers, Analytics Developers and Data Scientists. You will also need to collaborate with our Product & Engineering teams - as well as domain experts from across the business. You will need to be methodical, analytical, creative and collaborative, with very high attention to detail.

What you’ll focus on:

Data Integrations

• Developing and maintaining bidirectional data integrations with first and third-party systems:

  • ingesting and transforming data to enable our Analytics, Data Science and AI teams

  • exposing the outputs of their work back to all Multiverse systems

• Integrations utilising a range of REST APIs, GraphQL APIs, databases and event-driven systems

• Combining custom development (in Python) with re-use of third-party commercial or OSS technologies where appropriate

Cloud Infrastructure

• Supporting and extending our AWS infrastructure, deployed using Terraform and Serverless, which includes:

• A Data Lake, using S3, Glue and Athena

  1. Processing pipelines, using Lambda, ECS, EventBridge, SQS, DynamoDB, Airflow (AWS MWAA)

  2. A Data Warehouse, currently RDS (but likely to move to Redshift/Snowflake/BigQuery etc. in future)

• We prefer managed services, serverless technologies and event-driven processing, where possible

Orchestration, Monitoring & Observability

• Building scalable, efficient DAGs to manage our processing pipelines

• Ensuring that we have best-in-class monitoring and observability tooling:

• we know when there are issues before anybody else

  1. we are able to diagnose and fix issues quickly

  2. the tooling has high satisfaction scores from the D&I team

What we’re looking for:

Required:

• 3+ years of building and managing data integrations/pipelines within a commercial context

• Proven track record of producing high quality data integrations/pipelines, across a range of technologies (APIs, databases etc.)

• Experience using Python for data engineering use-cases

• Experience using orchestration tools (e.g. Airflow)

• Strong working knowledge of PostgreSQL

• Experience administering cloud infrastructure (e.g. AWS) using infrastructure-as-code (e.g Terraform)

• Experience administering an online codebase repository (e.g. GitHub)

• Experience administering a continuous integration/delivery server (e.g. GitHub Actions, CircleCI)

• Experience writing automated tests for data pipelines

• Meticulous attention to detail

• Commitment to Multiverse’s mission and values

Non-Required (But Desirable):

• Experience using Airflow

• Familiarity with building data models in dbt

• Familiarity with Tableau and/or Alteryx

• Working knowledge of education/skills sector

Our commitment to Diversity, Equity and Inclusion:

We’re an equal opportunities employer. And proud of it. Every applicant and employee is afforded the same opportunities regardless of race, colour, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender, gender identity or expression, or veteran status. This will never change.

Right to work:

Do you have the right to work in the UK? Unfortunately, at this time we cannot offer sponsorship for this role and we cannot consider overseas applications.

Safeguarding:

All posts in Multiverse involve some degree of responsibility for safeguarding. Successful applicants are required to complete a Disclosure Form from the Disclosure and Barring Service ("DBS") for the position. Failure to declare any convictions (that are not subject to DBS filtering) may disqualify a candidate for appointment or result in summary dismissal if the discrepancy comes to light subsequently.