Senior Data Engineer

About vidIQ

vidIQ builds software to help YouTube creators achieve their goals. Our mission is to advance the creator's journey with actionable data-driven insights and AI-powered tools. We are dedicated to our values of being creator-obsessed, lean and fast, and scientific. Having already supported millions of creators, we invite stunning new co-workers to join us in reaching millions more.

Imagine a product that reached over a million users without a traditional sales team. This product stands at the forefront of career evolution, where everyone can be their own brand with unlimited growth potential. That's the opportunity at vidIQ—an infinite market, a large and highly engaged customer base, and the chance to help scale the platform that delivers insights to millions of creators.

Discover more about vidIQ firsthand at www.vidiq.com.

About the Team

We're a team of 100+ people and growing quickly. Our vision is to be the smart copilot for every creator. We strive to create a welcoming environment where our team of smart, passionate people can share ideas with each other and do their best work. We are looking for team members who are excellent at their craft, communicate with kindness, and act like owners.

About the role

vidIQ is seeking a highly-motivated Senior Data Engineer with at least 5 years of hands-on data engineering experience to join our growing team. The ideal candidate will be a go-getter with the ability to work independently, be proactive and take full ownership. In this role, you will have oversight of working with data lake, events streaming, building ETL pipelines, wiring SQL, k8s.

You must be highly collaborative and a self-starter who is able to work in a fast-paced environment. Strong communication skills are essential in this role, as it will be integral in communicating to the back-end team where and how to implement data integration and persistence. Interest in AI and the potential it brings is a must.

This is a remote position open to candidates worldwide. Employment classification and engagement structure will be determined based on location, legal requirements, and mutual fit.

An overview of our technology stack

• AWS: S3, Glue, Athena, EKS, DynamoDB, OpenSearch, MSK

• SQL, Python, Spark (on k8s), pandas, dbt, Iceberg, Airflow, Superset, Airbyte, terraform, docker, Github Actions

What you will do:

• Building efficient, critical data pipelines and data platform

• Collaborate closely with ML/AI Engineers, Backend Engineers and product to create the data sets that power vidIQ’s algorithms

• Be an advocate for data quality, acquisition of new data sources, and data infrastructure tooling

• Work closely with cross-functional teammates, including product managers, designers, backend, DevOps and ML/AI Engineers to deliver the highest impact to our users

Requirements

Job Requirements:

• 5+ years of Python experience

• Experience working with cloud; AWS preferred

• Additional experience with DynamoDB, Athena, Iceberg, S3, Glue, OpenSearch

• Familiarity with Spark preferred

• Hands-on experience with data workflow orchestration (Airflow)

• Proven track record of working with cross-functional teams in an agile-like environment

• Ability to communicate data concepts, requirements, and risks clearly to cross-functional team members, especially product analysts, data scientists, and product managers

• Preferably, you have experience developing ML-based products and services

Nice to have:

• Vector databases, embeddings, semantic search

• Real time data ingestion and analytics

• Experience or familiarity with AI technologies such as GPT, OpenAI, Langchain, Hugging Face, etc.

• K8s

• dbt

• Backend experience

Perks and Benefits

• Work Remotely: Embrace the freedom to choose your ideal workspace anywhere in the world.

• Flexible Time Off: We support a flexible vacation policy allowing you to take time off when you need to recharge.

• Communication Stipend: Enjoy a monthly stipend to cover your phone and internet expenses, helping you stay connected effortlessly.

• Global Impact: Join our diverse team that’s shaping the future of the creator economy across the globe.

• Competitive Compensation: vidIQ believes in offering competitive compensation that reflects your skills, experience, and contributions. We ensure fairness through internal equity and geo-location based pay, with the upper range reserved for individuals demonstrating exceptional job knowledge and skills.

Our Commitment

At vidIQ, we dedicate ourselves to enabling creators of all kinds to succeed. We prioritize attracting diverse talent and fostering an environment that encourages collaboration, creativity, and personal growth. We are committed to building a company and a community where individuals thrive by being true to themselves and are inspired to do their best work every day.

If you share our enthusiasm for empowering creators, we would love to hear from you. Click the “Apply” button below to start your journey with us.