Principal Observability Engineer, Remote

Overview

As a Principal Observability Engineer, you'll help consult with teams to ensure their systems are reliable, and scalable. You'll operate at the intersection of systems engineering, platform operations, and tooling — providing expert guidance and making data-driven recommendations. You will design and teach best practices in observability, helping teams improve their system reliability and performance. This is a role for an engineer who enjoys solving complex infrastructure puzzles and being a connective force between Engineering, Support, and Product teams. You will report into the Sr. Director of Global SRE.

Responsibilities

• Design and implement comprehensive observability solutions that provide visibility into other teams' applications.
• Collaborate with development, operations, and SRE teams to identify performance metrics, such as latency, traffic, errors, and resource saturation.
• Ensure early resolution of issues to reduce downtime and improve user experience.
• Build dashboards, alerts, and reports that provide visibility into system performance.
• Implement solutions, ensuring that the right people are notified in case of issues.
• Fine-tune backend configurations to match customer traffic patterns, maintaining the stability and scalability of applications.
• Lead junior engineers and encourage a culture of continuous improvement.
• Configure applications to collect telemetry data, including metrics, logs, and traces.
• Define requirements and advocate for tooling improvements that reduce manual effort
• Guide system optimization to ensure that our practices evolve with the latest trends.
• Design and implement end-to-end observability solutions that integrate with Dynatrace and ServiceNow CMDB.

What You Bring:

• 10–15 years of experience in software engineering, DevOps, SRE, or platform operations roles.
• Programming background in Python, and experience with Dynatrace.
• History of implementing end-to-end observability architecture
• Proficiency in Infrastructure as Code (IaC) and configuration management
• Working knowledge of cloud-native infrastructure, distributed microservice architectures, and CI/CD pipelines.
• Hands-on experience with Jira or similar systems.
• Familiarity with observability tools (e.g., metrics, logging, and tracing platforms).
• Hands-on experience building dashboards
• Certifications in cloud platforms (e.g., AWS Certified DevOps Engineer, Azure DevOps Engineer Expert).
• History of developing technical teams.
• Proficiency in IT service management tools like ServiceNow.

Benefits/Perks:

• Great compensation package and bonus plan
• Core benefits including medical, dental, vision, and matching 401K
• Flexible work environment, ability to work remote, hybrid or in-office
• Flexible time off including volunteer time off, vacation, sick and 12-paid holidays
• Explore all our exciting benefits here: https://yourexperianbenefits.com/cand-index.html

Experian is proud to be an Equal Opportunity and Affirmative Action employer. Innovation is an important part of Experian's DNA and practices, and our diverse workforce drives our success. Everyone can succeed at Experian and bring their whole self to work, irrespective of their gender, ethnicity, religion, colour, sexuality, physical ability or age. If you have a disability or special need that requires accommodation, please let us know at the earliest opportunity.

Experian is a global data and technology company, powering opportunities for people and businesses around the world. We help to redefine lending practices, uncover and prevent fraud, simplify healthcare, create marketing solutions, and gain deeper insights into the automotive market, all using our unique combination of data, analytics and software. We also assist millions of people to accomplish their financial goals and help them save time and money.

We invest in people and new advanced technologies to unlock the power of data. As a FTSE 100 Index company listed on the London Stock Exchange (EXPN), we have a team of 22,500 people across 32 countries. Our corporate headquarters are in Dublin, Ireland. Learn more at experianplc.com.