Software Automation Engineer

Ontada is a leading oncology real‑world data and evidence, clinical education, and provider technology business dedicated to transforming the fight against cancer.Partof McKesson Corporation, we support science through our data, technology, and channels, accelerating innovation for life science companies, empowering community oncology providers, and advancing patient care. Together with our partners, we improve the lives of cancer patients.

Position Summary

The Software Automation Engineer (P3) is a senior individual contributor responsible for designing, developing, andmaintainingautomated test solutions acrossOntada’sproduct ecosystem,with a strong focus onAI enabledandGenAI poweredsystems.Reporting to the QA Lead, this role driveshigh qualityautomation for UI, API, backend, and data layers, while ensuring AI/ML features meet expectations for correctness, reliability, safety, and compliance.

This role partners closely with Product, Engineering, andML Opsteams tovalidateboth traditional software workflows andAI drivenbehaviors, includingprompt basedsystems,retrieval augmentedgeneration (RAG), andmodel integratedservices.

Key Responsibilities

Quality Engineering Ownership & Execution

• Own and execute test strategy, planning, and execution for assigned features, services, or product areas under the guidance of the QA Lead.

• Identifyfunctional, integration, and nonfunctional quality risks early; communicate risks, impacts, and recommendations clearly.

• Author comprehensive test strategies, test plans, and test cases aligned with product requirements and acceptance criteria.

• Perform exploratory testing to uncover complex,edge case, and systemic defects.

• Coordinateend to endvalidation across multiple environments to ensurereleasereadiness.

Test Automation & Framework Development

• Design, develop, andmaintainautomated test suites across UI, API, service, and data layers.

• Contribute to the enhancement and maintainability of automation frameworks using tools such as Selenium, Playwright, Cypress, TOSCA, or similar.

• Develop robust API automation usingRestAssured, Postman, or equivalent frameworks.

• Implement effective test data strategies, including synthetic data generation and environment setup.

• Integrate automated tests into CI/CD pipelines to support fast and reliable feedback cycles.

• LeverageAI assisteddevelopment tools(e.g.,GitHub Copilot, Claude Code, or similar) to accelerate test automation development, refactoring, and debugging whilemaintainingcode quality and security standards.

• Use AI tooling toassistwithtest case generation,edge caseidentification, anddata drivenscenario expansion,validatingall outputs through engineering judgment and established QA practices.

AI / GenAI Quality Engineering

• Design and execute test strategies for AI/ML andGenAI poweredfeatures, includingLLM basedworkflows.

• Validate prompt behavior, prompt templates, and prompt chaining across different scenarios and data contexts.

• Perform negative testing for AI systems, including prompt injection, jailbreak attempts, hallucination risks, and unsafe outputs.

• TestRetrieval AugmentedGeneration (RAG) pipelines, including:

• Embedding quality validation

• Retrieval accuracy, recall, and relevance

• Chunking and indexing strategies

• Validate AI outputs for accuracy, consistency, explain ability, and compliance in regulated environments.

• Collaborate with Engineering andML Opsteams to test model integrations, configuration changes, and inference pipelines.

• UtilizeAI poweredtools to supportprompt analysis, test scenario exploration, and hypothesis generationwhen validatingLLM basedfeatures and AI workflows.

• Critically evaluateAI generatedsuggestions and outputs to ensureaccuracy, safety, reproducibility, and regulatory compliance.

Backend & Data Testing

• Perform advanced backend testing acrossSQL and NoSQLdata systems.

• Validate data ingestion, transformations, persistence, and integrity across services and environments.

• Coordinate testing of asynchronous workflows and integrations (e.g., message queues, APIs, batchprocesses).

Agile Collaboration

• Work closely with Product Owners and Business Analysts to refine user stories, define acceptance criteria, and ensure testability.

• Partner with developers during design and implementation to supportshift lefttesting.

• Participate actively in sprint planning, grooming, retrospectives, and release readiness reviews.

• Collaborate with onshore and offshore QA team members to ensure consistent execution and quality standards.

Quality Governance, Compliance & Continuous Improvement

• Ensure testing activities align withHIPAAand other regulatory, security, anddata privacyrequirements.

• Contributeaudit readydocumentation, including test plans, execution evidence, and reports.

• Participate inroot causeanalysis for quality or performance issues and support corrective actions.

• Identifyopportunities for improving QA processes, tools, and documentation; contribute suggestions through established continuous improvement channels.

• Research and evaluate new QA, automation, or performancetesting,AI assistedtools and techniques as appropriate.

Minimum Requirement

Degree or equivalent and typically requires 4+ years of relevant experience.

Critical Skills

• 4+ years of progressive Software Quality Assurance experience, preferably in healthcare or regulated industries.

• 3+ years ofhands-ontest automation development experience.

• 2+ years of API testing and automation experience.

• 3+ years of backend testing experience using SQL and/or NoSQL databases.

• 3+ years of software performance testing experience, including test planning, execution, and analysis.

• 1+ yearsof experience testingAI/ML or GenAI systems, ordemonstrateddelivery ofAI adjacentquality frameworks (e.g., prompt testing, RAG evaluation, guardrails).

• Experience owning QA execution for complex product areas with limitedday to dayoversight.

• Experience mentoring or supporting junior QA engineers.

Additional Skills

• Strong experience working in Agile SDLC environments with CI/CD pipelines.

• Proficiencyin Java, JavaScript, or Python for test automation and scripting.

• Experience with CI/CD tools such as Jenkins, GitHub Actions, GitLab CI and build tools like Maven or Gradle.

• Solid understanding of QA methodologies, test design techniques, and quality metrics.

• Hands-on experience with performance testing tools (JMeter,NeoLoad, or similar).

• Experience usingprofiling and monitoring tools(Dynatrace, New Relic, AppDynamics, Splunk,JProfiler).

• Ability to analyze performance issues related to CPU, memory/heap, garbage collection, threads, databases, messaging systems, and network latency.

• Experience creating reusable, maintainable, and portable automation and performance test scripts.

• RAG testing experience, includingembedding quality, retrieval evaluation, and chunk strategy validation.

• Familiarity withvector databasesand semantic search concepts.

• Hands-on experience usingAI assistedcoding and analysis toolssuch asGitHub Copilot, Claude Code, or similar.

• Ability to apply AI tools effectively for:

• Test automation development and refactoring

• Debugging androot causeinvestigation

• Exploratory test design andedge casediscovery

• Strong understanding oflimitations and risks ofAI generatedoutputs, with the ability tovalidate, correct, and harden results forproduction qualityuse.

• Experience with source control tools such as GitHub, Bitbucket, GitBash.

• Experience with test management tools (qTest, TestRail, ALM,TestLink, or similar).

• Familiarity with microservices and distributed system architectures.

• Experience benchmarking, capacity planning, and release readiness reporting.

• Knowledge of healthcare software, data privacy, and regulatory compliance is a plus.

• Ability to manage multiple priorities and work independently in afast pacedenvironment.

Working Conditions

• Fully remote role within theEastern or Central United States.

• Collaboration with distributed,cross functionalteams.

Candidates must be authorized to work in USA. Sponsorship is not available for this role.