Senior Automation Tester (Python) – Agentic AI & LLM Systems (Hybrid)

Resillion is seeking a Senior Automation Engineer / SDET with strong Python skills and hands-on experience testing AI systems, including LLM-driven applications and agentic AI workflows.

Requirements

• Develop and maintain a Python-based automation framework for validating AI agents (chatbots, voicebots, agentic workflows).
• Design and execute automated tests for: LLM responses, conversational flows, retrieval-augmented generation (RAG), and end-to-end agentic scenarios.
• Apply modern AI testing strategies and evaluation methods.
• Integrate automated tests into CI/CD pipelines.
• Build and manage batch and scheduled executions using Docker, Kubernetes, cron jobs, or similar tooling.
• Manage code repositories using Git, including branching strategies, version control, and collaborative workflows.
• Collaborate closely with engineering, AI/ML, and product teams to ensure high-quality delivery of AI-powered features.