AI QA Automation Engineer

We are looking for an AI QA Automation Engineer (L5) to ensure the quality, reliability, and performance of AI-powered systems, including APIs, agentic workflows, and Retrieval-Augmented Generation (RAG) pipelines.
This role goes beyond traditional QA automation.
You will work at the intersection of quality engineering and AI systems, building automated validation frameworks for LLM-driven behavior and partnering closely with ML and full-stack teams to ship reliable AI features to production.
At Proximity, you won’t just test software - you’ll help define how AI systems are validated, evaluated, and trusted at scale.
Responsibilities: Build and maintain automation test suites for AI APIs, agentic workflows, and RAG-based systems.
Implement LLM evaluation tests, covering accuracy, consistency, determinism, and performance characteristics.
Collaborate closely with ML, backend, and full-stack engineering teams to validate releases and ensure production readiness.
Contribute to regression, reliability, and performance testing across AI and platform services.
Design test strategies for non-deterministic AI behavior, including guardrails, fallback paths, and quality thresholds.
Continuously improve test coverage, tooling, and automation frameworks for AI-driven systems.
Best in class compensation: We hire only the best, and we pay accordingly.
High-impact role: Help define quality standards for production AI systems.
Proximity Talks: Learn from experts across AI, engineering, and product.
Continuous learning: Grow your skills at the intersection of QA, automation, and AI.
About us: We are Proximity - a global team of coders, designers, product managers, geeks, and experts.
We solve complex problems and build cutting-edge technology at scale.
Our team of Proxonauts is growing quickly, which means your impact on the company’s success will be significant.
You’ll work with experienced leaders who have built and shipped AI-driven platforms used at scale.
Here’s a quick guide to getting to know us better: Watch our CEO, Hardik Jagda, tell you all about Proximity.
Read about Proximity’s values and meet some of our Proxonauts.
Explore our website, blog, and design wing — Studio Proximity.
Get behind the scenes with us on Instagram — follow @ProxWrks and @H.
Jagda. 5+ years of experience in QA automation or quality engineering roles.
Strong hands-on experience with Python and/or JavaScript, and modern testing frameworks.
Experience building and maintaining API-level and integration test suites.
Ability to design robust tests for complex, distributed systems.
Familiarity with LLM- or ML-based systems and their unique testing challenges is strongly preferred.
Strong analytical and problem-solving skills, with attention to detail.
Ability to work independently while collaborating effectively with cross-functional teams.
Desired Skills: Experience testing RAG pipelines, vector search, or AI agent workflows.
Exposure to evaluation frameworks for LLMs or ML models.
Familiarity with CI/CD integration for automated test execution.
Understanding of performance and load testing for backend services.