{"id":7734,"date":"2026-06-01T14:57:04","date_gmt":"2026-06-01T14:57:04","guid":{"rendered":"https:\/\/staging.elixirrdigital.com\/us\/vacancy\/senior-qa-engineer-ai-solutions-civn\/"},"modified":"2026-06-18T07:21:31","modified_gmt":"2026-06-18T07:21:31","slug":"senior-qa-engineer-ai-solutions-civn","status":"publish","type":"vacancy","link":"https:\/\/staging.elixirrdigital.com\/us\/vacancy\/senior-qa-engineer-ai-solutions-civn\/","title":{"rendered":"Senior QA Engineer (AI Solutions)"},"content":{"rendered":"
\n
\n

Elixirr Digital<\/strong> is a dynamic and innovative global consulting firm, recognized for delivering transformative solutions to our clients across a wide range of industries. As part of our growing AI practice, we are committed to developing enterprise-grade products, accelerators and agentic solutions that drive innovation for our clients and our people. We help leading organizations harness advanced AI technologies \u2014 from agentic workflows and retrieval-augmented generation (RAG) to multi-agent orchestration \u2014 grounded in their proprietary data and operating within the security and compliance standards their industries demand. <\/p>\n<\/div>\n

\n

We are looking for a Senior QA Engineer<\/strong> to take ownership of quality for Elixirr\u2019s AI-enabled solutions \u2014 from our internal agent platform to the growing portfolio of accelerators we build to deliver faster and grow the business. <\/p>\n<\/div>\n

\n

Quality on AI systems is a different thing. The fundamental shift is from deterministic assertions to probabilistic scoring: you\u2019re testing for meaning, grounding, accuracy and safety, not exact outputs. Behaviour drifts as models change, and a single regression in a prompt, tool or retrieval step can cascade across an entire agent workflow. We need a QA leader who embraces that reality, automates aggressively, and uses AI itself to accelerate how we surface, reproduce and triage issues. <\/p>\n<\/div>\n

\n

This is an SDET-shaped role: you\u2019ll shape our test strategy, build the automation and evaluation harnesses that keep our agents honest, and partner with engineering to bake quality into the way we build \u2014 shift-left into requirements and design, and shift-right into production observability. <\/p>\n<\/div>\n<\/div>\n

\n

Candidates applying for employment contract kindly note this position is a onsite working opportunity from our locations in Cape Town<\/strong> <\/span>or Johannesburg<\/strong>.<\/span><\/p>\n

What you will be doing as a Senior <\/strong>Back End Engineer<\/strong> at Elixirr Digital?<\/strong> <\/span><\/p>\n

\n
\n

QA Strategy for AI-Enabled Solutions <\/strong><\/p>\n

    \n
  • \n

    Own the end-to-end QA strategy for Elixirr\u2019s agent platform and AI-enabled accelerators \u2014 functional, non-functional, behavioural and safety. <\/p>\n<\/li>\n

  • \n

    Define what \u201cgood\u201d looks like for agent behaviour: accuracy, grounding, tool use, escalation, latency, cost, safety and user experience. <\/p>\n<\/li>\n

  • \n

    Shift left: engage at requirements and design so gaps, edge cases and testability issues are caught before code is written. <\/p>\n<\/li>\n

  • \n

    Shift right: use production traces, evals and user feedback as part of the regression loop, so quality keeps improving after release. <\/p>\n<\/li>\n

  • \n

    Establish deployment gates so AI features ship with evidence, not hope. <\/p>\n<\/li>\n<\/ul>\n

    Automation & Evaluation <\/strong><\/p>\n

      \n
    • \n

      Build and maintain automated test suites across UI, API, contract, integration and end-to-end layers, running as a first-class part of CI\/CD. <\/p>\n<\/li>\n

    • \n

      Design LLM and agent evaluation harnesses using a layered approach: automated checks, LLM-as-judge scoring, rubric-based evaluation and targeted human review. <\/p>\n<\/li>\n<\/ul>\n<\/div>\n

      \n
        \n
      • \n

        Maintain golden datasets, regression suites and red-teaming scenarios that evolve with the product. <\/p>\n<\/li>\n

      • \n

        Bring LLM quality metrics into the same unified reporting as functional, performance and security results. <\/p>\n<\/li>\n

      • \n

        Drive automation-first: if a check can be automated, it should be. <\/p>\n<\/li>\n<\/ul>\n

        AI-Accelerated QA <\/strong><\/p>\n

          \n
        • \n

          Use AI tooling (code-generation, LLM-based test generation, synthetic data, AI-assisted exploratory testing, self-healing test frameworks) to accelerate coverage and reduce manual toil. <\/p>\n<\/li>\n

        • \n

          Practice prompt engineering for QA: constrain AI tools with precise specifications so they produce tests and triage artefacts worth keeping. <\/p>\n<\/li>\n

        • \n

          Build or integrate AI-assisted triage \u2014 clustering failures, summarizing root causes, drafting repro steps and proposing fixes to engineers. <\/p>\n<\/li>\n

        • \n

          Continuously evolve the QA tooling stack as the AI developer ecosystem matures. <\/p>\n<\/li>\n<\/ul>\n

          Reliability, Performance & Safety <\/strong><\/p>\n

            \n
          • \n

            Run performance, load and chaos testing against agent workflows and backend services. <\/p>\n<\/li>\n

          • \n

            Stress-test guardrails, prompt injection defences, tool-use restrictions and data boundaries. <\/p>\n<\/li>\n

          • \n

            Cover accessibility, visual regression and security checks (SAST\/DAST) as part of the standard test pipeline. <\/p>\n<\/li>\n

          • \n

            Partner with security and compliance on safety reviews for client-facing AI features. <\/p>\n<\/li>\n<\/ul>\n

            Process & Collaboration <\/strong><\/p>\n