Q
AI Reliability QA Engineer
Accepting applicationsQualGent · United States
Full-Time Entry AIaiarmaterf
Posted
1d ago
Category
Test
Experience
Entry
Country
United States
We are seeking an AI Reliability QA Engineer to harden our autonomous AI systems for enterprise production environments.
AI systems are powerful, but inherently nondeterministic and stateful. This role focuses on engineering reliability into AI-driven workflows, ensuring stability, reproducibility, and trust at scale.
This is not a manual QA or script maintenance position. It is a systems-level reliability engineering role applied to AI-native infrastructure.
You will own the stability, determinism, and reliability layer of our platform.
Key Responsibilities
Deterministic AI Execution
Identify and eliminate flakiness in AI-generated workflows
Improve reproducibility across CI, staging, and production environments
Design validation layers and guardrails for AI agent behavior
Reduce regression escapes through structured reliability metrics
AI Test System Hardening
Evaluate AI-generated test cases for correctness and coverage gaps
Design stress-testing frameworks for AI workflows
Improve system resilience under concurrency and load
Define SLAs and reliability standards for autonomous execution
Observability & Root Cause Analysis
Instrument execution traces across AI decision paths
Build monitoring dashboards for reliability metrics
Reduce time-to-diagnosis for complex failures
Lead incident reviews focused on systemic improvements
Required Qualifications
2–10+ years of experience in QA, SDET, automation engineering, or reliability engineering
Proven experience reducing flakiness in CI/CD pipelines
Strong debugging capabilities across frontend, backend, and infrastructure layers
Experience supporting and improving production releases
Systems-level thinking with a focus on failure modes and edge cases
Preferred Qualifications
Mobile testing expertise (iOS/Android, emulators, device farms)
Experience with distributed systems
Observability tooling experience (Datadog, Prometheus, OpenTelemetry, Sentry, etc.)
Cloud infrastructure experience (AWS, GCP)
Exposure to LLM-based or agent-driven systems
What Success Looks Like
Significant reduction in AI system flakiness
Measurable improvements in regression catch rate
Deterministic behavior across environments
Faster root-cause analysis cycles
Increased enterprise trust in autonomous execution
Why This Role Matters
AI accelerates software development, but without reliability, it erodes trust.
The companies that succeed in the AI era will be those that engineer reliability into probabilistic systems. This role is critical to building that foundation.
Show more Show less
AI systems are powerful, but inherently nondeterministic and stateful. This role focuses on engineering reliability into AI-driven workflows, ensuring stability, reproducibility, and trust at scale.
This is not a manual QA or script maintenance position. It is a systems-level reliability engineering role applied to AI-native infrastructure.
You will own the stability, determinism, and reliability layer of our platform.
Key Responsibilities
Deterministic AI Execution
Identify and eliminate flakiness in AI-generated workflows
Improve reproducibility across CI, staging, and production environments
Design validation layers and guardrails for AI agent behavior
Reduce regression escapes through structured reliability metrics
AI Test System Hardening
Evaluate AI-generated test cases for correctness and coverage gaps
Design stress-testing frameworks for AI workflows
Improve system resilience under concurrency and load
Define SLAs and reliability standards for autonomous execution
Observability & Root Cause Analysis
Instrument execution traces across AI decision paths
Build monitoring dashboards for reliability metrics
Reduce time-to-diagnosis for complex failures
Lead incident reviews focused on systemic improvements
Required Qualifications
2–10+ years of experience in QA, SDET, automation engineering, or reliability engineering
Proven experience reducing flakiness in CI/CD pipelines
Strong debugging capabilities across frontend, backend, and infrastructure layers
Experience supporting and improving production releases
Systems-level thinking with a focus on failure modes and edge cases
Preferred Qualifications
Mobile testing expertise (iOS/Android, emulators, device farms)
Experience with distributed systems
Observability tooling experience (Datadog, Prometheus, OpenTelemetry, Sentry, etc.)
Cloud infrastructure experience (AWS, GCP)
Exposure to LLM-based or agent-driven systems
What Success Looks Like
Significant reduction in AI system flakiness
Measurable improvements in regression catch rate
Deterministic behavior across environments
Faster root-cause analysis cycles
Increased enterprise trust in autonomous execution
Why This Role Matters
AI accelerates software development, but without reliability, it erodes trust.
The companies that succeed in the AI era will be those that engineer reliability into probabilistic systems. This role is critical to building that foundation.
Show more Show less