TG
AI Tester
Accepting applicationsTMV Global Inc · Atlanta Metropolitan Area
Full-Time Mid_senior AIPythonaiarmasic
Posted
4d ago
Category
Test
Experience
Mid_senior
Country
N/A
Job Title - AI Tester
Location - Atlanta, GA
Ensure the quality, safety, reliability, and performance of Generative AI systems by developing and executing testing strategies that validate accuracy, fairness, security, and alignment with business and ethical standards.
Key Responsibilities
Design and execute test strategies for GenAI systems (LLMs, RAG pipelines, AI agents, copilots)
Validate accuracy, relevance, consistency, and factuality of AI-generated outputs
Perform prompt testing, adversarial testing, and edge-case validation
Test for hallucinations, bias, toxicity, and harmful content
Validate retrieval quality in RAG-based systems (chunking, embeddings, relevance)
Conduct regression testing for model updates, prompt changes, and data refreshes
Collaborate with data scientists, ML engineers, product managers, and compliance teams
Define and track AI quality metrics (BLEU, ROUGE, faithfulness, groundedness, latency)
Automate test cases for GenAI workflows where feasible
Required Skills
8+ years of experience in QA, testing, or quality engineering, with hands-on exposure to AI/ML or Generative AI systems
Experience testing with chatbots, NLP-based applications, and GenAI solutions, including prompt engineering and optimization
Strong understanding of AI evaluation techniques, including hallucination detection, factual accuracy, bias, and output consistency
Knowledge of Responsible AI principles, including fairness, transparency, and explainability
Experience validating data quality with a basic understanding of statistics and AI performance metrics
Proficient in API testing (REST, JSON) and testing AI model endpoints
Hands-on experience with test automation tools and scripting (Python preferred)
Familiarity with the ML lifecycle, model versioning, and regression testing for AI systems
Exposure to cloud-based AI platforms such as Azure OpenAI, AWS Bedrock, or Google Vertex AI
Strong foundation in software testing methodologies, including exploratory, negative, and adversarial testing
Ability to design test cases for non-deterministic AI systems
Strong analytical and critical-thinking skills, with the ability to objectively assess subjective AI outputs
Excellent documentation, communication, and collaboration skills, with experience working in Agile / DevOps, cross-functional AI teams
Show more Show less
Location - Atlanta, GA
Ensure the quality, safety, reliability, and performance of Generative AI systems by developing and executing testing strategies that validate accuracy, fairness, security, and alignment with business and ethical standards.
Key Responsibilities
Design and execute test strategies for GenAI systems (LLMs, RAG pipelines, AI agents, copilots)
Validate accuracy, relevance, consistency, and factuality of AI-generated outputs
Perform prompt testing, adversarial testing, and edge-case validation
Test for hallucinations, bias, toxicity, and harmful content
Validate retrieval quality in RAG-based systems (chunking, embeddings, relevance)
Conduct regression testing for model updates, prompt changes, and data refreshes
Collaborate with data scientists, ML engineers, product managers, and compliance teams
Define and track AI quality metrics (BLEU, ROUGE, faithfulness, groundedness, latency)
Automate test cases for GenAI workflows where feasible
Required Skills
8+ years of experience in QA, testing, or quality engineering, with hands-on exposure to AI/ML or Generative AI systems
Experience testing with chatbots, NLP-based applications, and GenAI solutions, including prompt engineering and optimization
Strong understanding of AI evaluation techniques, including hallucination detection, factual accuracy, bias, and output consistency
Knowledge of Responsible AI principles, including fairness, transparency, and explainability
Experience validating data quality with a basic understanding of statistics and AI performance metrics
Proficient in API testing (REST, JSON) and testing AI model endpoints
Hands-on experience with test automation tools and scripting (Python preferred)
Familiarity with the ML lifecycle, model versioning, and regression testing for AI systems
Exposure to cloud-based AI platforms such as Azure OpenAI, AWS Bedrock, or Google Vertex AI
Strong foundation in software testing methodologies, including exploratory, negative, and adversarial testing
Ability to design test cases for non-deterministic AI systems
Strong analytical and critical-thinking skills, with the ability to objectively assess subjective AI outputs
Excellent documentation, communication, and collaboration skills, with experience working in Agile / DevOps, cross-functional AI teams
Show more Show less
Similar Jobs
AM
Material Handler III
Applied Materials · Austin, United States, North America
N
Developer Advocate – Robotics and Physical AI
NVIDIA · Santa Clara, United States, North America
N
Principal Machine Learning Engineer, Accelerated Apache Spark
NVIDIA · Santa Clara, United States, North America
AD
Counsel, Corporate Attorney
Analog Devices · Wilmington, United States, North America