CJ
AI QA Engineer (Multilingual)
Accepting applicationsChatGPT Jobs · New York, NY
Full-Time Mid AIPythonaiate
Posted
2d ago
Category
Test
Experience
Mid
Country
United States
Job Description
Job Information
AI QA Engineer (Multilingual)
Company
Scaled Cognition
Location
New York, NY (Remote)
Key Responsibilities
Inspect, review, and grade LLM training data, evaluation test cases, and model outputs to ensure quality and accuracy.
Maintain local development environments, run test pipelines, investigate edge cases, and submit Git/GitHub PRs.
Analyze training data to identify error cases and technical failures.
Leverage LLMs for translation, verification, and maintenance of cross-lingual datasets.
Collaborate with engineering teams to refine evaluation criteria and improve data pipelines.
Key Qualifications
Strong technical background with hands-on coding experience (Python preferred) and proficiency with Git/GitHub.
Fluency in English and native or near-native proficiency in at least one other language.
Deep understanding of Large Language Models (LLMs), failure modes (hallucinations, formatting errors), and prompting techniques.
Proven experience in Quality Assurance, Data Quality, or Data Engineering, with a track record of auditing large datasets.
Exceptional written communication skills across multiple languages.
Required Skills & Attributes
Obsessive attention to detail for finding edge cases and translation errors.
Ability to handle repetitive data inspection tasks with a "builder" mentality.
Technical self-sufficiency: comfort with terminal usage, Python scripts, and version control.
Strong linguistic understanding of nuances required for high-quality cross-lingual evaluation.
Ability to thrive in a fast-paced, ownership-driven environment.
Show more Show less
Job Information
AI QA Engineer (Multilingual)
Company
Scaled Cognition
Location
New York, NY (Remote)
Key Responsibilities
Inspect, review, and grade LLM training data, evaluation test cases, and model outputs to ensure quality and accuracy.
Maintain local development environments, run test pipelines, investigate edge cases, and submit Git/GitHub PRs.
Analyze training data to identify error cases and technical failures.
Leverage LLMs for translation, verification, and maintenance of cross-lingual datasets.
Collaborate with engineering teams to refine evaluation criteria and improve data pipelines.
Key Qualifications
Strong technical background with hands-on coding experience (Python preferred) and proficiency with Git/GitHub.
Fluency in English and native or near-native proficiency in at least one other language.
Deep understanding of Large Language Models (LLMs), failure modes (hallucinations, formatting errors), and prompting techniques.
Proven experience in Quality Assurance, Data Quality, or Data Engineering, with a track record of auditing large datasets.
Exceptional written communication skills across multiple languages.
Required Skills & Attributes
Obsessive attention to detail for finding edge cases and translation errors.
Ability to handle repetitive data inspection tasks with a "builder" mentality.
Technical self-sufficiency: comfort with terminal usage, Python scripts, and version control.
Strong linguistic understanding of nuances required for high-quality cross-lingual evaluation.
Ability to thrive in a fast-paced, ownership-driven environment.
Show more Show less