AI/ML Engineer
Clera · San Francisco · onsite
ABOUT THE ROLE
Join an AI/ML team building tools to make production AI agents more reliable. You will own the intelligence that identifies agent failures, assesses confidence, and checks whether proposed fixes work.
WHAT YOU'LL DO
- Detect subtle failures, including omissions, compliant but incorrect responses, and patterns that emerge across large volumes of traces.
- Turn user behavior signals, such as rephrasing, abandonment, and retries, into evidence of agent failures.
- Build evaluation methods to measure detection precision when labeled ground truth is limited or unavailable.
- Make patch generation trustworthy by reproducing failures, verifying proposed fixes, and avoiding low-confidence changes.
- Improve the cost efficiency of detection, choosing appropriate models and approaches for each event.
- Review production customer traces regularly to identify problems and opportunities for improvement.
WHAT WE'RE LOOKING FOR
- Demonstrated AI/ML engineering experience; candidates with 0 to 10 years of experience are welcome, including new graduates and people without a traditional degree.
- Hands-on experience using LLMs in production, including evaluations, LLM-as-judge, embeddings, and deciding when a smaller model or no model is appropriate.
- An individual-contributor engineering background and a track record of shipping products that people use.
- Strong research judgment and the ability to distinguish meaningful improvements from noise, especially when ground truth is unavailable.
- Experience running agents in production and learning from their failures is a plus.
COMPENSATION & BENEFITS
Base salary: $120,000 to $200,000 annually. Visa sponsorship is not available.
LOCATION
On-site in San Francisco, United States.