Back to all jobs
C

Senior Software Engineer, Model Evaluation

Clera

San Francisco · us Full-time 180,000 – 250,000 USD 17h ago

Job description

ABOUT THE ROLE Join an early-stage AI team building agents for manufacturing operations. You will create benchmarks and train models against real procurement and supply-chain workflows, helping ensure agents can handle customer-facing tasks reliably. WHAT YOU'LL DO - Design and build evaluation suites for supplier sourcing, qualification, compliance documents, purchase orders, freight, and order tracking. - Work with procurement and manufacturing experts to scope tasks, write reference answers and rubrics, and build graders. - Train and fine-tune models, using evaluation results to guide experiments. - Identify agent failures, improve production systems, and measure whether changes work. - Help establish the team's research and evaluation practices. WHAT WE'RE LOOKING FOR - At least 3 years of experience, operating at a senior or staff level. - Hands-on experience designing benchmarks and evaluations, including authoring tasks, reference solutions or rubrics, and quality checks. - Practical experience training and fine-tuning models, not only using models through APIs. - Ability to work closely with non-ML domain experts and translate their knowledge into measurable tasks. - Supply-chain, procurement, manufacturing, or early-stage startup experience is helpful. COMPENSATION & BENEFITS Base salary is $180,000 to $250,000 annually, plus equity. Visa sponsorship details were not provided. LOCATION Hybrid in San Francisco, California, United States.

Similar open jobs