AI Reliability Engineer

arrow

New York City / $190000 - $210000 annum

INFO

Salary
SALARY:

$190000 - $210000

Location

LOCATION

New York City

Job Type
JOB TYPE

Permanent

AI Reliability Engineer

New York Metro Area - Hybrid

$190,000 - $210,000

THE COMPANY

Harnham is currently partnered with a data-driven fintech company that is spreading innovation across the construction industry. Their founding team comes from big tech organizations (LinkedIn, Uber, SpaceX) where they have successfully built and deployed software into the construction space.The ideal candidate will have a proven track record of leveraging machine learning and artificial intelligence to drive business insights and improve operational efficiency and a high level individual contributor.

THE ROLE

We are seeking an AI Reliability Engineer to build robust performance measurement systems for AI models, including retrieval-augmented generation (RAG) and agent-based architectures. This role is ideal for someone passionate about AI validation, system monitoring, and improving model reliability through data-driven insights.

THE RESPONSIBILITIES

  • Develop and refine evaluation frameworks to assess AI model performance, focusing on real-world accuracy and reliability
  • Build monitoring tools and dashboards that enhance visibility into AI systems throughout development and deployment
  • Work closely with cross-functional teams to integrate rigorous testing and validation processes into production workflows
  • Identify and resolve performance bottlenecks, ensuring AI systems operate at peak efficiency
  • Stay informed on emerging trends in AI evaluation, large language models, and system monitoring to continuously improve assessment methodologies

YOUR SKILLS AND EXPERIENCE

  • Advanced degree (MS/PhD) in Computer Science, Machine Learning, Artificial Intelligence, or a related field
  • 2+ years of experience in AI or ML system evaluation, focusing on performance analysis and validation
  • Hands-on experience with observability tools, analytics platforms, or data engineering pipelines for AI monitoring
  • Proficiency in Python and familiarity with ML frameworks such as TensorFlow, PyTorch, or scikit-learn
  • Understanding of RAG and agent-based AI workflows, with expertise in measuring and optimizing their performance
  • Experience with synthetic data generation or test automation for AI model validation
  • Strong analytical and problem-solving skills, with the ability to thrive in a fast-paced, collaborative environment

BENEFITS

As an AI Reliability Engineer, you can expect to earn up to $210,000 (depending on experience), highly competitive benefits and equity.

HOW TO APPLY

Please register your interest by sending your Resume to Grace McCarthy via the Apply link on this page

KEYWORDS

AI Evaluation, Model Validation, Machine Learning, Observability, System Monitoring, AI Performance Metrics, RAG, AI Testing, Python, TensorFlow, PyTorch, Data Engineering, Synthetic Data, Large Language Models, Reinforcement Learning, Startup Hiring

CONTACT

SIMILAR
JOB RESULTS

4k-Harnham_DA copy
CAN’T FIND THE RIGHT OPPORTUNITY?

STILL
LOOKING?

If you can’t see what you’re looking for right now, send us your CV anyway – we’re always getting fresh new roles through the door.