LLM Evaluation Engineer

Likely ghost job

πŸ•’ March 28

Where this job comes from

Posted on ThirdLaw Molecular's careers page β†—ThirdLaw Molecular's careers page, not through a job board

First seen by Remote Rocketship March 28

Still open on ThirdLaw Molecular's site. Checked 3 hours ago.

Apply Now
Logo of ThirdLaw Molecular

ThirdLaw Molecular

1 - 10 employees

Founded 2020

🧬 Biotechnology

πŸ’Š Pharmaceuticals

βš•οΈ Healthcare Insurance

Biotechnology β€’ Pharmaceuticals β€’ Healthcare Insurance

ThirdLaw Molecular is a pioneering biotechnology company that has developed a groundbreaking platform utilizing Spiroligomerβ„’ molecules for targeted drug discovery and diagnostics. Founded by Dr. Christian Schafmeister, the company focuses on creating a new class of therapeutics that are safer and more effective by employing a unique chemical platform for the design of synthetically engineered molecules. Their innovative approach enables highly selective binding to proteins, enhancing the potential for both drug therapies and diagnostic applications.

πŸ“‹ Description

β€’ Build the evaluation layer in the ThirdLaw platform for LLM prompts and responses β€’ Design and tune guardrails, classifiers, and semantic judgment systems in real-time β€’ Implement evaluation strategies with semantic similarity, foundation model scoring, and rule-based systems β€’ Integrate model outputs with downstream enforcement actions (e.g. redaction, escalation, blocking) β€’ Prototype, tune, and productize small language models for classification, labeling, or scoring β€’ Collaborate with data infrastructure engineers to connect evaluation logic with ingestion and storage β€’ Build tools to observe, debug, and improve evaluator performance across data distributions β€’ Define abstractions for reusable evaluation components that can scale across use cases

🎯 Requirements

β€’ 7+ years of experience in ML systems or AI engineering roles β€’ At least 1–2 years working directly with LLMs, NLP pipelines, or semantic search β€’ Deep understanding of foundation models (e.g. OpenAI, Claude, Mistral, Llama) and APIs β€’ Hands-on experience with vector search (e.g. FAISS, Qdrant, Weaviate) and embeddings pipelines β€’ Proven ability to implement real-time or near-real-time evaluation logic using semantic similarity, classifier scoring, or structured rules β€’ Strong in Python, with familiarity using libraries like Hugging Face Transformers, LangChain, and PyTorch or TensorFlow β€’ Ability to reason about model behavior, test prompt configurations, and debug complex decision logic in production

πŸ–οΈ Benefits

β€’ Generous benefits β€’ Market cash compensation β€’ Above-market equity β€’ Well-designed benefits

Apply Now

πŸ“Š Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Similar Jobs

πŸ•’ March 27

Railway

11 - 50

☁️ SaaS

Infrastructure Engineer at Railway building global datacenters with a focus on performance and reliability. Role emphasizes technical challenges and creative solutions in a high ownership environment.

πŸ•’ March 27

Novel

11 - 50

πŸ›’ Retail

πŸ›οΈ eCommerce

☁️ SaaS

Engineer developing a no-code NFT commerce platform backed by top investors. Collaborating with a world-class team to accelerate the adoption of NFTs.

πŸ•’ March 27

Commonwealth Associates, Inc.

201 - 500

πŸ’Ό Consulting

πŸ“¦ Logistics

⚑ Energy

Project Engineer overseeing large EPC projects, ensuring compliance with specifications and standards. Leading project teams and managing design, construction, and commissioning of power delivery infrastructure.

πŸ•’ March 27

Commonwealth Associates, Inc.

201 - 500

πŸ’Ό Consulting

πŸ“¦ Logistics

⚑ Energy

Seeking a Senior System Studies Engineer for complex electrical power projects. Collaborate with teams and clients to ensure project adherence to quality and budget standards.

πŸ•’ March 26

Partner Engineering & Science, Inc.

1001 - 5000

πŸ—οΈ Construction

πŸ’Ό Consulting

🏠 Real Estate

Building Science Project Engineer conducting Property & Facility Condition Assessments across the country for Partner Engineering and Science, Inc. Evaluating building systems, reporting findings, and providing recommendations for clients.

Built byΒ Lior Neu-ner. I'd love to hear your feedback β€” Get in touch via DM or support@remoterocketship.com
Follow Remote Rocketship on LinkedInFollow Remote Rocketship on InstagramFollow Remote Rocketship on FacebookFollow Remote Rocketship on TikTok