Technical Solutions Architect, Evals & Fine-Tuning

Emploi pas sur LinkedIn

🕒 il y a 1 mois

🇺🇸 États-Unis – Télétravail

💵 $140 000 - $160 000 / an

⏰ Temps Plein

🟠 Senior

🔴 Expert

💻 Ingénieur Solutions

👻 Score fantôme 39%

Ce que ça veut dire : la probabilité que cette offre reste ouverte sans recrutement actif.Comment le lire : 60 % ou plus, on pense que c'est une offre fantôme. Un score plus bas est moins inquiétant.Comment on la calcule : ancienneté, republications, postes ouverts vs taille de l'équipe, rapidité de clôture, et si le salaire est indiqué.À noter : une estimation à partir de données publiques, pas un jugement sur l'entreprise. De vraies offres peuvent avoir un score élevé, alors prenez-le comme un indice, pas comme un fait.

Postuler Maintenant

Trouver des Emplois à Distance Similaires

Configurer une alerte email pour les nouveaux emplois

📊 Vérifiez votre score de CV pour ce poste

Améliorez vos chances d'obtenir un entretien en vérifiant votre score de CV avant de postuler.

Télécharger le CV

InnoData

Site WebLinkedInTous les Emplois

2 - 10 employés

Fondée en 2019

🤝 B2B

💼 Conseil

🌍 Impact social

B2B • Consulting • Social Impact

InnoData est INNOvation DATA SCS, une coopérative sociale italienne basée à Foggia, spécialisée dans la fourniture de solutions technologiques. Le site web de l'entreprise (actuellement en maintenance) met en avant les "Soluzioni tecnologiche" (solutions technologiques) et l'impact social ("Impatto sociale"). Les coordonnées incluent Via Francesco Crispi 65, 71121 Foggia, Italie. D'après les informations disponibles, InnoData semble opérer à l'intersection de la technologie et de l'impact social, en proposant probablement des services axés sur la technologie à d'autres organisations.

Description

• Lead technical discovery with prospective and existing customers — foundation model labs, frontier AI teams, and large enterprises — to understand model objectives, gaps, and constraints. • Design end-to-end solutions across the post-training stack: SFT data curation, preference data collection for RLHF/DPO, golden datasets, custom benchmarks, LLM-as-judge pipelines, human-in-the-loop evaluation, red teaming, and multimodal eval (text, image, audio, video, long-context). • Architect engagements that combine Innodata’s platforms (GenAI Test & Evaluation Platform, Annotation Platform, GenAI Workbench) with our global SME workforce across 85+ languages and domains. • Author technical proposals, SOWs, solution diagrams, and pricing models in partnership with sales, delivery, and finance. • Run technical workshops, POCs, and pilot designs that de-risk larger programs and prove value quickly. • Serve as the ongoing technical advisor during delivery, partnering with applied research scientists, AI/ML research engineers, language data scientists, and program managers to keep solutions aligned with the original intent. • Feed customer signal back into Innodata’s R&D and product roadmap — what benchmarks customers actually want, where eval methodology is breaking, what new fine-tuning paradigms are gaining traction. • Stay current on the state of the art in evals (e.g., dynamic and agentic benchmarks, capability vs. safety evals, long-context and tool-use evaluation) and post-training (SFT, RLHF, DPO, RLAIF, rejection sampling, distillation). • Represent Innodata externally — at customer reviews, conferences, and in technical content.

🎯 Exigences

• 7+ years of experience in applied ML, ML engineering, ML research, or technical solutions roles, with at least 2+ years focused specifically on LLM evaluation and/or post-training. • Hands-on experience fine-tuning LLMs (SFT at minimum; preference optimization methods like RLHF, DPO, or KTO strongly preferred) and designing the data pipelines that feed them. • Deep familiarity with LLM evaluation methodology: public benchmarks and their limitations, custom benchmark construction, LLM-as-judge design and its failure modes, inter-annotator agreement, and human eval workflow design. • Strong fluency in Python and the modern LLM toolchain (Hugging Face, PyTorch, vLLM, evaluation frameworks such as lm-evaluation-harness, lighteval, or equivalents). • Excellent technical communication. You can hold your own in a room with research scientists at a frontier lab and, an hour later, brief a non-technical executive on the same engagement. • A consultative mindset: you ask sharp questions, you push back when a customer’s stated request won’t actually solve their problem, and you are comfortable owning a recommendation. • Bachelor’s or advanced degree in computer science, machine learning, computational linguistics, or related field — or equivalent demonstrated experience.