Technical Solutions Architect, Evals & Fine-Tuning

Vaga não está no LinkedIn

🕒 6 dias atrás

Candidatar-se

Encontrar Vagas Remotas Similares

Configurar Alerta de Email para Novas Vagas

📊 Verifique sua pontuação de currículo para esta vaga

Melhore suas chances de conseguir uma entrevista verificando sua pontuação de currículo antes de se candidatar.

Enviar Currículo

InnoData

SiteLinkedInTodas as Vagas

2 - 10 funcionários

Fundada em 2019

🤝 B2B

💼 Consultoria

🌍 Impacto Social

B2B • Consulting • Social Impact

InnoData é uma cooperativa social italiana denominada INNOvation DATA SCS, com sede em Foggia, que se identifica como fornecedora de soluções tecnológicas. O site da empresa (atualmente em manutenção) destaca "Soluzioni tecnologiche" (soluções tecnológicas) e enfatiza o impacto social ("Impatto sociale"). Os dados de contato listados incluem Via Francesco Crispi 65, 71121 Foggia, Itália. Com base nas informações disponíveis, a InnoData parece operar na interseção entre tecnologia e impacto social, provavelmente oferecendo serviços focados em tecnologia para outras organizações.

Descrição

• Lead technical discovery with prospective and existing customers — foundation model labs, frontier AI teams, and large enterprises — to understand model objectives, gaps, and constraints. • Design end-to-end solutions across the post-training stack: SFT data curation, preference data collection for RLHF/DPO, golden datasets, custom benchmarks, LLM-as-judge pipelines, human-in-the-loop evaluation, red teaming, and multimodal eval (text, image, audio, video, long-context). • Architect engagements that combine Innodata’s platforms (GenAI Test & Evaluation Platform, Annotation Platform, GenAI Workbench) with our global SME workforce across 85+ languages and domains. • Author technical proposals, SOWs, solution diagrams, and pricing models in partnership with sales, delivery, and finance. • Run technical workshops, POCs, and pilot designs that de-risk larger programs and prove value quickly. • Serve as the ongoing technical advisor during delivery, partnering with applied research scientists, AI/ML research engineers, language data scientists, and program managers to keep solutions aligned with the original intent. • Feed customer signal back into Innodata’s R&D and product roadmap — what benchmarks customers actually want, where eval methodology is breaking, what new fine-tuning paradigms are gaining traction. • Stay current on the state of the art in evals (e.g., dynamic and agentic benchmarks, capability vs. safety evals, long-context and tool-use evaluation) and post-training (SFT, RLHF, DPO, RLAIF, rejection sampling, distillation). • Represent Innodata externally — at customer reviews, conferences, and in technical content.

🎯 Requisitos

• 7+ years of experience in applied ML, ML engineering, ML research, or technical solutions roles, with at least 2+ years focused specifically on LLM evaluation and/or post-training. • Hands-on experience fine-tuning LLMs (SFT at minimum; preference optimization methods like RLHF, DPO, or KTO strongly preferred) and designing the data pipelines that feed them. • Deep familiarity with LLM evaluation methodology: public benchmarks and their limitations, custom benchmark construction, LLM-as-judge design and its failure modes, inter-annotator agreement, and human eval workflow design. • Strong fluency in Python and the modern LLM toolchain (Hugging Face, PyTorch, vLLM, evaluation frameworks such as lm-evaluation-harness, lighteval, or equivalents). • Excellent technical communication. You can hold your own in a room with research scientists at a frontier lab and, an hour later, brief a non-technical executive on the same engagement. • A consultative mindset: you ask sharp questions, you push back when a customer’s stated request won’t actually solve their problem, and you are comfortable owning a recommendation. • Bachelor’s or advanced degree in computer science, machine learning, computational linguistics, or related field — or equivalent demonstrated experience.