Master's Scholar, LLM, Model Evaluation, Python, Prompt Engineering

Job not on LinkedIn

🔥 0 minutes ago

Apply Now

Find Similar Remote Jobs

Receive Emails For Remote Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Sistema Fibra

WebsiteLinkedInAll Job Openings

1001 - 5000 employees

Founded 1972

Página oficial do Sistema Fibra

📋 Description

• Develop and run experiments to discover new methods for evaluating Large Language Models (LLMs) and agents, investigating metrics for quality, factuality, safety, and adherence to specific requirements. • Develop experimental pipelines for LLM-as-a-Judge, including prompt engineering, creation of rubrics, and automated evaluation protocols for scalable analysis of AI-generated responses, combining concepts from Reinforcement Learning and preference optimization. • Research and apply preference optimization and reinforcement learning techniques to continuously improve language models, analyzing impacts on quality, robustness, and alignment. • Perform quantitative analyses of experimental results, document findings, propose methodological improvements, and contribute to defining best practices for LLM evaluation and optimization.

🎯 Requirements

• Education: Master’s degree • Degree areas: Computer Science, Computer Engineering, Information Systems, Data Science, Statistics, Applied Mathematics, or related fields in Computing, Artificial Intelligence, and Machine Learning. • Proficiency in Python. • Knowledge of Machine Learning and Deep Learning fundamentals. • Familiarity with Large Language Models (LLMs) and Generative AI. • Experience with AI model development libraries (PyTorch and/or Hugging Face Transformers). • Ability to read and understand scientific papers in English. • Basic knowledge of experimentation, result analysis, and model evaluation. • Experience with LLM evaluation / LLM-as-a-Judge (desirable). • Experience with RLHF, DPO, or preference optimization (desirable). • Involvement in scientific research/publications in AI or Machine Learning (desirable).

🏖️ Benefits

• Stipend: BRL 9,000.00 • Availability: 40 hours per week; Duration: 12 months