Master's Scholar, LLM, Model Evaluation, Python, Prompt Engineering

Job not on LinkedIn

🕒 June 23

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Sistema Fibra

WebsiteLinkedInAll Job Openings

1001 - 5000 employees

Founded 1972

🤝 B2B

🤝 Non-profit

📚 Education

B2B • Non-profit • Education

Sistema Fibra is a civic organization based in Brasília that focuses on enhancing the competitiveness of industries and entrepreneurship through representation and support. It brings together various entities, including Sesi, Senai, and IEL, to promote industrial growth and workforce development in Brazil.

📋 Description

• Develop and run experiments to discover new methods for evaluating Large Language Models (LLMs) and agents, investigating metrics for quality, factuality, safety, and adherence to specific requirements. • Develop experimental pipelines for LLM-as-a-Judge, including prompt engineering, creation of rubrics, and automated evaluation protocols for scalable analysis of AI-generated responses, combining concepts from Reinforcement Learning and preference optimization. • Research and apply preference optimization and reinforcement learning techniques to continuously improve language models, analyzing impacts on quality, robustness, and alignment. • Perform quantitative analyses of experimental results, document findings, propose methodological improvements, and contribute to defining best practices for LLM evaluation and optimization.

🎯 Requirements

• Education: Master’s degree • Degree areas: Computer Science, Computer Engineering, Information Systems, Data Science, Statistics, Applied Mathematics, or related fields in Computing, Artificial Intelligence, and Machine Learning. • Proficiency in Python. • Knowledge of Machine Learning and Deep Learning fundamentals. • Familiarity with Large Language Models (LLMs) and Generative AI. • Experience with AI model development libraries (PyTorch and/or Hugging Face Transformers). • Ability to read and understand scientific papers in English. • Basic knowledge of experimentation, result analysis, and model evaluation. • Experience with LLM evaluation / LLM-as-a-Judge (desirable). • Experience with RLHF, DPO, or preference optimization (desirable). • Involvement in scientific research/publications in AI or Machine Learning (desirable).

🏖️ Benefits

• Stipend: BRL 9,000.00 • Availability: 40 hours per week; Duration: 12 months