Doctoral Research Fellow – Agents, Monitoring, AI Reliability, Model Evaluation

Job not on LinkedIn

🕒 July 3

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Sistema Fibra

WebsiteLinkedInAll Job Openings

1001 - 5000 employees

Founded 1972

🤝 B2B

🤝 Non-profit

📚 Education

B2B • Non-profit • Education

Sistema Fibra is a civic organization based in Brasília that focuses on enhancing the competitiveness of industries and entrepreneurship through representation and support. It brings together various entities, including Sesi, Senai, and IEL, to promote industrial growth and workforce development in Brazil.

📋 Description

• Map execution flows of agents and identify key points for log collection and monitoring. • Develop Python pipelines for collecting, storing, and processing execution logs. • Define and implement metrics to evaluate agents’ performance, quality, and consistency. • Build automated evaluation routines (evals) for different scenarios and use cases. • Produce analyses and reports on agent behavior, identifying failures and opportunities for improvement. • Support the implementation of dashboards or monitoring tools. • Document the architecture, metrics, and results obtained throughout the project.

🎯 Requirements

• PhD • Fields of study: Data Science, Statistics, Computer Engineering, Computer Science, or related areas • Programming in Python • Ability to organize and reproduce an LLM-based software project beyond casual use • Knowledge of multi-agent systems and how to orchestrate them • Extract, validate, and compare information in documents • Define and interpret evaluation metrics • Instrument the system with logs/traces/metrics • Explain failures technically • Read research articles and papers and translate ideas into experiments; write and present results clearly • Applied research experience with AI/ML/LLMs (desirable) • Able to build and evaluate end-to-end observable systems (desirable) • Able to translate an operational workflow into an agent architecture (desirable) • Clear scientific writing and strong verbal communication (desirable) • Demonstrates autonomy and experimental maturity (desirable)

🏖️ Benefits

• Stipend: BRL 11,000.00 • Availability: 40 hours per week • Duration: 12 months