Doctoral Research Fellow – Agents, Monitoring, AI Reliability, Model Evaluation

Job not on LinkedIn

🔥 2 minutes ago

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Sistema Fibra

WebsiteLinkedInAll Job Openings

1001 - 5000 employees

Founded 1972

Página oficial do Sistema Fibra

📋 Description

• Map execution flows of agents and identify key points for log collection and monitoring. • Develop Python pipelines for collecting, storing, and processing execution logs. • Define and implement metrics to evaluate agents’ performance, quality, and consistency. • Build automated evaluation routines (evals) for different scenarios and use cases. • Produce analyses and reports on agent behavior, identifying failures and opportunities for improvement. • Support the implementation of dashboards or monitoring tools. • Document the architecture, metrics, and results obtained throughout the project.

🎯 Requirements

• PhD • Fields of study: Data Science, Statistics, Computer Engineering, Computer Science, or related areas • Programming in Python • Ability to organize and reproduce an LLM-based software project beyond casual use • Knowledge of multi-agent systems and how to orchestrate them • Extract, validate, and compare information in documents • Define and interpret evaluation metrics • Instrument the system with logs/traces/metrics • Explain failures technically • Read research articles and papers and translate ideas into experiments; write and present results clearly • Applied research experience with AI/ML/LLMs (desirable) • Able to build and evaluate end-to-end observable systems (desirable) • Able to translate an operational workflow into an agent architecture (desirable) • Clear scientific writing and strong verbal communication (desirable) • Demonstrates autonomy and experimental maturity (desirable)

🏖️ Benefits

• Stipend: BRL 11,000.00 • Availability: 40 hours per week • Duration: 12 months