RESEARCHER IN STANDARDIZED METHODOLOGIES FOR BENCHMARKING RETRIEVAL-AUGMENTED GENERATION (RAG) SYSTEMS - PHD SCHOLARSHIP PROGRAMME VICENTE LOPEZ

The Applied Artificial Intelligence Unit of Eurecat focuses on developing and applying AI solutions to real-world industry and societal challenges.
It specializes in areas such as machine learning, deep learning, large language models (LLMs), computer vision, and reinforcement learning, integrating these technologies into sectors like manufacturing, energy, water management, climate change, food and space.
The unit works on projects related to predictive maintenance, optimization, intelligent decision support, digital twins, and generative AI, collaborating with businesses and public entities to drive innovation through AI-driven automation and efficiency improvements.
Eurecat in collaboration with Universidad de La Rioja, is offering a PhD position to develop standardized methodologies for benchmarking Retrieval-Augmented Generation (RAG) systems, enabling domain-agnostic evaluation and rapid adaptation across different contexts.
By defining robust evaluation metrics and adaptable benchmarking frameworks, the study seeks to facilitate reproducible and efficient assessments of RAG performance, ensuring consistency and comparability across diverse applications.
Functions and Responsibilities Literature Review Conduct an in-depth survey of existing RAG systems, benchmarking methodologies, and evaluation metrics.
Analyze gaps in current RAG benchmarking approaches and identify challenges in domain-agnostic benchmarking.
Benchmark Development and Methodology Design Develop novel methodologies to create adaptable and domain-independent RAG benchmarks.
Design standardized evaluation frameworks that allow seamless switching between domains.
Define and validate key performance metrics (e.
., retrieval effectiveness, generation accuracy, consistency, efficiency).
Experimentation Implement benchmark pipelines for different RAG architectures and datasets.
Develop automated tools for efficient benchmarking and evaluation.
Test and compare various RAG models using the proposed benchmark.
Explore automated or self-adaptive benchmarking approaches to handle domain shifts dynamically.
Collaboration & Dissemination Work with other researchers, engineers, and domain experts to validate benchmarks and assist in the development of RAG platforms.
Publish findings in ML/NLP conferences and journals.
Educational background.
MS degree in Artificial Intelligence, Natural Language Processing (NLP), Data Science or Machine Learning, Computer Engineering or Applied Mathematics, Computational Linguistics, or a related field.
Technical knowledge and skills.
Knowledge of transformer architectures (e.
., GPT, Llama, Mistral…) and fine-tuning methodologies.
Experience in benchmarking, text analysis and document summarization.
Strong programming skills in Python (Pandas, PyTorch, Hugging Face, LangChain).
Experience with NLP techniques.
Languages.
High oral and written communication skills in English Knowledge of Catalan or Spanish is desirable.
Submission of Applications.
To formalize their application, applicants must.
Complete the registration forms, making sure they provide all the information requested.
Provide the following documentation.
A photocopy of their valid NIF, NIE or passport.
A CV and covering letter explaining their reasons for applying to the program.
A photocopy of the official academic certificate (or payment receipt for it) that grants them access to the doctoral program.
The academic transcript of their master’s degree studies.
Applicants who have not yet completed their master’s degree studies must submit their most recent academic transcript.
Accreditation of any other relevant achievements.
Before March 31st, Selection Process The selection process will be led by a committee that includes Scientific Management department, a specialist from People and Organization department, and the person who will be responsible for supervising the thesis connected to the vacancy.
Once the applicants' compliance with the requirements has been verified, the Commission will evaluate the merits of each application based on the curriculum, academic transcript, previous research activity, previous experience in the field of the thesis in the industrial/company sector and their fluency in English.
The evaluation process will include selection interviews.
Curriculum.
Up to 20 points Motivation Letter.
Up to 10 points Academic Transcript.
Up to 20 points Previous research activity.
Up to 10 points Previous experience in the field of the thesis in the industrial sector or company.
Up to 10 points Fluency in English.
Up to 10 points Fluency in Spanish.
Up to 10 points Fluency in Catalan.
Up to 10 points The committee will then create a prioritized list of applications that received positive evaluations.
If no application reaches the minimum score of 60 points based on the criteria outlined in the previous section, the committee may void the call.
In this scenario, Eurecat may restart the selection process, establishing new deadlines for application submissions and the selection procedure.
Every applicant will be informed about the status of their application in the program.
By applying, the applicant agrees to these terms.
Non-compliance with these terms, incomplete documentation, or any other formal defects will result in the exclusion of the application from the selection process.
Decisions will be communicated to applicants via the email address provided during registration.