Institutional Repository of Vilnius University
Agentinių sistemų testavimas ir lyginimas /
Abstract
dc:descriptionThis thesis explores the evaluation of agent systems, focusing particularly on browserbased agents, as their realworld deployment is becoming increasingly relevant. To address both methodological and technical testing challenges, a dedicated evaluation platform WABW was developed. It enables systematic assessment of agent behavior in scenarios requiring accurate action execution, semantic understanding, UI state awareness, and safety compliance. Four agent systems were evaluated: BrowserUse with GPT-3.5-turbo and GPT-4o-mini, WebVoyager and NanoBrowser with GPT-4o-mini. Each agent was tested using a benchmark of 50 standardized tasks grouped into four categories: action capabilities, dynamic UI adaptation, semantic traps, and security risk management. Results showed that the BrowserUse GPT-4o-mini powered agent outperformed others in accuracy and consistency. However, none of the agents successfully identified phishing attempts or refused unethical requests. The experiments revealed both the strengths and critical weaknesses of LLM-based agents and emphasized the significant influence of language model choice. The proposed evaluation framework can serve future research and development in building safer and more robust AI agents.
Degree
thesis:*- Grantor dc:publisher
- Institutional Repository of Vilnius University
- Year dc:date
- 2025
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Čebatorius, Rokas,
Rights
dc:rights- Statement dc:rights
-
- info:eu-repo/semantics/openAccess
- Language dc:language
- lit
Identifiers
dc:identifier.*- Repository record dc:identifier
- https://repository.vu.lt/VU:ELABAETD239030056&prefLang=en_US