Queries submitted to search engines rarely provide a complete and precise description of a user's information need. Most queries are ambiguous to some extent, having multiple interpretations. For example, the seemingly unambiguous query ``tennis lessons'' might be submitted by a user interested in attending classes in her neighborhood, seeking lessons for her child, looking for online videos lessons, or planning to start a business teaching tennis. Search engines face the challenging task of satisfying different groups of users having diverse information needs associated with a given query. One solution is to optimize ranking functions to satisfy diverse sets of information needs. Unfortunately, existing evaluation frameworks do not support such optimization. Instead, ranking functions are rewarded for satisfying the most likely intent associated with a given query. In this thesis, we propose a framework and associated evaluation metrics that are capable of optimizing ranking functions to satisfy diverse information needs. Our proposed measures explicitly reward those ranking functions capable of presenting the user with information that is novel with respect to previously viewed documents. Our measures reflects quality of a ranking function by taking into account its ability to satisfy diverse users submitting a query. Moreover, the task of identifying and establishing test frameworks to compare ranking functions on a web-scale can be tedious. One reason for this problem is the dynamic nature of the web, where documents are constantly added and updated, making it necessary for search engine developers to seek additional human assessments. Along with issues of novelty and diversity, we explore one approximate approach to compare different ranking functions by overcoming the problem of lacking complete human assessments. We demonstrate that our approach is capable of accurately sorting ranking functions based on their capability of satisfying diverse users, even in the face of incomplete human assessments.

Queries submitted to search engines rarely provide a complete and precise description of a user's information need. Most queries are ambiguous to some extent, having multiple interpretations. For example, the seemingly unambiguous query ``tennis lessons'' might be submitted by a user interested in attending classes in her neighborhood, seeking lessons for her child, looking for online videos lessons, or planning to start a business teaching tennis. Search engines face the challenging task of satisfying different groups of users having diverse information needs associated with a given query. One solution is to optimize ranking functions to satisfy diverse sets of information needs. Unfortunately, existing evaluation frameworks do not support such optimization. Instead, ranking functions are rewarded for satisfying the most likely intent associated with a given query. In this thesis, we propose a framework and associated evaluation metrics that are capable of optimizing ranking functions to satisfy diverse information needs. Our proposed measures explicitly reward those ranking functions capable of presenting the user with information that is novel with respect to previously viewed documents. Our measures reflects quality of a ranking function by taking into account its ability to satisfy diverse users submitting a query. Moreover, the task of identifying and establishing test frameworks to compare ranking functions on a web-scale can be tedious. One reason for this problem is the dynamic nature of the web, where documents are constantly added and updated, making it necessary for search engine developers to seek additional human assessments. Along with issues of novelty and diversity, we explore one approximate approach to compare different ranking functions by overcoming the problem of lacking complete human assessments. We demonstrate that our approach is capable of accurately sorting ranking functions based on their capability of satisfying diverse users, even in the face of incomplete human assessments.

The focus of this study was to evaluate different search engines on Swedish text. Information retrieval is widely used by both people and organizations, and it is important to be able to efficiently retrieve needed information at the right time. The study determined that relevance and speed are the most important factors in search engines. The evaluation measures the precision and recall which are relevance measurements, and speed of two search engines, Elastic search and MarkLogic. The evaluation has determined that there is no significant difference in the relevance of the retrieved results between the engines. The evaluation has also determined that there is a statistically significant difference in speed between the engines, with Elastic search outperforming MarkLogic. Both search engines performed very well in terms of successful searches, meaning to return a relevant document in the first 20 results. Both engines succeeded in fulfilling the information need 96% of the time. / Fokus för denna studie var att utvärdera olika sökmotorer på svensk text. Informationshämtning används i stor utsträckning av både människor och organisationer, och det är viktigt att effektivt kunna hämta nödvändig information vid rätt tidpunkt. Studien fastställde att relevans och hastighet är de viktigaste faktorerna för sökmotorer. Utvärderingen mäter precision och recall som är relevansmätvärden och responstid som hastighetmätvärde för två sökmotorer, Elasticsearch och MarkLogic. Utvärderingen har visat att det inte finns någon signifikant skillnad i relevansen av de hämtade resultaten mellan motorerna. Utvärderingen har också visat att det finns en statistiskt signifikant skillnad i hastighet mellan motorerna, där Elasticsearch överträffar MarkLogic. Båda sökmotorerna presterade väldigt bra när det gäller lyckade sökningar, vilket innebär att returnera ett relevant dokument i de första 20 resultaten. Båda motorerna lyckas uppfylla informationsbehovet 96% av tiden.