The Web is a vast repository of data, and information on almost any subject can be found with the aid of search engines. Although the Web is international, the majority of research on finding of information has a focus on languages such as English and Chinese. In this thesis, we investigate information retrieval techniques for Indonesian. Although Indonesia is the fourth most populous country in the world, little attention has been given to search of Indonesian documents. Stemming is the process of reducing morphological variants of a word to a common stem form. Previous research has shown that stemming is language-dependent. Although several stemming algorithms have been proposed for Indonesian, there is no consensus on which gives better performance. We empirically explore these algorithms, showing that even the best algorithm still has scope for improvement. We propose novel extensions to this algorithm and develop a new Indonesian stemmer, and show that these can improve stemming correctness by up to three percentage points; our approach makes less than one error in thirty-eight words. We propose a range of techniques to enhance the performance of Indonesian information retrieval. These techniques include: stopping; sub-word tokenisation; and identification of proper nouns; and modifications to existing similarity functions. Our experiments show that many of these techniques can increase retrieval performance, with the highest increase achieved when we use grams of size five to tokenise words. We also present an effective method for identifying the language of a document; this allows various information retrieval techniques to be applied selectively depending on the language of target documents. We also address the problem of automatic creation of parallel corpora --- collections of documents that are the direct translations of each other --- which are essential for cross-lingual information retrieval tasks. Well-curated parallel corpora are rare, and for many languages, such as Indonesian, do not exist at all. We describe algorithms that we have developed to automatically identify parallel documents for Indonesian and English. Unlike most current approaches, which consider only the context and structure of the documents, our approach is based on the document content itself. Our algorithms do not make any prior assumptions about the documents, and are based on the Needleman-Wunsch algorithm for global alignment of protein sequences. Our approach works well in identifying Indonesian-English parallel documents, especially when no translation is performed. It can increase the separation value, a measure to discriminate good matches of parallel documents from bad matches, by approximately ten percentage points. We also investigate the applicability of our identification algorithms for other languages that use the Latin alphabet. Our experiments show that, with minor modifications, our alignment methods are effective for English-French, English-German, and French-German corpora, especially when the documents are not translated. Our technique can increase the separation value for the European corpus by up to twenty-eight percentage points. Together, these results provide a substantial advance in understanding techniques that can be applied for effective Indonesian text retrieval.

This thesis deals with Swedish full text retrieval and the problem of morphological variation of query terms in thedocument database. The study is an information retrieval experiment with a test collection. While no Swedish testcollection was available, such a collection was constructed. It consists of a document database containing 161,336news articles, and 52 topics with four-graded (0, 1, 2, 3) relevance assessments. The effects of indexing strategy-query term combination on retrieval effectiveness were studied. Three of five testedmethods involved indexing strategies that used conflation, in the form of normalization. Further, two of these threecombinations used indexing strategies that employed compound splitting. Normalization and compound splittingwere performed by SWETWOL, a morphological analyzer for the Swedish language. A fourth combinationattempted to group related terms by right hand truncation of query terms. A search expert performed the truncation.The four combinations were compared to each other and to a baseline combination, where no attempt was made tocounteract the problem of morphological variation of query terms in the document database. Two situations were examined in the evaluation: the binary relevance situation and the multiple degree relevancesituation. With regard to the binary relevance situation, where the three (positive) relevance degrees (1, 2, 3) weremerged into one, and where precision was used as evaluation measure, the four alternative combinationsoutperformed the baseline. The best performing combination was the combination that used truncation. Thiscombination performed better than or equal to a median precision value for 41 of the 52 topics. One reason for therelatively good performance of the truncation combination was the capacity of its queries to retrieve different partsof speech. In the multiple degree relevance situation, where the three (positive) relevance degrees were retained, retrievaleffectiveness was taken to be the accumulated gain the user receives by examining the retrieval result up to givenpositions. The evaluation measure used was nDCG (normalized cumulated gain with discount). This measurecredits retrieval methods that (1) rank highly relevant documents higher than less relevant ones, and (2) rankrelevant (of any degree) documents high. With respect to (2), nDCG involves a discount component: a discount withregard to the relevance score of a relevant (of any degree) document is performed, and this discount is greater andgreater, the higher position the document has in the ranked list of retrieved documents. In the multiple degree relevance situation, the five combinations were evaluated under four different user scenarios,where each scenario simulated a certain user type. Again, the four alternative combinations outperformed thebaseline, for each user scenario. The truncation combination had the best performance under each user scenario.This outcome agreed with the performance result in the binary relevance situation. However, there were alsodifferences between the two relevance situations. For 25 percent of the topics and with regard to one of the four userscenarios, the set of best performing combinations in the binary relevance situation was disjunct from the set of bestperforming combinations in the multiple degree relevance situation. The user scenario in question was such thatalmost all importance was placed on highly relevant documents, and the discount was sharp. The main conclusion of the thesis is that normalization and right hand truncation (performed by a search expert)enhanced retrieval effectiveness in comparison to the baseline, irrespective of which of the two relevance situationswe consider. Further, the three indexing strategy-query term combinations based on normalization were almost asgood as the combination that involves truncation. This holds for both relevance situations. / <p>QC 20150813</p>