Institutional Repository of Vilnius University
Automatizuoto duomenų surinkimo iš tinklalapių prevencijos būdai /
Abstract
dc:descriptionThis paper analyzes the possibilities of prevention against website scraping. The problem area is divided into five parts. First, the concept of an internet robot is defined by emphasizing the differences between a web crawler and a web scraper. Next, the existing control tools for internet robots are evaluated according to the characteristics defined during internet robots comparison. The results of the chapter include the identified criterias and the problems to be solved in order to practically and effectively apply prevention. After the definition of criterias the concept of prevention is presented, the working principles of a web scraper are explored and the theoretical analysis of the selected prevention methods is performed. It has been found that to cover all stages of web scraping, prevention techniques need to be combined. Existing commercial prevention measures are analyzed in order to assess the practical possibilities of prevention. Finally, the details of implementing a web scraping prevention prototype by integrating one of the selected commercial prevention measures are defined. In this case, the aim is to cover all stages of web scraping and to show that prevention can be applied to each of the stages analyzed practically. The results of the work include the comparisons made and the implementation of a prototype for the web scraping prevention. In the work it is concluded that prevention can be practically applied against unethical internet robots.
Degree
thesis:*- Grantor dc:publisher
- Institutional Repository of Vilnius University
- Year dc:date
- 2022
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Slėnys, Domantas,
Rights
dc:rights- Statement dc:rights
-
- info:eu-repo/semantics/openAccess
- Language dc:language
- lit
Identifiers
dc:identifier.*- Repository record dc:identifier
- https://repository.vu.lt/VU:ELABAETD146242387&prefLang=en_US