Back to search

Institutional Repository of Vilnius University

Automatizuoto duomenų surinkimo iš tinklalapių prevencijos būdai /

Abstract

dc:description

This paper analyzes the possibilities of prevention against website scraping. The problem area is divided into five parts. First, the concept of an internet robot is defined by emphasizing the differences between a web crawler and a web scraper. Next, the existing control tools for internet robots are evaluated according to the characteristics defined during internet robots comparison. The results of the chapter include the identified criterias and the problems to be solved in order to practically and effectively apply prevention. After the definition of criterias the concept of prevention is presented, the working principles of a web scraper are explored and the theoretical analysis of the selected prevention methods is performed. It has been found that to cover all stages of web scraping, prevention techniques need to be combined. Existing commercial prevention measures are analyzed in order to assess the practical possibilities of prevention. Finally, the details of implementing a web scraping prevention prototype by integrating one of the selected commercial prevention measures are defined. In this case, the aim is to cover all stages of web scraping and to show that prevention can be applied to each of the stages analyzed practically. The results of the work include the comparisons made and the implementation of a prototype for the web scraping prevention. In the work it is concluded that prevention can be practically applied against unethical internet robots.

Degree

thesis:*
Grantor dc:publisher
Institutional Repository of Vilnius University
Year dc:date
2022

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Slėnys, Domantas,

Rights

dc:rights
Statement dc:rights
  • info:eu-repo/semantics/openAccess
Language dc:language
lit

Identifiers

dc:identifier.*

Chain of custody

source
Harvested from
Vilnius University
Base URL
epublications.vu.lt/oai
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Slėnys, Domantas,. Automatizuoto duomenų surinkimo iš tinklalapių prevencijos būdai /. Institutional Repository of Vilnius University, 2022. https://repository.vu.lt/VU:ELABAETD146242387&prefLang=en_US