Web crawlers visit internet applications, collect data, and learn about new web pages from visited pages. Web crawlers have a long and interesting history. Quick expansion of the web, and the complexity added to web applications have made the process of crawling a very challenging one. Different solutions have been proposed to reduce the time and cost of crawling. New generation of web applications, known as Rich Internet Applications (RIAs), pose major challenges to the web crawlers. RIAs shift a portion of the computation to the client side. Shifting a portion of the application to the client browser influences the web crawler in two ways: First, the one-to-one correlation between the URL and the state of the application, that exists in traditional web applications, is broken. Second, reaching a state of the application is no longer a simple operation of navigating to the target URL, but often means navigating to a seed URL and executing a chain of events from it. Due to these challenges, crawling a RIA can take a prohibitively long time. This thesis studies applying distributed computing and parallel processing principles to the field of RIA crawling to reduce the time. We propose different algorithms to concurrently crawl a RIA over several nodes. The proposed algorithms are used as a building block to construct a distributed crawler of RIAs. The different algorithms proposed represent different trade-offs between communication and computation. This thesis explores the effect of making different trade-offs and their effect on the time it takes to crawl RIAs. We study the cost of running a distributed RIA crawl with client-server architecture and compare it with a peer-to-peer architecture. We further study distribution of different crawling strategies, namely: Breath-First search, Depth-First search, Greedy algorithm, and Probabilistic algorithm. To measure the effect of different design decisions in practice, a prototype of each algorithm is implemented. The implemented prototypes are used to obtain empirical performance measurements and to refine the algorithms. The ultimate refined algorithm is used for experimentation with a wide range of applications under different circumstances. This thesis finally includes two theoretical studies of load balancing algorithms and distributed component-based crawling and sets the stage for future studies.

Web applications have come a long way both in terms of adoption to provide information and services and in terms of the technologies to develop them. With the emergence of richer and more advanced technologies such as Ajax, web applications have become more interactive, responsive and user friendly. These applications, often called Rich Internet Applications (RIAs) changed the traditional web applications in two primary ways: Dynamic manipulation of client side state and Asynchronous communication with the server. At the same time, such techniques also introduce new challenges. Among these challenges, an important one is the difficulty of automatically crawling these new applications. Crawling is not only important for indexing the contents but also critical to web application assessment such as testing for security vulnerabilities or accessibility. Traditional crawlers are no longer sufficient for these newer technologies and crawling in RIAs is either inexistent or far from perfect. There is a need for an efficient crawler for web applications developed using these new technologies. Further, as more and more enterprises use these new technologies to provide their services, the requirement for a better crawler becomes inevitable. This thesis studies the problems associated with crawling RIAs. Crawling RIAs is fundamentally more difficult than crawling traditional multi-page web applications. The thesis also presents an efficient RIA crawling strategy and compares it with existing methods.

Web applications have come a long way both in terms of adoption to provide information and services and in terms of the technologies to develop them. With the emergence of richer and more advanced technologies such as Ajax, web applications have become more interactive, responsive and user friendly. These applications, often called Rich Internet Applications (RIAs) changed the traditional web applications in two primary ways: Dynamic manipulation of client side state and Asynchronous communication with the server. At the same time, such techniques also introduce new challenges. Among these challenges, an important one is the difficulty of automatically crawling these new applications. Crawling is not only important for indexing the contents but also critical to web application assessment such as testing for security vulnerabilities or accessibility. Traditional crawlers are no longer sufficient for these newer technologies and crawling in RIAs is either inexistent or far from perfect. There is a need for an efficient crawler for web applications developed using these new technologies. Further, as more and more enterprises use these new technologies to provide their services, the requirement for a better crawler becomes inevitable. This thesis studies the problems associated with crawling RIAs. Crawling RIAs is fundamentally more difficult than crawling traditional multi-page web applications. The thesis also presents an efficient RIA crawling strategy and compares it with existing methods.

Web applications have come a long way both in terms of adoption to provide information and services and in terms of the technologies to develop them. With the emergence of richer and more advanced technologies such as Ajax, web applications have become more interactive, responsive and user friendly. These applications, often called Rich Internet Applications (RIAs) changed the traditional web applications in two primary ways: Dynamic manipulation of client side state and Asynchronous communication with the server. At the same time, such techniques also introduce new challenges. Among these challenges, an important one is the difficulty of automatically crawling these new applications. Crawling is not only important for indexing the contents but also critical to web application assessment such as testing for security vulnerabilities or accessibility. Traditional crawlers are no longer sufficient for these newer technologies and crawling in RIAs is either inexistent or far from perfect. There is a need for an efficient crawler for web applications developed using these new technologies. Further, as more and more enterprises use these new technologies to provide their services, the requirement for a better crawler becomes inevitable. This thesis studies the problems associated with crawling RIAs. Crawling RIAs is fundamentally more difficult than crawling traditional multi-page web applications. The thesis also presents an efficient RIA crawling strategy and compares it with existing methods.

The generated HTTP traffic of users' interactions with a Web application can be logged for further analysis. In this thesis, we present the ``Session Reconstruction'' problem that is the reconstruction of user interactions from recorded request/response logs of a session. The reconstruction is especially useful when the only available information about the session is its HTTP trace, as could be the case during a forensic analysis of an attack on a website. New Web technologies such as AJAX and DOM manipulation have provided more responsive and smoother Web applications, sometimes called ``Rich Internet Applications''(RIAs). Despite the benefits of RIAs, the previous session reconstruction methods for traditional Web applications are not effective anymore. Recovering information from a log in RIAs is significantly more challenging as compared with classical Web applications, because the HTTP traffic contains often only application data and no obvious clues about what the user did to trigger that traffic. This thesis studies applying different techniques for efficient reconstruction of RIA sessions. We define the problem in the context of the client/server applications, and propose a solution for it. We present different algorithms to make the session reconstruction possible in practice: learning mechanisms to guide the session reconstruction process efficiently, techniques for recovering user-inputs and handling client-side randomness, and also algorithms for detections of actions that do not generate any HTTP traffic. In addition, to further reduce the session reconstruction time, we propose a distributed architecture to concurrently reconstruct a RIA session over several nodes. To measure the effectiveness of our proposed algorithms, a prototype called D-ForenRIA is implemented. The prototype is made of a proxy and a set of browsers. Browsers are responsible for trying candidate actions on each state, and the proxy, which contains the observed HTTP trace, is responsible for responding to browsers' requests and validating attempted actions on each state. We have used this tool to measure the effectiveness of the proposed techniques during session reconstruction process. The results of our evaluation on several RIAs show that the proposed solution can efficiently reconstruct use-sessions in practice.

In the last decade, the Web 2.0 brought technological changes in the manner of interaction and communication between users and applications, and among applications as well. Rich Internet Applications (RIA) offer user interfaces with a higher level of interactivity, similar to desktop interfaces, embed multimedia contents and minimise the communication between client and server components. Nonetheless, RIAs behave as black boxes that show the information in a user-friendly manner but this information can be only visualised gradually, according to the events triggered by the users on the Web browser, which limits the access of software agents, e.g., Web searchers. In the context of the present Internet, where the value has been moved from the Web applications to the data they manage, the use of open technological solutions is a need. In this way, the Semantic Web was aimed at solving issues of semantic incompatibility among systems by means of standard techniques and technologies (from knowledge representation and sharing to trust and security), which can be the key to solving the issues detected in RIA. Although some solutions exist, they do not cover all the possible types of RIA or they are dependent on the technology chosen for the implementation of the Web application. As a first contribution, this thesis introduces the concept of Semantic Rich Internet Application (SRIA), which can be defined as a RIA that extensively uses Semantic Web technologies to provide a representation of its contents and to reuse existing knowledge sources on the Web. The solution proposed is adapted to the existing RIA types and technologies. The thesis presents the architecture proposed for this type of application, describing its software modules and components. The evaluation of the solution was performed based on a collection of case studies. The development of Web applications, especially in the context of the Semantic Web, is a process traditionally performed manually and, given the complexity of the SRIA applications in this case, it is a process which might be prone to errors. The application of model-driven engineering techniques can reduce the cost of development and maintenance (in terms of time and resources) of the proposed applications, as demonstrated their use in other types of Web applications. Moreover, they can facilitate the adoption of the solution by the community. In the light of these issues, as a second contribution, this thesis presents the Sm4RIA methodology (Semantic Models for RIA) for the development of SRIA, as an extension of the OOH4RIA methodology. The thesis describes the development process, the models (with the corresponding metamodels) and the transformations included in the methodology. The evaluation of the methodology consisted in the development of the case studies proposed. The application of this model-driven methodology can speed up the development of these Web applications and simplify the reuse of external sources of knowledge. Finally, the thesis describes the Sm4RIA extension for OIDE, i.e., an extension of the OIDE CASE tool that implements all the elements of the Sm4RIA methodology.

Os SIG estão se popularizando cada vez mais e isso tem se dado principalmente através da Internet. Os assim chamados SIG-Web no entanto, quando desenvolvidos com as tecnologias tradicionais de web, apresentam as mesmas fraquezas daquelas, a saber: sincronicidade e pobreza na interação com o usuário. As tecnologias usadas para Rich Internet Applications (RIA) são uma alternativa que resolvem esses problemas. Na presente dissertação será demonstrada a factibilidade do seu uso para o desenvolvimento de SIG-Web, oferecendo um conjunto de códigos e estratégias para desenvolvimentos futuros, a partir de um conjunto básico de operações a se realizar em um SIG-Web. Adicionalmente será proposta a UWE-R, uma extensão a uma metodologia de engenharia web existente, para modelagem de RIA e SIG-Web. / GIS are getting more popular, mainly due to the Internet. Nonetheless the so called Web-GIS, when developed using traditional web technologies, inherit the same weaknesses from those, i.e.: sinchronicity and poor user interaction. Rich Internet Applications (RIA) technologies are an alternative for those issues. In the present work the feasibility of their usage for Web-GIS will be demonstrated, providing a set of codes and strategies for future developments, taking into account a basic set of Web-GIS operations. Additionally UWE-R, an extension to a web engineering methodology will be proposed for RIA modeling and Web-GIS.

Os SIG estão se popularizando cada vez mais e isso tem se dado principalmente através da Internet. Os assim chamados SIG-Web no entanto, quando desenvolvidos com as tecnologias tradicionais de web, apresentam as mesmas fraquezas daquelas, a saber: sincronicidade e pobreza na interação com o usuário. As tecnologias usadas para Rich Internet Applications (RIA) são uma alternativa que resolvem esses problemas. Na presente dissertação será demonstrada a factibilidade do seu uso para o desenvolvimento de SIG-Web, oferecendo um conjunto de códigos e estratégias para desenvolvimentos futuros, a partir de um conjunto básico de operações a se realizar em um SIG-Web. Adicionalmente será proposta a UWE-R, uma extensão a uma metodologia de engenharia web existente, para modelagem de RIA e SIG-Web. / GIS are getting more popular, mainly due to the Internet. Nonetheless the so called Web-GIS, when developed using traditional web technologies, inherit the same weaknesses from those, i.e.: sinchronicity and poor user interaction. Rich Internet Applications (RIA) technologies are an alternative for those issues. In the present work the feasibility of their usage for Web-GIS will be demonstrated, providing a set of codes and strategies for future developments, taking into account a basic set of Web-GIS operations. Additionally UWE-R, an extension to a web engineering methodology will be proposed for RIA modeling and Web-GIS.

This research examines the relationship between the development of a portfolio of interactive digital techniques and compositions, and its impact on user experiences of time and place. It is designed to answer two research questions: (i) What are some effective methods and techniques for evoking an enhanced awareness of past time and place using interactive digital technologies (IDTs)? (ii) How can users play a role in improving the development and impact of interfaces made with IDTs? The principal creative and thematic element of the portfolio is the concept of the palimpsest, and its artistic potential to reveal visual and aural layers that lie behind the landscapes and soundscapes around us. This research thus contributes to an evolving cadre of creative interest in palimpsests, developing techniques and compositions in the context of testing, collating user experience feedback, and improving the ways in which IDTs enable an artistic exploration and realisation of hidden layers, both aural and visual, of the past of place. An iterative theory-composition-testing methodology is developed and applied to optimise techniques for enabling users to navigate multiple layers of content, as well as in finding methods that evoke an increased emotional connection with the past of place. This iterative realisation cycle comprises four stages - of content origination, pre-processing, mapping and user interaction. The user interaction stage of this cycle forms an integral element of the research methodology, involving the techniques being subjected to formalised user experience testing, both to assist with their further refinement and to assess their value in evoking an increased awareness of time and place. Online usability testing gathered 5,451 responses over three years of iterative cycles of composition development and refinement, with more detailed usability labs conducted involving eighteen participants. Usability lab response categories span efficiency, accuracy, recall and emotional response. The portfolio includes a variety of interactive techniques developed and improved during its testing and refinement. User experience feedback data plays an essential role in influencing the development and direction of the portfolio, helping refine techniques to evoke an enhanced awareness of the past of place by identifying those that worked most, and least, effectively for users. This includes an analysis of the role of synthetic and authentic content on user perception of various digital techniques and compositions. The contributions of this research include: • the composition portfolio and the associated IDT techniques originated, developed, tested and refined in its research and creation • the research methodology developed and applied, utilising iterative development of aspects of the portfolio informed by user feedback obtained both online and in usability labs • the findings from user experience testing, in particular the extent to which various visual and aural techniques help evoke a heightened sense of the past of place • an exploration of the extent to which the usability testing substantiates that user responses to the compositions have the potential to establish an evocative connection that communicates a sense close to that of Barthes' punctum (something that pierces the viewer) rather than solely that of the studium • the role of synthetic and authentic content on user perception and appreciation of the techniques and compositions • the emergence of an analytical framework with the potential for wider application to the development, analysis and design of IDT compositions

<p>Ma’at Solution är ett Österrikiskt företag beläget i Linz, som utvecklar affärssystem specialiserade på aktieportföljsoptimering och värdepappers hantering. Företaget önskar komplettera sin nuvarande produkt Ma’at Invest med en Internetversion som delvis ska fungera som substitut men också som komplement till huvudprogrammet.Projektet kallades Ma’at Online. Man vill använda sig av de fördelar som finns i och med det nuvarande Java-utvecklingsverktyget- Eclipse. Exempel på fördelar är möjligheten att förstora och förminska önskade fönster vid behov. Genom det nyutvecklade Adobe (förut Macromedia) Flex är detta nu möjligt även på Internet. Ur denna idé utvecklades ett övergripande mål:</p><p>- Skapa ett användarvänligt gränssnitt för Ma’at Online med hjälp av Adobe Flex.</p><p>Initialt analyserades arbetsflödet i det existerande programmet med hjälp av UMLnotation. På så sätt var det enklare att förstå hur programmet var uppbyggt och vilka funktioner som ansågs nödvändiga. Vidare definierades målgruppen för att förstå hur programmet ska användas och vilka krav som bör ställas. Dessa krav utarbetades ytterligare och användes sedan för att omforma arbetsflödet från den nuvarande applikationen. Därefter utvecklades en alternativ design som mötte de utsatta kraven. Slutligen testades programmet på användarna för att säkerställa användbarheten.</p><p>Ytterligare lades kraft på att skapa ett gränssnitt som tog vara på fördelarna med animation och användarmanipulation för att skapa ett bättre och enklare arbetsflöde som förhindrar frustration och misstag och som istället hjälper användaren vid behov.</p><p>Ur datateknisk synpunkt undersöktes vilka komponenter som borde användas och hur de kunde skapas i Adobe Flex med hjälp av programmeringsspråket ActionScript.</p><p>Projektet resulterade i ett användarvänligt och estetiskt gränssnitt baserad på ny revolutionerande teknik som talar till rätt målgrupp och uppfyller de för programmet uppsatta målen.</p><p>Av utvecklingsarbetet med Adobe Flex har slutsatsen dragits att tekniken fortfarande har komplikationer, men att inom ett par år kommer de funktioner som är vanligt förekommande i standardprogram användas på Internet.</p> / <p>Ma’at Solutions is an Austrian software company located in Linz. The company is specialized in financial business applications such as portfolio management systems.The company wishes to extend its existing product Ma’at Invest with an Internet version that will work as a substitute or complement to the main program. The project is named Ma’at Online. Ma’at Solutions would like to use the advantages in the present program developed in the Java development environment -Eclipse. An example of features that the company would like to transfer to the online version is that the user can enlarge or minimize windows on request. Through the newly released Adobe (former Macromedia) Flex this is possible even on the Internet. From this a general goal was established:</p><p>- Create the Ma’at Online interface with an optimal usability approach in Adobe Flex.</p><p>The workflow in the existing application was analyzed with UML-modeling notation. In this way it was easier to understand the purpose of the program and which functions were required. Furthermore the target user was defined to acquire the needs of the users. These requirements were then developed even more and were used in the remodeling of the workflow to better suit an Internet version. In addition to that, an alternative design was developed, that met the requirements stated in the prior stage. At last an evaluation was performed and a test study of the finished prototype was conducted to make sure that the program meets the requirements.</p><p>During the development process, time was spent on trying to use the advantages of animations and interaction manipulations to create a better and easier workflow that prevents the user from frustration and errors. Instead it is supposed to provide help to the user when needed. The components in the Adobe Flex were investigated and tested to see how they could be created in the scripting language Action Script and if they could fit into the application.</p><p>The result of the project was that the application is user-friendly and has an aesthetic interface based on an edge-cutting technology.</p><p>The conclusion of the development process in Adobe Flex is that the technology still has complications, however in a few years, the eatures used in standalone programs will be used on the Internet.</p>