Better methods and tools are needed in the fight against child pornography. This thesis presents a method for file type categorisation of unknown data fragments, a method for reassembly of JPEG fragments, and the requirements put on an artificial JPEG header for viewing reassembled images. To enable empirical evaluation of the methods a number of tools based on the methods have been implemented. The file type categorisation method identifies JPEG fragments with a detection rate of 100% and a false positives rate of 0.1%. The method uses three algorithms, Byte Frequency Distribution (BFD), Rate of Change (RoC), and 2-grams. The algorithms are designed for different situations, depending on the requirements at hand. The reconnection method correctly reconnects 97% of a Restart (RST) marker enabled JPEG image, fragmented into 4 KiB large pieces. When dealing with fragments from several images at once, the method is able to correctly connect 70% of the fragments at the first iteration. Two parameters in a JPEG header are crucial to the quality of the image; the size of the image and the sampling factor (actually factors) of the image. The size can be found using brute force and the sampling factors only take on three different values. Hence it is possible to use an artificial JPEG header to view full of parts of an image. The only requirement is that the fragments contain RST markers. The results of the evaluations of the methods show that it is possible to find, reassemble, and view JPEG image fragments with high certainty.

High performance computing (HPC) is routinely employed in diverse domains such as life sciences, and Geology, to simulate and understand the behavior of complex phenomena. Big data driven scientific simulations are resource intensive and require both computing and I/O capabilities at scale. There is a crucial need for revisiting the HPC I/O subsystem to better optimize for and manage the increased pressure on the underlying storage systems from big data processing. Extant HPC storage systems are designed and tuned for a specific set of applications targeting a range of workload characteristics, but they lack the flexibility in adapting to the ever-changing application behaviors. The complex nature of modern HPC storage systems along with the ever-changing application behaviors present unique opportunities and engineering challenges. In this dissertation, we design and develop a framework for optimizing HPC storage systems by making them application-attuned. We select three different kinds of HPC storage systems - in-memory data analytics frameworks, parallel file systems and object storage. We first analyze the HPC application I/O behavior by studying real-world I/O traces. Next we optimize parallelism for applications running in-memory, then we design data management techniques for HPC storage systems, and finally focus on low-level I/O load balance for improving the efficiency of modern HPC storage systems. / Doctor of Philosophy / Clusters of multiple computers connected through internet are often deployed in industry and laboratories for large scale data processing or computation that cannot be handled by standalone computers. In such a cluster, resources such as CPU, memory, disks are integrated to work together. With the increase in popularity of applications that read and write a tremendous amount of data, we need a large number of disks that can interact effectively in such clusters. This forms the part of high performance computing (HPC) storage systems. Such HPC storage systems are used by a diverse set of applications coming from organizations from a vast range of domains from earth sciences, financial services, telecommunication to life sciences. Therefore, the HPC storage system should be efficient to perform well for the different read and write (I/O) requirements from all the different sets of applications. But current HPC storage systems do not cater to the varied I/O requirements. To this end, this dissertation designs and develops a framework for HPC storage systems that is application-attuned and thus provides much improved performance than other state-of-the-art HPC storage systems without such optimizations.

Överföring eller mottagande av datorfiler är en syssla som de flesta av oss ägnar ossåt regelbundet i ett eller annat sammanhang, oavsett om det sker privat eller om detär arbetsrelaterat. File Transfer Protocol (FTP) är ett protokoll för just detta ändamålsom lanserades under tiden då persondatorer hade blivit allt vanligare i hemmetoch på arbetsplatsen. Sedan dess har flera andra protokoll introducerats och i dagslägetfinns det flertalet valmöjligheter tillgängliga, alla med sina unika såväl som gemensammaegenskaper. Simple Mail Transfer Protocol (SMTP) kompletterad medMIME-standarden är ett populärt alternativ till de renodlade filöverföringsprotokollenpå grund av användarnas bekantskap och tidigare erfarenheter med e-posttjänster.Att skicka och ta emot filer via e-post innebär dock vissa begränsningar samt att vissasäkerhetsaspekter måste tas i beaktande, i synnerhet för företagsanvändare som dagin och dag ut måste hantera känslig data. I denna uppsats har tre olika filöverföringsprotokoll(SFTP, SMTP och HTTP) utvärderats ur ett användarvänlighetsperspektivsåväl som ett säkerhetsperspektiv. Resultatet indikerar att HTTP(S) tillsammans meduttökningen WebDAV kan anses vara ett lämpligt alternativ för organisationer somsamtidigt är enkelt att använda. De övriga protokollen, SMTP och SFTP, har ocksåsina användningsområden där SFTP verkar vara mer tilltalande för avanceradeanvändare och där SMTP förblir ett gångbart alternativ för privat bruk av mindreavancerade användare under vissa förutsättningar. / Transferring and receiving computer files are something most of us do on a regularbasis whether it is for private purposes or if it is work related. FTP is a protocolused for this very purpose which was launched during a time when personal computerswere growing increasingly popular in both home and workplace environments.Since then, more protocols have been introduced and today there are a number ofoptions available bringing new and common features alike to the market. The SMTPprotocol, extended with the MIME standard, is currently a widely used alternative tothe pure file transfer protocols due to the user familiarity and past experiences withthese mail services. Sending and receiving files through e-mail does however implysome limitations together with the fact that some security aspects need to be takeninto consideration, particularly for company users handling sensitive data. In thisthesis, three different file transfer protocols (SMTP, SFTP and HTTP) have all beenevaluated in account of their user friendliness as well as their security features. Theresults indicate that HTTP(S) combined with the WebDAV extension can be considereda viable alternative for organizations due to its user friendliness and security.The remaining protocols, SMTP and SFTP, also have their uses where SFTP wouldseem to be an appealing option for advanced users and where SMTP remains a passablealternative for private usage by regular users under the right conditions.

Peer-to-peer network applications has been a tremendous success among end users and has therefore received much attention in academia and industry, as have illegal public file sharing in media. However, private peer-to-peer file sharing between family, friends and co-workers have attracted little interest from the research community. Existing approaches also limit the users by not allowing for native interaction with userspace applications. In this paper we ex- -plore how private file sharing can be made safe, fast and scalable without constraining the users in this aspect. We demonstrate the concept of a private file sharing application utilizing a decentralized peer-to-peer network overlay by creating a prototype with extreme program- ming as methodology. To maximize the freedom of users the network is accessed through a virtual file-system interface. The prototype proves this to be a valid approach and we hope readers can use this paper as a platform for further developments in this area. / Fildelningsapplikationer som använder peer-to-peer teknik har varit en enorm framgång blandslutanvändare och har därmed erhållit mycket uppmärksamhet från akademi och indus- tri, liksom olaglig fildelning fått inom media. Däremot har inte privat fildelning mellan vän- ner, arbetskamrater och kollegor tilldelats samma uppmärksamhet från forskningssamfundet. Nuvarande tillämpningar begränsar användaren genom att inte tillåta naturlig interaktion med användarapplikationer. I denna uppsats utforskar vi hur privat fildelning kan göras snabb, skalbar och säker utan att begränsa användaren ur den aspekten. Vi demonstrerar ett koncept- för privat fildelning som nyttjar decentraliserad peer-to-peer arkitektur m.h.a en prototyp som tagits fram med extreme programming som metodologi. För att maximera användarnas frihet nyttjas ett virtuellt filsystem som gränssnitt. Prototypen visar att vår tillämpning fungerar i praktiken och vi hoppas att läsaren kan använda vårt arbete som en plattform för fortsatt utveckling inom detta område.

The main aim of the thesis is the reconstruction of three-dimensional surface from a~set of two-dimensional images. For the implementation of this application the programming language Java and its extension, that allows work with three-dimensional models, were chosen. First, viewing of three-dimensional models of two different file formats was necessary to allow. To create the three-dimensional models, the Marching Cubes algorithm was used. This algorithm is decribed theoretically in the text, description of the implementation and correction of deficiencies follows. Finally, the implementation of the inversion procedure of reconstruction of the three-dimensional surface, which is the extraction of two-dimensional images from the three-dimensional model, is described.

Master's thesis treats of implementation of big block file system for FreeBSD operating system. The introduction of this document is dedicated to theoretically analysis, which is oriented on history, sense and usage of file systems. Next chapter describes the architecture of UFS/FFS file system. Implementation part starts with introducing design of new file system, which was isnpired by MSDOSFS file system. Last part of this thesis presents implementation of file system with great emphasis on problems, which can happen during programming.

No description available.

„Studentų darbų įkėlimo ir kaupimo sistema“ - tai sistema kuri palengvina darbą dėstytojams su studentų darbų įkėlimu ir jų kaupimu. Dėstytojai kuria modulius ir atsiskaitymus jiems. Moduliams gali priskirti studentus, arba suteikti galimybe patiems studentams prisiregistruoti prie modulio, žinant modulio raktą. Parsisiųsti ar ištrinti atsiskaitymų failus. Importuoti, eksportuoti vartotojus. Nustatyti kokio dydžio, tipo failus galima įkelti į sistemą. Studentai gali užsiregistruoti moduliui, jei žino modulio raktą. Peržiūrėti kokie moduliai priskirti, kokie atsiskaitymai jiems sukurti. Įkelti atsiskaitymams failus. Peržiūrėti įkeltų failų sąrašą. / “Students Works Loading and Storage System“ a system that facilitate the work of students and university professors with file uploads and their accumulation. Teachers can create modules, assignments. Enroll users to modules. Set upload time limit, visibility, max file size and file type to assignments. Delete or download uploaded files. Import, export or delete users. Students may register to module. In one place see all attached modules and assignments, upload files. See the list of uploaded files.

Με αυτή την εργασία προτείνουμε και υποστηρίζουμε ένα νέο πρότυπο για τα συστήματα αρχείων νέας γενιάς. Σε αυτό το πρότυπο, η παραδοσιακή άποψη ενός αρχείου αντικαθίσταται από την έννοια της πληροφοριακής μονάδας (information unit) και η παραδοσιακή αντίληψη των ιεραρχικών συστημάτων αρχείων αντικαθίσταται από ένα συνεχώς εξελισσόμενο χώρο δυναμικά αλληλένδετων πληροφοριακών μονάδων. Ένα Οργανικό Σύστημα Αρχείων (OFS) ορίζεται ως ένα σύστημα το οποίο αναπτύσσεται φυσικά και δεν υπόκειται σε τεχνητούς κανόνες και προκαθορισμένους, στατικούς τρόπους προβολής προς τους χρήστες του. Στο επίκεντρο του OFS βρίσκονται νέες αφαιρέσεις που υποστηρίζουν ένα συνεχώς εξελισσόμενο σύνολο πληροφοριακών μονάδων, χαρακτηρισμών των χρηστών για αυτές, και σχέσεων που δημιουργούνται μεταξύ τους από την πρόσβαση των χρηστών σε αυτές. Οι αφαιρέσεις αυτές επιτρέπουν το ίδιο σύστημα και το περιεχόμενό του να είναι ορατό με διαφορετικό τρόπο από διαφορετικούς τύπους χρηστών, σύμφωνα με τις τρέχουσες πληροφοριακές τους ανάγκες. Το OFS είναι ανθρωποκεντρικό, καθώς απαιτείται ανθρώπινη συνεισφορά για το χαρακτηρισμό των πληροφοριακών μονάδων και για την ανακάλυψη και το σχολιασμό των μεταξύ τους σχέσεων. Δεδομένου αυτού, στην καρδιά του OFS βρίσκονται αλγόριθμοι για την αναζήτηση βάσει περιεχομένου στα αποθηκευμένα αρχεία. Στην εργασία αυτή εκθέτουμε τα αποτελέσματα της μέχρι τώρα έρευνάς μας, συμπεριλαμβανομένης μιας υλοποίησης σε επίπεδο πυρήνα του λειτουργικού συστήματος, των βασικών χαρακτηριστικών του OFS, καθώς και τις σχετικές μετρήσεις απόδοσης προς απόδειξη της βιωσιμότητας της προσέγγισής μας. Συζητάμε στη συνέχεια, τις προκλήσεις που παραμένουν και τον αντίκτυπο που μπορεί να έχει το OFS στις σχετικές προσπάθειες έρευνας και ανάπτυξης, επισημαίνοντας τη σχετική έρευνα από άλλους τομείς, όπως η Ανάκτηση Πληροφορίας, το Κοινωνικό Λογισμικό, οι Διεπαφές Χρηστών, και η Διαχείριση Δεδομένων. / We propose and advocate a new paradigm for the next-generation file systems. In it, the traditional view of a file is replaced by the notion of an information unit and the traditional notion of hierarchical filesystems is replaced by an ever-evolving space of dynamically inter-related information units. An Organic File System (OFS) is defined as a system, which develops naturally and which does not conform to artificial rules and predefined, static ways of being viewed by its users. At the core of OFS lie novel abstractions which support a continuously evolving set of information units, users' characterizations of them, and relationships established between them by users accessing them. The abstractional also facilitate the same system and its contents to be viewed differently by different types of users, based on the current information needs. OFS is human-centered, as human input is used to characterize information units and to discover and annotate relationships between units. Given this, at the heart of OFS lie algorithms for content-based search of stored files. We report our R&D efforts so far, including a kernel-level architecture and implementation of the basic features of OFS and relevant performance measures establishing the viability of our approach. We then discuss the large number of challenges that remain and the impact OFS can have in relevant R\&D efforts, highlighting relevant research from other fields, such as Information Retrieval, Social Software, User Interfaces, and Data Management.

[pt] Na execução de workflows científicos distribuídos, a principal forma de passar os dados entre os nós de execução do workflow é utilizando arquivos. Quando os arquivos são grandes, uma parte considerável do tempo de execução do workflow é gasto transferindo os arquivos entre o servidor de armazenamento compartilhado e os nós de execução. Este trabalho propõe uma estratégia de transferência de arquivos diretamente entre os nós de execução, antecipando as necessidades da próxima etapa do workflow e diminuindo a sobrecarga com tráfego dos arquivos para os servidores de armazenamento. O trabalho analisa cenários em que a estratégia se mostra vantajosa e em quais não. / [en] For distributed scientific workflows the main method of sharing data between the execution nodes is through files. When those files are large, a substantial portion of the workflow s execution time is spent transferring the files between the storage server and the execution nodes. This work proposes a strategy for transfering the files directly between the execution nodes, antecipating the requirements of the next step of the workflow and lowering the overhead from transfering the files to and of the storage server. This dissertation analises scenarios in which this strategy shows to be advantajous and in which it doesn t.