Big data processing frameworks utilizing distributed frameworks to parallelize the computing of datasets have become a staple part of the data engineering and data science pipelines. One of the more known frameworks is Dask, a widely utilized distributed framework used for parallelizing data processing jobs. In Dask, the main component that traverses and plans out the execution of the job is the scheduler. Dask utilizes a centralized scheduling approach, having a single server node as the scheduler. With no failover mechanism implemented for the scheduler, the work in progress is potentially lost if the scheduler fails. As a consequence, jobs that might have been executed for hours or longer need to be restarted. In this thesis, a highly available scheduler is designed, based on Dask. We introduce a highly-available scheduler that replicates the state of the job on a distributed key-value store. The replicated schedulers allow us to design an architecture where the schedulers are able to take over the job in case of a scheduler failure. To reduce the performance overhead of replication, we further explore optimizations based on partitioning typical task graphs and sending each partition to its own scheduler. The results show that the replicated scheduler is able to tolerate server failures and is able to complete the job without restarting but at a cost of reduced throughput due to the replication. This is mitigated by our partitioning, which achieves almost linear performance gains relative to our baseline fault-tolerant scheduler, through the utilization of a parallelized scheduling architecture. / Dataprocesseringsramverk av stora datamängder har blivit en viktig del inom Data Engineering och Data Science pipelines. Ett av de mer kända ramverken är Dask som används för att parallelisera jobb inom data processering. En av huvudkomponenterna i Dask är dess schemaläggare som traverserar och planerar exekveringen av av arbete. Dask använder en centraliserad schemaläggning, med en enda server nod som schemaläggare. Utan en implementerad felhanteringsmekanism innebär det att allt arbete är förlorat ifall schemaläggaren kraschar. I denna uppsats så skapar vi en schemaläggare baserad på Dask. Vi introducerar hög tillgänglighet till schemaläggaren genom att replikera statusen av ett jobb till en distribuerad Key-Value store. För att reducera kostnaden av replikationen så utforskas optimeringar genom att partitionera typiska uppgifts-grafer för att sedan skicka dem till varsin schemaläggare. Resultaten visar att en replikerad schemaläggare tolererar att schemaläggningsservarna kraschar, och att den kan slutföra ett jobb utan att behöva starta om, på en kostnad av reducerad schemaläggningseffektivitet på grund av replikationen. Denna reduktion av effektivitet mitigeras av vår partitioningsstrategi, som genom att använda en paralliserad schemaläggningsarkitektur, uppnår nästan linjära prestandaökningar jämfört med den simpla feltoleranta schemaläggaren.

Master of Science / Department of Computing and Information Sciences / Robby / Software defects cost our economy a significant amount of money. Techniques that can detect software defects before the software begins its operational life-cycle are therefore highly valuable. Unfortunately, as software is becoming more ubiquitous, it is also becoming more complex. Static analysis of software can be computationally intensive, and as software becomes more complex the computational demands of any analysis applied increase also. While increasingly complex software entails more computationally demanding analysis, the computational capabilities provided by computers have increased exponentially over the last half century of computing. Historically, the increase in computational capability has come by increasing the clock speed of the computer's central processing unit (CPU.) In the last several years, engineering limitations have made it increasingly difficult to build CPU's with progressively higher clock speeds. Instead, processor manufacturers now provide increased capability in the form of `multi-core' CPUs; where each processor package contains two or more processing units, enabling that processor to execute more than one task concurrently. This thesis describes the design and implementation of a parallel version of symbolic execution which can take advantage of modern multi-core and multi-processor systems to complete analysis of software units in a reduced amount of time.

International Telemetering Conference Proceedings / October 18-21, 2004 / Town & Country Resort, San Diego, California / Several algorithms have been proposed for solving constraint satisfaction and the more general constraint optimization problem in a distributed manner. In this paper we apply two such algorithms to the task of dynamic resource allocation in the sensor network domain using appropriate abstractions. The aim is to effectively track multiple targets by making the sensors coordinate with each other in a distributed manner, given a probabilistic representation of tasks (targets). We present simulation results and compare the performance of the DBA and DSA algorithms under varying experimental settings.

International Telemetering Conference Proceedings / October 18-21, 2004 / Town & Country Resort, San Diego, California / The Modular Range Interface (ModRI) is a reliable networked data acquisition system used to acquire and disseminate dissimilar data. ModRI’s purpose is to connect TSPI systems to a central computer network. The modular hardware design consists of an SBC, COTS network interfaces, and other COTS interfaces in a VME form factor. The modular software design uses C++ and OO patterns running under an RTOS. Current capabilities of ModRI include acquisition of Ethernet, PCM data, RS-422/232 serial data, and IRIG-B time. Future strategies might include stand-alone data acquisition, acquisition of digital video, and migration to other architectures and operating systems.

International Telemetering Conference Proceedings / October 18-21, 2004 / Town & Country Resort, San Diego, California / This paper describes the TENA architecture, which has been proposed by the Foundation Initiative 2010 (FI 2010) project as the basis for future US Test Range software systems. The benefits of this new architecture are explained by comparing the future TENA-enabled range infrastructure with the current situation of largely non-interoperable range resources. Legacy equipment and newly acquired off-the-shelf equipment that does not directly support TENA can be integrated into a TENA environment using TENA Gateways. This paper focuses on issues related to the construction of such gateways, including the important issue of real-time requirements when dealing with real-world data acquisition instruments. The benefits of leveraging commercial off-the-shelf (COTS) Data Acquisition Systems that are based on true real-time operating systems are discussed in the context of TENA Gateway construction.

Several technologies and languages exist for the development and implementation of distributed systems. Furthermore, several models for teaching computer programming and teaching programming in a distance-based educational environment exist. Limited literature, however, is available on models for teaching distributed computing in a distance-based educational environment. The focus of this study is to examine how distributed computing should be taught in a distance-based educational environment so as to ensure effective and quality learning for students. The required effectiveness and quality should be comparable to those for students exposed to laboratories, as commonly found in residential universities. This leads to an investigation of the factors that contribute to the success of teaching distributed computing and how these factors can be integrated into a distance-based teaching model. The study consisted of a literature study, followed by a comparative study of available tools to aid in the learning and teaching of distributed computing in a distance-based educational environment. A model to accomplish this teaching and learning is then proposed and implemented. The findings of the study highlight the requirements and challenges that a student of distributed computing in a distance-based educational environment faces and emphasises how the proposed model can address these challenges. This study employed qualitative research, as opposed to quantitative research, as qualitative research methods are designed to help researchers to understand people and the social and cultural contexts within which they live. The research methods employed are design research, since an artefact is created, and a case study, since “how” and “why” questions need to be answered. Data collection was done through a survey. Each method was evaluated via its own well-established evaluation methods, since evaluation is a crucial component of the research process. / Computing / M. Sc. (Computer Science)

ITC/USA 2005 Conference Proceedings / The Forty-First Annual International Telemetering Conference and Technical Exhibition / October 24-27, 2005 / Riviera Hotel & Convention Center, Las Vegas, Nevada / The adaptation of ubiquitous Ethernet technology to airborne FTI systems is a relatively recent development, offering multiple advantages to FTI applications, including a high data throughput and ability to integrate COTS equipment with ease. For large-scale FTI applications – such as on the Airbus A380 - the use of traditional PCM based data acquisition systems results in enormously complex system architectures, with difficulties in system design, implementation, commissioning, test and maintenance. However, on the A380, the use of the Ethernet-based, IENA protocol alleviated these problems, in addition to offering several additional advantages. This paper explores the theoretical and practical implications of using Ethernet-based data acquisition in an FTI application, with direct comparison to an equivalent PCM based system.

International Telemetering Conference Proceedings / October 20-23, 2003 / Riviera Hotel and Convention Center, Las Vegas, Nevada / Space vehicle telemetry requirements have evolved considerably over the past decade. More and more onboard data need to be acquired, not only for purposes of launch vehicle qualification in flight, but also for calibrating simulations. The flexibility required of the architecture has also changed the hardware considerably in order to fit into many different configurations without redoing the whole equipment development. EADS-LV accommodates these evolving needs and costs with a modular, distributed Telemetry channel architecture. This uses a single product line to meet the various needs ranging from the small launch vehicle to the heavy Ariane 5 vehicles.

ITC/USA 2009 Conference Proceedings / The Forty-Fifth Annual International Telemetering Conference and Technical Exhibition / October 26-29, 2009 / Riviera Hotel & Convention Center, Las Vegas, Nevada / It talks about a distributed net-based flight test raw data processing system, web-oriented and application oriented. The system likes a normal one, consists of database servers, web servers and NAS storage server, but with the particular distributed task scheduler servers and the calculation servers. Each type server can be a team. The user can use WEB browser with the help of OCX control to setup his own processing task according to his need, choose which plane, which flight no., and defining the parameters, flight time segments, extracting rate etc to be processed. The system can accomplish the processing using the embedded application middleware, various data processing modules in database, with the scheduler servers and processing servers. The system can meet many users' demand of huge quantity non-structural flight raw data quickly and efficient processing at the short time, ensure the flight data enhanced management, to keep from copying and distributing the great quantity raw data inefficiently and out-of-management.

ITC/USA 2007 Conference Proceedings / The Forty-Third Annual International Telemetering Conference and Technical Exhibition / October 22-25, 2007 / Riviera Hotel & Convention Center, Las Vegas, Nevada / In support of iNET maturation, NASA-JPL has collaborated with NASA-Dryden to develop, test and demonstrate an over-the-horizon vehicle-to-ground networking capability, using Iridium as the vehicle-to-ground communications link for relaying critical vehicle telemetry. To ensure reliability concerns are met, the Space Communications Protocol Standards (SCPS) transport protocol was investigated for its performance characteristics in this environment. In particular, the SCPS-TP software performance was compared to that of the standard Transmission Control Protocol (TCP) over the Internet Protocol (IP). This paper will report on the results of this work.