The increasing scale and complexity of network applications and protocols motivates the need for tools to aid in the understanding of network dynamics at similarly large scales. While current network simulation tools achieve large scale modeling, they do so by ignoring much of the intra-program state that plays an important role in the overall system's behavior. This work presents The Distributed Open Network Emulator, a scalable distributed network model that incorporates application program state to achieve high fidelity modeling. The Distributed Open Network Emulator, or DONE for short, is a parallel and distributed network simulation-emulation hybrid that achieves both scalability and the capability to run existing application code with minimal modification. These goals are accomplished through the use of a protocol stack extracted from the Linux kernel, a new programming model based on C, and a scaled real-time method for distributed synchronization. One of the primary challenges in the development of DONE was in reconciling the opposing requirements of emulation and simulation. Emulated code directly executes in real-time which progresses autonomously. In contrast, simulation models are forced ahead by the execution of events, an explicitly controlled mechanism. Relativistic time is used to integrate these two paradigms into a single model while providing efficient distributed synchronization. To demonstrate that the model provides the desired traits, a series of experiments are described. They show that DONE can provide super-linear speedup on small clusters, nearly linear speedup on moderate sized clusters, and accurate results when tuned appropriately. / Master of Science

The evolution of computer communications and the Internet has led to the emergence of a large number of communication technologies with widely different capabilities and characteristics. While this multitude of technologies provides a wide array of possibilities it also creates a complex and heterogeneous environment for higher-layer communication protocols. Specific link technologies, as well as overall network heterogeneity, can hamper user-perceived performance or impede end-to-end throughput. In this thesis we examine two transport layer centered approaches to improve performance. The first approach addresses the decrease in user satisfaction that occurs when web waiting times become too long. Increased transport layer flexibility with regards to reliability, together with error-resilient image coding, is used to enable a new trade-off. The user is given the possibility to reduce waiting times, at the expense of image fidelity. An experimental examination of this new functionality is provided, with a focus on image-coding aspects. The results show that reduced waiting times can be achieved, and user studies indicate the usefulness of this new trade-off. The second approach concerns the throughput degradations that can occur as a consequence of link and transport layer interactions. An experimental evaluation of the GSM environment shows that when negative interactions do occur, they are coupled to large variability in link layer round-trip times rather than simply to poor radio conditions. Another type of interaction can occur for link layers which expose higher layers to residual bit errors. Residual bit errors create an ambiguity problem for congestion controlled transport layer protocols which cannot correctly determine the cause for a loss. This ambiguity leads to an unnecessary throughput degradation. To mitigate this degradation, loss differentiation and notification mechanisms are proposed and experimentally evaluated from both performance and fairness perspectives. The results show that considerable performance improvements can be realized. However, there are also fairness implications that need to be taken into account since the same mechanisms that improve performance may also lead to unfairness towards flows that do not employ loss differentiation.

Discrete dynamic system models that track, maintain, utilize, and evolve virtual time are referred to as virtual time systems (VTS). The realization of VTS using virtual machine (VM) technology offers several benefits including fidelity, scalability, interoperability, fault tolerance and load balancing. The usage of VTS with VMs appears in two ways: (a) VMs within VTS, and (b) VTS over VMs. The former is prevalent in high-fidelity cyber infrastructure simulations and cyber-physical system simulations, wherein VMs form a crucial component of VTS. The latter appears in the popular Cloud computing services, where VMs are offered as computing commodities and the VTS utilizes VMs as parallel execution platforms. Prior to our work presented here, the simulation community using VM within VTS (specifically, cyber infrastructure simulations) had little awareness of the existence of a fundamental virtual time-ordering problem. The correctness problem was largely unnoticed and unaddressed because of the unrecognized effects of fair-share multiplexing of VMs to realize virtual time evolution of VMs within VTS. The dissertation research reported here demonstrated the latent incorrectness of existing methods, defined key correctness benchmarks, quantitatively measured the incorrectness, proposed and implemented novel algorithms to overcome incorrectness, and optimized the solutions to execute without a performance penalty. In fact our novel, correctness-enforcing design yields better runtime performance than the traditional (incorrect) methods. Similarly, the VTS execution over VM platforms such as Cloud computing services incurs large performance degradation, which was not known until our research uncovered the fundamental mismatch between the scheduling needs of VTS execution and those of traditional parallel workloads. Consequently, we designed a novel VTS-aware hypervisor scheduler and showed significant performance gains in VTS execution over VM platforms. Prior to our work, the performance concern of VTS over VM was largely unaddressed due to the absence of an understanding of execution policy mismatch between VMs and VTS applications. VTS follows virtual-time order execution whereas the conventional VM execution follows fair-share policy. Our research quantitatively uncovered the exact cause of poor performance of VTS in VM platforms. Moreover, we proposed and implemented a novel virtual time-aware execution methodology that relieves the degradation and provides over an order of magnitude faster execution than the traditional virtual time-unaware execution.

Network emulation has been widely used to aid in the development and evaluation of real-time applications. Many of today s applications and protocols need to be tested and evaluated in large scale network environments such as the Internet, which requires emulation tools that meet the requirements of scale, accuracy, timeliness. Due to physical resource constraints in network emulators, existing emulation tools fail to meet these requirements as they are either limited to small and static networks, use simplified network models, or fail to deliver timely emulation results. If more physical resources are devoted to network emulation by utilizing high performance computing facilities, the accuracy and scalability of network emulation can be greatly improved. However, for many users, high performance computing facilities may not be readily available in a local laboratory environment, and co-locating application code with a remote high performance computing facility may be cumbersome and inconvenient. This thesis proposes a network emulation approach called ROSENET (RemOte SErver-based Network EmulaTion) that utilizes a distributed server-based architecture in which local low-fidelity emulators provide real-time QoS predictions to distributed applications, coupled with a remote large scale high-fidelity simulator that continuously updates and calibrates the local low-fidelity emulators. A library-based modeling approach based on online simulation data collection is proposed and a system identification modeling technique is presented. Experimental results examining emulation end-to-end delay and loss show that ROSENET provides a promising approach to network emulation supporting accuracy and scale while meeting real-time constraints. Challenges faced in applying ROSENET to real world applications are addressed through two case studies including applying synthetic workload on DARPA s NMS network topology for large scale network simulation and a contemporary real-time distributed VoIP application Skype.

Mobile applications should work regardless of which type of wireless interface is used, and should be able to conceal unstable connections from the user to improve user experience. Therefore, network testing is important when developing mobile applications, but it is a challenge to reproduce network conditions when using real cellular networks since the test engineer has no control over the quality of the cellular network. Existing software tools can restrict bandwidth and add latency to the connection, but these tools do not accurately emulate cellular networks. This thesis proposes a system where it is possible to shape the network traffic for connected devices to mimic the network patterns of a real cellular connection when running on a WiFi connection. The design presented in this thesis is intended for testing mobile applications under diverse 3G connection parameters, such as latency, bandwidth and other characteristics. This thesis was conducted at Spotify, a company that provides a music streaming service which is a frequent user of network data traffic. The 3G emulator was evaluated using the Spotify Android application by measuring the correlation between packet traces from a real 3G connection and the 3G emulator. This correlation was compared to the correlation between packet traces from a real 3G connection and the current network emulator at Spotify. The evaluation shows that the proposed 3G emulator outperforms the current network emulator when performing tests on the Spotify application for Android. By using this emulator, we expect the network testing to become more effective as any 3G condition can be tested with repeatable results.

The thesis examines the impact of VoIP transmission channel characteristics on speech parameters. It seeks ways of emulating properties of a selected VoIP transmission channel and creating a network emulation environment. Several scripts have been created in the Matlab development environment and used to modify and divide a continuous recording into parts identical to the original speeches, before passing through the transmission channel. Subsequently a database of speech recordings is created, as affected by selected characteristics - jitter, bandwidth, loss. Within these databases, symptoms are sought as the most evident characteristics of the transmission channel. Using correlation, symptoms are selected that are best suited for automated determination of the properties of transmission channel characteristics such as jitter, loss and bandwidth.

Quality of Service is major factor for a successful business in modern and future network services. A minimum level of services is assured indulging quality of Experience for modern real time communication introducing user satisfaction with perceived service quality. Traffic engineering can be applied to provide better services to maintain or enhance user satisfaction through reactive and preventive traffic control mechanisms. Preventive traffic control can be more effective to manage the network resources through admission control, scheduling, policing and traffic shaping mechanisms maintaining a minimum level before it get worse and affect user perception. Accuracy, dynamicity, uniformity and reproducibility are objectives of vast research in network traffic. Real time tests, simulation and network emulation are applied to test uniformity, accuracy, reproducibility and dynamicity. Network Emulation is performed over experimental network to test real time application, protocol and traffic parameters. DummyNet is a network emulator and traffic shaper which allows nondeterministic placement of packet losses, delays and bandwidth changes. KauNet shaper is a network emulator which creates traffic patterns and applies these patterns for exact deterministic placement of bit-errors, packet losses, delay changes and bandwidth changes. An evaluation of KauNet with different patterns for packet losses, delay changes and bandwidth changes on emulated environment is part of this work. The main motivation for this work is to check the possibility to delay and drop the packets of a transfer/session in the same way as it has happened before (during the observation period). This goal is achieved to some extent using KauNet but some issues with pattern repetitions are still needed to be solved to get better results. The idea of history and trace-based traffic shaping using KauNet is given to make this possibility a reality.