The MPI_Barrier-collective operation, as a part of the MPI-1.1 standard, is extremely important for all parallel applications using it. The latency of this operation increases the application run time and can not be overlaid. Thus, the whole MPI performance can be decreased by unsatisfactory barrier latency. The main goals of this work are to lower the barrier latency for InfiniBand networks by analyzing well known barrier algorithms with regards to their suitability within InfiniBand networks, to enhance the barrier operation by utilizing standard InfiniBand operations as much as possible, and to design a constant time barrier for InfiniBand with special hardware support. This partition into three main steps is retained throughout the whole thesis. The first part evaluates publicly known models and proposes a new more accurate model (LoP) for InfiniBand. All barrier algorithms are evaluated within the well known LogP and this new model. Two new algorithms which promise a better performance have been developed. A constant time barrier integrated into InfiniBand as well as a cheap separate barrier network is proposed in the hardware section. All results have been implemented inside the Open MPI framework. This work led to three new Open MPI collective modules. The first one implements different barrier algorithms which are dynamically benchmarked and selected during the startup phase to maximize the performance. The second one offers a special barrier implementation for InfiniBand with RDMA and performs up to 40% better than the best solution that has been published so far. The third implementation offers a constant time barrier in a separate network, leveraging commodity components, with a latency of only 2.5 microseconds. All components have their specialty and can be used to enhance the barrier performance significantly.

Last summer, when I began thinking about my thesis, I tried to look back on my earlier work with a more critical eye. I noticed that my focus had largely been on trying to find my own idiom, and improving my construction skills. In the middle of my education I discovered the freedom in draping, and I challenged myself by putting the pen aside and instead make three-dimensional sketches. Looking back, I believe that I succeeded in producing the organic expression I had envisioned. I had still to explore materials more deeply, though, and therefore I made a summer course with exclusive focus on materials. These were the first steps towards beginning my thesis. There were a lot of things I wanted to explore in my thesis, and many techniques I wanted to try. I wanted to believe in myself and my strong sides, but also dare to explore completely new things and go one step further than before, without fear- ing failure. Being very comfortable with construction and cutting, I can quickly try my ideas to see if they work out or not. My sketching is uncontrolled, coarse and pretty abstract, it is through sketching my ideas evolve. I am fast when I make my sketches, be it by pen or three-dimensionally, but I work much more slowly towards the end when it’s time to work on the details, where I am meticulous.

Cyber-physical systems are dynamic physical systems that are controlled by computers for their safe and sound operations (e.g., cars, satellites, robots, elevators, and many others). Consequently, the programs running cyber-physical systems have real-time requirements, which require the programs to compute not only correctly but also timely because dynamic physical systems need to move to correct positions within certain duration to ensure safe and sound operations. To satisfy real-time requirements in better ways, many real-time languages have been proposed in the literature. Nevertheless, the general-purpose non-real-time languages C and C++ have remained the de facto languages to program cyber-physical systems, including Mars rovers and F-35 jet fighters. Given this reality, the better ways to satisfy real-time requirements have been the use of model-based tools (e.g., MATLAB/Simulink) that allow cyber-physical systems to be designed by modeling and simulating them and the resulting models to be translated automatically to C programs. Model-based tools, however, leave the resulting C programs for manual integration with other C/C++ programs, such as legacy/third-party device drivers and libraries. Since manual integration could slip in some inconsistencies, which proved fatal in the maiden flight of Ariane-5 rocket, this work shows how the standard features of C++, which support active libraries, can be used to embed a model-based real-time language, called Tice, as a C++ active library that can be used to declaratively express models of real-time systems that are processable by off-the-shelf standard C++ compilers (e.g., GCC and Clang) that automatically not only translate the models into C/C++ programs but also check both the validity of the models and the consistency of the models with other C/C++ programs. Furthermore, being compilable by off-the-shelf standard C++ compilers also sets Tice apart from other real-time languages already proposed in the literature because the other languages require either their own special compilers/interpreters or non-standard C/C++ compilers. Consequently, while Tice itself either uses no C++ features that are unsuitable for cyber-physical systems (e.g., exception) or uses some in judicious manner (e.g., template instantiations to generate programs), Tice prevents no usage that is permitted by standard C++ compilers. Beside that, as C++ active libraries are indeed ordinary C++ libraries, C++ active libraries are seamlessly composable as ordinary C++ libraries, and therefore, as models play an increasingly important role in software engineering, this work shows the potential of C++ as a platform for language-oriented programming where different languages that express different kinds of models and are embedded as C++ active libraries could be composed seamlessly.

The MPI_Barrier-collective operation, as a part of the MPI-1.1 standard, is extremely important for all parallel applications using it. The latency of this operation increases the application run time and can not be overlaid. Thus, the whole MPI performance can be decreased by unsatisfactory barrier latency. The main goals of this work are to lower the barrier latency for InfiniBand networks by analyzing well known barrier algorithms with regards to their suitability within InfiniBand networks, to enhance the barrier operation by utilizing standard InfiniBand operations as much as possible, and to design a constant time barrier for InfiniBand with special hardware support. This partition into three main steps is retained throughout the whole thesis. The first part evaluates publicly known models and proposes a new more accurate model (LoP) for InfiniBand. All barrier algorithms are evaluated within the well known LogP and this new model. Two new algorithms which promise a better performance have been developed. A constant time barrier integrated into InfiniBand as well as a cheap separate barrier network is proposed in the hardware section. All results have been implemented inside the Open MPI framework. This work led to three new Open MPI collective modules. The first one implements different barrier algorithms which are dynamically benchmarked and selected during the startup phase to maximize the performance. The second one offers a special barrier implementation for InfiniBand with RDMA and performs up to 40% better than the best solution that has been published so far. The third implementation offers a constant time barrier in a separate network, leveraging commodity components, with a latency of only 2.5 microseconds. All components have their specialty and can be used to enhance the barrier performance significantly.