Following the growth of high performance computing systems (HPC) in size and complexity, and the advent of faster and more complex Exascale systems, failures became the norm rather than the exception. Hence, the protection mechanisms need to be improved. The most de facto mechanisms such as checkpoint/restart or redundancy may also fail to support the continuous operation of future HPC systems in the presence of failures. Failure prediction is a new protection approach that is beneficial for HPC systems with a short mean time between failure. The failure prediction mechanism extends the existing protection mechanisms via the dynamic adjustment of the protection level. This work provides a prototype to analyze and predict system behavior using statistical analysis to pave the path toward resilience in HPC systems. The proposed anomaly detection method is noise-tolerant by design and produces accurate results with as little as 30 minutes of historical data. Machine learning models complement the main approach and further improve the accuracy of failure predictions up to 85%. The fully automatic unsupervised behavior analysis approach, proposed in this work, is a novel solution to protect future extreme-scale systems against failures.:1 Introduction 1.1 Background and Statement of the Problem 1.2 Purpose and Significance of the Study 1.3 Jam–e Jam: A System Behavior Analyzer 2 Review of the Literature 2.1 Syslog Analysis 2.2 Users and Systems Privacy 2.3 Failure Detection and Prediction 2.3.1 Failure Correlation 2.3.2 Anomaly Detection 2.3.3 Prediction Methods 2.3.4 Prediction Accuracy and Lead Time 3 Data Collection and Preparation 3.1 Taurus HPC Cluster 3.2 Monitoring Data 3.2.1 Data Collection 3.2.2 Taurus System Log Dataset 3.3 Data Preparation 3.3.1 Users and Systems Privacy 3.3.2 Storage and Size Reduction 3.3.3 Automation and Improvements 3.3.4 Data Discretization and Noise Mitigation 3.3.5 Cleansed Taurus System Log Dataset 3.4 Marking Potential Failures 4 Failure Prediction 4.1 Null Hypothesis 4.2 Failure Correlation 4.2.1 Node Vicinities 4.2.2 Impact of Vicinities 4.3 Anomaly Detection 4.3.1 Statistical Analysis (frequency) 4.3.2 Pattern Detection (order) 4.3.3 Machine Learning 4.4 Adaptive resilience 5 Results 5.1 Taurus System Logs 5.2 System-wide Failure Patterns 5.3 Failure Correlations 5.4 Taurus Failures Statistics 5.5 Jam-e Jam Prototype 5.6 Summary and Discussion 6 Conclusion and Future Works Bibliography List of Figures List of Tables Appendix A Neural Network Models Appendix B External Tools Appendix C Structure of Failure Metadata Databse Appendix D Reproducibility Appendix E Publicly Available HPC Monitoring Datasets Appendix F Glossary Appendix G Acronyms

Ces travaux se concentrent sur la résolution de problèmes de mécanique des fluides compressibles. De nombreuses méthodes numériques ont depuis plusieurs décennies été développées pour traiter ce type de problèmes. Cependant, l'évolution et la complexité des architectures informatiques nous poussent à actualiser et repenser ces méthodes numériques afin d'utiliser efficacement les calculateurs massivement parallèles. Au moyen de modèles de performance, nous analysons une méthode numérique de référence de type Lagrange-Projection afin de comprendre son comportement sur les supercalculateurs récents et d'en optimiser l'implémentation pour ces architectures. Grâce au bilan de cet analyse, nous proposons une formulation alternative de la phase de projection ainsi qu'une nouvelle méthode numérique plus performante baptisée Lagrange-Flux. Les développements de cette méthode ont permis d'obtenir des résultats d'une précision comparable à la méthode de référence. / This works are dedicated to hydrodynamics. For decades, numerous numerical methods has been developed to deal with this type of problems. However, both the evolution and the complexity of computing make us rethink or redesign our numerical solver in order to use efficiently massively parallel computers. Using performance modeling, we perform an analysis of a reference Lagrange-Remap solver in order to deeply understand its behavior on current supercomputer and to optimize its implementation. Thanks to the conclusions of this analysis, we derive a new numerical solver which by design has a better performance. We call it the Lagrange-Flux solver. The accuracy obtained with this solver is similar to the reference one. The derivation of this method also leads to rethink the Remap step.

abstract: This dissertation introduces FARCOM (Fortran Adaptive Refiner for Cartesian Orthogonal Meshes), a new general library for adaptive mesh refinement (AMR) based on an unstructured hexahedral mesh framework. As a result of the underlying unstructured formulation, the refinement and coarsening operators of the library operate on a single-cell basis and perform in-situ replacement of old mesh elements. This approach allows for h-refinement without the memory and computational expense of calculating masked coarse grid cells, as is done in traditional patch-based AMR approaches, and enables unstructured flow solvers to have access to the automated domain generation capabilities usually only found in tree AMR formulations. The library is written to let the user determine where to refine and coarsen through custom refinement selector functions for static mesh generation and dynamic mesh refinement, and can handle smooth fields (such as level sets) or localized markers (e.g. density gradients). The library was parallelized with the use of the Zoltan graph-partitioning library, which provides interfaces to both a graph partitioner (PT-Scotch) and a partitioner based on Hilbert space-filling curves. The partitioned adjacency graph, mesh data, and solution variable data is then packed and distributed across all MPI ranks in the simulation, which then regenerate the mesh, generate domain decomposition ghost cells, and create communication caches. Scalability runs were performed using a Leveque wave propagation scheme for solving the Euler equations. The results of simulations on up to 1536 cores indicate that the parallel performance is highly dependent on the graph partitioner being used, and differences between the partitioners were analyzed. FARCOM is found to have better performance if each MPI rank has more than 60,000 cells. / Dissertation/Thesis / Doctoral Dissertation Aerospace Engineering 2019

This thesis details the approach taken in developing PPerfGrid. Section 2 discusses other research related to this project. Section 3 provides general background on the technologies utilized in PPerfGrid, focusing on the components that make up the Grid services architecture. Section 4 provides a description of the architecture of PPerfGrid. Section 5 details the implementation of PPerfGrid. Section 6 presents tests designed to measure the overhead and scalability of the PPerfGrid application. Section 7 suggests future work, and Section 8 concludes the thesis.

No description available.

No description available.

Computational Neuroscience deals with spatio-temporal scales which vary considerably.For example interactions at synaptic contact regions occur on the scale of nanometers and nanoseconds to milliseconds (micro-scale) whereas networks of neurons can measure up to millimeters and signals are processed on the scale of seconds (macro-scale). Whole-cell calcium dynamics models (meso-scale) mediate between the multiple spatio-temporal scales. Of crucial importance is the calcium propagation mediated by the highly complex endoplasmic reticulum network. Most models do not account for the intricate intracellular architecture of neurons and consequently cannot resolve the interplay between structure and calcium-mediated function. To incorporate the detailed cellular architecture in intracellular Calcium models, a novel mesh generation methodology has been developed to allow for the efficient generation of computational meshes of neurons with a three-dimensionally resolved endoplasmic reticulum. Mesh generation routines are compiled into a versatile and fully automated reconstruct-and-simulation toolbox for multi-scale physics to be utilized on high-performance or regular computing infrastructures. First-principle numerical simulations on the neuronal reconstructions reveal that intracellular Calcium dynamics are effected by morphological features of the neurons, for instance a change of endoplasmic reticulum diameter leads to a significant spatio-temporal variability of the calcium signal at the soma. / Math & Science Education

The rapid growth of computers transformed the way in which information and data was stored. With this new paradigm of data access, comes the threat of this information being exposed to unauthorized and unintended users. Many systems have been developed which scrutinize the data for a deviation from the normal behavior of a user or system, or search for a known signature within the data. These systems are termed as Intrusion Detection Systems (IDS). These systems employ different techniques varying from statistical methods to machine learning algorithms. Intrusion detection systems use audit data generated by operating systems, application softwares or network devices. These sources produce huge amount of datasets with tens of millions of records in them. To analyze this data, data mining is used which is a process to dig useful patterns from a large bulk of information. A major obstacle in the process is that the traditional data mining and learning algorithms are overwhelmed by the bulk volume and complexity of available data. This makes these algorithms impractical for time critical tasks like intrusion detection because of the large execution time. Our approach towards this issue makes use of high performance data mining techniques to expedite the process by exploiting the parallelism in the existing data mining algorithms and the underlying hardware. We will show that how high performance and parallel computing can be used to scale the data mining algorithms to handle large datasets, allowing the data mining component to search a much larger set of patterns and models than traditional computational platforms and algorithms would allow. We develop parallel data mining algorithms by parallelizing existing machine learning techniques using cluster computing. These algorithms include parallel backpropagation and parallel fuzzy ARTMAP neural networks. We evaluate the performances of the developed models in terms of speedup over traditional algorithms, prediction rate and false alarm rate. Our results showed that the traditional backpropagation and fuzzy ARTMAP algorithms can benefit from high performance computing techniques which make them well suited for time critical tasks like intrusion detection.

Active matter represents a unique class of non-equilibrium systems, including examples ranging from cellular structures to large-scale biological tissues. These systems exhibit intriguing spatiotemporal dynamics, driven by the constituent particles’ continuous energy expenditure. Such active-matter systems, featuring complex hydrodynamics, are described by sophisticated mathematical models, typically using partial differential equations (PDEs). PDEs modeling hydrodynamics, such as the Navier-Stokes equations, are analytically intractable, and notoriously challenging to study computationally. The challenges include the need for consistent numerical methods along with their efficient and scalable high-performance computer implementation to solve the PDEs numerically. However, when considering new theoretical PDE models, such as active hydrodynamics, conventional approaches often fall short due to the specialization made in the numerical methods to study certain specific models. The inherent complexity and nonlinearity of active-matter PDEs add to the challenge. Hence, the computational study of such active-matter PDE models requires rapidly evolving high-performance computer software that can easily implement new numerical methods to solve these equations in biologically realistic three-dimensional domains. This presents a rich, yet underexplored territory demanding scalable computational frameworks that apply to a large class of PDEs. In this thesis, we introduce a computational framework that effectively allows for using multiple numerical methods through a context-aware template expression system akin to an embedded domain-specific language. This framework primarily aims at solving lengthy PDEs associated with active hydrodynamics in complex domains, while experimenting with new numerical methods. Existing PDE-solving codes often lack this flexibility, as they are closely tied to a PDE and domain geometry that rely on a specific numerical method. We overcome these limitations by using an object-oriented implementation design, and show experiments with adaptive and numerically consistent particle-based approach called Discretization-Corrected Particle Strength Exchange (DC-PSE). DC-PSE allows for the higher-order discretization of differential operators on arbitrary particle distributions leading to the possibility of solving active hydrodynamic PDEs in complex domains. However, the curse of dimensionality makes it difficult to numerically solve three-dimensional equations on single-core architectures and warrants the use of parallel and distributed computers. We design a novel template-expression system and implement it in the scalable scientific computing library OpenFPM. Our methodology offers an expression-based embedded language, enabling PDE codes to be written in a form that closely mirrors mathematical notation. Leveraging OpenFPM, this approach also ensures parallel scalability. To further enhance our framework's versatility, we employ a \textit{separation-of-concerns} abstraction, segregating the model equations from numerics, and domain geometry. This allows for the rapid rewriting of codes for agile numerical experiments across different model equations in various geometries. Supplementing this framework, we develop a distributed algebra system compatible with OpenFPM and Boost Odeint. This algebra system opens avenues for a multitude of explicit adaptive time-integration schemes, which can be selected by modifying a single line of code while maintaining parallel scalability. Motivated by symmetry-preserving theories of active hydrodynamics, and as a first benchmark of our template-expression system, we present a high-order numerically convergent scheme to study active polar fluids in arbitrary three-dimensional domains. We derive analytical solutions in simple Cartesian geometries and use them to show the numerical convergence of our algorithm. Further, we showcase the scalability of the computer code written using our expression system on distributed computing systems. To cater to the need for solving PDEs on curved surfaces, we present a novel meshfree numerical scheme, the Surface DC-PSE method. Upon implementation in our scalable framework, we benchmark Surface DC-PSE for both explicit and implicit Laplace-Beltrami operators and show applications to computing mean and Gauss curvature. Finally, we apply our computational framework to exploring the three-dimensional active hydrodynamics of biological flowing matter, a prominent model system to study the active dynamics of cytoskeletal networks, celluar migration, and tissue mechanics. Our software framework effectively tackles the challenges associated to numerically solving such non-equilibrium spatiotemporal PDEs. We perform linear perturbation analysis of the three-dimensional Ericksen-Leslie model and find an analytical expression for the critical active potential or, equivalently, a critical length of the system above which a spontaneous flow transition occurs. This spontaneous flow transition is a first realization of a three-dimensional active Fr\'eedericksz transition. With our expression system, we successfully simulate 3D active fluids, finding phases of spontaneous flow transitions, traveling waves, and spatiotemporal chaos with increasing active stress. We numerically find a topological phase transition similar to the Berezinskii–Kosterlitz–Thouless transition (BKT transition) of the two-dimensional XY model that occurs in active polar fluids after the spontaneous flow transition. We then proceed to non-Cartesian geometries and show the application of our software framework to solve the active polar fluid equations in spherical domains. We find spontaneous flows in agreement with recent experimental observations. We further showcase the framework to solve the equations in 3D annular domains and a `peanut' geometry that resembles a dividing cell. Our simulations further recapitulate the actin flows observed in \textit egg extracts within spherical shell geometries, showcasing our framework's versatility in handling complex geometrical modifications of model equations. Looking ahead, we hope our framework will serve as a foundation for further advancements in computational morphogenesis, fostering collaboration and using the present techniques in biophysical modeling.

The increased quality and quantity of sensors provide an ever-increasing capability to collect large quantities of high-quality data in the field. Research devoted to translating that data is progressing rapidly; however, translating field data into usable information can require high performance computing capabilities. While high performance computing (HPC) resources are available in centralized facilities, bandwidth, latency, security and other limitations inherent to edge location in field sensor applications may prevent HPC resources from being used in a timely fashion necessary for potential United States Army Corps of Engineers (USACE) field applications. To address these limitations, the design requirements for RECON are established and derived from a review of edge computing, in order to develop and evaluate a novel high-power, field-deployable HPC platform capable of operating in austere environments at the edge.