No description available.

Neural network models such as Transformer-based BERT, mBERT and RoBERTa are achieving impressive performance (Devlin et al., 2019; Lewis et al., 2020; Liu et al., 2019; Raffel et al., 2020; Y. Sun et al., 2019), but we still know little about their inner working due to the complex technique like multi-head self-attention they implement. Attention is commonly taken as a crucial way to explain the model outputs, but there are studies argue that attention may not provide faithful and reliable explanations in recent years (Jain and Wallace, 2019; Pruthi et al., 2020; Serrano and Smith, 2019; Wiegreffe and Pinter, 2019). Bastings and Filippova (2020) then propose that saliency may give better model interpretations since it is designed to find which token contributes to the prediction, i.e. the exact goal of explanation.  In this thesis, we investigate the extent to which syntactic structure is reflected in BERT, mBERT and RoBERTa trained on English and Chinese by using a gradient-based saliency method introduced by Simonyan et al. (2014). We examine the dependencies that our models and baselines predict.  We find that our models can predict some dependencies, especially those that have shorter mean distance and more fixed position of heads and dependents, even though all our models can handle global dependencies in theory. Besides, BERT usually has higher overall accuracy on connecting dependents to their corresponding heads, followed by mBERT and RoBERTa. Yet all the three model in fact have similar results on individual relations. Moreover, models trained on English have better performances than models trained on Chinese, possibly because of the flexibility of Chinese language.

Transport phenomena in heterogeneous media play a crucial role in numerous engineering applications such as hydrocarbon recovery from shales and material processing. Understanding and predicting these phenomena is critical for the success of these applications. In this dissertation, nanoscale transport phenomena in porous media are studied through physics-based simulations, and the effective solution of forward and inverse transport phenomena problems in heterogeneous media is tackled using data-driven, deep learning approaches. For nanoscale transport in porous media, the storage and recovery of gas from ultra-tight shale formations are investigated at the single-pore scale using molecular dynamics simulations. In the single-component gas recovery, a super-diffusive scaling law was found for the gas production due to the strong gas adsorption-desorption effects. For binary gas (methane/ethane) mixtures, surface adsorption contributes greatly to the storage of both gas in nanopores, with ethane enriched compared to methane. Ethane is produced from nanopores as effectively as the lighter methane despite its slower self-diffusion than the methane, and this phenomenon is traced to the strong couplings between the transport of the two species in the nanopore. The dying of solvent-loaded nanoporous filtration cakes by a purge gas flowing through them is next studied. The novelty and challenge of this problem lie in the fact that the drainage and evaporation can occur simultaneously. Using pore-network modeling, three distinct drying stages are identified. While drainage contributes less and less as drying proceeds through the first two stages, it can still contribute considerably to the net drying rate because of the strong coupling between the drainage and evaporation processes in the filtration cake. For the solution of transport phenomena problems using deep learning, first, convolutional neural networks with various architectures are trained to predict the effective diffusivity of two-dimensional (2D) porous media with complex and realistic structures from their images. Next, the inverse problem of reconstructing the structure of 2D heterogeneous composites featuring high-conductivity, circular fillers from the composites' temperature field is studied. This problem is challenging because of the high dimensionality of the temperature and conductivity fields. A deep-learning model based on convolutional neural networks with a U-shape architecture and the encoding-decoding processes is developed. The trained model can predict the distribution of fillers with good accuracy even when coarse-grained temperature data (less than 1% of the full data) are used as an input. Incorporating the temperature measurements in regions where the deep learning model has low prediction confidence can improve the model's prediction accuracy. / Doctor of Philosophy / Multiphysics transport phenomena inside structures with non-uniform pores or properties are common in engineering applications, e.g., gas recovery from shale reservoirs and drying of porous materials. Research on these transport phenomena can help improve related applications. In this dissertation, multiphysics transport in several types of structures is studied using physics-based simulations and data-driven deep learning models. In physics-based simulations, the multicomponent and multiphase transport phenomena in porous media are solved at the pore scale. The recovery of methane and methane-ethane mixtures from nanopores is studied using simulations to track motions and interactions of methane and ethane molecules inside the nanopores. The strong gas-pore wall interactions lead to significant adsorption of gas near the pore wall and contribute greatly to the gas storage in these pores. Because of strong gas adsorption and couplings between the transport of different gas species, several interesting and practically important observations have been found during the gas recovery process. For example, lighter methane and heavier ethane are recovered at similar rates. Pore-scale modeling are applied to study the drying of nanoporous filtration cakes, during which drainage and evaporation can occur concurrently. The drying is found to proceed in three distinct stages and the drainage-evaporation coupling greatly affects the drying rate. In deep learning modeling, convolutional neural networks are trained to predict the diffusivity of two-dimensional porous media by taking the image of their structures as input. The model can predict the diffusivity of the porous media accurately with computational cost orders of magnitude lower than physics-based simulations. A deep learning model is also developed to reconstruct the structure of fillers inside a two-dimensional matrix from its temperature field. The trained model can predict the structure of fillers accurately using full-scale and coarse-grained temperature input data. The predictions of the deep learning model can be improved by adding additional true temperature data in regions where the model has low prediction confidence.

Massive proliferation of wireless devices throughout world in the past decade comes with a host of tough and demanding design problems. Noise at receivers and wireless interference are the two major issues which severely limits the received signal quality and the quantity of users that can be simultaneously served. Traditional approaches to this problems are known as Power Control (PC), SINR Balancing (SINRB) and User Selection (US) in Wireless Networks respectively. Interestingly, for a large class of wireless system models, both this problems have a generic form. Thus any approach to this generic optimization problem benefits the transceiver design of all the underlying wireless models. In this thesis, we propose an Eigen approach based on the Joint Numerical Range (JNR) of hermitian matrices for PC, SINRB and US problems for a class of wireless models. In the beginning of the thesis, we address the PC and SINRB problems. PC problems can be expressed as Homogeneous Quadratic Constrained Quadratic Optimization Problems (HQCQP) which are known to be NP-Hard in general. Leveraging their connection to JNR, we show that when the constraints are fewer, HQCQP problems admit iterative schemes which are considerably fast compared to the state of the art and have guarantees of global convergence. In the general case for any number of constraints, we show that the true solution can be bounded above and below by two convex optimization problems. Our numerical simulations suggested that the bounds are tight in almost all scenarios suggesting the achievement of true solution. Further, the SINRB problems are shown to be intimately related to PC problems, and thus share the same approach. We then proceed on to comment on the convexity of PC problems and SINRB problems in the general case of any number of constraints. We show that they are intimately related to the convexity of joint numerical range. Based on this connection, we derive results on the attainability of solution and comment on the same about the state-of-the-art technique Semi-De nite Relaxation (SDR). In the subsequent part of the thesis, we address the US problem. We show that the US problem can be formulated as a combinatorial problem of selecting a feasible subset of quadratic constraints. We propose two approaches to the US problem. The first approach is based on the JNR view point which allows us to propose a heuristic approach. The heuristic approach is then shown to be equivalent to a convex optimization problem. In the second approach, we show that the US is equivalent to another non-convex optimization problem. We then propose a convex approximation approach to the latter. Both the approaches are shown to have near optimal performance in simulations. We conclude the thesis with a discussion on applicability and extension to other class of optimization problems and some open problems which has come out of this work.

Thesis (MSc)--Stellenbosch University, 2011. / ENGLISH ABSTRACT: Active materials are a subset of soft matter that is constantly being driven out of an equilibrium state due to the energy input from internal processes such as the hydrolysis of adenosine triphosphate (ATP) to adenosine diphosphate (ADP), as found in biological systems. Firstly, we construct and study a simple model of a flexible filament with an active crosslinker/molecular motor. We treat the system on a mesoscopic scale using a Langevin equation approach, which we analyse via a functional integral approach using the Martin-Siggia-Rose formalism. We characterise the steady state behaviour of the system up to first order in the motor force and also the autocorrelation of fluctuations of the position of the active crosslink on the filament. We find that this autocorrelation function does not depend on the motor force up to first order for the case where the crosslinker is located in the middle of the contour length of the filament. Properties that characterise the elastic response of the system are studied and found to scale with the autocorrelation of fluctuations of the active crosslink position. Secondly, we give a brief overview of the current state of dynamical polymer network theory and then propose two dynamical network models based on a Cayley-tree topology. Our first model takes a renormalisation approach and derive recurrence relations for the coupling constants of the system. The second model builds on the ideas of an Edwards type network theory where Wick’s theorem is employed to enforce the constraint conditions. Both models are examined using a functional integral approach. / AFRIKAANSE OPSOMMING: Aktiewe stelsels is ’n subveld van sagte materie fisika wat handel oor sisteme wat uit ekwilibruim gedryf word deur middel van interne prossesse, soos wat gevind word in biologiese stelsels. Eerstens konstruëer en bestudeer ons ’n model vir ’n buigbare filament met ’n aktiewe kruisskakelaar of molekulêre motor. Ons formuleer die stelsel op ’n mesoskopiese skaal deur gebruik te maak van ’n Langevin vergelyking formalisme en bestudeer die stelsel deur gebruik te maak van funksionaal integraal metodes deur middel van die Martin-Siggia-Rose formalisme. Dit laat ons in staat om die tydonafhankle gedrag van die stelsel te bestudeer tot op eerste orde in die motorkrag. Ons is ook in staat om die outokorrelasie fluktuasies van die posisie van die aktiewe kruisskakelaar te karakteriseer. Ons vind dat die outokorrelasie onafhanklink is van die motorkrag tot eerste orde in die geval waar die kruisskakelaar in die middel van die filament geleë is. Die elastiese eienksappe van die sisteem word ook ondersoek en gevind dat die skaleer soos die outokorrelasie van die fluktuasies van die aktiewe kruisskakelaar posisie. Tweedens gee ons ’n vlugtige oorsig van die huidige toestand van dinamiese polimeer netwerk teorie en stel dan ons eie twee modelle voor wat gebasseer is op ’n Caylee-boom topologie. Ons eerste model maak gebruik van ’n hernormering beginsel en dit laat ons toe om rekurrensierelasies vir die koppelingskonstates te verkry. Die tweede model bou op idees van ’n Edwards tipe netwerk teorie waar Wick se teorema ingespan word om die beperkingskondisies af te dwing. Beide modelle word met funksionaal integraal metodes bestudeer.

In general, this dissertation is concerned with modeling of mechanical behavior of protein molecules. In particular, we focus on coarse-grained models, which bridge the gap in time and length scale between the atomistic simulation and biological processes. The dissertation presents three independent studies involving such models. The first study is concerned with a rigorous coarse-graining method for dynamics of linear systems. In this method, as usual, the conformational space of the original atomistic system is divided into master and slave degrees of freedom. Under the assumption that the characteristic timescales of the masters are slower than those of the slaves, the method results in Langevin-type equations of motion governed by an effective potential of mean force. In addition, coarse-graining introduces hydrodynamic-like coupling among the masters as well as non-trivial inertial effects. Application of our method to the long-timescale part of the relaxation spectra of proteins shows that such dynamic coupling is essential for reproducing their relaxation rates and modes. The second study is concerned with calibration of elastic network models based on the so-called B-factors, obtained from x-ray crystallographic measurements. We show that a proper calibration procedure must account for rigid-body motion and constraints imposed by the crystalline environment on the protein. These fundamental aspects of protein dynamics in crystals are often ignored in currently used elastic network models, leading to potentially erroneous network parameters. We develop an elastic network model that properly takes rigid-body motion and crystalline constraints into account. This model reveals that B-factors are dominated by rigid-body motion rather than deformation, and therefore B-factors are poorly suited for identifying elastic properties of protein molecules. Furthermore, it turns out that B-factors for a benchmark set of three hundred and thirty protein molecules can be well approximated by assuming that the protein molecules are rigid. The third study is concerned with the polymer mediated interaction between two planar surfaces. In particular, we consider the case where a thin polymer layer bridges two parallel plates. We consider two models of monodisperse and polydisperse for the polymer layer and obtain an analytical expression for the force-distance relationship of the two plates. / text

Les instabilités thermoacoustiques, également appelées instabilités de combustion, sont un problème majeur pour la production d’électricité ainsi que dans l’industrie aérospatiale. Ces instabilités sont dues à un transfert d’énergie entre une source chaude, le plus souvent une flamme stabilisée dans un brûleur, et le champ acoustique environnant. Les instabilités de combustion peuvent avoir de nombreuses conséquences délétères telles que l’extinction de la flamme, l’augmentation des flux de chaleur pariétaux, l’émission d’ondes sonores de grande amplitude à certaines fréquences, des vibrations importantes, des dégâts structurels et même l’explosion du moteur dans certains cas. Étant donné les conséquences potentielles de tels phénomènes, d’importants moyens de recherche ont été consacrés à la prédiction de l’apparition d’instabilités de combustion dans les chaudières, les moteurs de fusée et les turbines à gaz ces dernières décennies. Néanmoins, le cadre théorique associé à l’étude de ces instabilités est complexe et nécessite l’emploi de nombreuses disciplines de la physique. De plus, les brûleurs industriels sont constitués de nombreuses cavités tridimensionnelles interagissant entre elles d’un point de vue acoustique. Pour toutes ces raisons, la prédiction de la stabilité thermoacoustique d’un brûleur demeure une tâche ardue à ce jour... (Voir le texte de la thèse pour la suite du résumé) / Thermoacoustic instabilities, also known as combustion instabilities, are a major concern in the aerospace and energy production industries. They are due to an energy transfer that occurs between a heat source, usually a flame stabilized inside a combustor, and the surrounding acoustic field and may lead to undesirable phenomena such as flame extinction, increased heat fluxes, very large sound emissions at certain frequencies, vibration, structural damage and even catastrophic failure in some cases. Given the potential consequences of such phenomena, a large research effort has been devoted to predicting the onset of combustion instabilities in modern boilers, rocket engines and gas turbines during the past few decades. Unfortunately, the theoretical framework associated with the study of thermoacoustic instabilities is complex and multi-physics and the geometry of practical combustors is an intricate arrangement of 3D cavities. As a consequence, predicting the thermoacoustic stability of a combustor at an early design stage is a challenging task to date... (See inside the manuscript for the remainder of the abstract)

Modelling is an important technique in the comprehension and management of complex systems. Queueing network models capture most relevant information from computer system and network behaviour. The construction and resolution of these models is constrained by many factors. Approximations contain detail lost for exact solution and/or provide results at lower cost than simulation. Information at the resource and interactive command level is gathered with monitors under ULTRIX'. Validation studies indicate central processor service times are highly variable on the system. More pessimistic predictions assuming this variability are in part verified by observation. The utility of the Generalised Exponential (GE) as a distribution parameterised by mean and variance is explored. Small networks of GE service centres can be solved exactly using methods proposed for Generalised Stochastic Petri Nets. For two centre. systems of GE type a new technique simplifying the balance equations is developed. A very efficient "building bglloocbka"l. is presented for exactly solving two centre systems with service or transfer blocking, Bernoulli feedback and load dependent rate, multiple GE servers. In the tandem finite buffer algorithm the building block illustrates problems encountered modelling high variability in blocking networks. A parametric validation study is made of approximations for single class closed networks of First-Come-First-Served (FCFS) centres with general service times. The multiserver extension using the building block is validated. Finally the Maximum Entropy approximation is extended to FCFS centres with multiple chains and implemented with computationally efficient convolution.

Groundwater flow and transport in deep crystalline rock is an important area of research. This is partly due to its relevance for constructing a long term repository for storing radioactive spent nuclear fuel in deep bedrock. Understanding the behavior of flow and transport processes in deep crystalline rock is crucial in developing a sustainable solution to this problem. This study aims to increase the understanding of how channel network models (CNM) can be applied to represent groundwater flow and solute transport in sparsely fractured crystalline rock under site specific conditions. A main objective was to determine how to incorporate structural and hydrogeological site characterization data in the construction of the CNMs. In addition to this, the associated key parameters of the CNMs were investigated to gain further understanding of model site application. To that end, a scripting approach with the python scripting library Pychan3d was used to create alternative channel network representations of a field site. A conceptual discrete fracture network (DFN) model was constructed using field site data obtained from a structural model of the fractures present at the site of the Tracer Retention Understanding Experiments (TRUE) - Block Scale at the Äspö Hard Rock Laboratory (HRL). This conceptual model was used as a base for constructing two different alternatives, denoted respectively as sparse and dense, of a CNM. The sparse CNM consisted of a limited amount of channels for each fracture, while the dense CNM acted as a DFN proxy, taking the full extent of the fracture areas into account and creating a dense, large network of flow channels for each fracture. In order to verify the performance of the generated CNMs, a reproduction of tracer tests performed at the same specific field site was attempted using a particle tracking technique. In addition to this, long term predictions of solute transport without the interference of the pumps used during the tracer tests were done in order to estimate transport time distributions. Pychan3d and the scripting approach was successfully used to create CNMs respecting specific conditions from the TRUE-Block Scale site. The sparse CNM was found to give very adequate flow and transport responses in most cases and to be relatively easier to calibrate than its dense counterpart. The long term transport predictions at the site according to the models seem to follow a channelized pattern, with only a few select paths for transport. The difficulties encountered in matching the dense CNM with the tracer tests most likely stem from difficulties in flow calibration, as well as certain key parameters being assigned too generically.

The investigation of the latent geometrical space behind complex network topologies is a fervid topic in current network science and the hyperbolic space is one of the most studied, because it seems associated to the structural organization of many real complex systems. The popularity-similarity-optimization (PSO) generative model is able to grow random geometric graphs in the hyperbolic space with realistic properties such as clustering, small-worldness, scale-freeness and rich-clubness. However, it misses to reproduce an important feature of real complex systems, which is the community organization. Here, we introduce the nonuniform PSO (nPSO) generative model, a generalization of the PSO model with a tailored community structure, and we provide an efficient algorithmic implementation with a O(EN) time complexity, where N is the number of nodes and E the number of edges. Meanwhile, in recent years, the inverse problem has also gained increasing attention: given a network topology, how to provide an accurate mapping into its latent geometrical space. Unlike previous attempts based on a computationally expensive maximum likelihood optimization (whose time complexity is between O(N^3) and O(N^4)), here we show that a class of methods based on nonlinear dimensionality reduction can solve the problem with higher precision and reducing the time complexity to O(N^2).