With the growth of online media, surveillance and mobile cameras, the amount and size of video databases are increasing at an incredible pace. For example, YouTube reported that over 400 hours of video are uploaded every minute to their servers. Arguably, people are the most important and interesting subjects of such videos. The computer vision community has embraced this observation to validate the crucial role that human action recognition plays in building smarter surveillance systems, semantically aware video indexes and more natural human-computer interfaces. However, despite the explosion of video data, the ability to automatically recognize and understand human activities is still somewhat limited. In this work, I address four different challenges at scaling up action understanding. First, I tackle existing dataset limitations by using a flexible framework that allows continuous acquisition, crowdsourced annotation, and segmentation of online videos, thus, culminating in a large-scale, rich, and easy-to-use activity dataset, known as ActivityNet. Second, I develop an action proposal model that takes a video and directly generates temporal segments that are likely to contain human actions. The model has two appealing properties: (a) it retrieves temporal locations of activities with high recall, and (b) it produces these proposals quickly. Thirdly, I introduce a model, which exploits action-object and action-scene relationships to improve the localization quality of a fast generic action proposal method and to prune out irrelevant activities in a cascade fashion quickly. These two features lead to an efficient and accurate cascade pipeline for temporal activity localization. Lastly, I introduce a novel active learning framework for temporal localization that aims to mitigate the data dependency issue of contemporary action detectors. By creating a large-scale video benchmark, designing efficient action scanning methods, enriching approaches with high-level semantics for activity localization, and an effective strategy to build action detectors with limited data, this thesis is making a step closer towards general video understanding.

The recent surge in Type 2 Diabetes (T2D) has renewed interest in the study of cellular metabolism – which mitochondria tightly control. Previous work has shown mitochondrial dysfunction plays a critical role in the development of metabolic diseases, such as T2D. The pancreatic β-cell synthesizes and secretes insulin in vivo in response to diverse fuel signals such as glucose, fatty acids, and amino acids; failure or loss of β-cell mass is a hallmark of T2D. Pancreatic β-cell mitochondria are dynamic organelles living a life of fusion, fission, and movement collectively called mitochondrial dynamics. Mitochondrial fusion is impaired in obesity and models of obesity, while basal secretion of insulin is elevated. Previous studies demonstrate that hyperinsulinemia alone is sufficient to induce insulin resistance, yet the relationship between mitochondrial morphology and basal insulin secretion has not yet been studied. Here, we investigated the link between loss of mitochondrial fusion and insulin secretion at basal glucose concentrations by reducing the expression of mitofusin 2 (Mfn2), which controls mitochondrial morphology and metabolism. We found that forced mitochondrial fragmentation caused increased insulin secretion at basal glucose concentrations. In addition, fragmentation of mitochondria enhanced the secretory response of islets to palmitate at nonstimulatory glucose concentrations and increased fatty acid uptake and oxidation in a cell model of pancreatic β-cells. We developed unique solutions to challenges posed by the measurement of mitochondrial dynamics via confocal microscopy by using novel image analysis techniques, including a novel method of mitochondrial segmentation. This technique also revealed novel biology of brown adipose tissue mitochondria dependent on their localization within the cell. Our findings demonstrate that changes to mitochondrial dynamics in the β-cell can lead to increased insulin secretion at basal glucose concentrations. These data support the possibility that hyperinsulinemia and the downstream outcome of insulin resistance can be initiated by altered mitochondrial function in the β-cell independently of other tissues. By uncovering a new process that governs basal insulin secretion, we provide novel targets for regulation, such as mitochondrial morphology or fatty acid induced insulin secretion that may present new approaches to treatment of diabetes.

Researchers have developed many visual saliency models in order to advance the technology in computer vision. Neural networks, Convolution Neural Networks (CNNs) in particular, have successfully differentiate objects in images through feature extraction. Meanwhile, Cummings et al. has proposed a proto-object image saliency (POIS) model that shows perceptual objects or shapes can be modelled through the bottom-up saliency algorithm. Inspired from their work, this research is aimed to explore the imbedding features in the proto-object representations and utilizing artificial neural networks (ANN) to capture and predict the saliency output of POIS. A combination of CNN and a bi-directional long short-term memory (BLSTM) neural network is proposed for this saliency model as a machine learning alternative to the border ownership and grouping mechanism in POIS. As ANNs become more efficient in performing visual saliency tasks, the result of this work would extend their application in computer vision through successful implementation for proto-object based saliency.

No description available.

The advent of advanced high-throughput biological technologies provides opportunities to measure the whole genome at different molecular levels in biological systems, which produces different types of omic data such as genome, epigenome, transcriptome, translatome, proteome, metabolome and interactome. Biological systems are highly dynamic and complex mechanisms which involve not only the within-level functionality but also the between-level regulation. In order to uncover the complexity of biological systems, it is desirable to integrate multi-omic data to transform the multiple level data into biological knowledge about the underlying mechanisms. Due to the heterogeneity and high-dimension of multi-omic data, it is necessary to develop effective and efficient methods for multi-omic data integration. This thesis aims to develop efficient approaches for multi-omic data integration using machine learning methods and network theory. We assume that a biological system can be represented by a network with nodes denoting molecules and edges indicating functional links between molecules, in which multi-omic data can be integrated as attributes of nodes and edges. We propose four network-based approaches for multi-omic data integration using machine learning methods. Firstly, we propose an approach for gene module detection by integrating multi-condition transcriptome data and interactome data using network overlapping module detection method. We apply the approach to study the transcriptome data of human pre-implantation embryos across multiple development stages, and identify several stage-specific dynamic functional modules and genes which provide interesting biological insights. We evaluate the reproducibility of the modules by comparing with some other widely used methods and show that the intra-module genes are significantly overlapped between the different methods. Secondly, we propose an approach for gene module detection by integrating transcriptome, translatome, and interactome data using multilayer network. We apply the approach to study the ribosome profiling data of mTOR perturbed human prostate cancer cells and mine several translation efficiency regulated modules associated with mTOR perturbation. We develop an R package, TERM, for implementation of the proposed approach which offers a useful tool for the research field. Next, we propose an approach for feature selection by integrating transcriptome and interactome data using network-constrained regression. We develop a more efficient network-constrained regression method eGBL. We evaluate its performance in term of variable selection and prediction, and show that eGBL outperforms the other related regression methods. With application on the transcriptome data of human blastocysts, we select several interested genes associated with time-lapse parameters. Finally, we propose an approach for classification by integrating epigenome and transcriptome data using neural networks. We introduce a superlayer neural network (SNN) model which learns DNA methylation and gene expression data parallelly in superlayers but with cross-connections allowing crosstalks between them. We evaluate its performance on human breast cancer classification. The SNN provides superior performances and outperforms several other common machine learning methods. The approaches proposed in this thesis offer effective and efficient solutions for integration of heterogeneous high-dimensional datasets, which can be easily applied to other datasets presenting the similar structures. They are therefore applicable to many fields including but not limited to Bioinformatics and Computer Science.

Gallium Nitride (GaN) power devices have the potential to jump-start the next generation of power converters which are smaller, faster, denser, and cheaper. They are thus expected to meet the increasing 21st Century need for power density and efficiency, while at the same time reducing pollution. With the commercialisation of 600 V GaN power devices, which the industry is keen to adopt, come significant challenges. Since there are a number of such devices which are new to the power community, there is a steep learning curve involved, with dispersed information on how best to employ these devices. This work aims to solve this problem through the development of a universal GaN power device and circuit model and the formulation of design rules and guidelines. Through this contribution, designers will be able to better understand and work with these novel devices with relative ease. This will aid the need for faster adoption of GaN devices by the industry solving the barriers to commercialisation. This research demonstrates the use of machine learning (ML) algorithms for behavioural modelling of GaN power devices. Introducing ML as the key to developing a general behavioural and circuit model for GaN power devices combined with understanding, learning, customizing and successfully demonstrating it is the major contribution of this research work. This research first presents a comprehensive investigation into the parasitic effect on the GaN device switching performance. A simple process based on RF techniques is introduced to approximately extract the impedances of the GaN device to develop a behavioural model. The switching behaviour of the model is validated using simulation and double pulse test experiments at 450 V, 10 A test conditions. The developed behavioural model for Transhporm GaN HEMT is 95.2% accurate as the existing LT-spice manufacturer model, and is very much easier for power designers to handle, without the need for knowledge about the physics or geometry of the device. However, given that separate models would need to be developed for each commercial GaN device, the need for a generalized and accurate GaN behavioural model was identified, and it is this generalised model that the remainder of this thesis focuses on. In the next part of this research, a GaN platform test bench is built through bridging RF and power electronics design methodologies to achieve a gate loop and power loop inductance of around 1.8nH with switching waveforms with rise time and fall time around 2.5ns at 450V, 15A, 500KHz test conditions. The double pulse test circuits are customized using different off the shelf gate drives and analysed for collecting switching data for training the ML model. ML modelling using supervised learning is used to predict the switching voltage and current waveforms thus making it possible to construct a generic GaN black box model. Different architectures with single and multi- layer neural networks are explored for modelling. The ability to demonstrate a GaN device ML model that maps both voltage and current inputs and outputs is another characteristic and novel feature of this work. This research demonstrates different types of GaN ML models. The developed voltage and current prediction models are based on feed forward neural network (FFNN), long short-term memory unit (LSTM) and gated recurrent unit (GRU). Several parameters are quantified and compared for validating the models. They are the network architectures, parameters, training time, validation loss and error loss. The ML models are also compared with the demonstrated model of chapter 3 and existing LT-Spice manufacturer models. The results show that the author has been able to develop a GaN LSTM ML model with an error rate of 0.03, and convergence at 3s with excellent stability. The ML based modelling is then translated from GaN power devices to GaN based circuits. Among the different neural network architectures trained and tested, a multi FFNN with 5 hidden layers and 30 neurons, was found to be the best for prediction and optimization. The switching behaviour comparison results shows the benefits and value of ML modelling in opening up whole new possibilities of employing advanced control algorithms for very efficient, reliable and scalable performance of GaN power electronics systems. Finally, the findings of this work have been generalized to frame machine learning based techniques to address the need for generic modelling of power electronic devices. These solutions are presented as an information manual to researchers, engineers and students interested in benefiting from adopting machine learning for power electronics applications.

Technological advances have enabled profiling gene expression variability, both at the RNA and the protein level, with ever increasing throughput. In addition, miniaturisation has enabled quantifying gene expression from small volumes of the input material and most recently at the level of single cells. Increasingly these technologies also preserve context information, such as assaying tissues with high spatial resolution. A second example of contextual information is multi-omics protocols, for example to assay gene expression and DNA methylation from the same cells or samples. Although such contextual gene expression datasets are increasingly available for both popu- lation and single-cell variation studies, methods for their analysis are not established. In this thesis, we propose two modelling approaches for the analysis of gene expression variation in specific biological contexts. The first contribution of this thesis is a statistical method for analysing single cell expression data in a spatial context. Our method identifies the sources of gene expression variability by decomposing it into different components, each attributable to a different source. These sources include aspects of spatial variation such as cell-cell interactions. In applications to data across different technologies, we show that cell-cell interactions are indeed a major determinant of the expression level of specific genes with a relevant link to their function. The second contribution is a latent variable model for the unsupervised analysis of gene expression data, while accounting for structured prior knowledge on experimental context. The proposed method enables the joint analysis of gene expression data and other omics data profiled in the same samples, and the model can be used to account for the grouping structure of samples, e.g. samples from individuals with different clinical covariates or from distinct experimental batches. Our model constitutes a principled framework to compare the molecular identities of these distinct groups.

Tumours are highly heterogeneous collections of tissues characterised by a repertoire of heavily mutated and rapidly proliferating cells. Evading immune destruction is a fundamental hallmark of cancer, and elucidating the contextual basis of tumour-infiltrating leukocytes is pivotal for improving immunotherapy initiatives. However, progress in this domain is hindered by an incomplete characterisation of the regulatory mechanisms involved in cancer immunity. Addressing this challenge, this thesis is formulated around a fundamental line of inquiry: how do we quantitatively describe the immune system with respect to tumour heterogeneity? Describing the molecular interactions between cancer cells and the immune system is a fundamental goal of cancer immunology. The first part of this thesis describes a three-stage association study to address this challenge in pancreatic ductal adenocarcinoma (PDAC). Firstly, network-based approaches are used to characterise PDAC on the basis of transcription factor regulators of an oncogenic KRAS signature. Next, gene expression tools are used to resolve the leukocyte subset mixing proportions, stromal contamination, immune checkpoint expression and immune pathway dysregulation from the data. Finally, partial correlations are used to characterise immune features in terms of KRAS master regulator activity. The results are compared across two independent cohorts for consistency. Moving beyond associations, the second part of the dissertation introduces a causal modelling approach to infer directed interactions between signaling pathway activity and immune agency. This is achieved by anchoring the analysis on somatic genomic changes. In particular, copy number profiles, transcriptomic data, image data and a protein-protein interaction network are integrated using graphical modelling approaches to infer directed relationships. Generated models are compared between independent cohorts and orthogonal datasets to evaluate consistency. Finally, proposed mechanisms are cross-referenced against literature examples to test for legitimacy. In summary, this dissertation provides methodological contributions, at the levels of associative and causal inference, for inferring the contextual basis for tumour-specific immune agency.

Key to achieving more effective machine intelligence is the capability to generalise knowledge across different contexts. In this thesis, we develop a new and very general perspective on knowledge sharing that unifi es and generalises many existing methodologies, while being practically effective, simple to implement, and opening up new problem settings. Knowledge sharing across tasks and domains has conventionally been studied disparately. We fi rst introduce the concept of a semantic descriptor and a flexible neural network approach to knowledge sharing that together unify multi-task/multi-domain learning, and encompass various classic and recent multi-domain learning (MDL) and multi-task learning (MTL) algorithms as special cases. We next generalise this framework from single-output to multi-output problems and from shallow to deep models. To achieve this, we establish the equivalence between classic tensor decomposition methods, and specifi c neural network architectures. This makes it possible to implement our framework within modern deep learning stacks. We present both explicit low-rank, and trace norm regularisation solutions. From a practical perspective, we also explore a new problem setting of zero-shot domain adaptation (ZSDA) where a model can be calibrated solely based on some abstract information of a new domain, e.g., some metadata like the capture device of photos, without collecting or labelling the data.

In this thesis we are designing machine learning methodologies for solving the problem of video annotation and retrieval using either pre-defined semantic concepts or ad-hoc queries. Concept-based video annotation refers to the annotation of video fragments with one or more semantic concepts (e.g. hand, sky, running), chosen from a predefined concept list. Ad-hoc queries refer to textual descriptions that may contain objects, activities, locations etc., and combinations of the former. Our contributions are: i) A thorough analysis on extending and using different local descriptors towards improved concept-based video annotation and a stacking architecture that uses in the first layer, concept classifiers trained on local descriptors and improves their prediction accuracy by implicitly capturing concept relations, in the last layer of the stack. ii) A cascade architecture that orders and combines many classifiers, trained on different visual descriptors, for the same concept. iii) A deep learning architecture that exploits concept relations at two different levels. At the first level, we build on ideas from multi-task learning, and propose an approach to learn concept-specific representations that are sparse, linear combinations of representations of latent concepts. At a second level, we build on ideas from structured output learning, and propose the introduction, at training time, of a new cost term that explicitly models the correlations between the concepts. By doing so, we explicitly model the structure in the output space (i.e., the concept labels). iv) A fully-automatic ad-hoc video search architecture that combines concept-based video annotation and textual query analysis, and transforms concept-based keyframe and query representations into a common semantic embedding space. Our architectures have been extensively evaluated on the TRECVID SIN 2013, the TRECVID AVS 2016, and other large-scale datasets presenting their effectiveness compared to other similar approaches.