Conventional feed forward Neural Networks have used the sum-of-squares cost function for training. A new cost function is presented here with a description length interpretation based on Rissanen's Minimum Description Length principle. It is a heuristic that has a rough interpretation as the number of data points fit by the model. Not concerned with finding optimal descriptions, the cost function prefers to form minimum descriptions in a naive way for computational convenience. The cost function is called the Naive Description Length cost function. Finding minimum description models will be shown to be closely related to the identification of clusters in the data. As a consequence the minimum of this cost function approximates the most probable mode of the data rather than the sum-of-squares cost function that approximates the mean. The new cost function is shown to provide information about the structure of the data. This is done by inspecting the dependence of the error to the amount of regularisation. This structure provides a method of selecting regularisation parameters as an alternative or supplement to Bayesian methods. The new cost function is tested on a number of multi-valued problems such as a simple inverse kinematics problem. It is also tested on a number of classification and regression problems. The mode-seeking property of this cost function is shown to improve prediction in time series problems. Description length principles are used in a similar fashion to derive a regulariser to control network complexity.

This thesis is about the study of relationships between experimental dynamical systems. The basic approach is to fit radial basis function maps between time delay embeddings of manifolds. We have shown that under certain conditions these maps are generically diffeomorphisms, and can be analysed to determine whether or not the manifolds in question are diffeomorphically related to each other. If not, a study of the distribution of errors may provide information about the lack of equivalence between the two. The method has applications wherever two or more sensors are used to measure a single system, or where a single sensor can respond on more than one time scale: their respective time series can be tested to determine whether or not they are coupled, and to what degree. One application which we have explored is the determination of a minimum embedding dimension for dynamical system reconstruction. In this special case the diffeomorphism in question is closely related to the predictor for the time series itself. Linear transformations of delay embedded manifolds can also be shown to have nonlinear inverses under the right conditions, and we have used radial basis functions to approximate these inverse maps in a variety of contexts. This method is particularly useful when the linear transformation corresponds to the delay embedding of a finite impulse response filtered time series. One application of fitting an inverse to this linear map is the detection of periodic orbits in chaotic attractors, using suitably tuned filters. This method has also been used to separate signals with known bandwidths from deterministic noise, by tuning a filter to stop the signal and then recovering the chaos with the nonlinear inverse. The method may have applications to the cancellation of noise generated by mechanical or electrical systems. In the course of this research a sophisticated piece of software has been developed. The program allows the construction of a hierarchy of delay embeddings from scalar and multi-valued time series. The embedded objects can be analysed graphically, and radial basis function maps can be fitted between them asynchronously, in parallel, on a multi-processor machine. In addition to a graphical user interface, the program can be driven by a batch mode command language, incorporating the concept of parallel and sequential instruction groups and enabling complex sequences of experiments to be performed in parallel in a resource-efficient manner.

Magíster en Ciencias de la Ingeniería, Mención Eléctrica. Ingeniero Civil Eléctrico / El modelamiento de series de tiempo es un problema transversal a diferentes áreas de ingeniería y ciencias. Este tópico, visto a través del foco de aprendizaje de máquinas o aprendizaje estadístico, se reduce a elegir un modelo de regresión que sea lo suficientemente flexible sin que sobreajuste al conjunto de entrenamiento y, por ende, permita generalizar. No obstante, la elección de modelos flexibles suele venir de la mano de poca interpretabilidad de los mismos, como por ejemplo en modelos con estructura tipo \textit{caja negra}. Los modelos más flexibles son preferidos para problemas de alta complejidad, debido a su ajuste con mayor precisión a las observaciones. Más aún, el ajuste de los modelos predictivos es una componente crìtica para la regresión en línea aplicada a problemas reales. Es por ello que se decide abordar el tema del aprendizaje en línea para series de tiempo no lineales a través de un modelo flexible, que extiende la teoría del filtrado adaptivo lineal, al caso no lineal, haciendo uso de transformación de espacio de características basadas en \textit{kernel} reproductivos. Los objetivos de la investigación realizada son (i) presentar e interpretar el estimador de filtro de \textit{kernel} adaptivo (KAF) al contexto de regresión no lineal de series de tiempo, (ii) extender, en términos de mejoras sobre el algoritmo y el ajuste de sus hiperparámetros, la aplicación estándar de KAF validada sobre series sintéticas y datos reales y (iii) acercar la interpretabilidad y aplicabilidad de los métodos KAF para usuarios, validando la mejora tanto en desempeño predictivo como en ajuste de modelos con las extensiones propuestas. Para ello, este trabajo de investigación reúne los resultados principales de dos investigaciones previas, la primera enfocada en mejorar la predicción de KAF utilizando una entrada exógena de un sistema. En ese contexto se estudió el comportamiento de descarga de batería de ion-litio para una bicicleta eléctrica que utilizaba como entrada exógena mediciones de altitud derivadas a partir de coordenadas de geolocalización. El objetivo era caracterizar la posible dependencia oculta a través del descubrimiento automático de relevancia de las variables al momento de la predicción; para lo cual se usó un \textit{kernel} Gaussiano de Determinación de Relevancia Automática (ARD). Por otro lado, la segunda investigación se centró en la validación de una metodología para la inicialización de KAF extendiendo el estimador a una variante probabilística para mejorar su desempeño y entrenamiento, proponiendo hibridar la estimación en línea adicionando un entrenamiento en \textit{batch} que permite encontrar los hiperparámetros óptimos de la extensión propuesta. Adicionalmente, este enfoque permitió proponer un regularizador novedoso para abordar dos de los problemas más desafiantes de diseño según el estado del arte para KAF: el ajuste del hiperparámetro del \textit{kernel} Gaussiano y el tamaño del diccionario usado por el estimador. La metodología fue validada tanto en datos sintéticos, específicamente para el caso del atractor caótico de Lorentz, como en datos reales, los cuales correspondieron a una serie de viento extraída a partir de mediciones de anemométro. Ambos estudios mostraron resultados prometedores, acercando el uso de KAF a usuarios neófitos, tanto por las metodologías desarrolladas que quedan como guías metodológicas aplicadas, como por la interpretabilidad proporcionada a través de toda la investigación, caracterización y desarrollo del uso de KAF. Finalmente se dejan desafíos futuros con respecto a promover más aún la automatización con respecto a la selección de hiperparámetros del modelo, lo que culminaría con un desarrollo completamente adaptivo de estos métodos, vale decir, con intervención mínima del usuario en la selección de los hiperparámetros.

Thesis advisor: Dubi Kelmer / We prove a second moment formula for incomplete Eisenstein series on the homogeneous space Γ\G with G the orientation preserving isometry group of the real (n + 1)-dimensional hyperbolic space and Γ⊂ G a non-uniform lattice. This result generalizes the classical Rogers' second moment formula for Siegel transform on the space of unimodular lattices. We give two applications of this moment formula. In Chapter 5 we prove a logarithm law for unipotent flows making cusp excursions in a non-compact finite-volume hyperbolic manifold. In Chapter 6 we study the counting problem counting the number of orbits of Γ-translates in an increasing family of generalized sectors in the light cone, and prove a power saving estimate for the error term for a generic Γ-translate with the exponent determined by the largest exceptional pole of corresponding Eisenstein series. When Γ is taken to be the lattice of integral points, we give applications to the primitive lattice points counting problem on the light cone for a generic unimodular lattice coming from SO₀(n+1,1)(ℤ\SO₀(n+1,1). / Thesis (PhD) — Boston College, 2018. / Submitted to: Boston College. Graduate School of Arts and Sciences. / Discipline: Mathematics.

University of Macau / Faculty of Science and Technology. / Department of Computer and Information Science

by Wai-sum Chan. / Thesis (M.Ph.)--Chinese University of Hong Kong, 1986 / Includes bibliographical references.

Au Tsz-yin. / Thesis (M.Phil.)--Chinese University of Hong Kong, 2000. / Includes bibliographical references (leaves 76-79). / Abstracts in English and Chinese. / Chapter 1 --- Introduction --- p.1 / Chapter 1.1 --- Forecasting Time Series --- p.1 / Chapter 1.2 --- Importance of Multiple Forecasts --- p.2 / Chapter 1.3 --- Methodology of Forecasting for Autoregressive Models --- p.3 / Chapter 1.4 --- Bootstrap Approach --- p.9 / Chapter 1.5 --- Objectives --- p.12 / Chapter 2 --- "Bootstrapping Simultaneous Prediction Intervals, Case A: p known" --- p.15 / Chapter 2.1 --- TS Procedure --- p.16 / Chapter 2.2 --- CAO Procedure --- p.18 / Chapter 2.3 --- MAS Procedure --- p.20 / Chapter 3 --- "Bootstrapping Simultaneous Prediction Intervals, Case B: p unknown" --- p.24 / Chapter 3.1 --- TS Procedure --- p.25 / Chapter 3.2 --- CAO Procedure --- p.27 / Chapter 3.3 --- MAS Procedure --- p.28 / Chapter 4 --- Simulation Study --- p.29 / Chapter 4.1 --- Design of The Experiment --- p.29 / Chapter 4.2 --- Simulation Results --- p.33 / Chapter 5 --- A Real-Data Case --- p.36 / Chapter 5.1 --- Case A --- p.37 / Chapter 5.2 --- Case B --- p.42 / Chapter 6 --- Conclusion --- p.46 / Chapter A --- Tables of Simulation Results for Case A --- p.49 / Chapter B --- Tables of Simulation Results for Case B --- p.62 / Chapter C --- References --- p.76

Wong Yiu Kei. / Thesis (M.Phil.)--Chinese University of Hong Kong, 2006. / Includes bibliographical references (leaves 68-70). / Abstracts in English and Chinese. / Chapter 1 --- Introduction --- p.1 / Chapter 1.1 --- The Importance of Multiple Forecasting and Examples --- p.2 / Chapter 1.2 --- Previous Literature on Prediction Interval and Simultaneous Prediction Interval --- p.3 / Chapter 1.3 --- Objectives --- p.7 / Chapter 2 --- The Holt-Winters forecasting procedure --- p.8 / Chapter 2.1 --- Exponential Smoothing --- p.8 / Chapter 2.2 --- Holt-Winters Forecasting Procedure --- p.10 / Chapter 2.2.1 --- Additive Holt-Winters Model --- p.14 / Chapter 2.2.2 --- Multiplicative Holt-Winters Model --- p.16 / Chapter 2.3 --- Some Practical Issues --- p.18 / Chapter 2.3.1 --- Choosing Starting Values --- p.19 / Chapter 2.3.2 --- Choosing the Smoothing Parameters --- p.20 / Chapter 3 --- Constructing Simultaneous Prediction Intervals Method --- p.23 / Chapter 3.1 --- Bonferroni Procedure --- p.24 / Chapter 3.2 --- The 'Exact' Procedure --- p.25 / Chapter 3.3 --- Summary --- p.25 / Chapter 3.4 --- Covariance of forecast errors --- p.26 / Chapter 3.4.1 --- Yar and Chatfield's approach --- p.26 / Chapter 3.4.2 --- "Koehler, Snyder and Ord Approach" --- p.28 / Chapter 3.5 --- Simulation Study --- p.31 / Chapter 4 --- An Illustrative Example --- p.37 / Chapter 5 --- Simulation --- p.56 / Chapter 5.1 --- Conclusion --- p.62 / Appendix --- p.64 / References --- p.68

This thesis designed and realized a two-degree of freedom wrist and two finger manipulator that completes the six-degree of freedom Sensitive Manipulation Platform, the arm of which was previously developed. This platform extends the previous research in the field of robotics by covering not only the end effector with deformable tactile sensors, but also the links of the arm. Having tactile sensors on the arm will improve the dynamic model of the system during contact with its environment and will allow research in contact navigation to be explored. This type of research is intended for developing algorithms for exploring dynamic environments. Unlike traditional robots that focus on collision avoidance, this platform is designed to seek out contact and use it to gather important information about its surroundings. This small desktop platform was designed to have similar proportions and properties to a small human arm. These properties include compliant joints and tactile sensitivity along the lengths of the arms. The primary applications for the completed platform will be research in contact navigation and manipulation in dynamic environments. However, there are countless potential applications for a compliant arm with increased tactile feedback, including prosthetics and domestic robotics. This thesis covers the details behind the design, analysis, and evaluation of the two degrees of the Wrist and two two-link fingers, with particular attention being given to the integration of series elastics actuators, the decoupling of the fingers from the wrist, and the incorporation of tactile sensors in both the forearm motor module and fingers.

Early time series classification is the task of predicting the class label of a time series before it is observed in its entirety. In time-sensitive domains where information is collected over time it is worth sacrificing some classification accuracy in favor of earlier predictions, ideally early enough for actions to be taken. However, since accuracy and earliness are contradictory objectives, a solution to this problem must find a task-dependent trade-off. There are two common state-of-the-art methods. The first involves an analyst selecting a timestep at which all predictions must be made. This does not capture earliness on a case-by-case basis, so if the selecting timestep is too early, all later signals are missed, and if a signal happens early, the classifier still waits to generate a prediction. The second method is the exhaustive search for signals, which encodes no timing information and is not scalable to high dimensions or long time series. We design the first early classification model called EARLIEST to tackle this multi-objective optimization problem, jointly learning (1) to decide at which time step to halt and generate predictions and (2) how to classify the time series. Each of these is learned based on the task and data features. We achieve an analyst-controlled balance between the goals of earliness and accuracy by pairing a recurrent neural network that learns to classify time series as a supervised learning task with a stochastic controller network that learns a halting-policy as a reinforcement learning task. The halting-policy dictates sequential decisions, one per timestep, of whether or not to halt the recurrent neural network and classify the time series early. This pairing of networks optimizes a global objective function that incorporates both earliness and accuracy. We validate our method via critical clinical prediction tasks in the MIMIC III database from the Beth Israel Deaconess Medical Center along with another publicly available time series classification dataset. We show that EARLIEST out-performs two state-of-the-art LSTM-based early classification methods. Additionally, we dig deeper into our model's performance using a synthetic dataset which shows that EARLIEST learns to halt when it observes signals without having explicit access to signal locations. The contributions of this work are three-fold. First, our method is the first neural network-based solution to early classification of time series, bringing the recent successes of deep learning to this problem. Second, we present the first reinforcement-learning based solution to the unsupervised nature of early classification, learning the underlying distributions of signals without access to this information through trial and error. Third, we propose the first joint-optimization of earliness and accuracy, allowing learning of complex relationships between these contradictory goals.