Research Doctorate - Doctor of Philosophy (PhD) / This thesis presents results on optimal coding and decoding of discrete-time stochastic signals, in the sense of minimizing a distortion metric subject to a constraint on the bit-rate and on the signal transfer function from source to reconstruction. The first (preliminary) contribution of this thesis is the introduction of new distortion metric that extends the mean squared error (MSE) criterion. We give this extension the name Weighted-Correlation MSE (WCMSE), and use it as the distortion metric throughout the thesis. The WCMSE is a weighted sum of two components of the MSE: the variance of the error component uncorrelated to the source, on the one hand, and the remainder of the MSE, on the other. The WCMSE can take account of signal transfer function constraints by assigning a larger weight to deviations from a target signal transfer function than to source-uncorrelated distortion. Within this framework, the second contribution is the solution of a family of feedback quantizer design problems for wide sense stationary sources using an additive noise model for quantization errors. These associated problems consist of finding the frequency response of the filters deployed around a scalar quantizer that minimize the WCMSE for a fixed quantizer signal-to-(granular)-noise ratio (SNR). This general structure, which incorporates pre-, post-, and feedback filters, includes as special cases well known source coding schemes such as pulse coded modulation (PCM), Differential Pulse-Coded Modulation (DPCM), Sigma Delta converters, and noise-shaping coders. The optimal frequency response of each of the filters in this architecture is found for each possible subset of the remaining filters being given and fixed. These results are then applied to oversampled feedback quantization. In particular, it is shown that, within the linear model used, and for a fixed quantizer SNR, the MSE decays exponentially with oversampling ratio, provided optimal filters are used at each oversampling ratio. If a subtractively dithered quantizer is utilized, then the noise model is exact, and the SNR constraint can be directly related to the bit-rate if entropy coding is used, regardless of the number of quantization levels. On the other hand, in the case of fixed-rate quantization, the SNR is related to the number of quantization levels, and hence to the bit-rate, when overload errors are negligible. It is shown that, for sources with unbounded support, the latter condition is violated for sufficiently large oversampling ratios. By deriving an upper bound on the contribution of overload errors to the total WCMSE, a lower bound for the decay rate of the WCMSE as a function of the oversampling ratio is found for fixed-rate quantization of sources with finite or infinite support. The third main contribution of the thesis is the introduction of the rate-distortion function (RDF) when WCMSE is the distortion metric, denoted by WCMSE-RDF. We provide a complete characterization for Gaussian sources. The resulting WCMSE-RDF yields, as special cases, Shannon's RDF, as well as the recently introduced RDF for source-uncorrelated distortions (RDF-SUD). For cases where only source-uncorrelated distortion is allowed, the RDF-SUD is extended to include the possibility of linear-time invariant feedback between reconstructed signal and coder input. It is also shown that feedback quantization schemes can achieve a bit-rate only 0.254 bits/sample above this RDF by using the same filters that minimize the reconstruction MSE for a quantizer-SNR constraint. The fourth main contribution of this thesis is to provide a set of conditions under which knowledge of a realization of the RDF can be used directly to solve encoder-decoder design optimization problems. This result has direct implications in the design of subband coders with feedback, as well as in the design of encoder-decoder pairs for applications such as networked control. As the fifth main contribution of this thesis, the RDF-SUD is utilized to show that, for Gaussian sta-tionary sources with memory and MSE distortion criterion, an upper bound on the information-theoretic causal RDF can be obtained by means of an iterative numerical procedure, at all rates. This bound is tighter than 0:5 bits/sample. Moreover, if there exists a realization of the causal RDF in which the re-construction error is jointly stationary with the source, then the bound obtained coincides with the causal RDF. The iterative procedure proposed here to obtain Ritc(D) also yields a characterization of the filters in a scalar feedback quantizer having an operational rate that exceeds the bound by less than 0:254 bits/sample. This constitutes an upper bound on the optimal performance theoretically attainable by any causal source coder for stationary Gaussian sources under the MSE distortion criterion.

La surveillance de phénomènes physiques à l’aide d’un réseau de capteurs (autonomes mais communicants) est fortement contrainte en consommation énergétique, principalement pour la transmission de données. Dans ce cadre, cette thèse propose des méthodes de traitement du signal permettant de réduire les communications sans compromettre la précision des calculs ultérieurs. La complexité de ces méthodes est réduite, de façon à ne consommer que peu d’énergie supplémentaire. Deux éléments servent à leur synthèse : la compression dès l’acquisition (Acquisition compressive) et la quantification grossière (sur 1 bit). D’abord, on étudie le corrélateur compressé, un estimateur qui permet d’évaluer les fonctions de corrélation, temps de retard et densités spectrales en exploitant directement des signaux compressés. Ses performances sont comparées au corrélateur usuel. Si le signal à traiter possède un support spectral étroit, l’estimateur proposé s’avère sensiblement meilleur que l’usuel. Ensuite, inspirés par les corrélateurs à forte quantification des années 50 et 60, deux nouveaux corrélateurs sont étudiés : le compressé sur 1 bit et le compressé hybride, qui peuvent également surpasser les performances de leurs contreparties non-compressées. Finalement, on montre la pertinence de ces méthodes pour les applications envisagées à travers l’exploitation de données réelles. / Monitoring physical phenomena by using a network of sensors (autonomous but interconnected) is highly constrained in energy consumption, mainly for data transmission. In this context, this thesis proposes signal processing tools to reduce communications without compromising computational accuracy in subsequent calculations. The complexity of these methods is reduced, so as to consume only little additional energy. Our two building blocks are compression during signal acquisition (Compressive Sensing) and CoarseQuantization (1 bit). We first study the Compressed Correlator, an estimator which allows for evaluating correlation functions, time-delay, and spectral densities directly from compressed signals. Its performance is compared with the usual correlator. As we show, if the signal of interest has limited frequency content, the proposed estimator significantly outperforms theconventional correlator. Then, inspired by the coarse quantization correlators from the 50s and 60s, two new correlators are studied: The 1-bit Compressed and the Hybrid Compressed, which can also outperform their uncompressed counterparts. Finally, we show the applicability of these methods in the context of interest through the exploitation of real data.

With the current demand for mixed-signal SoCs, an increasing number of designers are looking for ADC architectures that can be easily implemented over digital substrates. Since ADC performance is strongly dependent upon physical and electrical features, it gets more difficult for them to benefit from more recent technologies, where these features are more variable. This way, analog signal acquisition is not allowed to follow an evolutionary trend compatible with Moore’s Law. In fact, such trend shall get worst, since newer technologies are expected to have more variable characteristics. Also, for a matter of economy of scale, many times a mixed-signal SoC presents a good amount of idle processing power. In such systems it is advantageous to employ more costly digital signal processing provided that it allows a reduction in the analog area demanded or the use of less expensive analog blocks, able to cope with process variations and uncertainty. Besides the technological concerns, other factors that impact the cost of the design also advise to transfer problems from the analog to the digital domain whenever possible: design automation and self-test requirements, for instance. Recent surveys indicate that the total cost in designer hours for the analog blocks of a mixed-signal system can be up to three times the cost of the digital ones. This manuscript explores the concept of bottom-up analog acquisition design, using statistical sampling as a way to reduce the analog area demanded in the design of ADCs within mixed-signal systems. More particularly, it investigates the possibility of using digital modeling and digital compensation of non-idealities to ease the design of ADCs. The work is developed around three axes: the definition of target applications, the development of digital compensation algorithms and the exploration of architectural possibilities. New methods and architectures are defined and validated. The main notions behind the proposal are analyzed and it is shown that the approach is feasible, opening new paths of future research. Keywords:

Este trabalho investiga a influência da quantização em sistemas de controle em rede. São tratados problemas de estabilidade e estabilização de sistemas lineares de tempo discreto envolvendo quantização finita nas entradas da planta controlada, considerando dois tipos de quantizadores: os uniformes e os logarítmicos. Como consequência da quantização finita, ocorrem também efeitos de saturação e zonamorta dos sinais de entrada. Tais comportamentos não-lineares são considerados explicitamente na análise. Para plantas instáveis, o objetivo é estimar a região onde os estados estarão confinados em regime permanente. Esta região, denominada atrator dos estados, é estimada por meio de um conjunto elipsoidal. Ao mesmo tempo, determina-se um conjunto elipsoidal de condições iniciais admissíveis, para o qual se garante a convergência das trajetórias para o atrator em tempo finito. Primeiramente, esses conjuntos são determinados para o caso de um controlador dado e, posteriormente, sintetiza-se um controlador que minimiza o atrator. Em se tratando de plantas estáveis, investiga-se como o desempenho dinâmico é afetado pela quantização. Para tanto, utiliza-se como critério o coeficiente de decaimento exponencial que é garantido para o sistema. Nesta parte, excluem-se os comportamentos na região de saturação e na região da zona-morta. Primeiramente, o coeficiente de decaimento garantido é estimado para um sistema com controlador dado. Neste caso, faz-se uma análise de degradação de desempenho induzida pela quantização com relação ao comportamento do sistema em malha fechada sem quantização. Posteriormente, sintetiza-se um controlador que minimiza este coeficiente na presença da quantização. Na obtenção dos resultados, utilizam-se condições de setor respeitadas pelas não linearidades e formulam-se os problemas na forma de inequações matriciais que podem ser resolvidas a partir de problemas de otimização baseados em LMIs. / This work investigates the in uence of quantization over networked control systems. At rst, we tackle stability and stabilization problems of discrete-time linear systems involving nite quantization on the input of the controlled plant, considering two kinds of quantizers: uniform and logarithmic. As a consequence of the nite quantization, saturation and dead-zone e ects on the input signals are also present. These non-linear behaviors are explictly considered in the analysis. For unstable plants, the objective is to estimate the region where the states will be ultimately bounded. This region, which we call the attractor of the states, is estimated through an ellipsoidal set. Simultaneously, we determine an ellipsoidal set of admissible initial conditions, for which the trajectories will converge to the attractor in nite time. At rst, the sets are determined for the case where the controller is given and, in the sequel, a controller that minimizes the attractor is designed. When dealing with stable plants, we investigate how the dynamic performance is a ected by the quantization. To do that, we use as criterion the exponential decay rate which is guaranteed for the system. At this point, we exclude the behaviour in the saturation and deadzone regions. At rst, the guaranteed decay rate is estimated for a system where the controller is given. In this case, we analyze the deterioration of the performance in uenced by the quantization, compared to the behavior of the closed-loop system without quantization. In the sequel, a controller that minimizes that rate in the presence of quantization is designed. To obtain the results, we use sector conditions which are respected by the nonlinearities and we state the problems as matrix inequalities which can be solved using LMI-based optimization problems.

With the current demand for mixed-signal SoCs, an increasing number of designers are looking for ADC architectures that can be easily implemented over digital substrates. Since ADC performance is strongly dependent upon physical and electrical features, it gets more difficult for them to benefit from more recent technologies, where these features are more variable. This way, analog signal acquisition is not allowed to follow an evolutionary trend compatible with Moore’s Law. In fact, such trend shall get worst, since newer technologies are expected to have more variable characteristics. Also, for a matter of economy of scale, many times a mixed-signal SoC presents a good amount of idle processing power. In such systems it is advantageous to employ more costly digital signal processing provided that it allows a reduction in the analog area demanded or the use of less expensive analog blocks, able to cope with process variations and uncertainty. Besides the technological concerns, other factors that impact the cost of the design also advise to transfer problems from the analog to the digital domain whenever possible: design automation and self-test requirements, for instance. Recent surveys indicate that the total cost in designer hours for the analog blocks of a mixed-signal system can be up to three times the cost of the digital ones. This manuscript explores the concept of bottom-up analog acquisition design, using statistical sampling as a way to reduce the analog area demanded in the design of ADCs within mixed-signal systems. More particularly, it investigates the possibility of using digital modeling and digital compensation of non-idealities to ease the design of ADCs. The work is developed around three axes: the definition of target applications, the development of digital compensation algorithms and the exploration of architectural possibilities. New methods and architectures are defined and validated. The main notions behind the proposal are analyzed and it is shown that the approach is feasible, opening new paths of future research. Keywords:

Made available in DSpace on 2015-05-08T14:57:18Z (GMT). No. of bitstreams: 1 arquivototal.pdf: 3795959 bytes, checksum: 7a11a6cf0b41c67297d55642c2b80df3 (MD5) Previous issue date: 2014-02-21 / Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - CAPES / In this work, we analyse different architectures of analog-to-digital converters (ADC) and propose an architecture based on sampling by crossing levels and adaptive quantization step, aiming at reducing the energy required to convert and process specific signals. The proposed architecture has parameters which can be dynamically configured by the user, as to adapt the conversion process to the signal being sampled and to the requirements of power consumption of the target application. The architecture was modeled and simulated using Matlab, and used to convert several test signals, of which an ECG signal. The use of the proposed architecture resulted in SNR improvements of up to 10dB if compared against uniform (periodic) sampling. The digital logic was implemented in FPGA from a SystemVerilog description functionally compatible with the Matlab model, and the analog part was implemented with discrete components. / Neste trabalho, faz-se uma análise de diferentes arquiteturas de conversores analógico-digitais, e propõe-se uma arquitetura de conversor analógico-digital baseado em amostragem por cruzamento de níveis (não-uniforme) com adaptação do passo de quantização, com o objetivo de reduzir o consumo de energia requerido pela conversão analógica-digital e processamento de sinais com características específicas. A arquitetura proposta possui parâmetros que podem ser configurados dinamicamente pelo usuário, a fim de que o processo de conversão se adeque às características do sinal a ser amostrado e aos requerimentos de consumo de energia da aplicação. A arquitetura foi modelada e simulada em MatLab, tendo sido utilizada na conversão de diversos sinais de teste, dentre os quais um sinal típico de eletrocardiograma. Verificou-se que a amostragem não-uniforme com adaptação do passo de quantização proposta resultou em um aumento da relação sinal-ruído do sinal amostrado de até 10dB quando comparado com a amostragem uniforme. A implementação da parte digital foi feita em FPGA a partir de uma descrição em SystemVerilog funcionalmente compatível com o modelo em Matlab, e a parte analógica foi implementada com componentes discretos.

Este trabalho investiga a influência da quantização em sistemas de controle em rede. São tratados problemas de estabilidade e estabilização de sistemas lineares de tempo discreto envolvendo quantização finita nas entradas da planta controlada, considerando dois tipos de quantizadores: os uniformes e os logarítmicos. Como consequência da quantização finita, ocorrem também efeitos de saturação e zonamorta dos sinais de entrada. Tais comportamentos não-lineares são considerados explicitamente na análise. Para plantas instáveis, o objetivo é estimar a região onde os estados estarão confinados em regime permanente. Esta região, denominada atrator dos estados, é estimada por meio de um conjunto elipsoidal. Ao mesmo tempo, determina-se um conjunto elipsoidal de condições iniciais admissíveis, para o qual se garante a convergência das trajetórias para o atrator em tempo finito. Primeiramente, esses conjuntos são determinados para o caso de um controlador dado e, posteriormente, sintetiza-se um controlador que minimiza o atrator. Em se tratando de plantas estáveis, investiga-se como o desempenho dinâmico é afetado pela quantização. Para tanto, utiliza-se como critério o coeficiente de decaimento exponencial que é garantido para o sistema. Nesta parte, excluem-se os comportamentos na região de saturação e na região da zona-morta. Primeiramente, o coeficiente de decaimento garantido é estimado para um sistema com controlador dado. Neste caso, faz-se uma análise de degradação de desempenho induzida pela quantização com relação ao comportamento do sistema em malha fechada sem quantização. Posteriormente, sintetiza-se um controlador que minimiza este coeficiente na presença da quantização. Na obtenção dos resultados, utilizam-se condições de setor respeitadas pelas não linearidades e formulam-se os problemas na forma de inequações matriciais que podem ser resolvidas a partir de problemas de otimização baseados em LMIs. / This work investigates the in uence of quantization over networked control systems. At rst, we tackle stability and stabilization problems of discrete-time linear systems involving nite quantization on the input of the controlled plant, considering two kinds of quantizers: uniform and logarithmic. As a consequence of the nite quantization, saturation and dead-zone e ects on the input signals are also present. These non-linear behaviors are explictly considered in the analysis. For unstable plants, the objective is to estimate the region where the states will be ultimately bounded. This region, which we call the attractor of the states, is estimated through an ellipsoidal set. Simultaneously, we determine an ellipsoidal set of admissible initial conditions, for which the trajectories will converge to the attractor in nite time. At rst, the sets are determined for the case where the controller is given and, in the sequel, a controller that minimizes the attractor is designed. When dealing with stable plants, we investigate how the dynamic performance is a ected by the quantization. To do that, we use as criterion the exponential decay rate which is guaranteed for the system. At this point, we exclude the behaviour in the saturation and deadzone regions. At rst, the guaranteed decay rate is estimated for a system where the controller is given. In this case, we analyze the deterioration of the performance in uenced by the quantization, compared to the behavior of the closed-loop system without quantization. In the sequel, a controller that minimizes that rate in the presence of quantization is designed. To obtain the results, we use sector conditions which are respected by the nonlinearities and we state the problems as matrix inequalities which can be solved using LMI-based optimization problems.

abstract: Deep neural networks (DNN) have shown tremendous success in various cognitive tasks, such as image classification, speech recognition, etc. However, their usage on resource-constrained edge devices has been limited due to high computation and large memory requirement. To overcome these challenges, recent works have extensively investigated model compression techniques such as element-wise sparsity, structured sparsity and quantization. While most of these works have applied these compression techniques in isolation, there have been very few studies on application of quantization and structured sparsity together on a DNN model. This thesis co-optimizes structured sparsity and quantization constraints on DNN models during training. Specifically, it obtains optimal setting of 2-bit weight and 2-bit activation coupled with 4X structured compression by performing combined exploration of quantization and structured compression settings. The optimal DNN model achieves 50X weight memory reduction compared to floating-point uncompressed DNN. This memory saving is significant since applying only structured sparsity constraints achieves 2X memory savings and only quantization constraints achieves 16X memory savings. The algorithm has been validated on both high and low capacity DNNs and on wide-sparse and deep-sparse DNN models. Experiments demonstrated that deep-sparse DNN outperforms shallow-dense DNN with varying level of memory savings depending on DNN precision and sparsity levels. This work further proposed a Pareto-optimal approach to systematically extract optimal DNN models from a huge set of sparse and dense DNN models. The resulting 11 optimal designs were further evaluated by considering overall DNN memory which includes activation memory and weight memory. It was found that there is only a small change in the memory footprint of the optimal designs corresponding to the low sparsity DNNs. However, activation memory cannot be ignored for high sparsity DNNs. / Dissertation/Thesis / Masters Thesis Computer Engineering 2018

Nesta tese apresentamos três problemas interligados: a quântização de teorias não-Lagrangianos, a mecânica quântica não-comutativa (MQNC) e a construção do produto estrela atravéz do ordenamento de Weyl. No contexto do primeiro problema foi elaborada uma abordagem da quantização canônica de sistemas com as equações de movimento não-Lagrangianas. Construímos um princípio da ação mínima para um sistema equivalente das equações diferenciais de primeira ordem. Existe uma ambiguidade não-trivial (que não se reduz a uma derivada total) na definição da função de Lagrange para os sistemas de equações de primeira ordem. Apresentamos uma descrição completa desta ambiguidade. O esquema proposto é aplicado para a quantização da teoria quadrática geral. Também foi construida a quantização do oscilador harmônico amortecido e da carga elétrica com radiação. No contexto da MQNC elaboramos uma formulação da integral de trajetória da MQNC relativística e construímos a generalização não-comutativa da ação da super-partícula. A quantização da ação proposta fornece as equações de Klein-Gordon e de Dirac nas teorias de campo não-comutativas. No contexto do terceiro problema desenvolvemos uma abordagem para a quantização por deformação no plano real com uma estrutura de Poisson arbitrária baseada no ordenamento simétrico dos produtos dos operadores. É formulado um procedimento iterativo simples e efetivo para a construção do produto estrela. Este procedimento nos permitiu calcular o produto estrela em ordens altas (em terceira e quarta ordens), algo que foi feito pela primeira vez. Exceto por uma análise da cohomologia, que não consideramos no artigo, o método proposto dá uma descrição explicita, na linguagem matemática usual da física, do produto estrela. / We present here three interrelated problems: quantization of non-Lagrangian theories, noncommutative quantum mechanics (NCQM) and a constructions of the star product trough the the Weyl ordering. In the context of the first problem an approach to the canonical quantization of systems with non-Lagrangian equations of motion is proposed. We construct an action principle for an equivalent first-order equations of motion. There exists an ambiguity (not reducible to a total time derivative) in associating a Lagrange function with the given set of equations. We give a complete description of this ambiguity. The proposed scheme is applied to quantization of a general quadratic theory. Also the quantization of a damped oscillator and a radiating point-like charge is constructed. In the context of NCQM we propose a path integral formulation of relativistic NCQM and construct a noncommutative generalization of superparticle action. After quantization, the proposed action reproduces the Klein-Gordon and Dirac equations in the noncommutative field theories. In the context of the third problem we develop an approach to the deformation quantization on the real plane with an arbitrary Poisson structure which based on Weyl symmetrically ordered operator products. A simple and effective iterative procedure of the construction of star products is formulated. This procedure allowed us to calculate the third and the fourth order star products. Modulo some cohomology issues which we do not consider here, the method gives an explicit and physics-friendly description of the star products.

With the current demand for mixed-signal SoCs, an increasing number of designers are looking for ADC architectures that can be easily implemented over digital substrates. Since ADC performance is strongly dependent upon physical and electrical features, it gets more difficult for them to benefit from more recent technologies, where these features are more variable. This way, analog signal acquisition is not allowed to follow an evolutionary trend compatible with Moore’s Law. In fact, such trend shall get worst, since newer technologies are expected to have more variable characteristics. Also, for a matter of economy of scale, many times a mixed-signal SoC presents a good amount of idle processing power. In such systems it is advantageous to employ more costly digital signal processing provided that it allows a reduction in the analog area demanded or the use of less expensive analog blocks, able to cope with process variations and uncertainty. Besides the technological concerns, other factors that impact the cost of the design also advise to transfer problems from the analog to the digital domain whenever possible: design automation and self-test requirements, for instance. Recent surveys indicate that the total cost in designer hours for the analog blocks of a mixed-signal system can be up to three times the cost of the digital ones. This manuscript explores the concept of bottom-up analog acquisition design, using statistical sampling as a way to reduce the analog area demanded in the design of ADCs within mixed-signal systems. More particularly, it investigates the possibility of using digital modeling and digital compensation of non-idealities to ease the design of ADCs. The work is developed around three axes: the definition of target applications, the development of digital compensation algorithms and the exploration of architectural possibilities. New methods and architectures are defined and validated. The main notions behind the proposal are analyzed and it is shown that the approach is feasible, opening new paths of future research. Keywords: