The sheer increase in the volume of data over the last decade has triggered research in cluster computing frameworks that enable web enterprises to extract big insights from big data. While Apache Spark defines the state of the art in big data analytics platforms for (i) exploiting data-flow and in-memory computing and (ii) for exhibiting superior scale-out performance on the commodity machines, little effort has been devoted to understanding the performance of in-memory data analytics with Spark on modern scale-up servers. This thesis characterizes the performance of in-memory data analytics with Spark on scale-up servers.Through empirical evaluation of representative benchmark workloads on a dual socket server, we have found that in-memory data analytics with Spark exhibit poor multi-core scalability beyond 12 cores due to thread level load imbalance and work-time inflation (the additional CPU time spent by threads in a multi-threaded computation beyond the CPU time required to perform the same work in a sequential computation). We have also found that workloads are bound by the latency of frequent data accesses to the memory. By enlarging input data size, application performance degrades significantly due to the substantial increase in wait time during I/O operations and garbage collection, despite 10% better instruction retirement rate (due to lower L1cache misses and higher core utilization).For data accesses, we have found that simultaneous multi-threading is effective in hiding the data latencies. We have also observed that (i) data locality on NUMA nodes can improve the performance by 10% on average,(ii) disabling next-line L1-D prefetchers can reduce the execution time by upto14%. For garbage collection impact, we match memory behavior with the garbage collector to improve the performance of applications between 1.6xto 3x and recommend using multiple small Spark executors that can provide up to 36% reduction in execution time over single large executor. Based on the characteristics of workloads, the thesis envisions near-memory and near storage hardware acceleration to improve the single-node performance of scale-out frameworks like Apache Spark. Using modeling techniques, it estimates the speed-up of 4x for Apache Spark on scale-up servers augmented with near-data accelerators. / <p>QC 20171121</p>

La localisation de source Distribuée Cohérente (DC) présente un défi du traitement d'antenne. Les contributions de cette thèse s’articulent principalement autour de trois aspects. Premièrement, un estimateur conjoint de l'angle, la distance, la dispersion et la forme de la source appelée JADSSE est proposé pour le cas champ proche. L’estimation d’un paramètre de forme de distribution de la dispersion permet d’éviter des erreurs de modèles sur l’a priori de la forme de la distribution. Deuxièmement, on généralise l'estimateur Decoupled DSPE en champ proche. Cette approche permet de découpler l'estimation de la Direction D’Arrivée (DDA) et de la distance de l'estimation de la dispersion. Afin de permettre l’estimation de la dispersion sans connaître a priori les formes de distribution, on propose le DADSSE qui consiste à estimer successivement la DDA, la distance et ensuite la dispersion et la forme de la distribution de la source. Troisièmement, on généralise le modèle DC avec une dispersion spatiale bidimensionnelle de la source ainsi que l’estimateur JADSSE. Deux approches sont proposées pour l’estimation de la puissance prenant en compte le modèle d’étalement des sources. Les méthodes proposées sont testées sur les données expérimentales de la soufflerie de Renault. Les résultats mettent en évidence des sources aéro-acoustiques proches et de faibles puissances. L’ensemble de ces travaux permet de fournir un outil pour une meilleure cartographie et caractérisation des sources aéro-acoustiques grâce à l’estimation de la position, l'étalement, la puissance et la forme. / Localization of Coherently Distributed (CD) source presents a challenge in the array signal processing. Our work motivates the localization of aero-acoustic source based on its spatial extension. This challenge is practically ignored in the literature of acoustic imaging field where many applications consist in mapping noisy source to reduce its contribution. The thesis presents the three following contributions. First, we propose a Joint Angle, Distance, Spread and Shape Estimator called JADSSE. The estimation of the so-called spread shape distribution parameter proposed by JADSSE avoids the modeling error due to the required a priori knowledge on the source shape when using classical estimators. Second, we expand the Decoupled DSPE to the near field. This method decouples the Direction of Arrival (DoA) and the range estimation from the spread estimation. Meanwhile, this method prevents the spread estimation for unknown shape distribution. Therefore, we propose the DADSSE to successively estimate the DOA, the range and then the spread and the shape distribution of the source. Third, we generalize the CD model and the JADSSE to consider the bi-dimensional spread of the source. Next, we propose two source power estimation approaches accounting the spatial spread of the source. The proposed methods are tested using a set of experimental data of the Renault wind tunnel application. Results show the presence of new aero-acoustic sources especially the overlapped ones with weak powers. We provide a tool to better map and characterize the aero-acoustic source by estimating the position, spread, power and shape.

La localisation de source Distribuée Cohérente (DC) présente un défi du traitement d'antenne. Les contributions de cette thèse s’articulent principalement autour de trois aspects. Premièrement, un estimateur conjoint de l'angle, la distance, la dispersion et la forme de la source appelée JADSSE est proposé pour le cas champ proche. L’estimation d’un paramètre de forme de distribution de la dispersion permet d’éviter des erreurs de modèles sur l’a priori de la forme de la distribution. Deuxièmement, on généralise l'estimateur Decoupled DSPE en champ proche. Cette approche permet de découpler l'estimation de la Direction D’Arrivée (DDA) et de la distance de l'estimation de la dispersion. Afin de permettre l’estimation de la dispersion sans connaître a priori les formes de distribution, on propose le DADSSE qui consiste à estimer successivement la DDA, la distance et ensuite la dispersion et la forme de la distribution de la source. Troisièmement, on généralise le modèle DC avec une dispersion spatiale bidimensionnelle de la source ainsi que l’estimateur JADSSE. Deux approches sont proposées pour l’estimation de la puissance prenant en compte le modèle d’étalement des sources. Les méthodes proposées sont testées sur les données expérimentales de la soufflerie de Renault. Les résultats mettent en évidence des sources aéro-acoustiques proches et de faibles puissances. L’ensemble de ces travaux permet de fournir un outil pour une meilleure cartographie et caractérisation des sources aéro-acoustiques grâce à l’estimation de la position, l'étalement, la puissance et la forme. / Localization of Coherently Distributed (CD) source presents a challenge in the array signal processing. Our work motivates the localization of aero-acoustic source based on its spatial extension. This challenge is practically ignored in the literature of acoustic imaging field where many applications consist in mapping noisy source to reduce its contribution. The thesis presents the three following contributions. First, we propose a Joint Angle, Distance, Spread and Shape Estimator called JADSSE. The estimation of the so-called spread shape distribution parameter proposed by JADSSE avoids the modeling error due to the required a priori knowledge on the source shape when using classical estimators. Second, we expand the Decoupled DSPE to the near field. This method decouples the Direction of Arrival (DoA) and the range estimation from the spread estimation. Meanwhile, this method prevents the spread estimation for unknown shape distribution. Therefore, we propose the DADSSE to successively estimate the DOA, the range and then the spread and the shape distribution of the source. Third, we generalize the CD model and the JADSSE to consider the bi-dimensional spread of the source. Next, we propose two source power estimation approaches accounting the spatial spread of the source. The proposed methods are tested using a set of experimental data of the Renault wind tunnel application. Results show the presence of new aero-acoustic sources especially the overlapped ones with weak powers. We provide a tool to better map and characterize the aero-acoustic source by estimating the position, spread, power and shape.