Context. High-contrast scattered light observations have revealed the surface morphology of several dozen protoplanetary disks at optical and near-infrared wavelengths. Inclined disks offer the opportunity to measure part of the phase function of the dust grains that reside in the disk surface which is essential for our understanding of protoplanetary dust properties and the early stages of planet formation. Aims. We aim to construct a method which takes into account how the flaring shape of the scattering surface of an optically thick protoplanetary disk projects onto the image plane of the observer. This allows us to map physical quantities (e.g., scattering radius and scattering angle) onto scattered light images and retrieve stellar irradiation corrected images (r(2)-scaled) and dust phase functions. Methods. The scattered light mapping method projects a power law shaped disk surface onto the detector plane after which the observed scattered light image is interpolated backward onto the disk surface. We apply the method on archival polarized intensity images of the protoplanetary disk around HD 100546 that were obtained with VLT/SPHERE in the R' band and VLT/NACO in the H and K-s bands. \Results. The brightest side of the r(2)-scaled R-0 band polarized intensity image of HD 100546 changes from the far to the near side of the disk when a flaring instead of a geometrically flat disk surface is used for the r(2)-scaling. The decrease in polarized surface brightness in the scattering angle range of similar to 40 degrees-70 degrees is likely a result of the dust phase function and degree of polarization which peak in different scattering angle regimes. The derived phase functions show part of a forward scattering peak, which indicates that large, aggregate dust grains dominate the scattering opacity in the disk surface. Conclusions. Projection effects of a protoplanetary disk surface need to be taken into account to correctly interpret scattered light images. Applying the correct scaling for the correction of stellar irradiation is crucial for the interpretation of the images and the derivation of the dust properties in the disk surface layer.

L'objectif principal de cette thèse est de proposer de nouveaux mécanismes de transport de solides dans les disques protoplanétaires afin de résoudre le problème de la dérive radiale des solides causée par la friction du gaz. En effet, malgré d'importants efforts théoriques et expérimentaux, il reste difficile à expliquer comment de petites particules de poussière submillimétriques forment des blocs kilométriques dans les conditions qui règnent au sein des disques protoplanétaires. Je montre que les mécanismes de transport proposés dans cette thèse sont en mesure de résoudre ce problème de dérive et j'étudie leurs effets sur la formation des premiers solides. Dans un premier temps, je considère les effets de la photophorèse et des jets magnétiques sur le mouvement radial des grains dans les disques protoplanétaires. Le premier est dû aux effets thermiques du rayonnement stellaire sur la surface des grains, tandis que le deuxième est provoqué par les lignes de champ magnétique stellaire qui traversent le disque. Les résultats sont obtenus en résolvant les équations du mouvement des particules de façon numérique. Le transport induit par ces mécanismes a d'importantes conséquences pour la composition des météorites qui sont discutées dans le contexte de la nébuleuse solaire. Dans un deuxième temps, j'étudie la formation de pièges à particules causés par la présence de plusieurs planètes dans le disque grâce à des simulations hydrodynamiques. Ces résultats incluent la croissance des grains et sont directement comparés aux travaux similaires considérant une seule planète dans le disque. Le cas de l'étoile HD 100546, pour lequel les observations récentes suggèrent la présence de deux planètes dans le disque, est examiné en détail. L'évolution du disque en considérant différentes tailles de grain est étudiée au moyen de simulations hydrodynamiques SPH. Les distributions de la poussière et du gaz dans le disque sont particulièrement révélatrices car elles permettent de mettre à l'épreuve les différents scenarios proposés par les observations. L'étude de ces mécanismes montre que, selon leur taille et leur composition, les grains s'accumulent à différentes distances radiales dans le disque. Ces processus empêchent donc l'accrétion des solides par l'étoile et résolvent ainsi le problème de la barrière de dérive radiale. Les futures observations avec des instruments tels que ALMA, SPHERE et MATISSE permettront de mieux contraindre l'efficacité de ces mécanismes dans les disques protoplanétaires / The main goal of this work is to study new transport mechanisms of solids in protoplanetary disks and its implications for the composition of the first solids. The motion of solids inside the disk leads to the so-called radial-drift barrier caused by the gas aerodynamic drag, which is a severe problem for planet formation theory. In this context, it is hard to explain how sub-mm grains reach planetesimal sizes during the disk lifespan. First of all, I study the effects of photophoresis on the dust grains illuminated by the stellar radiation and quantify the efficiency of radial transport as a function of the particle properties. Then, I study the ejection of particles from the inner regions of the disk via the so-called stellar fountain model. Due to the stellar magnetic field which threads the disk, solid particles enter a jet that sends them outwards up to a few astronomical units. Both processes, photophoresis and jets, have important implications for the composition of meteorites which are discussed within the Solar Nebula scenario. In the last chapter, I study dust dynamics in multi-planetary systems through SPH simulations. The formation of particle traps in a disk with two planets is treated in detail and compared to previous work considering a single planet. Then I consider the case of HD 100546, a star with a disk which might harbor two planets according to recent observations, and study the disk evolution in different scenarios. By considering different grains sizes it is then possible to establish a link with interferometric observations of the system. We consider models with different planetary masses and radial distances in order to better constrain these quantities. The study of these mechanisms reveals that, according to particle size and composition, grains can pile up at different radial distances in the disk. This prevents the accretion by the central star by stopping the radial drift of solids, which shows that these mechanisms are good candidates to solve the radial-drift barrier. Future observations using ALMA, SPHERE and MATISSE will provide insights into the efficiency of these transport processes in protoplanetary disks

We present optical and near-infrared high-contrast images of the transitional disk HD 100546 taken with the Magellan Adaptive Optics system (MagAO) and the Gemini Planet Imager (GPI). GPI data include both polarized intensity and total intensity imagery, and MagAO data are taken in Simultaneous Differential Imaging mode at Ha. The new GPI H-band total intensity data represent a significant enhancement in sensitivity and field rotation compared to previous data sets and enable a detailed exploration of substructure in the disk. The data are processed with a variety of differential imaging techniques (polarized, angular, reference, and simultaneous differential imaging) in an attempt to identify the disk structures that are most consistent across wavelengths, processing techniques, and algorithmic parameters. The inner disk cavity at 15 au is clearly resolved in multiple data sets, as are a variety of spiral features. While the cavity and spiral structures are identified at levels significantly distinct from the neighboring regions of the disk under several algorithms and with a range of algorithmic parameters, emission at the location of HD 100546 "c" varies from point-like under aggressive algorithmic parameters to a smooth continuous structure with conservative parameters, and is consistent with disk emission. Features identified in the HD 100546 disk bear qualitative similarity to computational models of a moderately inclined two-armed spiral disk, where projection effects and wrapping of the spiral arms around the star result in a number of truncated spiral features in forward-modeled images.

We present H band spectroscopic and H alpha photometric observations of HD 100546 obtained with the Gemini Planet Imager and the Magellan Visible AO camera. We detect H band emission at the location of the protoplanet HD 100546 b, but show that the choice of data processing parameters strongly affects the morphology of this source. It appears point-like in some aggressive reductions, but rejoins an extended disk structure in the majority of the others. Furthermore, we demonstrate that this emission appears stationary on a timescale of 4.6 years, inconsistent at the 2 sigma level with a Keplerian clockwise orbit at 59 au in the disk plane. The H band spectrum of the emission is inconsistent with any type of low effective temperature object or accreting protoplanetary disk. It strongly suggests a scattered-light origin, as this is consistent with the spectrum of the star and the spectra extracted at other locations in the disk. A nondetection at the 5 sigma level of HD 100546 b in differential H alpha imaging places an upper limit, assuming the protoplanet lies in a gap free of extinction, on the accretion luminosity of 1.7 x 10(-4) L-circle dot and M(M) over dot < 6.3 x 10(-7) M-Jup(2) yr(-1) for 1 R-Jup. These limits are comparable to the accretion luminosity and accretion rate of T-Tauri stars or LkCa 15 b. Taken together, these lines of evidence suggest that the H band source at the location of HD 100546 b is not emitted by a planetary photosphere or an accreting circumplanetary disk but is a disk feature enhanced by the point-spread function subtraction process. This non-detection is consistent with the non-detection in the K. band reported in an earlier study but does not exclude the possibility that HD 100546 b is deeply embedded.