Context. High-contrast scattered light observations have revealed the surface morphology of several dozen protoplanetary disks at optical and near-infrared wavelengths. Inclined disks offer the opportunity to measure part of the phase function of the dust grains that reside in the disk surface which is essential for our understanding of protoplanetary dust properties and the early stages of planet formation. Aims. We aim to construct a method which takes into account how the flaring shape of the scattering surface of an optically thick protoplanetary disk projects onto the image plane of the observer. This allows us to map physical quantities (e.g., scattering radius and scattering angle) onto scattered light images and retrieve stellar irradiation corrected images (r(2)-scaled) and dust phase functions. Methods. The scattered light mapping method projects a power law shaped disk surface onto the detector plane after which the observed scattered light image is interpolated backward onto the disk surface. We apply the method on archival polarized intensity images of the protoplanetary disk around HD 100546 that were obtained with VLT/SPHERE in the R' band and VLT/NACO in the H and K-s bands. \Results. The brightest side of the r(2)-scaled R-0 band polarized intensity image of HD 100546 changes from the far to the near side of the disk when a flaring instead of a geometrically flat disk surface is used for the r(2)-scaling. The decrease in polarized surface brightness in the scattering angle range of similar to 40 degrees-70 degrees is likely a result of the dust phase function and degree of polarization which peak in different scattering angle regimes. The derived phase functions show part of a forward scattering peak, which indicates that large, aggregate dust grains dominate the scattering opacity in the disk surface. Conclusions. Projection effects of a protoplanetary disk surface need to be taken into account to correctly interpret scattered light images. Applying the correct scaling for the correction of stellar irradiation is crucial for the interpretation of the images and the derivation of the dust properties in the disk surface layer.

We present optical and near-infrared high-contrast images of the transitional disk HD 100546 taken with the Magellan Adaptive Optics system (MagAO) and the Gemini Planet Imager (GPI). GPI data include both polarized intensity and total intensity imagery, and MagAO data are taken in Simultaneous Differential Imaging mode at Ha. The new GPI H-band total intensity data represent a significant enhancement in sensitivity and field rotation compared to previous data sets and enable a detailed exploration of substructure in the disk. The data are processed with a variety of differential imaging techniques (polarized, angular, reference, and simultaneous differential imaging) in an attempt to identify the disk structures that are most consistent across wavelengths, processing techniques, and algorithmic parameters. The inner disk cavity at 15 au is clearly resolved in multiple data sets, as are a variety of spiral features. While the cavity and spiral structures are identified at levels significantly distinct from the neighboring regions of the disk under several algorithms and with a range of algorithmic parameters, emission at the location of HD 100546 "c" varies from point-like under aggressive algorithmic parameters to a smooth continuous structure with conservative parameters, and is consistent with disk emission. Features identified in the HD 100546 disk bear qualitative similarity to computational models of a moderately inclined two-armed spiral disk, where projection effects and wrapping of the spiral arms around the star result in a number of truncated spiral features in forward-modeled images.

We present H band spectroscopic and H alpha photometric observations of HD 100546 obtained with the Gemini Planet Imager and the Magellan Visible AO camera. We detect H band emission at the location of the protoplanet HD 100546 b, but show that the choice of data processing parameters strongly affects the morphology of this source. It appears point-like in some aggressive reductions, but rejoins an extended disk structure in the majority of the others. Furthermore, we demonstrate that this emission appears stationary on a timescale of 4.6 years, inconsistent at the 2 sigma level with a Keplerian clockwise orbit at 59 au in the disk plane. The H band spectrum of the emission is inconsistent with any type of low effective temperature object or accreting protoplanetary disk. It strongly suggests a scattered-light origin, as this is consistent with the spectrum of the star and the spectra extracted at other locations in the disk. A nondetection at the 5 sigma level of HD 100546 b in differential H alpha imaging places an upper limit, assuming the protoplanet lies in a gap free of extinction, on the accretion luminosity of 1.7 x 10(-4) L-circle dot and M(M) over dot < 6.3 x 10(-7) M-Jup(2) yr(-1) for 1 R-Jup. These limits are comparable to the accretion luminosity and accretion rate of T-Tauri stars or LkCa 15 b. Taken together, these lines of evidence suggest that the H band source at the location of HD 100546 b is not emitted by a planetary photosphere or an accreting circumplanetary disk but is a disk feature enhanced by the point-spread function subtraction process. This non-detection is consistent with the non-detection in the K. band reported in an earlier study but does not exclude the possibility that HD 100546 b is deeply embedded.