Background: Few studies have delved into the relationship of the social environment with children’s physical activity and outdoor play in public playgrounds by considering gender differences. The aim of the present study was to examine gender differences and the relationship of the social environment with children’s physical activity and outdoor play in public playgrounds. Methods: A quantitative, observational study was conducted at ten playgrounds in one district of a middle-sized town in Germany. The social environment, physical activity levels, and outdoor play were measured using a modified version of the System for Observing Play and Leisure Activity in Youth. Results: In total, 266 observations of children (117 girls/149 boys) between four and 12 years old were used in this analysis. Significant gender differences were found in relation to activity types, but not in moderate-to-vigorous physical activity (MVPA). The presence of active children was the main explanatory variable for MVPA. In the models stratified by gender, the presence of opposite-sex children was a significant negative predictor of MVPA in girls but not in boys. Conclusions: The presence of active children contributes to children’s physical activity levels in public playgrounds. Girls’ physical activity seems to be suppressed in the presence of boys.