While site effects are accounted for in most modern U.S. seismic design codes for building structures, there exist no standardized procedures for the computationally efficient integration of nonlinear ground response analyses in broadband ground motion simulations. In turn, the lack of a unified methodology affects the prediction accuracy of site-specific ground motion intensity measures, the evaluation of site amplification factors when broadband simulations are used for the development of hybrid attenuation relations and the estimation of inelastic structural performance when strong motion records are used as input in aseismic structural design procedures. In this study, a set of criteria is established, which quantifies how strong nonlinear effects are anticipated to manifest at a site by investigating the empirical relation between nonlinear soil response, soil properties, and ground motion characteristics. More specifically, the modeling variability and parametric uncertainty of nonlinear soil response predictions are studied, along with the uncertainty propagation of site response analyses to the estimation of inelastic structural performance. Due to the scarcity of design level ground motion recording, the geotechnical information at 24 downhole arrays is used and the profiles are subjected to broadband ground motion synthetics. For the modeling variability study, the site response models are validated against available downhole array observations. The site and ground motion parameters that govern the intensity of nonlinear effects are next identified, and an empirical relationship is established, which may be used to estimate to a first approximation the error introduced in ground motion predictions if nonlinear effects are not accounted for. The soil parameter uncertainty in site response predictions is next evaluated as a function of the same measures of soil properties and ground motion characteristics. It is shown that the effects of nonlinear soil property uncertainties on the ground-motion variability strongly depend on the seismic motion intensity, and this dependency is more pronounced for soft soil profiles. By contrast, the effects of velocity profile uncertainties are less intensity dependent and more sensitive to the velocity impedance in the near surface that governs the maximum site amplification. Finally, a series of bilinear single degree of freedom oscillators are subjected to the synthetic ground motions computed using the alternative soil models, and evaluate the consequent variability in structural response. Results show high bias and uncertainty of the inelastic structural displacement ratio predicted using the linear site response model for periods close to the fundamental period of the soil profile. The amount of bias and the period range where the structural performance uncertainty manifests are shown to be a function of both input motion and site parameters.