In this dissertation, I consider the problem of model reduction in both oscillatory and networked systems. Previously, the Manifold Boundary Approximation Method (MBAM) has been demonstrated as a data-driven tool for reducing the parametric complexity of so-called sloppy models. To be effective, MBAM requires the model manifold to have low curvature. I show that oscillatory models are characterized by model manifolds with high curvature in one or more directions. I propose methods for transforming the model manifolds of these models into ones with low curvature and demonstrate on a couple of test systems. I demonstrate MBAM as a tool for data-driven network reduction on a small model from power systems. I derive multiple effective networks for the model, each tailored to a specific choice of system observations. I find several important types of parameter reductions, including network reductions, which can be used in large power systems models. Finally, I consider the problem of piecemeal reduction of large systems. When a large system is split into pieces that are to be reduced separately using MBAM, there is no guarantee that the reduced pieces will be compatible for reassembly. I propose a strategy for reducing a system piecemeal while guaranteeing that the reduced pieces will be compatible. I demonstrate the reduction strategy on a small resistor network.