In applications spanning across aerospace, marine, automotive, energy, and space travel domains, composite materials have become ubiquitous because of their superior stiffness-to-weight ratios as well as corrosion and fatigue resistance. However, from a manufacturing perspective, these advanced materials have introduced new challenges that demand the development of new tools. Due to the complex anisotropic and nonlinear material properties, composite materials are more difficult to model than conventional materials such as metals and plastics. Furthermore, there exist ultra-high precision requirements in safety critical applications that are yet to be reliably met in production. Towards developing new tools addressing these challenges, this dissertation aims to (i) build high-fidelity numerical simulations of composite assembly processes, (ii) bridge these simulations to machine learning tools, and (iii) apply data-driven solutions to process control problems while identifying and overcoming their shortcomings. This is accomplished in case studies that model the fixturing, shape control, and fastening of composite fuselage components. Therein, simulation environments are created that interact with novel implementations of modified proximal policy optimization, based on a newly developed reinforcement learning algorithm. The resulting reinforcement learning agents are able to successfully address the underlying optimization problems that underpin the process and quality requirements. / Doctor of Philosophy / Within the manufacturing domain, there has been a concerted effort to transition towards Industry 4.0. To a large degree, this term refers Klaus Schwab's vision presented at the World Economic Forum in 2015, in which he outlined fundamental systemic changes that would incorporate ubiquitous computing, artificial intelligence (AI), big data, and the internet-of-things (IoT) into all aspects of productive activities within the economy. Schwab argues that rapid change will be driven by fusing these new technologies in existing and emerging applications. However, this process has only just begun and there still exist many challenges to realize the promise of Industry 4.0. One such challenge is to create computer models that are not only useful during early design stages of a product, but that are connected to its manufacturing processes, thereby guiding and informing decisions in real-time. This dissertation explores such scenarios in the context of composite structure assembly in aerospace manufacturing. It aims to link computer simulations that characterize the assembly of product components with their physical counterparts, and provides data-driven solutions to control problems that cannot typically be solved without tedious trial-and-error approaches or expert knowledge.