Deep Learning Networks (DLN) is a relatively new artificial intelligence algorithm that gained popularity quickly due to its unprecedented performance. One of the key elements for this success is DL’s ability to extract a high-level of information from large amounts of raw data. This ability comes at the cost of high computational and memory requirements for the training process. Estimation algorithms such as the Extended Kalman Filter (EKF) and the Smooth Variable Structure Filter (SVSF) are used in literature to train small Neural Networks. However, they have failed to scale well with deep networks due to their excessive requirements for computation and memory size. In this thesis the concept of using EKF and SVSF for DLN training is revisited. A New family of filters that are efficient in memory and computational requirements are proposed and their performance is evaluated against the state-of-the-art algorithms. The new filters show competitive performance to existing algorithms and do not require fine tuning. These new findings change the scientific community’s perception that estimation theory methods such as EKF and SVSF are not practical for their application to large networks. A second contribution from this research is the application of DLN to Fault Detection and Diagnosis. The findings indicate that DL can analyze complex sound and vibration signals in testing of automotive starters to successfully detect and diagnose faults with 97.6% success rates. This proves that DLN can automate end-of-line testing of starters and replace operators who manually listen to sound signals to detect any deviation. Use of DLN in end-of-line testing could lead to significant economic benefits in manufacturing operations. In addition to starters, another application considered is the use of DLN in monitoring of the State-Of-Charge (SOC) of batteries in electric cars. The use of DLN for improving the SOC prediction accuracy is discussed. / Thesis / Doctor of Science (PhD) / There are two main ideas discussed in this thesis, both are related to Deep Learning (DL). The first investigates the use of estimation theory in DL network training. Training DL networks is challenging as it requires large amounts of data and it is computationally demanding. The thesis discusses the use of estimation theory for training of DL networks and its utility in information extraction. The thesis also presents the application of DL networks in an end-of-line Fault Detection and Diagnosis system for complex automotive components. Failure of appropriately testing automotive components can lead to shipping faulty components that can harm a manufacturer’s reputation as well as potentially jeopardizing safety. In this thesis, DL is used to detect and analyze complex fault patterns of automotive starters, complemented by sound and vibration measurements.