Matrix and tensor accelerators play indispensable roles in the field of artificial intelligence (AI). Although most of the matrix accelerators, such as graphic processing units (GPUs) and tensor processing units (TPUs), are still electronics based, the energy efficiency and scalability limits of electronic accelerators have presented an opportunity for photonics to perform matrix and tensor acceleration. This dissertation explores silicon photonics as an enabling and cost-effective platform for developing photonic systems, in particular, photonic tensor accelerators. The thesis presents a detailed design procedure for active and passive components, forming a comprehensive Process Design Kit (PDK) in a foundry-compatible silicon photonic platform. The PDK library includes passive waveguide building blocks as well as active components such as micro ring modulators with an EO bandwidth of more than 20GHz and Ge-on-Si photodetectors with >25GHz bandwidth. Having our own PDK ensures consistency in the layout and fabrication of silicon photonic integrated circuits (PIC) across different foundries. We designed and fabricated multidimensional photonic tensor accelerators, each of which consists of many waveguides, splitters/couplers, coherent modulators, and balanced detectors, and successfully demonstrated PIC-based matrix-vector multiplications.