Partial differential equations (PDEs) and stochastic partial differential equations (SPDEs) are powerful tools in modeling real-world phenomena in many fields such as geo-engineering. For instance processes such as oil or gas recovery from hydrocarbon reservoirs and mining heat from geothermal reservoirs can be modeled by PDEs or SPDEs. An important task is to understand the behavior of such phenomena. This can be achieved through explicit solutions of equations. Since explicit solutions of many PDEs and SPDEs are rarely known, developing numerical schemes is a good alternative to provide approximations of these explicit solutions. The study of numerical solutions of PDEs and SPDEs is therefore an active research area and has attracted a lot of attentions since at least two decades. The aims of this dissertation is to develop numerical schemes to approximate semilinear parabolic PDEs and SPDEs in space and in time. The approximation in space is done via the standard Galerkin finite element method and the approximation in time, which is the core of our work is done via various numerical integrators. This dissertation consists of two general parts. The first part of this thesis deals with autonomous PDEs and SPDEs. Here, our main interest is on semilinear PDEs and SPDEs where the nonlinear part is stronger than the linear part also called (stochastic) reactive dominated transport equations, or stiff problems. For such problems, many numerical techniques in the current scientific literature lose their good stability properties. We develop a new explicit exponential integrator (called exponential Rosenbrock-type method) and a new semi-implicit method (called linear implicit Rosenbrock-type method), appropriate for such PDEs and SPDEs. We analyze the strong convergence of our novel fully discrete schemes towards the mild solution of the (S)PDE and obtain convergence orders similar to existing ones in the literature. The second part of this thesis focuses on numerical approximations of semilinear non-autonomous parabolic PDEs and SPDEs. Such equations are more realistic than the autonomous ones and find applications in many fields such as fluid mechanics, quantum field theory, electromagnetism, etc. Numerics of non-autonomous semilinear parabolic PDEs and SPDEs are far from being well understood in the literature. We fill that gap in this thesis by developing a new exponential integrator (called Magnus-type method) and the semi-implicit method for such problems and provide their strong convergence towards the mild solution. Moreover, for both autonomous and non-autonomous SPDEs driven by additive noise, we achieve optimal convergence order in time 1 or approximately 1. Numerical simulations are provided to illustrate our theoretical findings in both autonomous and non-autonomous cases.