In recent years, the discontinuous Galerkin (DG) method has been successfully applied to solving hyperbolic conservation laws. Due to its compactness, high order accuracy, and versatility, the DG method has been extensively applied to convection-diffusion problems. In this dissertation, a numerical package, texttt{PHORCE}, is introduced to solve a number of convection-diffusion problems in neutral fluids and plasmas. Unstructured grids are used in order to randomize grid errors, which is especially important for complex geometries. texttt{PHORCE} is written in texttt{C++} and fully parallelized using the texttt{MPI} library. Memory optimization has been considered in this work to achieve improved efficiency. DG algorithms for hyperbolic terms are well studied. However, an accurate and efficient diffusion solver still constitutes ongoing research, especially for a nodal representation of the discontinuous Galerkin (NDG) method. An affine reconstructed discontinuous Galerkin (aRDG) algorithm is developed in this work to solve the diffusive operator using an unstructured NDG method. Unlike other reconstructed/recovery algorithms, all computations can be performed on a reference domain, which promotes efficiency in computation and storage. In addition, to the best of the authors' knowledge, this is the first practical guideline that has been proposed for applying the reconstruction algorithm on a nodal discontinuous Galerkin method. TVB type and WENO type limiters are also studied to deal with numerical oscillations in regions with strong physical gradients in state variables. A high-order positivity-preserving limiter is also extended in this work to prevent negative densities and pressure. A new interface tracking method, mass of fluid (MOF), along with its bound limiter has been proposed in this work to compute the mass fractions of different fluids over time. Hydrodynamic models, such as Euler and Navier-Stokes equations, and plasma models, such as ideal-magnetohydrodynamics (MHD) and two-fluid plasma equations, are studied and benchmarked with various applications using this DG framework. Numerical computations of Rayleigh-Taylor instability growth with experimentally relevant parameters are performed using hydrodynamic and MHD models on planar and radially converging domains. Discussions of the suppression mechanisms of Rayleigh-Taylor instabilities due to magnetic fields, viscosity, resistivity, and thermal conductivity are also included. This work was partially supported by the US Department of Energy under grant number DE-SC0016515. The author acknowledges Advanced Research Computing at Virginia Tech for providing computational resources and technical support that have contributed to the results reported within this work. URL: http://www.arc.vt.edu / Doctor of Philosophy / High-energy density (HED) plasma science is an important area in studying astrophysical phenomena as well as laboratory phenomena such as those applicable to inertial confinement fusion (ICF). ICF plasmas undergo radial compression, with an aim of achieving fusion ignition, and are subject to a number of hydrodynamic instabilities that can significantly alter the implosion and prevent sufficient fusion reactions. An understanding of these instabilities and their mitigation mechanisms is important allow for a stable implosion in ICF experiments. This work aims to provide a high order accurate and robust numerical framework that can be used to study these instabilities through simulations. The first half of this work aims to provide a detailed description of the numerical framework, texttt{PHORCE}. texttt{PHORCE} is a high order numerical package that can be used in solving convection-diffusion problems in neutral fluids and plasmas. Outstanding challenges exist in simulating high energy density (HED) hydrodynamics, where very large gradients exist in density, temperature, and transport coefficients (such as viscosity), and numerical instabilities arise from these region if there is no intervention. These instabilities may lead to inaccurate results or cause simulations to fail, especially for high-order numerical methods. Substantial work has been done in texttt{PHORCE} to improve its robustness in dealing with numerical instabilities. This includes the implementation and design of several high-order limiters. An novel algorithm is also proposed in this work to solve the diffusion term accurately and efficiently, which further enriches the physics that texttt{PHORCE} can investigate. The second half of this work involves rigorous benchmarks and experimentally relevant simulations of hydrodynamic instabilities. Both advection and diffusion solvers are well verified through convergence studies. Hydrodynamic and plasma models implemented are also validated against results in existing literature. Rayleigh-Taylor instability growth with experimentally relevant parameters are performed on both planar and radially converging domains. Although this work is motivated by physics in HED hydrodynamics, the emphasis is placed on numerical models that are generally applicable across a wide variety of fields and disciplines.