Low mass, self-gravitating accretion disks admit quasi-steady, "gravito-turbulent" states in which cooling balances turbulent viscous heating. However, numerical simulations show that gravito-turbulence cannot be sustained beyond dynamical timescales when the cooling rate or corresponding turbulent viscosity is too large. The result is disk fragmentation. We motivate and quantify an interpretation of disk fragmentation as the inability to maintain gravito-turbulence due to formal secondary instabilities driven by: (1) cooling, which reduces pressure support; and/or (2) viscosity, which reduces rotational support. We analyze the axisymmetric gravitational stability of viscous, non-adiabatic accretion disks with internal heating, external irradiation, and cooling in the shearing box approximation. We consider parameterized cooling functions in 2D and 3D disks, as well as radiative diffusion in 3D. We show that generally there is no critical cooling rate/viscosity below which the disk is formally stable, although interesting limits appear for unstable modes with lengthscales on the order of the disk thickness. We apply this new linear theory to protoplanetary disks subject to gravito-turbulence modeled as an effective viscosity, and cooling regulated by dust opacity. We find that viscosity renders the disk beyond similar to 60 au dynamically unstable on radial lengthscales a few times the local disk thickness. This is coincident with the empirical condition for disk fragmentation based on a maximum sustainable stress. We suggest turbulent stresses can play an active role in realistic disk fragmentation by removing rotational stabilization against self-gravity, and that the observed transition in behavior from gravito-turbulent to fragmenting may reflect instability of the gravito-turbulent state itself.