We present spectrophotometry spanning 1-5 mu m of 51 Eridani b, a 2-10 M-Jup planet discovered by the Gemini Planet Imager Exoplanet Survey. In this study, we present new K1 (1.90-2.19 mu m) and K2 (2.10-2.40 mu m) spectra taken with the Gemini Planet Imager as well as an updated L-P (3.76 mu m) and new M-S (4.67 mu m) photometry from the NIRC2 Narrow camera. The new data were combined with J (1.13-1.35 mu m) and H (1.50-1.80 mu m) spectra from the discovery epoch with the goal of better characterizing the planet properties. The 51 Eri b photometry is redder than field brown dwarfs as well as known young T-dwarfs with similar spectral type (between T4 and T8), and we propose that 51 Eri b might be in the process of undergoing the transition from L-type to T-type. We used two complementary atmosphere model grids including either deep iron/silicate clouds or sulfide/salt clouds in the photosphere, spanning a range of cloud properties, including fully cloudy, cloud-free, and patchy/intermediate-opacity clouds. The model fits suggest that 51 Eri. b has an effective temperature ranging between 605 and 737 K, a solar metallicity, and a surface gravity of log(g) = 3.5-4.0 dex, and the atmosphere requires a patchy cloud atmosphere to model the spectral energy distribution (SED). From the model atmospheres, we infer a luminosity for the planet of -5.83 to -5.93 (logL/L circle dot),leaving 51 Eri b in the unique position of being one of the only directly imaged planets consistent with having formed via a cold-start scenario. Comparisons of the planet SED against warm-start models indicate that the planet luminosity is best reproduced by a planet formed via core accretion with a core mass between 15 and 127 M-circle plus.