Diaconis and Ylvisaker (1979) give necessary conditions for conjugate priors for distributions from the natural exponential family to be proper as well as to have the property of linear posterior expectation of the mean parameter of the family. Their conditions for propriety and linear posterior expectation are also sufficient if the natural parameter space is equal to the set of all d-dimensional real numbers. In this paper their results are extended to characterize when conjugate priors are proper if the natural parameter space is bounded. For the special case where the natural exponential family is through a spherical probability distribution n,we show that the proper conjugate priors can be characterized by the behavior of the moment generating function of n at the boundary of the natural parameter space, or the second-order tail behavior of n. In addition, we show that if these families are non-regular, then linear posterior expectation never holds. The results for this special case are also extended to natural exponential families through elliptical probability distributions.