Boundary value problems on manifolds with conical singularities or edges contain potential operators as well as trace and Green operators which play a similar role as the corresponding operators in (pseudo-differential) boundary value problems on a smooth manifold. There is then a specific asymptotic behaviour of these operators close to the singularities. We characterise potential operators in terms of actions of cone or edge pseudo-differential operators (in the neighbouring space) on densities supported by sbmanifolds which also have conical or edge singularities. As a byproduct we show the continuity of such potentials as continuous perators between cone or edge Sobolev spaces and subspaces with asymptotics.