The First item we need to have you explain is the three letters 'HEP', do you mean High Energy Physics or something else?
I think your questions refer to the original "Nuclear Shell Model" found here at CERN: http://home.cern/about/updates/2013/06/cerns-isoltrap-reveals-new-magic-atomic-nucleus If not it is a great asset to find someone to answer your questions directly.
It is a kind of "jargon" used In the elementary particle physics. 'On-shell' and 'off-shell' expressions are identical to the 'on-mass-shell' and 'off-mass-shell' expressions.
When we say 'on-shell' or 'on-mass-shell' it means that (for a particle under discussion) the particle energy (E) and the particle 3-momentum (p) obey to the famous relation E^2-p^2=m^2 (in units when velocities are measured in units of the speed of light in vacuum (c), i.e. c=1). Here 'm' is the mass of this particle (given in the Particle Properties Tables). When a particle is on-shell, it is a real detectable particle.
When a particle is off-shell, it is a virtual particle which cannot be detected by any real detector.