Sleep paralysis happens when you're conscious but your brain hasn't recovered control of your entire body. You're still dreaming (hence the hallucinations) and things like breathing are still being regulated by your nervous system (which is why the experience of something pressing on your chest is so common, it's actually your proprioception wrestling with the fact that you aren't breathing the way you consciously want to, but instead still in the regulated and even breathing pattern of sleep).

Everyone's experience is a little different but most people can reduce the frequency simply by avoiding sleeping in the supernal position (that is sleeping on your back).

Sleep paralysis is the most plausible explanation for alien abduction and demonic visitation experiences where restraint or a weighted chest are common features. This is born out by the fact that completely separate individuals can describe extremely similar events in their sleep paralysis dreams. There is a milder form that can occur when you're awake where, while you're relatively relaxed, you'll suddenly have the distinct feeling that you're not alone even when you are. There's no supernatural aspect to this, it's just your brain not quite making the whole transition between sleep and wakefullness.