When I used to shave regularly I would (very often) get some ingrown hairs (I was shaving against the grain to get that really close shave with pretty much 0 stubble) , I´m not exagerating by saying that 20-30% of my beard hairs were ingrown hairs, that was bothering me very much and it hurted quite a lot too, so many red-pink bumps and I was not helping myself as I was always picking or plucking the hairs, then I decided to change the way I was shaving by adding shaving cream and shaving with the grain and those ingrown hairs pretty much disappeared. Nowadays I very rarely get one ( I have one particular ingrown hair that has been growing abnormally for many months). Your case seems pretty uncomfortable, I would go see a dermatologist if I were you.

also my skin is very oily and unlike most guys here, it is not sensitive.