I experienced a similar thing. It was a reason why I shaved. However, I think you just need to wash your face more often and try to stop touching your beard. The beard suits you - don't shave it!