P.D.A. - What would you do?

I'm dating this great guy, we've been together for 5 months now. He keeps trying to hold my hand / put his arm around me when we're walking down the street. He gets really close to me when we ride the subway and I get super paranoid. I know it's ridiculous, but I'm way too anti PDA. I know he just really cares for me and wants to show affection and that's normal and if we weren't gay it'd be natural and no one would care, but I do! I know we as LGBT are supposed to not think twice about it because we have to be natural and let other people be more exposed to us, but I think i'm somewhat of a phobic gay man. Am I a jerk? What do I do?!