Left nipple constantly gets hard?

[initialize]

Hello all,
I'm roughly 7 months post-op from my gyno surgery and overall I'm fairly happy with my results.
However, for some reason my left areola/nipple gets hard at random times throughout the day while my right areola doesn't get hard. I find this super annoying... sometimes I can tell my left nipple gets hard under my shirt when I'm just going about, doing my day-to-day activities. I guess the thing that bothers me the most is the aesthetics of it... by having one nipple hard and the other one not hard, it looks very asymmetric.
Does anyone else have this problem? I told my plastic surgeon about this and he said it could have something to do with the nerves going to my areola.