It'll be harder to heal at your age, but it's still a very realistic possibility.
However, I always wondered how gynecomastia effected someone over 50. I would feel like at that point, you've been through the real tough years of dealing with it (teens through 20's) where your peers would be most critical or where you'd be the most self-conscious.
I'm only 24 and I decided that after 10 years, I just had to get it removed for my own mental well being. I couldn't live these years of my life after wasting an entire decade, having to hide it all the time.. not being able to enjoy summers.. hating the beach, etc.. I'm slim and my gynecomastia was the only thing really keeping me back from doing a lot of things I wanted to do in warm weather.
It's hard for me to say what it'd be like at age 56.. I feel like maybe you'd just have gotten used to it after all this time and decide that after dealing with it through the hardest times, that you don't even need the surgery.
In the end you have to do what makes you happy. But hey.. if you've gone this long with it, maybe you don't really even NEED to get it surgically removed.