I don't think it is too late if it's a healthy tree and you don't go overboard. Some people even think it's better to prune trees in the summer, because they will make fewer water sprouts if they are pruned at that time. I've snipped around on fruit trees at all times of the year and haven't killed any yet. It's better to do a little every year rather than go hog wild and try to do too much at once.