I have some thoughts I need to get out of my head. First, I don't think anybody that's written in response to the initial post has had an "ah" moment. We all seem to be coming at the question from our own unique perspectives (and that's being kind).
The original question was, "Does soil quality affect deer size?" It's a simple question, but one that begs more development. Is it a question of a direct and complete correlation and causation?
I think the answer is a resounding NO! It sure helps but it isn't necessary to grow a (single?) big deer. Maybe the answer is a resounding YES if you want to grow a lot of big deer. I think Ben had it right in his post above.
It isn't the soil quality, it's the quality and quantity of the potential food sources that soil productivity supports. Nobody argues that point. And that leads to the subject of genetics. It's mentioned in a lot of posts above. I thought that had been demystified some time ago. Whitetail genetics are whitetail genetics. Sometimes those characteristics are suppressed by any number of environmental stress. Quality and quantity (in relation to herd size?) of food, cold stress, social stress, heat stress, financial inequality, car - deer collision stress, and how many others can you think of?
If you really want deer to get big, just leave them alone! You know what I mean. Change the original question to Does not killing deer lead to bigger deer?
Here's my last thought - and I'll leave the door open to argument - If, somehow, we were able to study every square mile of whitetail habitat, capture every deer on each of those square miles, and weigh them - or measure antlers, or capture shoe size or some other measurement to determine deer size, we could then create a list of the elements the environments have in common. High soil "quality" might be one of the factors. We could even give it a weight. Perhaps it accounts for 40% or 10% of all the reasons. I don't think we've answered that question.
Now, with that list in hand, your hand, I would contend you could not recreate those conditions even remotely. You might be able to mimic three out of 10 or maybe four out of ten, but not all of them. For that, you need to go where it happens. What I just asserted applies to almost everyone, but for a few. Those folks are the pros. You'll find a number of them here.