Thinking is good, thats what leads to knowledge. Perhaps. First indeed a mature doe will have higher survival rate of her fawns in theory as she has the better bedding and feeding areas in the pecking order of doehood. In theory.
Second, studies show thru DNA testing that sub bucks contribute to breeding as much if not more than mature bucks.
And yes it should hold true that early/normal dropping times would encourage a greater success of fawn survival. My example of the young fawn and mother would make one think they are less equiped to survive a harsh winter. But that is to assume a natural predator or man does not end the life of the mature ones early on. So lot of factors going on.
Now, as for the gene pool. Obviously two contributors, the doe and buck, both having say in the socalled best genes. And one can't assume that a mature buck has a better gene donation. Even a young suborbinate buck or doe my have in its package a prime group of genes that could lead to a new world record, that is if we are looking at racks as being a dominate gene trait. It's no different than someone like me giving offspring that are highly intelligent or good looking. Neither of those are my shown traits but could exist within my gene pool, or the mothers.
So while " normal " breeding and birthing times may improve results, so many other factors enter within the habitat, its hard to control. The magnificent 10 pointer I killed with my Jeep last year didn't have much an advantage with his gene pool.
