Trees are different - people are different. Look at a family of six siblings - they will have similarities but they will be different.
I am collecting under 45 trees and I do my best to mix up orders so that the 52 I put in the box has diversity. Now I have tubs of chestnuts and my diversity will vary based upon my supply but I know the best insurance for nature is a diverse landscape.
I specifically went today to my 30 tree grove and collected from the late droppers.
Now to Your Yard:
Feed the deer with both sources of chestnuts. A whitetail deer will eat nuts we will not eat.
Collect under the trees separately. Name the trees and do not mix those chestnuts together. How many do you wish to grow for yourself?
If the answer is you want to grow 20 trees - then keep 30 good chestnuts from each tree. Cherry pick them - size is not everything. Firmness and when they drop matters. I hunt far and wide for late dropping trees. I have two trees holding 90% of their chestnut burs right now. I have to knock on the door before I collect - I knocked three times today - they are out of town.
If you follow what I say - you have 30 chestnuts bagged from one tree (L for large) and 30 chestnuts in a separate bag from (S for small).
If you have weevils - your 30 is insurance against losses.
Chestnuts planted by wife's grandfather all have different sizes and colors. They drop at different times - my father-in-law can almost pick the tree it came from. Those characteristics / genetics exist. It is just how the world works. I say use it to your advantage.
My son purchased the wife's grandfathers home place about one month ago. Those trees have weevils bad. My son will develop that land and those trees will be cut down. I know this and I am hitting those trees selecting the best chestnuts to pick from. I have going to get trees from that line of genetics. I will plant them on our hunting farm. There is one tree in that group that has closed burs when they drop - they are big - but slow to open.
Sorry for the long response. Collect 30 good nuts from each and store them separately. Later - grow them separately and planted them in a mixing setting. Diversity is great in nature!