That's probably the case. Remember if you find a smaller one you can topwork it to whatever mulberry that you want. Below is a picture of one I topworked to an Illinois Everbearing this spring - and YES, it produced mulberries off the scion wood.
It would be for me with a tree that large. You could try maybe doing some of the lower limbs, but I doubt you would be satisfied with the results. Personally, I would just buy a couple of Illinois Everbearing trees from a nursery and set them out the same as you would an apple tree. It won't be many years until you will have great fruit.
I was looking to get a couple of those Mulberry trees. Im in Virginia , but a tree nursery that I trust near me said that zones 7 and 8 are experiencing " popcorn disease " and they recommend not planting them . Im not sure if you are in that planting zone but just FYI .