I have also had this problem in the past, and my solution was to sacrifice the rye in early spring for one spring season to kill off the grass and save the clover, which still meets the main objective of an almost year round food supply with minimal maintenance, only sacrificing the spring rye fawning cover. It usually only takes one early spring spraying at 3" growth height to get the cool season grasses under control for quite a few years. It's also possible to spray Imazethapyr 2SL and kill the grass while saving the cereal rye because cereal rye has a higher tolerance for this particular herbicide, but the application rate is very touchy, and it needs to be applied when the early growth of everything is about exactly 3" high.I’ve had really good results following this regimen. The only struggle I have is cool season grasses competing with the rye and wheat. I’m left with either spraying clethodim early and killing the cereals with the grass before they mature. Or letting the cereals mature and the clover suffering from battling the cool season grass, which then rears its ugly head again the following fall. If the rye suppresses the cool season grass, I haven’t noticed it.
Sometimes I also have trouble with warm season grasses like cattails in late summer, I like a midsummer application of clethodim to control warm season grasses.
It seems like I can't get away from 1 herbicide application per year for this plot combo, if it isn't the cool season grasses, it's the warm season grasses. I don't worry too much about broadleaf weeds, as they are often just more deer food, but even a few blades of grass get my attention very quickly, as grass can quickly partially destroy a clover plot, and even a few blades of grass left go to seed will be an explosion like a house fire the next year.
For this Rye&Ladino combo plot I usually average two mowings, one herbicide application, and one rye planting per year. If the clover is looking sparce I add a little more ladino seed right in the spinner with the rye.