Letting the clover grow and go to seed, will pay dividends later on. Some people like to mow clover and have fresh new tender grow, which does attract deer. It's a 6 of 1, 1/2 dozen of another kind of thing.
If you mow and your plot has grass in it, it will sometimes let the grass over take the clover. If you've got a good thick stand of clover now, it will keep the grasses and weeds at bay.