I hear you. To be honest, I don't blindly follow the recommendations of the EPA and similar orgs either, but I also am not a conspiracy theorist. My dad, who I trust, used to be afraid of milorganite for the reasons you listed - recently his opinion has changed. I will see what changed his mind.
In my field of veterinary medicine, new drugs are introduced constantly. I've seen how the public and industries can react to certain things, and those reactions spread like wildfire. Once the public has made up their mind on something, it's hard to change that perception.
With milorganite and other biosolids, I put my faith in the regulatory agents and hope they are doing their jobs. I can't question everything. If I were organic I would feel differently.