health info

Any reason not to sign an authorization for the disclosure of personal health information when applying for life insurance?

I 50, good health, in good condition, so I had a cancer scare a few years ago, but it turned out he did not. I'm thinking of applying for a policy whole new life to replace another that I have. They are asking me to sign this form and explain to all my doctors to release all my records. Does this is normal and necessary in all cases? What is the downside? How do I know that the agent did not disclose the information inappropriately?