Unsure to accept or turning down a management position in public sector

Has anyone here turned down any managerial positions before?

What are your reasons?

Personally afraid that I’m unable to handle the team as most co-workers are more senior than me…