Salesforce Chat button showing offline when agents are online

1. Check the below article.

https://help.salesforce.com/articleView?id=000316800&language=en_US&type=1&mode=1

2. Make sure the user is Service Cloud user.

3. Make sure the user is Chat User.

4. Also, use the Supervisor tab(very handy) to check the Agent's capacity and availability.