The first area in Stone Caverns has a path on the left where you can go back to previous areas, such as River Town to get supplies. Then, head into the caverns by taking the path to the right.
There are multiple areas in Stone Caverns, shown below with chests marked.
One difference between Stone Caverns and other areas is that even if you stand still, you'll still get attacked by bats that swoop down and start battles, so keep moving.
At the end of Stone Caverns, you'll find the Flea Friend again, along with 3 rooms that contain chests. You need to open the chests in the correct order to open a path. The correct order is MIDDLE, RIGHT, LEFT. The path opens to the right of the Flea Friend. Head there to fight the final boss. There is one last chest in the final area.
A good strategy is timing up the giant ant and jumping at him and attacking. You're bound to get hit a few times so stock up on Health Vials/Jars before fighting him.
Enemies in this section are machete ants, scorpions, grenade ants and bats.
Bats drain your magic when they attack, unless you have the red ring equipped.