It's not panic he feels. Nothing like panic. Because any situation he can say as being panic-inducing is running from angry soldiers. Or running from angry women. Or running from angry merchants that got confused over the idea that Jack should pay them for their goods. So this can't be panic. He's not running. There's not an angry someone chasing him. It just kind of feels like he should be, but he doesn't know what exactly he feels the urge to run from.

Or why he thinks maybe he needs to stay this time.

There's a loud scraping sound from just behind. Like a giant coconut falling from the tree. A really giant coconut. That makes oofing sounds. He turns around just in time to see a crumpled Norrington at the base of the tree. It's a slightly comical image. Jack didn't know a man could get his knee that close to his nose. He watches with what he wants to be a smirk on his face as Norrington rights himself, prepared to crack some weak joke at the mess he's made of his limbs.

Then shuts himself up at what comes out of the man's mouth.

Bugger. Jack looks wildly about at all the coconuts at his feet and decides he really does not want to answer that seriously. So he doesn't.

"Hm?" Wide smile. "Don't want to be infringing on your territory, mate. Stretching necks and all. Navy's idea of a good time. Me, I prefer pillaging."

There we are. Just avoid. As good a plan as any.