I strongly suspect you have at least that part of the story right.
Well, it could have been pirates. After sailing to Jamaica, the previous owner had to fight off pirates who ended up taking his boat and made him walk the plank. The plank was made from the port side seatback. Thinking they were done with him, they tossed the useless seatback overboard and sailed off to Port Royal. They then had their new prize outfitted with cannons to improve their fleet. This meant removing the starboard seatback as well.
Unbeknownst to the hapless pirates, the PO was a retired marine. He floated around in the Caribbean for half a day before bumping into his port side seatback. He grabbed onto it just in-time to defend himself from a hungry 14' bull shark. He smacked the shark across that blunt snout with the seatback them paddled it like a surfer until her drifted to shore on a deserted stretch of beach. Carrying the lifesaving seatback with him, he made his way through the jungle. Again that seatback served him well when he used it as a shield to stop the poison darts flung at him from the headhunter's trip-wired trap. He then used it to bridge a small crevass and escaped the cannibal territory. That's when he saw his hijacked boat tied to a quay surrounded by a pirate camp.
He waited until dark and snuck past a group of drunken pirates singing "A pirates life for me" and bidding on captured booty. He quietly dispatched the pirate watch aboard his boat and made it to the barrier reef just when they spotted him and fired their shore batteries to try and stop him. They would have succeeded too, if he hadn't still had that seatback and had propped it up on the stern rail while navigating the tretcherous reef lined pass to open ocean. That seatback deflected an otherwise deadly shot from a 4 lb ball.
When he managed to land in Key West, he found nearly a dozen bullets lodged in that piece of plywood. He replace the port seatback after repairing it, of course, but there was no cherry plywood to replace the starboard one. He sold the cannons and saved the money by using a standard balsam plywood on the starboard side. Then he sold the boat and moved to the mountains.
- Will (Dragonfly)