I've been using a 1-1/2" wide very sharp wood chisel to remove vinyl and paint from gel coat. There is no reason why it wouldn't take the barnacle and any residue right off. I know that most would be horrified, but I can testify that it is almost impossible to set it on edge in a manner that would gouge the gel coat. I can put full pressure on it and it simply glides right across the gelcoat and removes all foreign substance with the least amount of effort. It takes adhesive off pretty effectively and lately, I've found that it is my last resort in shaving off that final thin layer of acrylic polyglo that stubbornly resists ALL chemical application. Try it gingerly if you must. I've found that you would have to be very uncoordinated to actually turn that blade so that it gouges.
BTW, I've never found a plastic scraper that can remove anything effectively. The plastic "razor blades" are an absolute joke and a complete waste of time. Don't use thin razor blades, they do gouge very easily. The secret is that a wood chisel is extremely stiff and you would have to purposely set it on edge so that it gouges. Set flat, the blade will not penetrate the gelcoat while at the same time it gets right under any foreign substance. You'd have to set it at a very high angle to actually penetrate the gelcoat, and then it isn't effective for scraping either. It's pretty easy to get the exact right angle and it is non-damaging to the gel. The irony is that I would never actually use it on wood, unless I intended to chisel the wood! It's the only way I was able to get a painted boot stripe off without damaging the underlying gel coat. With that aggressive use, I do have to aggressively polish the gelcoat to get the color of the boot stripe restored, but I had no gouges to deal with. (I have no idea why the P.O. painted the boot stripe - he must have thought a darker blue would look better, but he only made a mess that had to be removed).