When on a swift motor vessel, bypassing (well) ahead of a sailboat, you are passing him in the one place he can't possibly hit you from! Any other place and it is possible that the driver of the sailboat could, put himself in harm's way. And really I could care less if you want to surf my wake.
I find that ignorance especially apparent in the ditch. I used to deliver 6 gold plated sport fishermen up and down the coast every year, spring and fall Fla to NY or north. Most had the power to easily cruise at 35 knots, and outside the no wake zones (very few and far between back then) perfectly legal. Then I'll run up on a sailboat dawdling along at 5 knots. From a mile or so back I'll give the proper whistle signals and slowly decrease speed until I'm doing around 5 knots when I get to his stern. That's about as slow as I can go without using the trolling knobs, or bumping one engine or the other in and out of gear (really hard on those sorts of engines) and minimum wake.
Normally, not even a glance back, but if so, it's usually rather hate-filled. "Gee dude, it's not my fault I can make it from Miami to Norfolk in 4 days, sorry." Anyway, on he continues at 5 knots and there I am stuck behind him because I don't want to be rude and speed up and make a wake! 5 minutes, 10 and now I'm getting a bit frustrated. I call on the radio; nada.
Finally, if he won't slow down to make this more pleasant for himself, I've gotta speed up and get on my way. I pull as far away from him as I can get in the ditch and run her up to about 7.5 or 8 knots, obviously throwing A LOT more wake. He gets thrown around like a cork and most often my patience and attempt at courtesy is met with a middle finger, unbeknownst to him all at his fault. Had he slowed to idle for just a minute I could have slipped by at minimum wake and we all would have been happy. And I'm sure it resulted in another "rude motorboat captain" story in the next anchorage or marina. By late afternoon, it's really hard to slow down at all, because I know I'll be met with the same vitriol as I was by the first guy, so might just as well go by at speed.
I have encountered people like the OP several times over my career and sadly they rarely realize in time that I won't risk the lives of my passengers or the wellbeing of the vessel I'm commanding, so things usually turn out very poorly for them in a confined channel.
No matter what the rules say, common sense and a tiny bit of courtesy go a long way to making it so we can all use the water without incidents as posted above.