Agree with hp...
I was in the same situation last fall as I was crossing the mouth of a large bay to starboard in my 26 footer. Tons of water, little traffic. A larger, faster sailboat came from astern, came along on my port side (no more than 8' separating us). He slowly pulled ahead, and when his stern was barely clear of my bow (about 3 or 4'), turned to starboard toward the bay. I turned to port, not liking the "direct hit, T-bone" position. It was too close, and there was no need, as he easily could have ducked us. Aside from wanting to follow and pummel the guy, a debate broke out among the "racers" I had as guests that day. I went to the local Coast Guard to ask. They said he was an overtaking vessel, and if they had seen this occur, they would have fined him ($500, I think). Some seem confused between The Racing Rules and COLREGS, which differ. Here's the governing COLREG: Rule 13Overtaking(a) Notwithstanding anything contained in the Rules of Part B, Sections I and II, any vessel overtaking any other shall keep out of the way of the vessel being overtaken.(b) A vessel shall be deemed to be overtaking when coming up with a another vessel from a direction more than 22.5 degrees abaft her beam, that is, in such a position with reference to the vessel she is overtaking, that at night she would be able to see only the sternlight of that vessel but neither of her sidelights.(c) When a vessel is in any doubt as to whether she is overtaking another, she shall assume that this is the case and act accordingly.(d) Any subsequent alteration of the bearing between the two vessels shall not make the overtaking vessel a crossing vessel within the meaning of these Rules or relieve her of the duty of keeping clear of the overtaken vessel until she is finally past and clear. As others have stated, being "right" is of little comfort if a collision occurs, so avoiding one is "THE" rule.