When I refinished our mahogany rudder, I used epoxy, thinned maybe 10% with acetone, for the 1st coat. The second coat was straight epoxy, and the top finish was 2 coats of Epiphanes 2-part clear finish. That's been waterproof for 5 years.
Caveat - we trailer-sail, though the rudder has been immersed for up to a week continuously at times.
I think both polyester and epoxy give good results for boat structural work. I will say that I think epoxy is a bit more forgiving for a newbie to work with, and a bit less dangerous/toxic.