My thoughts... If the rub rail is sitting in a grove not physically attached in position then the rub rail is not designed to be sealed. It is meant to let water flow through and out. The deck to hull joint is where the sealant is located. The rub rail is meant to protect the hull to deck joint. You do not want to seal water in under the rub rail where it can find it's way through the deck hull joint.
That said, if you have screws or thru bolts that secure the rub to the deck then use of buytl or something like 3M's 4000/4200 are good to stop water from around the screw/bolt hole.
If you find that the place where the PO used sealant has or had a leak into your cabin, then the issue is more likely the screw/bolt holes or damage occurred to the deck/hull joint in that area and sealant on the rub rail would be like putting a band-aid on your shin when you have a cut on your arm. You will need to remove the rub rail and examine the site for the cause not the symptom.
Best of luck and let us know what you find.