Two ways to prevent it
Ideally, the way to prevent backflow from the tank to the toilet and spills out the vent when the boat is heeled is to spec inlet and vent fittings toward the centerline of the boat, away from the hull. When the inlet and vent fittings on the tank are toward the centerline of the boat, tank contents can't flow back toward the head or out the vent when the boat is heeled...because when heeled to one side, waste runs away from the inlet...when heeled to the other side, the vent and head discharge hoses run uphill from the tank.When the inlet fitting is toward the hull, a VENTED loop in the head discharge isn't needed, only a loop/arch in the hose that becomes a "hill" higher than waste can climb over at any angle of heel. However, it's often simpler to just install a vented loop immediately after the toilet. Unfortunately, when the tank vent fitting is toward the hull, there's not much that can be done to prevent waste from overflowing out the vent when heeled on toward the vent thru-hull.