How does a traveler help keep the boom over the boat when you're close-hauled but still allow the boom to swing out when you're headed downwind?
Think of your mainsheet going to a fixed point on the centerline of the boat. When you sheet in, you get to a point where the boom is roughly, what, 10 or 15º off the centerline? And no matter how hard you pull the mainsheet in, the boom will not go any closer to the centerline.
If you have a traveler, you can pull the car to windward; basically move the point of attachment of the mainsheet to the boat up to windward. Now, when you set your mainsheet, the traveler pulls the boom to the centerline.
However, there is a major drawback to using just the mainsheet to control the boom angle (and thus angle of attack,) and that is the more you pull on mainsheet, the more tension you pull on the luff of the sail. The more you pull down the leech, the less twist you have aloft. Twist is kinda hard to understand for many people. At the top of the mast, the wind moves faster, because there isn't as much friction with slower air below it, until you hit the surface of the water, where the air is the slowest due to friction with the ground. Due to this, the air is actually moving at slightly different angles as you go up the mast. In light wind, this differential is more pronounced, as there isn't as much velocity to overcome that friction down lower. In higher wind, there is less differential. So, for lighter wind, it is preferred to have the boom angle over the centerline, but higher up the sail, the angle goes further off to leeward; the sails twists off to leeward as you go higher. And for stronger wind, less twist is needed aloft.
Ok, got that? Now, if you have light air, and you pull lots of leech tension, the top of the sail comes closer to the centerline of the boat, which makes that part of the sail overtrimmed. You can see this because a telltale at the leech of the top batten will blow around to the leeward side of the sail, but a telltale down lower on the sail might stream back properly, as the lower end of the sail is trimmed properly for that wind angle down lower. So, reducing leech tension would allow that top batten to twist off, allowing the top of the sail to trim for the angle of that faster wind, and then the top telltale and the bottom telltale will stream.
So, a traveler allows you to de-couple leech tension (mainsheet pulling down) and angle of attack (traveler goes side to side independently of mainsheet pulling down.) You can set the mainsheet tension for the correct amount of twist, and then move the traveler back and forth for angle of attack. You can pull the traveler to windward to get the boom on the centerline for pointing to windward, and then let it down for puffs, or if you bear off to a close reach, let the traveler down to leeward for correct trim for that point of sail.
Without a traveler, you can approximate the effect using vang sheeting. Ideally, you need either a topping lift paired with a line vang, or you need a rigid vang such as a GNAV, or a line vang paired with a Boomkicker. Sailing in light air, your boom weight is often enough to pull too much tension in the leech, and effectively de-twist the main. So, if you balance the boom between an easily adjustable topping lift and the vang, you can lift the boom to set leech tension/twist, and then just use the mainsheet to adjust angle of attack. It will not be sufficient to get the boom to the centerline for better pointing, but it will allow the main to set properly and drive as best as possible in light wind. The Boomkicker is a pair of flexible fiberglass rods under the boom that lift the boom up, and the line vang pulls down against that. A rigid GNAV either lifts or pulls the boom down to the correct leech tension. With appropriate controls, lifting up the boom, you can actually induce a bit of twist for those really light days.
Hope this helps and doesn't confuse. This book helps A LOT with sail trim. However, it doesn't discuss vang sheeting, so you need to translate traveler discussions to vang sheeting.
https://shop.sailboatowners.com/prod.php?51998
