I'd like to add to the thoughtful comments already written by others in this thread.
I've got 20+ years worth of experience raising and lowering masts on a regular basis and, in my experience, the biggest risk is damaging the mast hinge/tabernacle/step when that one of the following happens: a) the mast rotates about its long axis, particularly if there is a furler that rests on the spreaders or if there's a side wind or b) the mast head moves laterally, off center line. Or both simultaneously.
If you are not going to use bridles to stabilize the mast laterally, you still need to keep the mast on centerline in the event something unexpected happens.
To keep the mast on center line, I would recommend the following:
Assign one person to hold onto a spare halyard to guide the mast laterally as it is lowered and raised.
If possible face the boat stern to the wind, avoid side winds
Do it when the water is calm. If the boat rocks sideways, the mast will rotate and/or leave the centerline.
As for preventing the mast from rotating along it's axis, the only sure way I know of to do that is with a gin pole with a bridle on the end that keeps the mast from rotating.
My current boat has a mast that's about 35 feet long, and weighs around 100 pounds including all the hardware, cables, radar reflector, instruments, lights, and halyards attached to it. The system has one bridle to keep the mast from deviating laterally off center line, and one to keep it from rotating on the long axis. We raise and lower it every time we trailer it. One person cranks the winch, the other person watches for shrouds and lines that get caught. I can do it single handed, but it's heavy lifting for me to slide the mast off the carrier and into position to pin the foot in place.