I am currently in the middle of that project as well. It depends on what type of reefing system you have and whether the lines are brough back to the cockpit or not. The fancy system looks like this inside the boom:
View attachment 187305
View attachment 187305
I don't have the 2:1 system shown above so calculating the line length is pretty straightforward: the boom is 12 feet long, I'm allowing 6 feet to go from the aft end of the boom to the first reefing point and back, 12' to go fro the boom to the second reefing point and back. 3' from the boom to the deck and 15' to go from the mast base back to the rope clutches in the cockpit and then add in some extra - I will trim the line to the optimal length once everything is installed and tried out. I will probably also run the reefing lines to the forward reefing reefing points using some cheek blocks on the outside of the boom so you would need to add another 12' for the boom and 6/12' to get to the forward reefing point. I am thinking 3/8" diameter line.