The 23 can leak through a crack in the centerboard trunk, and that sounds like what is happening on your boat. I would try to shine a strong flashlight in hte area where you see the weeping to get a better look at the leak. It may be that yo uwill need to have this professionally repaired it you can't acces the leak yourself.
Yes, you COULD sail with the CENTERBOARD retracted all the time, but you will most likely find that you make more leeway while trying the beat to windward and steering will be less precise. The Centerboard (NOT swing-keel, it is a CENTERBOARD) provides resistance to leeway (sideslip) and improves directional control and adjusting the centerboard allows you to fine tune the balance (lee-helm vs weather helm) of the boat on various points of sail. If you do decide to eliminate the centerboard, you may reduce the resale value of the boat since most owners will want to use the centerboard. If you still wish to permanantly fix it in the up position, it would be better to remove it and then glass over the bottom of the keel to close the slot, perhaps glue/screw a piece of plywood over the CB slot in the keel so that can be reopened and hte CB reinserted if you change your mind or sell the boat.
However, I think you would be very disappointed by the performance of the boat after eliminating hte CB.