Some ideas:
I agree that the Maptech Guide to LI Sound would be very useful. If you are looking for a harbor on the CT shore as a destination on the CT shore, there a couple of great options. However, if you want to be within walking distance of your shoreside destinations, those options narrow considerably. Milford is one of the few harbors where the marinas are within walking distance of the actual town. The downtown area there is filled with restaurants, a park or two, beaches and shops. There are three marinas there, one of which caters only to transients, and offers morning coffee, newspapers, etc. The only marina I know of in New Haven is not near downtown at all. Branford has several marinas, including one in the Brewer's chain. It has a restaurant and pool along with showers, laundry, etc. It is also within walking distance of a couple of other restaurants, but you would need a ride into town (and a nice walking downtown it is). Stony Creek (near the Thimble Islands) is technically part of Branford, but is really its own little world. It is like a small Maine village. Restaurants, some antiquey-kind of shops, but mostly a place where new and old mansions are side by side with older smaller houses. Quaint with a capital K. I don't know if there are moorings or dock space available for transients, but there is certainly a lot of anchorage space there if you have a dinghy. You can take a guided boat tour of the Thimbles from the town dock in an excursion boat. Worth the money. Further east, Westbrook has a lot of dock space, and there are a bunch of restaurants and stores that have sprung up around them. Brewer's is there too. No real town though. If you want a real "Newport" style yacht-town sort of place, the choice is Essex. It is up a few miles from the mouth of the Conn. River, but it is the real deal. Great boating facilities and real New England walking town. It combines chi-chi stores with real old fashioned flinty New England stores. The Griswold Inn is there, a must stop place to eat or get a drink among the nautical decor. Also the CT River Museum.The Maptech Guide is worth the money: charts, piloting advice, lists/contact info for marinas, restauarants and services. Have fun.