If you're a history buff, Cedar Key will not disappoint. You can even take a horse-and-buggy historical tour.

For me, the most poignant moment was visiting the old cemetery on Atsena Otie Key. People only lived into their 30's and 40's back then. The oldest individual I found there there died in his mid 60's, and his large monument celebrates his "old" age proudly. Many, many children and infants are buried there. It provides a fresh new appreciation for the pampered existence we enjoy today.
You may be buried there YOURSELF very soon, if you do not BATHE in bug repellent before exploring the island. Please trust me on this. The copperheads someone mentioned are actually quite rare this far south, but this is Florida ... so ...
Our go-to anchorage is along the beach on the inside of Atsena Otie. It is not always well protected (NOTHING is always well protected in CK) and the current reverses with the tide, but the holding is excellent. The extreme north end of the beach is constantly shoaling, but you're golden just a few yards south of the end.
Beware the boat ramp if there is a strong SE wind. You'll have to pretty much
surf your boat onto the trailer. We always launch at B's Marina in Yankeetown, motor out the Withlacoochee River (lots of manatees), and sail to CK from there. It's about 18 miles once you get out the channel.