We did that trip last year, about this time, in our 19' boats, starting from Leamington (Ontario, Canada)
It was a good trip. Being in a smaller boat, we have to be a bit more aware of the weather, and prepared to abort or change destination if the weather is forecast to be threatening. Also, we were doing a 'guy' sail, so we were up for inclement weather. For a family, you might be more choosy about the conditions.
Going from Pelee Island to Put-in Bay, the forecast was for 20 - 25 kt and rain, and the waves were in the 3' to 5' range, but we dressed for it, we had PFDs and harnesses, we reefed the mains and used standard jib, and we had an exhilarating yet safe journey. The other days were breezy, but nicer weather.
With planning, I'm sure you can successfully make the trip.