Hi, I'm in your situation. I sailed smaller boats - 16' Lightnings and small cats as a teenager, but haven't been in a boat for 30 years up until I went sailing with a friend on a 27' Hunter and got the bug. This spring I took a 3-day sailing course (let me know if you want the name of the school) which was terrific. We spent 3 days on the water, 6-8 hours each day, and it was worthwhile. I am now sailing a borrowed 22' Catalina single-handed on the Chesapeake and have a great time. Looking to buy a 26' - 28' something in the late fall or early spring.
Sounds like you've had more experience that I did going in, so you should be fine. Others have said that a larger boat will be more forgiving and, in some respects, easier to handle, and I would agree with that.
Good luck, and great sailing!