I don't know your situation and you did mention not really wanting to trailer anything, but I've been up and down the Delaware Bay several times and the area you are talking about and others make me think a Hobie Cat or the like would be an absolute blast. They go fast enough that even the annoying 2-4 knot current wouldn't really make a difference so timing current wouldn't be so big a deal. You said you're not really thinking about destinations just going out and sailing around (or I misread). Something at least 16' can have you outrunning flies and screaming back and forth.
Just a thought!