Our sailing club is aging and causes a lot of concern for many of us. We are among the younger set, for sure, even at 47/46.
From what I know, many of the members joined in the 70s and sailed smaller boats- Thistles, Flying Scots, Flying Dutchman, etc. There was a very active racing program, too. As they aged, they moved on to cruising and a much more leisurely (and expensive) approach to sailing. Unfortunately, the club became more of a cruising club with little support or interest in small boat sailing. To me, that is the problem because our club doesn't seem to support the less expensive side of the sport.
The club is working on that, trying to find more small boats to add to the fleet and working on the sailing instruction program. Oddly enough, all sessions of the week long sailing camp they provide are full of kids, but few parents seem to be interested in getting involved.
I have never felt that sailing is too expensive or time intensive. For us it really is no more expensive than a two week vacation to the beach, which is an adequate trade off, when you consider that the lake is there all Spring/Summer/Fall. Granted we are close by- a half an hour from our driveway to hoisting the main. My guess is that the barrier is more the perception that sailing is work; work that I happen to greatly enjoy, but a little more complicated to learn than many people feel they have time for.