First for me, protection. You can get that with geography and/or a benign forecast. Second, beauty,...close second.
That’s an excellent point about forecasts. Some of my favorites are “good” only in the prevailing westerlies of the west coast here, if there is to be much wind. Toward late fall and into winter, they are not so good because of the risk of strong easterly winds arising. They are still scenic, of course, but might no longer provide the shelter. Puts one a little on edge.
All weather anchorages which also offer solitude AND high scenic quality are basically nowhere to be found in Southern California. That’s because the all weather anchorages are found only in harbors, so are what I call urban anchorages, and offer virtually no solitude as they are not remote. Many power boats around with all that brings.
The one possible exception is Cat Harbor on the south coast (“back side”) of Catalina Island. It’s scenic, well sheltered if deep inside the harbor, and remote to a certain extent in that it’s the most distant from our coastal cities, minimally 35 n.mi., so it’s not a good “weekend destination” for many boaters. It’s also a bit rougher out there rounding the (north)west end of the Island than going straight to destinations on the north coast. There is a dinghy dock and services in the village of Two Harbors, including a propane fill station and outboard engine repair shop if you wish to lug stuff that far. Better have a cart! No water or fuel at Cat Harbor. Actually, a great “off season” destination when there are the fewest visitors.

But do not go there on a 3-day summertime weekend of settled weather (e.g. Labor Day) if you wish solitude.
The best and closest destination for solitude then would be Santa Barbara Island @ Landing Cove, about 45 n.mi. from San Pedro. Another 20 n.mi. from the west end of Catalina Island. Our first time there, Labor Day weekend in 2004, was a beautiful weekend. There were altogether 9 boats in the anchorage over the whole time! Imagine, only 45 to 50 n.mi. from a coastline of millions of people and literally thousands of yachts on a gorgeous 3-day summer weekend and only 8 other yachts besides us in the very roomy anchorage! Amazing!