Draft is important if you aren't going to run the NJ Coast on the outside but recognize that at least 50% of the run must be done on the outside and you will probably want to run the entire thing on the outside.
Thus, the good news is that you really don't need to choose between safety or time in planning.
You have a handful of choices to duck in due to sloppy weather or seas. In order from north to south: Sandy Hook Bay/Atlantic Highlands, Shark River Inlet/Belmar, Manasquan Inlet, Barnegat Inlet, Little Egg Inlet, Absecon Inlet (Atlantic City), and Cape May Inlet. If your draft is less than 5 ft and your mast is less than 65 feet, you can run from Manasquan to Atlantic City on the inside. South of Barnegat eel grass in your strainer, thin water, and twisting channels are a challenge.
I would not try to run Barnegat or Little Egg in rough conditions (especially wind opposing tide). I would also try to avoid Shark River in rough conditions as there is a drawbridge across the inlet and there is nowhere to go once inside (other than laying over).
If you can do a 120nm day/night you can make Cape May on Day 1 (I've done Cape May to Atlantic Highlands in a straight shot, its about 18 hours), otherwise its Manasquan or Absecon for Day 1. Time the outgoing currents in Ambrose and run inside of the Ambrose to Barnegat shipping lane (but not too close to shore). Watch out for fishing vessels at Shrewsbury Rocks (off Sea Bright) and around each of the inlets. Too, be aware of near-shore dredging operations along the shore.
Layover in Cape May and time the currents in the Delaware Bay to the C&D canal.
If there is tropical activity (such as we have now with Florence) plan to have rather large but rolling seas off your port bow which may slow you down a bit. Too, you probably don't want to be travelling when we have these sustained easterly winds.
In fact, checking Buoy 44065 (NY Harbor Entrance 15 NM south of Breezy Point) right now shows 9 foot waves out of the ESE with a period of 6-8 seconds and 25 knot winds gusting to 33 knots out of the East. Only a small part of this is due to Florence and suffice to say today is not a good travel day down the NJ Coast. Off Barnegat, Buoy 44091 is reporting 11 foot waves with a similar period. Bookmarking these two buoys for quick reference (or having access to them via an app) keeps the planning time minimal and the information maximized.