There a number of “Long Coves” in that area, but it seems the most likely one is the one near New Harbor - the other “Long Coves” I am familiar with either dry at low tide, or do not have enough water at high tide to support sailboats.
If it does, in fact, turn out to be the “Long Cove” near New Harbor, I have a few suggestions for places to buy or eat lobster. All have suitable anchorages right there.
In New Harbor itself (a very colorful / picturesque Maine fishing village whose picture could - and probably has - graced the cover of some magazine), there is Shaw’s Fish & Lobster Wharf. Shaw’s is a very popular eating and watering hole in that area. It was where part of Kevin Costner’s movie, Message in a Bottle, was filmed. There is indoor seating as well as a second-floor deck to watch the harbor action. The menu goes from lobster to burgers and many things in between.
Up the coast (North-ish) from New Harbor, is the harbor at Round Pond, another spot that looks very “Maine-like”. In the harbor at Round Pond is the Anchor Inn - an all-inside seating restaurant with a much-expanded menu over the various outdoor lobster pounds. Next door to the Anchor Inn, are the outdoor-seating-only Muscongus Bay Lobster Company and the outdoor-seating-only Round Pond Lobstermen's Co-op. In order of menu variety, it goes from the Co-Op (the simplest, lobsters and clams), then Muscongus Bay Lobster (adds on shrimp, crab cakes), and finally, the Anchor Inn (adds on sword fish and other fishes, steaks, pasta, et cetera). All three are right there together, all basically touching each other. Right up the road, about a five-minute walk, is the local ice cream spot. Every Maine lobster needs a little Maine ice cream to go along with it - or at least, that’s how our trips to Round Pond go.
Still along the same shoreline, but even a little further North is BCMS (Broad Cove Marine Services). Although their name says “Broad Cove”, they are not actually in Broad Cove, but south of the cove on the Hockomock Channel between the mainland (the village of Medomak) and Bremen Long Island. They are a full service marina that also has a lobster-cooking shed right on the dock. You can eat there (lobsters are the only thing on the menu) or purchase cooked or live lobsters to take with you.
There are many great anchorages in that part of the world waiting to be discovered. You may even see some of the Outward Bound sailing crowd doing their thing - we have run into them numerous times on Muscongus Bay.
Enjoy your trip!