Depends.
When there is enough power in the wind or diesel, there is no appreciable decrease in boat speed. For us, in winds above ~12 knots we are already close to hull speed and the dinghy has little or no effect, below 12 knots we may lose up to a half knot or so.
When motoring, we have to increase the power slightly to maintain speed.
Where you tow the dinghy makes a difference. When we can, we try to tow the dinghy on top of the second stern wave. If it is in front of the stern wave, the boat tends to surf down the wave, behind the wave we are dragging it up hill.
The size and type of the dinghy also matters. We have an 8' RIB that tows very nicely and in a straight line, larger heavier boats will require more energy (wind or diesel) to tow. Taking the motor off helps a lot, on our boat it reduces the weight by about 40%. Dinghies that do not track well will slow the boat down more, as they will veer around behind the boat.