Yep, go with a small one.
We have a 6hp Suzuki. It is about 20 years old, and it still runs great. I have it mounted on the stern rail. It is a bit heavy lowering it onto the dinghy, so I have a block setup on the boom. Doing it usually require both my wife and I to get the job done. A real pain.I would prefer to go with something smaller, but I really don't want to spend any additional money if I don't have to. The Suzuki is a bit heavy, needs an external tank, and the only time I use it is going from the anchorage to the shore, usually only for 2 people. Because it is a pain to lower and raise the outboard to/from the dinghy, we often don't even put it on the dinghy and just put up with rowing. Sometimes, it can be a long way from the anchorage to shore or marina. Especiall when the wind kicks up.Also, our current 2.5 gal external fuel tank would often last us all summer, sometimes even 2 summers. Then we will have to find a way to dispose of the dead gas.I think something along 2-3 hp unit with an internal tank would be ideal. Easy to lift and stow, and 2-3 knots is plenty fast.