Loose connection
It certainly seems obvious that you have a bad/loose connection. Wiggling the wires may jostle the one that loose, and it may only be one wire. A good digital VOM will be a big help. That one wire may be tough to find out of a bunch. I would recommend that you take each and every wire off, and check each one. There are lots of possibilities. A crimp on connector may be loose or corroded, the screw or nut that holds it may be loose or corroded. The possibilities are many. Just keep in mind that DC electrical is not rocket science. Each wire is either a positive or a negative/ground. Usually all it takes is some thought and common sense. And usually, after you consider all the possibilities, it will be the first and most simple thing you thought of.