Oh, MrUnix, I am so glad it's not something like that!
Turns out it was a double quote in the html that had been copy/pasted into most of the site templates. The confusing part is that the sites are actually a mash of two different applications: vBulletin forums and a content management system. And on the other axis, the code is common but the templates are different for each site. It can be difficult to tell when you jump from one to the other which makes things like this kind of a nightmare to find.
Anyway, thank you all for the good clues. It should now be fixed everywhere.