Thanks for all the feedback and suggestions. I went back to the drawing board and came up with this. sorry the bad pics, it started raining as I finished up so I was in a hurry to finish and didn't look and see how they turned out until I got home.
Coming off the main battery 1/2/B switch I am going straight into the ANL fuse block (top left in first pic). I reduced the fuse size to 100A from the 130A I was planning based on suggestions.
From there I split the output to a single switch, and to the 100A buss bar. The buss bar handles all positive loads except for the inverter, and the switch goes to the inverter. This allows me to disconnect the inverter from the battery to prevent phantom loads when it isn't being used. From the switch it is another 3 feet of wire to where I mounted the inverter. For the ground I ran it straight to the battery post.
All of the wire is 4AWG, and I also upgraded the ground wire that linked the two batteries to 4 AWG from 6 AWG.
I was also took the opportunity to shorten the cable from the main battery switch after installing the ANL fuse block, it used to go straight to the buss bar, and had about 12 extra inches of cable length.
I powered up the inverter and plugged in what is probably going to be the highest load I will ever use for any length of time, my heat gun for heat shrinking cables, and it worked great. The inverter reported that the input voltage dropped and stabilized between 11.7v -11.8v while it was on so I think that's more than acceptable given it had a 300-400w load on it, particularly since I was running off of only one battery at the time.
Last thing I need to figure out now is if I need 12 AWG wire to wire the 110v outlets, or can I safely use 14 AWG? it's $10 cheaper to run 14 AWG if I can...