Most likely stainless steel to aluminum corrosion. It looks like you may have hammered on the pin which has caused burrs to the end of the pin which, when you do get it to move will stop all further progress. You will need to file down the burrs and follow with emory cloth to get a smooth surface. You may need to try alternating heat and cold to see if it will break free. Be sure to not overheat the aluminum quadrant. A suggestion I got from Edson was to drill a small hole vertically down from the top to the through pin and soak, soak, soak in a solvent of your choice (PB Blaster, brake fluid and transmission fluid, or whatever you think will work best. There are threads on the "best" fluid to use. When you do go to drive the pin out, do not strike directly onto the end of the pin. This will only peene over the metal. For mine, the through pin was accessible from the top and we cut a section of it out and used a kind of press that pushed apart the pin at the cut coupled with mild heat onto the radial. However, it looks like you have a fiberglass rudder shaft and the quadrant is below decks and there is no access to be able to cut the pin.
For the bolt that is shown sticking up through the quadrant arm (I am assuming a autopilot drive) that looks a lot like a ball joint on a car. Maybe an auto parts store will have a ball joint removal tool you can use. Just spitballing it here.
Best of luck and patience will be necessary.