If you removed the set screws the bearing should come out easy with a punch and a hammer. I had the thru shaft out when I did mine. That may have made it earier. But there is a tool to do this.
You might check with some of the Marine supply stores and see if they carry them.
The one Joe shows in his pic is nice. That will do the job.
Robert 47