I have used sheet plastic cutting board material for multiple purposes on multiple boats including tiller shims.I've had success making shims out of a plastic kitchen cutting board. They come in a variety of thicknesses and tend to be durable. My Cal 29 had a squeak in the mast step I quieted this way.
Looks to me the bracket attached to the handle is moving and the bolt is not. Make me think the holes are enlarged but could be the bolt has worn too. As previously mentioned could go up to a larger bolt. (would need to drill holes larger). If the holes in the bracket that attaches to the tiller are worn, drill them larger and add a bushing or weld them smaller and drill it the right size again, then and it should be good again.