When I got a degree in mechanical engineering, I took one semester of structural engineering, and two of fluid mechanics. They included all the tools to understand wave mechanics, and the design of hull shapes, air flow/wings/sails, and beams (mast, booms, and rigging.) Those courses required calculus - another two semesters. Ohio State offers free auditing of all those courses for those of us over 60. I bet they would consider allowing you to audit them for free for your avocational purpose. So I would ask engineering universities in areas that interest you what are the chances of auditing such courses.
Application of the thought tools you would learn in those courses to the nautical application could happen in the lab section of the courses, or you could take a job as an assistant in a design or research operation. My university had wind tunnels and a wave lab I could use while taking fluid mechanics. At that time, I ran a small, well instrumented wind tunnel to understand how wings ( that is sails) work. In fact, “trimming a sail” is a continuous effort in redesigning a flexible wing.
My Professor/Tutor, Peter P. Wegener, wrote an excellent book called “What Makes Airplanes Fly?” that clearly explains the aerodynamics of wings/sails for non-engineers. As a note: rudders, keels, and sails are all wings, and operate on the same principles.
Finally, I am a life long learner. I encourage you in your pursuit of the fluid and structural mechanics of sailing. What I have learned so far adds immensely to my joy of sailing our boat!