I started sailing even later than you, when I was 30. After a few years, I bought a 30ft keel boat to race. I had some great mentors that tried to teach me everything I needed to know, but I still was in the back of the fleet.
A friend invited me to race on a two man dinghy, and I loved it. So much so, I went a bought one the next month. I just lied. I really didn't love racing the dinghy, and I still don't. What I really love about it is that i allows me to sharpen the skills I am missing since I didn't start sailing while young.
It also makes sailing easier, which translates to more stick time. I need only one other person to go race with, instead of 5. At our lake, we are lucky that the dinghy's and the big boats race on opposite days, so I race more...more practice...more stick time...more time to catch up to those that started racing earlier.
When racing the dinghy, I concentrate on the 3 controls that I have instead of the 15 on the big boat. I learn about the tactics and the strategy instead of concentrating on the crew of 4. These are the skills that make you a better sailor.
I would never give up the big boat sailing, I like the team aspect. I like sharing the experience with friends that don't sail, on the dinghy it is too hard to bring an inexperienced sailor to a race. So the answer to your question is YES. Get on a dinghy and learn to race, the skills and confidence you will carry back to the big boat will be immeasurable.