Yup, I have done that more than once and yanked them out of my ear! I have to turn down every new pair to zero to make sure now. I almost did it with this when i plugged it into my DX5 II when I was only listening with my XS before.
I did that a few weeks ago with the TRUTHEAR x Crinacle ZERO:RED that I have. Plugged them into one of the AMPs and I had to rip the things out of my ears. Even slightly over volume was too much. You do have to be careful as you say.
Similar to you I have the above IEMs which do me. Were brilliant on the holiday to the US last year, do well for walks and occasionally come out for Switch/Steam Deck gaming but that's it. Don't need more. Maybe in a few years I'll replace them with the new hotness in the budget arena.