i'm happy with my eye sight at the moment and don't want to make it seem bad in comparison.
You're happy because your eyes deteriorate slowly and so you don't notice the gradual decline. This is why when you start wearing them you'll notice it wasn't as good as you thought it was as others have said
if i don't get glasses soon, or do get them but rarely wear them, is there potential my sight could get worse because of not wearing them?
Your eyes will fluctuate in their performance over time regardless of whether you wear them or not. Generally age brings on long sightedness. If your eyes were generally fine then you this normally manifests itself into needing reading glasses and if you're short sighted then it ends up needing bifocals or a combination of glasses for different activities. As a child I'd always hoped that age would correct my short sightedness but it doesn't work that way.
Get them and enjoy better vision.
