Indeed. I've been "sporty" all my life. A dozen years of playing rugby. Lots of rowing. Squash and basketball were 2 massive sports in my life as well. As such, my body has probably developed a more resilient approach to exercise and probably deals with it better. However, weightlifting is a discipline in it's own sense.
I honestly believe in "**** the numbers" - as long as you improve, and you get better at doing what you do. i.e. even if your weights don't increase that much, if you're able to move them more efficiently, better form, and more reps or if it becomes "easier" - then so what? Numbers are numbers. Ultimately, although my numbers may not be as big as some, I have a tremendous amount of power, which, whilst not necessarily translated into my lifting, means that I can perform at 80% of my max with relative ease. Heck even ice said, that although he can now out lift me, he still thinks in a strongman type of event he feels I'd out perform him in some aspects (I doubt that but it's kind of him to say so).
Fundamentally, it is also down to genetics, some people will always be big boned, large mass type of individuals able to do certain things well. Some will be the same, but be totally opposite in their abilities. Some will be slim and waif like, but control phenomenal amounts of power / strength, relative to what they "look" like.
It's a bitch, but a more "alpha" male in the genetic sense, will with the same training, outperform a lesser male (in terms of genetic capacity). All things being equal in terms of diet, training intensity and so on. These are just the facts. It's not fair maybe, but that's nature.
It's up to you to define, what you're willing to achieve. I know I'll never hit 300kg on deads - I have the capacity for it I'm sure, but I don't have the will power, time, or dedication to do it. However, you may have the determination to do so and as such will achieve it - or you will also accept that there are some limits to your ability.
The world's strongest men, are all genetic freaks - sure there are artificial enhancements too - however, as a base, they are genetically superior in terms of muscle, size and ability. You can't judge yourself against someone else - it's just unfair on you.
Some people will be the strongest, some will be the brightest, some will be the more fertile... it's just the way of the world. Natural selection and so on.
As long as you give it 100% effort and you are patient and work hard you will progress. IT will be finite - make no illusions about that. However, so what? Just better yourself and keep at it.