You can say that about most shows though - it just takes good writing to still make the concept 'work'. Although S3 clearly isn't the best, still doesn't mean it can't continue to work.
Look at Dexter, for example. Just as the poor S3 made you think it was getting tired - bang - S4 comes along putting it back on course.
The whole 'Chuck' storyline for most people seems to revolve around the Chuck/Sarah will-they-wont-they, and people seem to be under the impression that once they're together, people will lose interest. I say get them together and start developing other areas of the show. There's plenty of scope there.
I'm not 100% sure thats the case. In "The Big Bang Theory", the first two series seem to be largely based around Penny and Leonard's will-they-wont-they. They got together and the show still works, you've got Sheldon stepping up to be the 'central' character.
Again, its all down to good writing. Chuck seems to suffer from some major plot flaws, and they need to start adressing that, although the budget cuts and the constant changing series (S1 was cut short, S3 was extended late on) probably don't help.