Considering my terrible state of fitness, i'd hate to sit on the sidelines and denigrate any particular athlete, or group of athletes. My own granny could probably outrun me at this point.
Tennis is largely burst effort rather than sustained. Five hours of Tennis, with it's myriad sprints, rapid changes of direction and overall work rate is grueling for anyone, and punishing on the body. I've never seen Novak look particularly fatigued though, and he seems able to keep going day in, day out with barely a sign that he's ever out of breath.
There are probably people who are fitter. Rowing and Cross Country Skiing are known to have the highest demands in terms of cardio output. Pro Cycling on a grand tour is mental for the GC contenders in the mountain stages. Any of those sports is trumps for sustained output, with the likes of Chris Froome being able to throw in short bursts of greater effort, even on top of the ridiculously high level already being put in.