Pardon my ignorance, I'm not very clued up on Internet connections, but is the 96ms the time it takes for you to contact their server, or the time it takes for you to contact their server and for their server to send a reply to you?
Yup - time for the "round trip". Time between your request and receiving a response.
http://en.wikipedia.org/wiki/Ping