Start button -> Run... -> type "cmd" into the box (without the quotes)
Now you have a command shell open, type "ipconfig" (without the quotes)
take note of the Default Gateway address
Open a web browser and type the IP address into the address bar (for example, 192.168.1.1)
This should open up the routers built in html configuration page
If it asks for a username and password you may need to consult the manual for the default settings (if you havnt changed it that is), or try:
admin / admin
admin / password
your isp email address / your isp password
admin / [nothing]
admin / administrator
If you get it in, great! If not, you need to do a hard reset (button on the back of the router? Press it in for up to ten secs probably)