Have you tried fiddling with pipelining and page rendering delay in about:config?
network.http.pipelining true
network.http.proxy.pipelining true
network.http.pipelining.maxrequests 30
nglayout.initialpaint.delay 0
To be honest I haven't personally noticed much difference with these settings, and in fact all the main browsers seem about equally speedy to me for all practical purposes. I think the breadth and quality of FF's add-ons are pretty much its USP, and if you have no use for them then it doesn't really offer anything worthwhile over the other contenders.