I am really shocked that anyone thinks it's acceptable to have a loud party until 4am ever! :O
Weekdays 10pm (but do consider neighbours with young children)
Friday / Saturday: Midnight at the latest
Super special occasion - speak to the neighbours first, if you tell them an exact time that the noise will end, they are likely to be far more accomodating. There is nothing worse than lying there not knowing when it's going to end. Make sure if you do this, you stick absolutely to the time you set! Even on a really special occasion anything beyond 1am is starting to take the ****.
Even when I was a student and used to have large parties at my place, I'd always make sure I let the neighbours know first, told them the very latest I would make any noise outside and stuck to it. No matter how well the party was going, I still insisted that we took it on to a club etc if it went on beyond 11pm. If I had a smaller group indoors, music and talking kept low enough not to disturb. We often drank through to the early hours (and had plenty of fun doing so

) but kept it quiet enough not to **** anyone off.
Although the neighbours were an old lady in her late 80s and a family with young children both quite noise averse, many years later they both commented on how grateful they were for the way I considered them in my student days.
Britboy - It's entirely in your control to be a decent neighbour. It doesn't need to compromise your fun at all, but if anyone should be inconvenienced (expense of taking it to a bar, hiring out a venue, taking it indoors and cutting out the music) then it should be you and not your neighbours losing their sleep!