Dont just buy 10 fish and whack them in there! You need to buy them slowly, i'd say max of 2 at a time for a tank that size.
This will give the water bacteria's time to adjust to the waste being produced all of a sudden. If you don't do this your waters amonia levels may become to high and kill your fish - thus wasting your time and money.
It's also a good idea to fill the tank up 3 or more days before buying your fish, as it takes up to 52 hours for the chlorine in tap water to be removed. I say only 3 days as they are hardy goldfish... If your going to have a filter then its a good to set it all up 7 days in advance and leave it running to cycle your tank.
Hope that helps!
Ignore the above. Most of it is wrong. (No offence Ciphon, but I'm a pretty passionate fishkeeper - and what you've posted is mostly outdated, though you're close with some of it.)
For starters, chlorine. Buy a dechorinator - any place that sells fish will sell it. It costs about a fiver for a small bottle that will do you for ages doing weekly water changes. You add a small amount when you first fill the tank and whenever you add new tapwater (when doing water changes.) Chlorine kills the bacteria that breaks down ammonia, so you need dechlor - or to leave water standing for days before you do a water change (which is much more of a pain in the arse than adding a few drops of dechlor.)
Cycling next - a much simplified explanation. Cycling is in my opinion essential. Fish poo. That composes of ammonia - which is harmful to fish. There are bacteria which break down the ammonia into nitrite (still harmful) and other bacteria breaks nitrite down into nitrate (harmful, but only in high quantities.
The bacteria that breaks the ammonia and nitrite down takes a long time to build up. It builds up in the filter media (which is precisely the reason you must have one.) There are three main ways to cycle a tank - fishless, fish in, or using mature media. Go to the forums I mentioned above for more detail - they've got stickies. Simplified breakdown here:
Fishless cycling means setting up the tank and filter, then adding ammonia daily and using a testing kit for ammonia, nitrite and nitrate to tell when the tanks cycled. Once its cycled its ready for fish. The amount of time varies, but is normally about 3-4 weeks.
Fish in cycling means setting up the tank and filter and adding a small number of fish, and letting the ammonia from their poo attract the nitrifying bacteria. To be sure that ammonia doesn't build up to unsafe levels daily water changes are recommended, though it still isn't as safe a fishless cycling.
The third option is the one I use when setting up a new tank - using mature media. Basically you find somoene with an established tank and filter, and pinch some of their filter media. Keeping it soaked in a small amount of tank water you move it across to your own filter, and voila! Instantly cycled tank ready for fish.
If you don't do one of the three above you're risking dead fish. Hardy fish may survive but personally I wouldn't (and won't) risk it when setting up a new tank. There are a lot of urban myths about fish keeping (such as just running the filter for a few days to cycle the tank - you need more than just the filter, you need ammonia too of some form - its this that attracts the nitrifying bacteria.) And I'm a firm believer in not trusting employee's of fish shops - I've been into more than I could count while I've been fishkeeping, and I've overheard a lot of bad advice. Not all is bad, but enough that I'd encourage you to verify anything they tell you.
Sorry for the rant, but hope its helpful.