There's been some mixed advice in the thread so far. I currently race in single seaters (FF2000) but raced in various kart series before that. I'll try and break down your options, but ultimately it's going to be your budget that determines the best way to go!
Karting:
Option 1. Buy a cheap TKM kart to have some fun at test days
This is probably the cheapest option, a decent kart can cost as little as £500and a test day around £40 entry for the whole day. Add fuel + whatever you break and that's all it'll cost. The kart will fit in the back of a large hatchback or estate or on a roof rack of even a fiesta. A small trailer will only cost £100 or so if you have a tow bar.
You will however need some mechanical knowledge yourself. They're very simple machines, but if you're starting from zero it can be a little daunting.
The karts are extremely quick though and an absolute blast to drive - a whole world apart from your usual 4 stroke rental karts.
Option 2: Club100 - Arrive and Drive 2 stoke karting
This is probably going to be your best option to get into racing. I started here - it's been running since 1994 and is still a superb series. They use the same 115cc TKM 2 stroke motor as in option 1 and a slightly stronger Birel chassis. It caters for everyone from the complete novice through to top level by offering different championships based on experience/ability. It's absolutely the best place to learn race craft and the best bit is, unlike all the other options it doesn't cost you any extra if you get it wrong!
Start off with a test day, then you have a choice of sprint or endurance racing. It's around £160 a race which considering you just get to turn up and drive is excellent value. Even if you have a whole load of money burning a hole in your pocket - I'd still say a season of club100 first would be invaluable because the racing is so competitive you will learn quicker and more than in anything else.
Option 3. Club karting
If you've got a kart circuit local to you, you could join one of the MSA club series. There are various classes (TKM, Rotax, Formula Blue etc) and the one to go for would depend on the grid sizes at the tracks near you. The problem is that although it can be done very cheaply, as soon as you want to be competitive you can literally spend any amount trying to keep up with the lads with rich parents! Once you get under the MSA umbrella too, you also have a number of extra costs - ie MSA approved race kit, helmet, ARKS test, medical, license etc.
You mentioned that you thought that karting was the most dangerous options? I'd disagree, karting is actually pretty safe. In 10+ years of karting the worst injury I suffered was a broken rib despite many spectacular crashes! Ironically the broken rib came from driving a kart with sticky tyres in an ill fitting seat - not from an accident!
Track days can be a bit worrying at times as you have very mixed abilities on track often in normal road cars without the proper safety equipment (roll cage, 5 point harness etc). Single seaters actually have very strong chassis and not much momentum (low weight) so you can usually walk away from huge accidents. There's always going to be a level of risk associated with all of the options though, only you can decide what you are ok with.
Cars:
Option 4. Track days
Track days are extremely popular these days. I can understand why - buy a cheap car, strip it out and have a laugh chucking it around some circuits. It will be more expensive than running a kart though - tyres, brakes, fuel etc. Unless you spend mega money it won't feel anything like as fast or alive as a kart either. It's a great way to get a taste of circuit driving, but after a while track day rules, the mixed abilities and the limitations of a road car are either going to lead you down the route of spending £1000s modifying the car or a little frustrated.
Option 5. Circuit racing.
There are loads of different club racing series out there that are relatively cost effective but great fun. Mx5s, BMW Compact Cup, Formula Ford 1600, Formula Vee etc just to name a few. The problem is that it's never going to be cheap in the way that karting can be no matter what you do.
Personally I prefer single seaters - they are very pure racing machines, quick, lively and exciting to drive, but don't have to cost any more to run than even an MX5. Formula Fords use standard ford road engines, in a welded steel spaceframe chassis. If you bend something it's usually just a case of welding it back together, but as they're designed to be crashed and repaired they are often cheaper to do so than a road car based series where you can easily need a new shell after a prang.
I've just won a championship on a budget smaller than what many people spend on their track day car in a year, however realistically unless you do have a reasonable budget to buy the car (5k+) I'd strongly advise getting out in karts first. Learn the ropes, make some mistakes and have a lot of fun - if it's still for you then start looking into cars to race
Option 6: Sprint racing, autotests etc
There's a lot of other grass roots options like those above. I don't know as much about them as circuit racing so I'll leave it to vipernet etc to fill you in
