[..] Wow, but hang on, I'm a complete nube here, how are their different prices for the same game through Steam? How does Good Old Games compare to Steam? Can anyone provide a decent explanation on how to buy games these days? Last game I bought came on a CD from Electronic Boutique!
PC gaming is pretty much entirely digital distribution nowadays. Even if you buy a physical copy, it will probably require digital distribution anyway. You might get a disc with just a downloader on it. You might get a box with a code in it to redeem at a digital distribution site.
Almost all digital distribution companies have their own client that you must use for the games you got from them. This usually includes bloatware and spyware because of course it does. It also makes it possible to buy the same game twice by mistake by buying it on different clients. Plenty of people have done that.
That's one of the ways in which GOG (which no longer calls itself Good Old Games because it no longer only sells old games) is different. They have a client, but you don't have to use it. You can install and run games from them like people did back when people bought games rather than effectively renting them for as long as the publishers allow. You buy a game from them, it's yours. No client required, no DRM, etc.
There's a trend towards different publishers having their own clients, so you might end up having to install and use half a dozen different clients, each with their own seperate library. In some cases, this is used to force publisher exclusives in order to ensure that everyone who wants to play a particular game must do it through one specific client and only that client, trying to turn the open platform of PC gaming into a closed platform. Epic is particularly infamous for this, especially as their client was spectacularly defective at launch and lacked most of the basic functions of a gaming client.
There's a trend towards having meta-library software that can display the contents of libraries from different distributors and run the different clients for each one. That at least solves the problems of buying a game more than once by mistake or wondering which library a particular game is in...as long as the different clients are supported.
The reason for the different prices in Steam is that you can buy a key code for a game and apply it to Steam to get the game added to your account. Which is effectively what you're doing anyway, it's just more obvious when you buy the code as a standalone purchase. You might be able to do the same with other distributers, but it's at least far more common with Steam. There's a grey market in game codes acquired from various sources, which vary from legit to outright theft and fraud. It's a contentious issue. At one extreme, there have been devs who have publically stated that they would prefer people to outright pirate their games rather than buy a key from a keyseller business. There are various keyseller businesses and they have very varied reputations.