Quality Assurance in Games
- Thread starter BFCTom
- Start date
Associate
- Joined
- 22 Dec 2005
- Posts
- 138
this site maybe of some help, it covers anything game development related. There's also contact staff on there who you can email with specific questions if there isn't an article on a certain topic.
http://www.gamasutra.com/
http://www.gamasutra.com/
With all the patches/hotfixes/break fixes/specific hardware fixes I'd say as little as possible!
Up until the late 1990's most PC games were perfect out of the box and rarely did you need to apply any additional patch/fix.
Nowadays most games are being released unfinished and many require a patch as soon as the game hits retail (Splinter Cell3 is a recent high profile example requiring a 75MB patch).
Even console games (PS2 especially are being released in bug ridden states Tomb Raider AOD is a good example of that).
I just think its lazyness/poor coding/Q&A. The cost is high so the executives usually pressure the release date so something has to go.
Up until the late 1990's most PC games were perfect out of the box and rarely did you need to apply any additional patch/fix.
Nowadays most games are being released unfinished and many require a patch as soon as the game hits retail (Splinter Cell3 is a recent high profile example requiring a 75MB patch).
Even console games (PS2 especially are being released in bug ridden states Tomb Raider AOD is a good example of that).
I just think its lazyness/poor coding/Q&A. The cost is high so the executives usually pressure the release date so something has to go.
Associate
Well I work in QA, ask me some questions and i'll do my best to answer them.
Associate
Well there are many stages, here is a brief overview what happens when a game is tested
Alpha, Beta, Submission, Gold etc Most games start getting properly tested at the Alpha stage, but obviously there is some checking before this. It really depends on the size of company, resources etc Different companies test in different ways. For large games there is usually a testplan created by the Lead Tester/s in which he/she lays out what needs to be looked at. There may be a big overall test plan or lots of smaller ones updated now and again.
There are different types of testing that have carried out: Standards (Very important) These are created by Sony, Microsoft, Nintendo etc they describe how a game should react to certain situations on their platform (Removing a memory card, inserting a controller) these standards are very strict and game cannot be released if it does not meet them.
Then you have Playability Testing (How hard is it, can it be completed etc, Functionality Testing (Covers a number of things, similar to standards, testing the functionality of the menu's, in-game stuff. Does the sound/graphics/controls work as they should. Is all the text readable etc etc
Compatibility Testing (Important for PC games) - Running the game with different processors, ram, sound cards, graphics cards etc Usually helps to decide what should be the minimum spec recommended for running a certain game.
Multiplayer testing - Obvious (Balanced gameplay?)
Localization testing - Making sure all the text in the foreign language versions is correct/readable, not cut-off.
Free testing - Playthroughs etc
If a bug is found then it is logged, entered into a database, along with as much detail as possible -
Who entered the bug,
Time and date,
Platform - PC, Xbox, PS2 etc
OS - PC only really
Version number: What build?
What type of bug it is (Functionality, sound, graphics etc)
Where the bug was found (Boot, Main Menu, In-game etc)
How serious the bug is (A, B, C - although different companies use different ways of describing severity)
Priority - Does it need to be fixes urgently?
The reproducibility - How often does it happen? (Always, sometimes, randomly, only happened once etc)
Summary of the bug - Quick description
Description - Detailed account of the bug - where it happens, why etc
Steps to reproduce - A detailed step by step guide to reproducing the bug
Attached files - Screenshots or videos of the bug.
This is then entered into the database, obviously the tester has to check that the bug hasnt already been entered, sometimes then need to link bugs together.
The bug is then often checked and verified by the lead.
Eventually the programmers get hold of it and try to fix the problem.
Sometimes the bug cannot be fixed and this is where all the politics kicks in, generally the developers and publishers will argue for a long time over a solution. Quite often bugs are not fixed and the game is released.
Anyway....if the bug is resolved then it’s up to the tester to regress it (i.e. see if it’s fixed.)
If the bug is fixed then it’s closed in the database, if not the bug is re-opened and it goes back to the programmers. (Add more politics)
Obviously the programmers will be working on new content, so every now then new build are sent out with bug fixes which are tested (Usually following some sort of test plan).
Eventually all the bugs are fixed or at least dealt with, the final code is tested by the testers, lead testers, head of QA etc, its then sent of to Sony/Microsoft etc who check it for standards and any other problems (This costs quite a bit of money) if everything is ok then the game goes into production, if not it goes back and the bugs fixed, retested and sent off again until everything is fine.
Quite often the game won’t be completely finished and the developer/publisher will continue to patch the game (if its PC) for some time. Or if a member of the public reports a problem.
Well that’s basically, its far more complex than this though, apologies for my poor grammar; I’m typing this at work on the sly
Alpha, Beta, Submission, Gold etc Most games start getting properly tested at the Alpha stage, but obviously there is some checking before this. It really depends on the size of company, resources etc Different companies test in different ways. For large games there is usually a testplan created by the Lead Tester/s in which he/she lays out what needs to be looked at. There may be a big overall test plan or lots of smaller ones updated now and again.
There are different types of testing that have carried out: Standards (Very important) These are created by Sony, Microsoft, Nintendo etc they describe how a game should react to certain situations on their platform (Removing a memory card, inserting a controller) these standards are very strict and game cannot be released if it does not meet them.
Then you have Playability Testing (How hard is it, can it be completed etc, Functionality Testing (Covers a number of things, similar to standards, testing the functionality of the menu's, in-game stuff. Does the sound/graphics/controls work as they should. Is all the text readable etc etc
Compatibility Testing (Important for PC games) - Running the game with different processors, ram, sound cards, graphics cards etc Usually helps to decide what should be the minimum spec recommended for running a certain game.
Multiplayer testing - Obvious (Balanced gameplay?)
Localization testing - Making sure all the text in the foreign language versions is correct/readable, not cut-off.
Free testing - Playthroughs etc
If a bug is found then it is logged, entered into a database, along with as much detail as possible -
Who entered the bug,
Time and date,
Platform - PC, Xbox, PS2 etc
OS - PC only really
Version number: What build?
What type of bug it is (Functionality, sound, graphics etc)
Where the bug was found (Boot, Main Menu, In-game etc)
How serious the bug is (A, B, C - although different companies use different ways of describing severity)
Priority - Does it need to be fixes urgently?
The reproducibility - How often does it happen? (Always, sometimes, randomly, only happened once etc)
Summary of the bug - Quick description
Description - Detailed account of the bug - where it happens, why etc
Steps to reproduce - A detailed step by step guide to reproducing the bug
Attached files - Screenshots or videos of the bug.
This is then entered into the database, obviously the tester has to check that the bug hasnt already been entered, sometimes then need to link bugs together.
The bug is then often checked and verified by the lead.
Eventually the programmers get hold of it and try to fix the problem.
Sometimes the bug cannot be fixed and this is where all the politics kicks in, generally the developers and publishers will argue for a long time over a solution. Quite often bugs are not fixed and the game is released.
Anyway....if the bug is resolved then it’s up to the tester to regress it (i.e. see if it’s fixed.)
If the bug is fixed then it’s closed in the database, if not the bug is re-opened and it goes back to the programmers. (Add more politics)
Obviously the programmers will be working on new content, so every now then new build are sent out with bug fixes which are tested (Usually following some sort of test plan).
Eventually all the bugs are fixed or at least dealt with, the final code is tested by the testers, lead testers, head of QA etc, its then sent of to Sony/Microsoft etc who check it for standards and any other problems (This costs quite a bit of money) if everything is ok then the game goes into production, if not it goes back and the bugs fixed, retested and sent off again until everything is fine.
Quite often the game won’t be completely finished and the developer/publisher will continue to patch the game (if its PC) for some time. Or if a member of the public reports a problem.
Well that’s basically, its far more complex than this though, apologies for my poor grammar; I’m typing this at work on the sly
Soldato
I work in QA and everything he said is spot on. Also, sometimes an issue is fixed so is closed in the database only for a few builds later it to become broken again. It's a bit of a nightmare really, if you could see what goes on you'd probably be a bit more forgiving of bugs in games! 
Associate
Well i can give a more in depth explanation on other parts if you need it. I cant explain every in and out of the process because i only deal with the testing side of it. Obviously i had to miss out a few little things otherwise it would take up pages.
I'm doing Mobile phone game testing at the moment and thats even more complex.
I'm doing Mobile phone game testing at the moment and thats even more complex.
Going a little off topic here.
If i go out and buy a game, and on the box it says something like "You've never experienced a FPS like this before, you eon't be disappointed" and then i get it back home, play it for 1 month and complete it, and it turns out the graphics are rubbish, it plays bad, it has no features and the ending was rubbish, would they accept the game back?
If i go out and buy a game, and on the box it says something like "You've never experienced a FPS like this before, you eon't be disappointed" and then i get it back home, play it for 1 month and complete it, and it turns out the graphics are rubbish, it plays bad, it has no features and the ending was rubbish, would they accept the game back?
Associate
Maybe not after a month. That's marketing for you, always talking about things that are not in the game. Probably cos marketing never actually play any of the games they market.
Last edited: