The Machine Learning Thread

mid_gen

mid_gen

mid_gen

Soldato
Joined
20 Dec 2004
Posts
16,187
Thought it would be good to have a thread on this.

For background, I'm a game AI programmer. So not really AI in what's the contemporary use of the word.

I did a Stanford course on ML a few years back and learned the fundamentals of neural networks, using Octave etc, so I have a decent grasp of the basics.

The frameworks and languages are a bit of a minefield.

Spent an evening looking at Tensorflow and got some the basic NIST image classification tutorials working. Initially just a basic python environment locally, but then a docker image using Jupyter, which seemed kinda neat.

As I'm interesting in expanding my employment options, I did some more reading, and seems like the industry is moving towards PyTorch, so I've ordered a couple of books to get to grips with it.

Short term I'm just wanting to get familiar with tools and frameworks. Then I want to start developing some AI agents in Unreal using reinforcement learning systems.
 
Thanks for starting the thread, wouldn't mind learning more myself (as a laymen), particularly as some of my work revolves around ML (fortunately my lack of knowledge in it is fine).
 
The free ML Stanford courses -

www.coursera.org

Machine Learning

Offered by Stanford University and DeepLearning.AI. #BreakIntoAI with Machine Learning Specialization. Master fundamental AI concepts and ... Enroll for free.
www.coursera.org www.coursera.org
 
Chaos said:
The free ML Stanford courses -

www.coursera.org

Machine Learning

Offered by Stanford University and DeepLearning.AI. #BreakIntoAI with Machine Learning Specialization. Master fundamental AI concepts and ... Enroll for free.
www.coursera.org www.coursera.org
Click to expand...
Andrew Ng's courses are very good. I did his general ML one a few years ago, but it was first principles stuff in Octave, rather than using any modern frameworks.
 
Books finally arrived. Been getting through the reinforcement learning one. Remarkable how basically straightforward the systems are, all just common sense really.

I've had a few prototype ideas knocking around for a while I'll get onto soon.
 
When it comes to machine learning the one thing I am really interested in is how it relates to robotics and how to combine the two. I'll have to do some research as I'll also have to look into mechanical engineering.
 
Cromulent said:
When it comes to machine learning the one thing I am really interested in is how it relates to robotics and how to combine the two. I'll have to do some research as I'll also have to look into mechanical engineering.
Click to expand...
I would imagine it's a combination of things. Neural nets for pattern recognition etc. Reinforcement learning models trained in virtual environments for movement etc.
 
mid_gen said:
For background, I'm a game AI programmer. So not really AI in what's the contemporary use of the word.
Click to expand...

I find the AI in most games these days so boring - hardly any studios seem to do anything interesting with AI any more and so often it is just stand and fight like things have barely moved on from Doom :(

Unless you are fairly familiar with the stock enemy behaviour in Quake 2 it might not mean much but I managed to get them moving around in a much more interesting way and not just running straight at you - without implementing any modern AI features https://www.youtube.com/watch?v=Ub3uLoEyvkk (I was just toying with it here to show it) still using the old wall following and player trail code with some little additions of my own, no way finding, no navigation mesh, no goal system, etc. etc.
 
Last edited:
Rroff said:
I find the AI in most games these days so boring - hardly any studios seem to do anything interesting with AI any more and so often it is just stand and fight like things have barely moved on from Doom :(

Unless you are fairly familiar with the stock enemy behaviour in Quake 2 it might not mean much but I managed to get them moving around in a much more interesting way and not just running straight at you - without implementing any modern AI features https://www.youtube.com/watch?v=Ub3uLoEyvkk (I was just toying with it here to show it) still using the old wall following and player trail code with some little additions of my own, no way finding, no navigation mesh, no goal system, etc. etc.
Click to expand...
Thing with game AI is that you're not trying to make 'advanced' AI, you're trying to make a good game, and much of that is actually about making the player feel powerful, and making the AI agents predictable so the player can reason about what they're going to do and come up with strategies to beat them. Souls games are a prime example.

Making genuinely smart AI is just super frustrating as a player, outside niches like deathmatch bots.

I'm looking forward to trying to reinforcement learning agents out though, but I'll be setting them against each other, rather than me, mainly so I can leave them training unattended.
 
mid_gen said:
Thing with game AI is that you're not trying to make 'advanced' AI, you're trying to make a good game, and much of that is actually about making the player feel powerful, and making the AI agents predictable so the player can reason about what they're going to do and come up with strategies to beat them. Souls games are a prime example.

Making genuinely smart AI is just super frustrating as a player, outside niches like deathmatch bots.

I'm looking forward to trying to reinforcement learning agents out though, but I'll be setting them against each other, rather than me, mainly so I can leave them training unattended.
Click to expand...

What I find though is that approach tends to funnel games into being about mastering a specific pattern and that just kills the game for me personally.

In general with game design predictability is unfortunately a big one - I've got a gameplay modified for my Quake 2 RTX mod which is kind of fun but also quite unpredictable - where every enemy kill gives you a 30% chance of losing 25 health (but won't take you below 1 health) and a 30% chance of getting a 5 second quad damage. In actual combat it is almost impossible to track it and apply it strategically especially the health loss when you suddenly drop to 1 health in combat hah. It does combine with other gameplay modifiers to do interesting things though. (It would probably be better with some kind of shuffling card counter type approach so the player could judge whether/when the next kill would result in a health loss and adjust accordingly).

It is quite possible to combine the feeling of the player being powerful with challenging AI though, it just takes working other parts of the game around it, though it may put off players who don't want to put in effort to get results.
 
Rroff said:
What I find though is that approach tends to funnel games into being about mastering a specific pattern and that just kills the game for me personally.
Click to expand...
Oh definitely, I find Souls-type games incredibly dull personally.
Rroff said:
In general with game design predictability is unfortunately a big one - I've got a gameplay modified for my Quake 2 RTX mod which is kind of fun but also quite unpredictable - where every enemy kill gives you a 30% chance of losing 25 health (but won't take you below 1 health) and a 30% chance of getting a 5 second quad damage. In actual combat it is almost impossible to track it and apply it strategically especially the health loss when you suddenly drop to 1 health in combat hah. It does combine with other gameplay modifiers to do interesting things though. (It would probably be better with some kind of shuffling card counter type approach so the player could judge whether/when the next kill would result in a health loss and adjust accordingly).
Click to expand...
You could probably have whole conference dedicated to the topic of random-ness in games :D It's very complicated, obviously having the player stomped by a roll of the dice on the face of it sucks, but depends what the consequences are to a large extent, and what agency the player has to mitigate it.
Rroff said:
It is quite possible to combine the feeling of the player being powerful with challenging AI though, it just takes working other parts of the game around it, though it may put off players who don't want to put in effort to get results.
Click to expand...
Definitely, depends massively on what kind of game you're trying to make, what emotions you're trying to instil in the player. To take two extremes, Alien Isolation the player should be terrified and try desperately to avoid the alien, Serious Sam the player should feel like a machine-gun toting god, and both nail it. Neither have particularly interesting or 'clever' AI mind. It's quite rare that you get a game where you're given free reign to make truly smart AI, because 'smart' AI isn't good gameplay in and of itself.....which is why I'm doing this personal research on my own time.

The dream would be to get into one of our R&D teams where you can do this stuff, but alas, I do prefer the production side of things.
 
Cromulent said:
I'm very interested in ML as well. Thinking about getting this book:


but like most tech books it is ridiculously expensive.
Click to expand...

Are you going to read it though? :D

It's available online btw, albeit not in pdf format:


It is pretty much the go-to book for deep learning.

Are RL was mentioned too, this is the go-to RL textbook, also available online:


Ideally do buy these books too if you use them a lot.

For practical stuff, there's this ML book by a guy from the former Google Brain team; covers a bit of classic ML before getting into deep learning for which it uses TensorFlow. (For theory re: classic ML see books by Murphy and Bishop. Also the book Elements of Statistical Learning is relevant too or the Introductory version (both free online).)

And there's this one by Jeremy Howard from fast.ai which uses PyTorch + the fast.ai framework.


Note, also free online: https://course.fast.ai/Resources/book.html

See the website for the courses too and some of his YouTube stuff, it's quite practical. And on the subject of YouTube See also:

www.youtube.com

Andrej Karpathy

SuperThanks: very optional, goes to Eureka Labs.
www.youtube.com www.youtube.com

mid_gen said:
I'm looking forward to trying to reinforcement learning agents out though, but I'll be setting them against each other, rather than me, mainly so I can leave them training unattended.
Click to expand...

Replicating the DeepMind atari games paper can be a fun exercise, just build a DeepRL agent and train it on old-school video games.

MTA99 said:
@mid_gen how you finding that pytorch book? Would you recommend it to a data scientist? :D
Click to expand...

Don't worry too much about which framework to use, you just want to get a good grasp of the basics initially.
 
MTA99 said:
@mid_gen how you finding that pytorch book? Would you recommend it to a data scientist? :D
Click to expand...
Sorry, missed this. Honestly I haven't opened it yet, I've been working through the RL one and it's got all the PyTorch examples I need for now. It did seem to be the best PyTorch manual around from reviews.
 
Very interested in getting a discussion going here. I work extensively with LLMs in my day job (market research data processing and insights) from a ML Ops perspective but want to get a better understanding of the underlying technology.

For those who are curious ML Ops is like Devops for ML. I’m deploying models into production environments, scaling platforms to meet load, building model evaluation frameworks and pipelines to record metrics, building LLM support tooling (API abstraction layers that perform rate limiting, prompt optimisation, usage attribution, safety checks and prompt injection protection).
 
You must log in or register to reply here.
Back
Top Bottom