My AI powered NPC Meta Quest 3 mixed reality project

Tykey

Tykey

Tykey

Associate
Joined
13 Jun 2013
Posts
2,089
Meet my new friend in the Mixed Reality world ! (not that I'm lonely or anything :cry:) . I whipped this up over the last 2/3 months, it's been really fun and educational. AI powered NPC's in VR/MR aren't new but I've not known any that can walk around the real world like this (and without guardian/boundary warning messages).


 
Last edited:
gavinh87 said:
I'd get mugged if I wore a quest in my local park!
Click to expand...
lol this was a football field very near to where I live, I did try the local park first as it's massive and very scenic but turned back as it was surprisingly busy, didn't want the scrutiny of dozens of people wondering what I was doing.
 
I've been working on this project and have some updates to show. The NPC can now navigate by itself around my home (or atleast the rooms that have been scanned), I've added gesture animations, and the time it takes the NPC to reply has come down a lot. Note in this vid the lip sync'ing is off, the video recording audio/video is out of sync, in reality it's fine.

 
Last edited:
its impressive, i can see a lot more of these types of apps becoming more common.
either for those poeple that can for one reason or another have a partner or friends.
the only sort of down side is peope relying on it to much in the future and resulting in lossing social skills.
some of hte image gets obfuscated but that going to be better mapping and time i guess.
the gpt responses are interesting boarder on beliveable to "cant you see the headset on my face?"
not sure if thats something your able to add in or not either way can be immersion breking.
 
Augmented reality has a powerful part to play in the future IMO, but it really needs an order of magnitude breakthrough in convenience of wearables and interface, be interesting as well when things like door sensors can be used to direct the AI navigation heh, or even more spookily automated doors they could activate to walk through.
 
LostCorpse said:
its impressive, i can see a lot more of these types of apps becoming more common.
either for those poeple that can for one reason or another have a partner or friends.
the only sort of down side is peope relying on it to much in the future and resulting in lossing social skills.
some of hte image gets obfuscated but that going to be better mapping and time i guess.
the gpt responses are interesting boarder on beliveable to "cant you see the headset on my face?"
not sure if thats something your able to add in or not either way can be immersion breking.
Click to expand...
Thanks, I'm pleased with it as I was newcomer to this sort of thing until recently. Yeh I agree, it could be of use to people who are lonely etc. I think I got the idea because I knew my dog who I was very attached to wouldn't be with me for much longer due to ill health, so yeah maybe I was craving companionship that I knew I was going to lose. The biggest thing I have gained from it is learning, I enjoy working on it more than using it and have spent far more time working on it than using it. I have got irl friends, but due to family commitments and work etc I seldom see them although we speak often. The believe-ability of the NPC responses can vary yeah, it will get better with time as the AI they use is constantly being improved upon.
 
Last edited:
Rroff said:
Augmented reality has a powerful part to play in the future IMO, but it really needs an order of magnitude breakthrough in convenience of wearables and interface, be interesting as well when things like door sensors can be used to direct the AI navigation heh, or even more spookily automated doors they could activate to walk through.
Click to expand...
Yeh, until the day comes that VR/MR hmd's are not much bigger than sunglasses I think this sort of thing will remain a niche.
 
  • Like
Reactions: SPG
In a more general sense until these systems are performing closer to human responses the delay to the responses could probably be masked with some sort of stock relevant "pre-amble" using 2 stages of AI. Seen a few takes of trying to do it with NPCs in games and people seem to lack thinking outside the box when it comes to working around the limitations - granted better to not have to patch over the limitations.
 
Last edited:
Poor girl said a number of times how lonely she is and thankful you’re there to talk to, and you dropped her as soon as she took a keen interest in one of your hobbies. Hope she’s there next time you boot up the app :D
 
MoNkeE said:
Poor girl said a number of times how lonely she is and thankful you’re there to talk to, and you dropped her as soon as she took a keen interest in one of your hobbies. Hope she’s there next time you boot up the app :D
Click to expand...
Guilty as charged, what can I say? I'm definitely not a ladies man :D
 
I don't suppose anyone here would try out what I've made? To see if the NPC can walk around a room(s) you've scanned with Meta's Space Setup (where you scan the room and the mesh appears on it etc). You'd be helping me out and in return you'll be getting a free copy of my app which is worth loads :D (believe it or not it looks like some people are willing to pay for what I've been doing). If you've never done a Meta Space setup for mixed reality games/apps before then maybe give it a miss.
 
Last edited:
people are and do pay for what your doing. its also part of the whole Virtual Girl/boy freind as well to make it as real an expirene as possible.

is Meta Spaces setup that hard?
also assume one of the pre-requisists is a gpt login?
 
LostCorpse said:
people are and do pay for what your doing. its also part of the whole Virtual Girl/boy freind as well to make it as real an expirene as possible.

is Meta Spaces setup that hard?
also assume one of the pre-requisists is a gpt login?
Click to expand...
It's not difficult tbh, it's something that needs to be done before a lot of Mixed Reality games/apps. I've done a video on it which could guide you if necessary. You can find the option to do the Space Setup if you go the main settings inside your Quest 3, then Physical Space, then select Space Setup. Once you start the process, there's onscreen instructions. If you feel like you might be ok doing it and have a bit of time spare I could message you a google drive link to my app. No probs if not though of course.

Edit to add, no pre-requisites for ChatGPT or anything, you can just simply use it, won't cost you anything.
 
Last edited:
Tykey said:
The NPC can now find and sit on my sofa :D plus I've added a menu and named the app TykeAI.

Click to expand...

I wonder what else she can sit on.
50743141726_ec3ab89e9b_o_d.gif
 
will have a look at weekend a bit more, i did see the space mapping options. always interested inthis sort of stuff.

www.uploadvr.com

Zuckerberg: Neural Wristband To Ship In 'Next Few Years'

Mark Zuckerberg says Meta's finger-tracking neural wristband for AR/VR input will ship "in the next few years". Explanation here:
www.uploadvr.com www.uploadvr.com
this is also something i see meta integrating in to the eco system it looks promising.
 
LostCorpse said:
will have a look at weekend a bit more, i did see the space mapping options. always interested inthis sort of stuff.

www.uploadvr.com

Zuckerberg: Neural Wristband To Ship In 'Next Few Years'

Mark Zuckerberg says Meta's finger-tracking neural wristband for AR/VR input will ship "in the next few years". Explanation here:
www.uploadvr.com www.uploadvr.com
this is also something i see meta integrating in to the eco system it looks promising.
Click to expand...
Yeh no problem, there's a chap in Denmark who has the app, he's tested it a bit & gave some feedback but not the AI navigation (it works without the AI navigation bit). He seemed to be going well with it for the 1st 24 hours and then he went quiet, maybe the NPC attacked him or something :D and I got an email from a professor from a VR research lab in a German University yesterday evening :eek: :cool: asking to try it how cool is that! If u get a bit of time spare etc then we'll see but no probs if not.
 
You must log in or register to reply here.
Back
Top Bottom