My AI powered NPC Meta Quest 3 mixed reality project

[Tykey]

Meet my new friend in the Mixed Reality world ! (not that I'm lonely or anything ) . I whipped this up over the last 2/3 months, it's been really fun and educational. AI powered NPC's in VR/MR aren't new but I've not known any that can walk around the real world like this (and without guardian/boundary warning messages).

 

[Tykey]

I'd get mugged if I wore a quest in my local park!

lol this was a football field very near to where I live, I did try the local park first as it's massive and very scenic but turned back as it was surprisingly busy, didn't want the scrutiny of dozens of people wondering what I was doing.

 

[Tykey]

I've been working on this project and have some updates to show. The NPC can now navigate by itself around my home (or atleast the rooms that have been scanned), I've added gesture animations, and the time it takes the NPC to reply has come down a lot. Note in this vid the lip sync'ing is off, the video recording audio/video is out of sync, in reality it's fine.

 

[Tykey]

its impressive, i can see a lot more of these types of apps becoming more common.
either for those poeple that can for one reason or another have a partner or friends.
the only sort of down side is peope relying on it to much in the future and resulting in lossing social skills.
some of hte image gets obfuscated but that going to be better mapping and time i guess.
the gpt responses are interesting boarder on beliveable to "cant you see the headset on my face?"
not sure if thats something your able to add in or not either way can be immersion breking.

Thanks, I'm pleased with it as I was newcomer to this sort of thing until recently. Yeh I agree, it could be of use to people who are lonely etc. I think I got the idea because I knew my dog who I was very attached to wouldn't be with me for much longer due to ill health, so yeah maybe I was craving companionship that I knew I was going to lose. The biggest thing I have gained from it is learning, I enjoy working on it more than using it and have spent far more time working on it than using it. I have got irl friends, but due to family commitments and work etc I seldom see them although we speak often. The believe-ability of the NPC responses can vary yeah, it will get better with time as the AI they use is constantly being improved upon.

 

[Tykey]

Augmented reality has a powerful part to play in the future IMO, but it really needs an order of magnitude breakthrough in convenience of wearables and interface, be interesting as well when things like door sensors can be used to direct the AI navigation heh, or even more spookily automated doors they could activate to walk through.

Yeh, until the day comes that VR/MR hmd's are not much bigger than sunglasses I think this sort of thing will remain a niche.

 

[Tykey]

Poor girl said a number of times how lonely she is and thankful you’re there to talk to, and you dropped her as soon as she took a keen interest in one of your hobbies. Hope she’s there next time you boot up the app

Guilty as charged, what can I say? I'm definitely not a ladies man

 

[Tykey]

I don't suppose anyone here would try out what I've made? To see if the NPC can walk around a room(s) you've scanned with Meta's Space Setup (where you scan the room and the mesh appears on it etc). You'd be helping me out and in return you'll be getting a free copy of my app which is worth loads (believe it or not it looks like some people are willing to pay for what I've been doing). If you've never done a Meta Space setup for mixed reality games/apps before then maybe give it a miss.

 

[Tykey]

people are and do pay for what your doing. its also part of the whole Virtual Girl/boy freind as well to make it as real an expirene as possible.

is Meta Spaces setup that hard?
also assume one of the pre-requisists is a gpt login?

It's not difficult tbh, it's something that needs to be done before a lot of Mixed Reality games/apps. I've done a video on it which could guide you if necessary. You can find the option to do the Space Setup if you go the main settings inside your Quest 3, then Physical Space, then select Space Setup. Once you start the process, there's onscreen instructions. If you feel like you might be ok doing it and have a bit of time spare I could message you a google drive link to my app. No probs if not though of course.

Edit to add, no pre-requisites for ChatGPT or anything, you can just simply use it, won't cost you anything.

 

[Tykey]

The NPC can now find and sit on my sofa plus I've added a menu and named the app TykeAI.

 

[Tykey]

will have a look at weekend a bit more, i did see the space mapping options. always interested inthis sort of stuff.

Zuckerberg: Neural Wristband To Ship In 'Next Few Years'

Mark Zuckerberg says Meta's finger-tracking neural wristband for AR/VR input will ship "in the next few years". Explanation here:

www.uploadvr.com

this is also something i see meta integrating in to the eco system it looks promising.

Yeh no problem, there's a chap in Denmark who has the app, he's tested it a bit & gave some feedback but not the AI navigation (it works without the AI navigation bit). He seemed to be going well with it for the 1st 24 hours and then he went quiet, maybe the NPC attacked him or something and I got an email from a professor from a VR research lab in a German University yesterday evening asking to try it how cool is that! If u get a bit of time spare etc then we'll see but no probs if not.

 

[Tykey]

im not anywere near that level
i is just a lowly underclocker.

but sounds way cool dude getting some good exposure there by the sounds of it!

i'll have time just on the weekend though. my attention span is low though.. i can finally live my dream of having a Blonde in my house and pretend shes my mistriss

Hey you seem like a decent guy who comes across well on the forum which is far important to me than being a professor in a uni (i'm not saying the professor isn't decent! idk, he probably is, but u get what i mean i hope). yeh if u get a bit of spare time on the weekend just let me know and i'll get it sorted. i have written up some quite good (imo) instructions too into a pdf (only 1 page so not blabbering on too much). yeh this app is the nearest i'll ever get to a slim blue eyed blonde....tbh that's a fib, at 1 time (a looong time ago) i was able to actually interact successfully with attractive females who were real

 

[Tykey]

ow no, you mis understood me sorry, my humour & style didnt translate t well. i hnestly dont know what i'd be doing with it but, lets learn and experiment.
i was only meaning im very proud for you its a great achievement in my book. People are actively talking about the stuff your doing in important circles. its impactfull and has applications in differant fields.
we'll catch up a bit laters and sort something out

No worries, it's probably me missing obvious humour! I'm slow on the uptake sometimes and may miss things! Yeh we'll catch up n sort something. Yeh it's cool some people are talking about my app, my little late doggo Tyke is still having a positive effect on me even after his passing

 

[Tykey]

I can't believe there hasn't been a literal stampede to try out my uber super duper cool app ok it's probably a bit rubbish but it's free! LostCorpse has kindly offered to test it for me, are there any other takers before I hike up the price to £99.99? If you DM me your Meta account email address I can set it up so that you can easily install it through the Meta app on your phone so no messing around with SideQuest etc. Being able to do Meta Space Setup (the room scan) will help but is not essential.

 

[Tykey]

Sorry bud, not sure how my wife would feel about me talking privately with another woman.

Understandable! She's not real though and doesn't do sexy talk

 

[Tykey]

No sexy talk? Definitely out now

I realise I've cut myself out of 90% of the market for these AI NPC's by having no sexy talk, but I have morals, honest

 

[Tykey]

its not cheating if shs doenst find out... or if shes out the house when your talking

@LostCorpse I've uploaded another version with better feedback for when sitting on the couch so whenever you're ready just install the app again please from the Meta app. No rush on actually testing it. As the Chief Beta Tester I will add your name to the credits

 

[Tykey]

My app has been approved for SideQuest so I can list it on there if I want. I've submitted a request for App Lab listing, App Lab is cool because app installation is via the Meta app which is super convenient. It might yet go on Patreon, a subscription based payment makes sense as it incurs monthly API AI call costs, however I'm not a fan of recurring payments for software. If it does go on Patreon once 1 month is paid then it'll be ok to use it without keeping up the subscription, I just hope atleast a few people do keep up their sub (assuming anyone subs to it!).

 

[Tykey]

In one of your videos the NPC asked if you've heard of the Q3, while you were wearing one. If you stood in front of a mirror, would the NPC then know that you're wearing a Q3?

Nope, not now. In an earlier iteration of the app it would have but a couple of months ago I changed the AI product it uses for reasons of reduced latency, ease of integration and cost, but it doesn't have some of the abilities of the old AI unfortunately.

 

[Tykey]

My app is released now on Patreon as of yesterday, and some people have subbed (not many, but even just 1 person makes it worth the effort because I'm still working on it). I found out yesterday a Youtuber called Obscure Nerd with 12.1k subs briefly covered my app in a livestream, it's pretty funny see the vid if u get chance (see below), I hope he'll do a video on it it'd be surreal!

 

[Tykey]

Don't forget us when you become rich and famous!!

I've earned $30 up to now, not bad for 4.5 months work lol, i'll buy everyone here a tube of smarties! Nah i've enjoyed doing it that's enough reward.