Hello everyone! I wanted to share some thoughts and information about today’s launch, but before I do I want to thank all of you that showed us some patience and kindness in our first big release. Even though we felt we were prepared and came in with a high level of confidence - even feeling that we over prepared - we were shown that in a real-world situation things can go wrong that just simply do not in simulated scale testing. You can see a post I wrote yesterday about this that has of course “aged like milk” but I still think it’s informative and gives insight into our feelings the day before launch:
https://www.reddit.com/r/LastEpoch/s/v87zrzUa3m
Within 20 minutes of launch 150,000 people had joined us. We are thrilled that so many people are excited about what we’ve poured our hearts and souls into and joined us for launch day. Truly and deeply. This did mean that all of our scale testing efforts were immediately put to the test, and unfortunately a service failed in a way that we didn’t suspect, and we immediately went to work to investigate and resolve it. Many people just played offline and this didn’t bother them, but many of course want to play online (me included).
What does that look like? As someone who was not in the games industry 6 years ago, I always wondered and now that I’m on the other side I can share with you all - at least what it looks like in our scenario. Launch day we had our senior engineers, backend team, leadership, infrastructure/server/services providers in the “war room”, which is just a silly name for a zoom/Google call where we monitor and address issues that crop up with all of our dashboards and tooling in front of us. Dashboards showing what’s happening with server connections, timeouts, regional data, player data, databasing calls, etc. People involved are calling out what they’re seeing, potentials of what may be causing a problem and potential solutions, determining if we should go down the route of trying a solution that may take X amount of time and solve an issue or leave us in the same position, etc. Then you have the rest of the internal team anxiously awaiting updates so we can communicate with you all what’s going on as that’s a lot of people who are pretty upset with you and many being quite vocal about it. “War room” makes a lot of sense after you’ve been in it during a launch.
One thing I wanted to ensure today is that if we did have problems, which we did, that we would stay as communicative with you all as possible. I tried hard to do this by keeping a log of updates and posting every 15-30 minutes in our Discord. I’ve been on the other side of this where as a player I just wanted communication - any communication - on what is going on. I can certainly see how large studios struggle with this as not every update is a PR win, but I’d rather stay transparent and hopefully there’s a net PR win by building trust between us and you all, knowing that we’ll communicate and care deeply when something isn’t going as intended.
Tomorrow as we aim to deploy another hotfix, reduce the too-often scene transition times being longer than they should be, and fix any other issues that crop up, I’ll be keeping everyone up to date with a log in the same way I did today. See the #news channel in our discord.
https://discord.gg/lastepoch
I will also work with the team to sometime next week put out a more technical retrospective on today as it sounds like many of you are interested in that.
At the end of the day I’m very happy to see so many of you, mainly through Twitch, enjoying the game, even online,after we have things in a somewhat more stable state - and visiting Reddit to see some posts that put a much appreciated smile on my face after 14 hours in the “war room”. I even got to get a few levels on my Multishot Falconer before starting to write this.
Again, thank you all for your patience and excitement for the game. We are excited as heck that we have built a community of this size that we get to create content for, for years to come, and be a studio that you guys can trust from game design to communication and listening to your feedback. I look forward to seeing you all in game!