Now that we’ve gotten the title update out the door, I wanted to respond to some of the opinions and comments I've been reading about the release of the Zero.
Before I go into that, I want to make clear that we’re very proud of the work that went into making sure the Zero meets our high quality bar for the way our planes look, sound and feel, and we’re pleased about the response as indicated by purchase numbers.
But since there's a lot of speculation in various community channels as to our motives and decision-making in releasing content, here are some answers. I realize it’s long, but if you’d like a peek behind the curtain into our decisions, why we talk about data so much, etc., here it is. I’m going to try to paraphrase some of the questions, comments, objections, anxieties and so on that I’m hearing from parts of our player community (not necessarily from you guys, but I figure you'd like to know too
).
Why release another basic model when people have been commenting on your blog and Facebook page that they prefer deluxe planes (with cockpits)?
Please keep in mind that it’s been only four weeks since we launched (and released the P-51). Work on the Zero was started before Flight was released, because we felt it was important to set a regular rhythm to our content releases, as opposed to releasing some content then going dark while we gathered data and feedback and reacted to it. The release of a plane is preceded not just by the actual production of the content, but by testing, the certification process, and the process of getting it onto LIVE. There’s also the time it takes to get reliable data and interpret it.
So, it was always going to be the Zero that came next. Our next DLC plane is also already locked.
Have you been sensitive to potential concerns about the Zero?
Every word, image, outfit and gesture in our games gets reviewed by our geopolitical risk assessment team who assesses the potential to offend people in any region in which we're shipping the product. The product team then decides whether the level of risk is acceptable, or whether to remove the content from the game.
The Zero was no different. In the end, it was decided that since the Zero in our game has no weapons and can't do any damage to the environment, and it’s been over half a century since Pearl Harbor, it represented an acceptable level of risk. In addition, a number of historical societies fly Zeros with historically-accurate livery similar to that featured in Flight. Ultimately, we’re a game about flying, and the Zero is an interesting plane to fly.
You keep saying you’re “gathering data.” Why is it taking so long?
We are listening and adjusting as necessary. However, the type of data that's useful to us takes time to gather.
One of the reasons for this is a statistical concept called sample size. (Sorry in advance to people who already know this – please be patient with me!) Broadly speaking, when you’re trying to understand the behavior of a group, the more members of that group you observe, the greater the precision with which you can draw conclusions about their behavior. Similarly, in many cases, the greater the length of the study, the more data you have on behavior trends. More data = more reliable conclusions.
Flight’s been available for a month, and we’re just now reaching the point where we have enough data about our launch content to see trends rather than blips.
But your users are telling you what they want. Isn’t user feedback the best type of data to have?
I genuinely believe that most of the people who write to us or about us are 100% sincere. The problem is that sometimes what they’re saying doesn’t reflect what they’re doing, or what they believe to be a strongly-held majority opinion isn’t actually in sync with what the majority of our players are doing. (I don’t want to get sidetracked into a statistics/psychology discussion, but if you’re interested, you can do a search on some of the following terms: self-reporting inaccuracy, confirmation bias, attitude polarization, and group polarization, which are all things I have to account for when I try to summarize community attitudes.)
For example:
Out of all the people who’ve written in to give us a list of essential features for Flight’s success, I have yet to see two lists that match exactly.
There are quite a few people claiming that they bought the P-51 but won’t buy the Zero, for various reasons. Yet 90% of the people who buy the Zero have also bought the P-51, and the Zero’s numbers for the length of time it’s been out are comparable to those of the P-51.
That’s why, in order to make sound business decisions, we compare what people say to what they actually do. We also look at how the data changes over time.
So does this mean it’s useless to talk to you about the game? You’re only listening to your in-game data and not to us?
Absolutely not. We are absolutely listening to our users.
User feedback is itself a source of data. It’s also useful for interpreting the other data we get about user behavior. Those in-game data streams can tell us if people are playing one mission way more than another, for example, but they can’t necessarily tell us why. That’s where the art and science of collecting the data, interpreting it based on what users are saying and other sources of information, and deciding how to change the game in response based on the expertise and creativity of our team members comes into play.
Okay, then how do I ensure I get listened to?
Well, the above should give you some sense of how we use customer feedback. Game data shows us what people do, feedback helps us understand why they do it. If people aren’t finishing a mission, for example, bug reports and support requests/feedback emails can tell us if that’s because there’s a bug that’s stopping them.
So, if you want to give us feedback that’s useful, letting us know when things don’t work, versus when you’re just not enjoying them, is helpful, as well as telling us what you’d like to see in future updates. Jon’s feedback threads are a really helpful example of this (thanks, Jon!), as are some of the threads on AVSIM.
But if you are listening, why haven’t you made any changes?
We have made changes, and are continuing to; big changes take time, and before we invest that time we need enough information to make good decisions on whether they’re the right changes. One change that’s in process is that – although at this time we have nothing to announce –
due to the consistent and numerous requests for TrackIR, we’re actively working on putting it in a future update.
Finally, the quickest feedback-result loop continues to be fixing issues with Flight that are making it difficult for users to play, as we’ve done in our title update. We’ll continue to make changes as necessary to make Flight the best product it can be.
