Are You Using AI In Your Job?

We want to understand the real-world applications of AL and ML in business and the impact it will have on all our jobs.

Want to help? Complete the survey, your insights could make a big difference. It will just take one minute.
You'll be the first to get access to the final report.-->

What We Can Learn from the MultiVersus Open Beta

| December 12, 2022

Before releasing a digital product to the world, there is a test period that sometimes includes the general public, commonly referred to as an “open beta.”

With the coming and going of high-profile names like Call of Duty: Modern Warfare 2, Overwatch 2, and the free-to-play (F2P) crossover fighting title, MultiVersus, we have found some great talking points from the chatter surrounding these titles.

Here, we’re going to focus on the upcoming MultiVersus title to discuss some of the conversations and changes that we’ve seen online (mostly on Reddit and Twitter) with this title that are also relevant for other products outside of those geared solely toward entertainment.

What is a beta and why is it important?

Developing and maintaining video games shares some similarities with business software. Both require layers of design and development as well as user feedback to fine-tune a product.

In Mario’s earlier Goomba stomping days, backend analytics were nowhere to be found.

Back in the day, when games came on glorious cartridges, once you released a game, that was that. If your release had bugs or available exploits that allow people to “cheese” (i.e., take advantage of some mechanic not intended by the designers to circumvent a challenge) portions of the game, then these issues would persist indefinitely.

multiversus open beta

Minor issues were in just about everything, like this classic with two different ways to duplicate items.

Now, games and software are far more complex than their older counterparts. And by quite a bit.

Creating a game requires additional, specialized design and development when compared to most apps, but almost all products are developed iteratively, beginning with an MVP.

Your prototype (and alpha, in certain cases) act as a learning tool by giving teams enough insight to create a usable beta version for testing with real users.

Most apps produced here at Blue Label Labs use a closed beta process that varies substantially between products we create for different industries. Game development is similar, but in other cases (especially with live services), they open the beta to players meeting certain conditions (e.g., those that preordered the game, members of specific organizations, etc.) And in MultiVersus’s case, the beta is open to everyone on all major platforms.

Though feedback from the MultiVersus community can seem dramatic when stacked against most non-public betas, we can extract some good talking points that also apply to software development.

Takeaways from conversations during the MultiVersus beta

Even on a 2D plane like in MultiVersus, getting everything right is challenging.

Some shortcomings and optimization opportunities reveal themselves in behavioral analytics, but only some things, hence why user feedback and observation are critical for future development. This is much the same with other digital products, so here, we’re going to look at public conversations on social media and how these conversations have impacted the game.

1. Everyone wants everything, but not everything has value

MultiVersus is developed by a newer company called Player First Games that’s working with the publisher Warner Bro Games to deliver this unique fighting game that spins off the gameplay formula first popularized by the original Super Smash Bros.

Unlike Nintendo, Player First Games is small, with less than 50 employees at the time of this writing. Compared to their ambition – supporting a F2P, fully cross-platform game that theoretically could tap into anything in the Warner Bros collection – the company is tiny.

multiversus open beta

That means all these guys are possibilities for future characters in the game, among many other IPs. | Source: Mark Robinson Writes

Their size and the game’s appeal create a fun problem for the company: having a massive catalog of potential characters attracts highly-critical fanbases that want their favorite character to be playable – like, right now – and in way they find enjoyable.

The nice part is that this opens the doors for all kinds of possibilities. And with active, public communities like those in the fan-maintained and official subreddit, there’s no shortage of user ideas.

multiversus open beta

Many users put together these kinds of “wish lists” for characters they’d like to see in the game, like this awesome creation from Reddit user BahamutAXIOM. | Source: Reddit

Of course, between Player One Games’ small team and MultiVersus’s massive user base (it’s called Steam’s most popular fighting game ever), only so much can be functionally added at a time.

And like all other well-planned digital products, there is a roadmap first to consider.

multiversus open beta

Dramatic, yes, but a fun language lesson for those unfamiliar with great French colloquialisms from Reddit user Deadlysloths2. | Source: Reddit

Because of the large audience, some feedback is overlooked because of the sheer volume. The same holds true for bugs and optimization-related projects, which are tackled based on severity, as these can and will regularly surface for all games and products.

Essentially, the folks working on MultiVersus will be forever burdened by new and exciting bugs because of the constant change in all the underlying mechanisms that bring this and other games to life. It’s a delicate balance of staying on top of maintenance while updating the game with new content while working quickly to address “unknown-unknowns” once they reveal themselves.

A good example of this is the balancing required each time a new character is brought into the mix: balancing issues can affect just the new character, but other times, certain characters shed new light on global mechanics, which mandates further testing and tweaks.

multiversus open beta

Unlike a mobile or web app for non-gaming business uses, you can’t use certain tricks like A/B testing we sometimes use when testing an app either during the beta or in the real world. Simultaneously providing two different mechanics in a real-time game will be immensely frustrating for most users.

During a beta test, we’re typically able to work in changes based on prioritization quickly – it becomes a much different (i.e., slower) process once an app has launched to a storefront which is true for any software.

Though some are critical of the MultiVersus team for not moving fast enough, users need to recognize that Player First Games is currently accomplishing quite the feat given the circumstances.

2. Make big changes like significant “nerfs” during the beta

There are many psycho-social phenomena we see occur in virtual communities just as they do in the living flesh: the larger an organization, the slower it is to change.

multiversus open beta

This is satire from Reddit user apenasumfa but this wonderfully nostalgia-filled thread perfectly demonstrates how the noisiest of users react after a change is made. | Source: Reddit

Both seemingly small and noticeably substantial changes can radically change a person’s experience (which is the point), but for some, these changes aren’t well received.

After every notable change, frustrations are often voiced by people who, for whatever reason, seem to be highly competitive but have little mastery of the core game mechanics and wholly rely on a particular tactic to win.

These are casualties of beta testers, but it’s to be expected: changing core mechanics down the road – even for the better! – is, by definition, still a change and will forever be subject to the perspective of those affected.

Taking something familiar and making it “different” is always divisive, and people will drop off as the group acclimates. For MultiVersus, it’s important these changes are done now rather than in the future when it would disrupt a larger number of players.

multiversus open beta

Changes have become less dramatic recently but can still be “felt” by people who play regularly and/or are properly dialed into the subtleties of the game’s mechanics.

Further, it’s essential to understand that certain changes cause a much more significant kind of discord among users, and that’s when you mess with people’s time or money.

It further demonstrates notable social psychological phenomena detailed in chapters 9 and 10 in the book Predictably Irrational by renowned author Dan Ariely. People’s perception of pricing is mostly relative because we all carry biases based on the pricing of similar products (and quality) as well as any historical fluctuations of price for a product in question. Further, people also demonstrate unique behavior when a product is free, which is another factor present in MultiVersus and other F2P games.

In app design, such as with retailers or QSR restaurants, speed is everything: faster processes almost always yield more revenue (and profit, hopefully) than their slower counterparts.

Back in August, a change in Multiversus reduced the rate at characters earned experience and gold (an in-game currency everyone earns), which generated several discussions. Though the game didn’t take much of a hit, other digital products may suffer under the same circumstances.

multiversus open beta

When a thing that used to cost a certain amount becomes more expensive, some of us take it personally, like Reddit user xX-Coffee-Eater-Xx. | Source: Reddit

For example, when issues surface in Starbucks loyalty rewards systems after changes transpire, it usually becomes topical on Reddit, Twitter, and other platforms where people “stick it” to businesses.

The caveat here is that, eventually, prices go up. Fortunately, the people building MultiVersus recognized that people were earning faster than they had intended and made adjustments now rather than later when halving rewards would impact a much larger number of users.

When making significant changes to your rewards, try to get these ironed out before you open your doors to the general public. Some will never be able to shake the perspective of earning fewer rewards for the same amount of work.

And rightly so.

We keep users happy by not keeping everyone happy.

It sounds terrible, but that’s life. Building a digital product is often wrought with tough decisions.

Fortunately, you can keep all hell from breaking loose by getting off on the right foot with good planning and a complete breakfast every morning.

We won’t make you a meal (well, some of us might), but we do know how to build and maintain digital products for the long term – get in touch to learn more.

+ posts

Get the latest from the Blue Label Labs’ blog in your inbox


* indicates required