Purple Unicorns and Gaming Brush Bristles

By Adonis Adoni | Photo by TASPHO

“Games are deeply human artistic pieces; nobody ever devised a fantastic game on a spreadsheet.”

Jeremy Ballenger, the witty Head of Analytics at Wargaming, told me this, deep into our kaleidoscopic conversation, while trying to prise his computer out of the claws of his curious cat, This statement would have found him at odds with the late film critic Roger Ebert, who made several enemies a decade ago when he declared that videogames could never be art. Ebert’s argument takes us back to the famed art critic Louis Leroy, who lampooned the art of Monet and Pissarro and invented the term ‘impressionism’ as an insult. In a post-modern interpretation, though, art is in the eye of the beholder and Ebert and Leroy are symbols of the perpetual human trait that builds dams against the flood of new. “For the most part,” Ballenger went on, “Analytics is like the bristles of the paintbrush in an artist’s hand.”

In 2010, Wargaming rolled out a new online game in which mid-20th century tanks were locked in combat. A free-to-play title, World of Tanks surpassed its predecessors in the Massively Multiplayer Online (MMO) genre with such striking detail and interactivity that, a year after its release, the Guinness World of Records would register the game as the record-holder for the highest number of players simultaneously logged onto a single server.

Victor Kislyi, the CEO and founder of the small Belarusian development company, did not rest on his laurels. Instead, he started looking into ways that would fortify the company’s future in an increasingly competitive industry. At the time, having greater and better information required a considerable investment. The finance industry had for some time been developing algorithms that could give their models remarkable accuracy. “We have Siri today, in part because of machine learning algorithms analysing the news to provide even the most miniscule predictive accuracy to financial models,” Ballenger explained. In 2012, Kislyi took a punt on moving Wargaming away from Excel-based spreadsheets towards actionable business intelligence. Ballenger was one of the first people he approached to build the initial analytics competence in Austin, Texas. “I had learned early on in my career that I was a pretty good analyst and a terrible engineer! So, I found a fantastic engineer, Alex Ryabov, who was working in a different part of the company and was also fascinated by this stuff,” he said. According to Ballenger, the people working in the world of analytics can be split into three distinct personalities, each playing their own game: (1) the engineers, for whom structuring data is similar to an abstract form of complicated algebra that requires a brain generally good at puzzles; they play a game of chess. Then, (2) the data scientists, who have an affinity for formulating and exploring hypothesis by being able to delve into the minute. They play a game closer to tic tac toe. And, (3) the product or game analysts, who are natural born storytellers and, as such, are required to read their audience to understand how much cognitive overload they can give before none of it makes sense. These people would fare well around a poker table. “You’ll very rarely see someone that combines all three abilities; we call them purple unicorns,” he explained.

When devising their action plan, Ballenger and Ryabov took a rather slow path since they wanted to build a solution that would become the foundation of everything they did over the next ten years. “Some thought that two years was too long to be planning but we stuck to our guns because we wanted to be able to extract anything we wanted from the game,” he told GOLD, adding, “We now know a player’s journey from start to finish.”

In the gaming industry, with most people having worked their entire lives without using data, the novelty of analytics naturally invoked scepticism (shades of Ebert and Leroy) and Ballenger had to be patient. “The more educated people became, though, the more you saw them taking to analytics. For a long time, we were so far ahead of everyone else, as regards what we knew about our players and what we could track, that other companies were credibly jealous.”

When it came to game features, analytics allowed Wargaming to validate all its assumptions about whether a new feature would be embraced by players and to design the next feature better if the company’s assumptions proved wrong. What Ballenger found more fascinating, though, was the fact that players would take a new feature and turn it on its head. “We released three tanks, and there was this small speedier one that we all thought was going to do poorly and which turned out to be the most fun for our players,” he said. “This interest inside a feature was something we didn’t know and couldn’t have known without our players’ feedback.”
With players coming in different shapes and sizes, the game developer also went on to adopt user testing to gather quantitative feedback. “We want to give the best experience for the greatest number of people and the only way we can do it is by having the information that tells us if they are actually feeling that way,” Ballenger noted.

Analytics at Wargaming extends way beyond improving in-game mechanics. Ballenger set up a smaller internal team to handle competitive intelligence. Initially used for the marketing teams and designers eyeing the competition for inspiration, it then became a tool to answer much more nuanced questions. “Right now, we are experiencing COVID-19 and people are at home playing video games, so, by and large, the industry is doing fantastic. But, how do we know that are we doing fantastic compared to everyone else?” he asked. In an industry where the market ultimately picks the winner, competitive intelligence gives companies confidence that they are moving in the right direction. And, delving deeper, it can support corporate development in making better investment decisions and creating better forecasts. “Predicting how many games you will sell is not as easy as, for example, figuring out how many Toyotas will be sold,” he said. “We are impacted by unpredictable environmental factors, like cold weather or a global pandemic, which drive people home, where they play more games.”

The reasons why people stop playing games are also unquantifiable. However, user-churn, a term coined by European telecoms, is not a defining factor of success for MMO games. “You will see mobile game companies being affected more, because they have player lifespans of days or weeks. We care about creating a relationship with our players that will last a lifetime,” Ballenger said. This long-term relationship, he explained, is forged by repetition: it’s a routine in which players will go home after work, crack open a beer and spend a few hours on World of Tanks instead of going to the cinema. The information that lets Wargaming know what kind of lifestyle choices its players are making is extremely valuable. “If they play a few hours this week, then they will probably do the same next week as well. The routine is there for a reason; it’s comfortable. And it’s super-important for us to understand that if they start going to the movies more, it may be because our games are falling behind,” he noted.

Having enlisted 160 million users since World of Tanks was first released, Wargaming’s decision to invest in analytics intelligence has clearly paid dividends. However, even the most successful analysts are unlikely to become the recipients of fame and fortune. Jeremy Ballenger ended our conversation by noting that the person who receives all the praise for a job well done is the stakeholder who drove the decision, not the analyst who devised the system. Similarly, when things go wrong, the stakeholder will be the one who gets the blame. “The analyst’s job is to provide the tools that offer the best possible solution, not to take the final decision,” he clarified. “If you want to be rewarded, become a stakeholder. But, if you’re obsessed with helping people be amazing at what they do, then you need to have the humility to step back and be proud of what will always be seen as someone else’s accomplishment, not yours.” Behind every great game developer, there is evidently a great – but quiet – data analyst!