Press "Enter" to skip to content

How Google AI implemented: the history of one bug

Stepan Frolov 0

 

For several years Google has been consistently implementing machine learning technologies into theirs services. It’s the turn of the Google Play app store where the US Internet company plans to promote better games and applications, based on internal metrics such as user return, internal purchases, application deletions, and so on. This has been expected by users for a long time, but mass failures show that developers need to be ready for unpredictable behavior of artificial intelligence (AI).

Something went wrong

Since June 19th, the Google Play’s app Stores around the world has experienced serious disruptions in their important services — application searches, native A \ B tests, showing downloads statistics, and the “Similar apps” block.

The consequence of failures were massive complaints of developers for large losses in the installation of the application by new users from the application store and so-called organic traffic decreased. For some, this figure reached 90% of the daily volume of new users. The problem has affected not only Russian companies, but also foreign ones. The problem is discussed at many forums, for example, Reddit.

example one

 

example two

For example, the failure occurred in the service of built-in application search, the most significant source of new organic users in the Store, accounting for more than 50% of all application installations. Service stopped issuing the correct search output, there was a loop of the first 16–20 applications, the remaining 230+ applications in each search query disappeared from the search and stopped getting users.

As an example of reduced impressions in the store — and a game called: Get over it PvP: Hammer hit.

The number of impressions also fell. From 18th to 24th of June there were 350,000 hits:

example 3

From 25 June to 1 July, there was a drop to 50,000 hits:

example 4

Also install conversion dropped from 15.5% to 11.8% (It is obvious that it was targeted auditory, this shows that there is an error in “Similar apps” and searches)

An application Hyperloop is an example of falls in installs.

example 5

An application Monster Evolution:

example 6

“Approximately on 20th of June there was little fall in installs but on 22nd installs just dropped. Data on traffic sources are not displayed, according to estimates (the last for June 1st )”, chronology of the failure is described by the individual developer of the application Hyperloop — Anton Slyusar. He also mentioned that he lost an ability to conduct A/B tests because of the lack of data on traffic. Project conversion has fallen by 3 times (or statistics has shown wrong data). From the developer side there was not any changes that could affect it.

At the moment, serious disruptions in Google Play services are fixed, at least the search works as it was before. For those who continue to have fall in installations, it is worthwhile to wait, when all the services are backed up, especially “Similar apps”, on which the quality of incoming organic traffic depends, and statistics will start to be displayed again.

Unfortunately, Google representatives at the time of the preparation of the material did not provide information about the failures inside Google Play, while sharing the results of the introduction of new recommendatory algorithms of applications based on machine learning.

Bright hopes

According to Chi Huai-hsin, Google’s chief AI researcher — “Google already implements computer-based learning technologies on Google Play to create the most accurate personalized recommendations for mobile applications, which has already led to an increase in conversion to the installation of mobile applications by 3.3%.”

Example, Vitaly Kozlov, CEO & Founder KB Production (Fixiki applications and Masha and the Bear):
“Since June 19, some projects have literally failed in the settings, and some on the contrary — have grown several times and continue to grow.

Analyzing the main indicators of various games, such as: retention, the presence of payments, the number of technical errors, the average score, the number of deletions, we cannot say with certainty — which factor or their combination has an impact on the installation of applications on Google Play since June 19. Probably, we are talking about a combination of a certain number of internal project metrics that Google sees through its systems. Presumably, the growth is observed in higher-quality applications, in which there is value and depth for the user, perhaps has a weight of monetization through intra-game purchases.

It seems that Google in a new algorithm of recommendations is trying to solve the problem of promoting better games and applications. In the future, this is a good trend for the consumer, who will receive recommendations for better games and applications. For developers, these are the new conditions that you will have to adjust to remain successful on this platform, if Google leaves the current algorithm in effect. A separate issue concerns the segment of children’s games and applications, where the main indicators differ to the worse, in comparison with adult projects, because of the behavior of the children’s audience. It is not entirely clear whether this point is taken into account in the new algorithm or not. “ — innovations concern expressed by Kozlov.

example 7

Results for developers and users

New, stronger algorithms of recommendations can solve 3 current problems:

  1. Better applications will receive more users, which will stimulate their further growth and growth of the whole ecosystem of mobile applications in general.
  2. There will be a blurring in the level of income between applications “giants” and the rest. Current statistics say that 1% of applications earns 95% of all revenue. With a new algorithm of recommendations based on the personal needs of users, the share of “giants” should decrease slightly.
  3. Disappearing of gray promotion, based on ratings cheating and the number of installations to keep applications in the tops of categories, and search. For the end user and high-quality applications — is good. The consumer will receive an application that will close their needs here and now, the developer will receive high-quality audience for further monetization and will not spend budgets on gray promotions to keep positions, while less quality applications take their places on top at the expense of their budgets and gray promotions. The budget will be channeled to more formal and measurable traffic sources.

Google’s ranking mechanisms is constantly improved. Perhaps in the AI training there are such rapid changes of parameters that you do not know — what is going to be moved where. Developers should be prepared for the fact that sometimes their applications are dramatically losing popularity. Do not immediately stress. If this happened abruptly and for no apparent reason, it might be worthwhile to wait for the next “teachings” and wait until the AI ​​comes to its senses.

On the part of the Internet company, I would like a little more openness about such processes, to understand that the “storm” will soon end.

Comments are closed.