True Machine Learning is the Strongest Weapon to Fight App Install Ad Fraud - A Fireside Chat with Draftkings. MAU 2019

Irina Seals:

My name is Irina Seals and I'm focusing on user growth and customer success at Scalarr. Scalarr’s mission is to empower more mobile app developers like Draftkings, for example, to acquire new users without wasting any money on fraudulent and non-human traffic. And we actually have been working with a lot of different app developers in the app space, different verticals including gaming. Gaming is actually one of our biggest and largest verticals at this point. That's why I also have Daniel today with me.

Daniel Lopez is one of the leaders in UA (user acquisition) over DraftKings, coming from Machine Zone and having a lot of experience in that space. He's joining me today on stage to help us understand a little bit more what his experience is in general in mobile ad fraud, also what his experience is in working together with Scalarr when it comes to ad fraud.

Before I jump into this conversation with Daniel, I'd love to give you a very-very brief understanding over the state of mobile ad fraud nowadays and why it is so important for mobile app developers to actually look into this topic.

In 2018 we had the ability to investigate over 170 million installs globally, and we realized that within all these installs we investigated, over 22% of the installs were fraudulent. That actually means for app developers that every fifth install you are buying nowadays could be fraudulent. And, it does not just mean that you end up losing money on fake installs. The bigger issue here is that the whole effectiveness of your business model is actually degraded because you are basing your success KPIs of low performing inventory sources that are appearing to be high performing to you.

Additional to these findings we realized that over 77% of all the installs we have marked as fraudulent are highly sophisticated. The sophistication of the fraudulent types becomes visible by the time fraudsters need to deviate from their strategy once they realized that we have busted them. Sometimes we see them reverse-engineer us as quickly as 18 hours. So pretty much on a daily basis there could be new fraudulent patterns on the market. And we're not just talking about basic types of fraud like attribution fraud but we're talking about very smart types of fraud like smart bots and mixes which are the most sophisticated in the industry.

You might have heard about smart bots in the past but those are the guys or the bots that are able to emulate real user behavior within your app up to 30 days. And we're talking here not just about clicks and opens but we're also talking about different things like achieving different levels within your app or also having real purchases.

What does that mean for app developers? Why am I talking about these things?

Well, it means that there is a need of highly sophisticated tool in the market to be able to catch those types of frauds, the sophisticated ones. Also stay ahead of the fraudsters and actually catch all the newly developing fraud types in the industry.

It also becomes visible that fraudsters are not a one-man show but huge organizations. And we are predicting that there will be a loss of marketing budget of about $12.6 billion in 2019 globally just in mobile alone.

So what can you do if you are a mobile app developer?

You might be asking yourself how do I secure myself from that. After talking to a lot of different app developers globally we realized that there tends to be a shift from a lot of game and app developers to utilizing so-called rules-based technologies when it comes to mobile ad fraud protection. Rules-based simply means you have a couple of rules that you set up and define to fight some well-known patterns in the industry. For example, if you have a rule as if my time to install is less than 30 seconds then this is supposed to be marked as a click injection. The issue with rules is that they tend to be outdated very quickly. Fraudsters are able to reverse-engineer them very quickly and you are not protected let alone from any sophisticated types of fraud on newly developing patterns especially. That is the second issue.

Rules-based technologies are not able to investigate post-install analysis. But due to the new fraud patterns that I just mentioned, where bots are actually emulating real user behavior 30 days after the install, you absolutely have to look at everything that happens after the install as well to catch those sophisticated types of fraud.

How do we solve this problem?

We are focusing on Artificial Intelligence and we're utilizing machine learning to understand how we can fight against those types of fraudulent patterns. One of the things we are utilizing our unsupervised and semi-supervised machine learning algorithms. Those are highly sophisticated and we don’t set any types of rules to find specific patterns of fraud. We are actually never teaching them to do that.

What we are doing, we're training and programming our algorithms to go out there and to literally look at any kind of abnormal behavior within your app and flag those. Afterwards we are looking at least 72 hours post-install events to make sure we can make a decision if there is fraud or if there is no fraud in that specific install.

Without going too deep into the nitty-gritty here if you would like to talk to Scalarr a little bit more about the product as such you can reach out to us here.

Now I want to jump into the discussion with Daniel to give you a little bit more of a real user case here. Daniel and I have been working together for a while now and before we started to work together DraftKings was also using a simple rules-based technology for their fraud detection. And I'm very curious to hear from Daniel how it resonates with him what I just mentioned, were there needs to be a shift from a rules-based technology towards a machine learning technology?

Daniel Lopez:

It resonates with me pretty deeply. Trying to keep up with fraudsters, you're always going to be left in the dust. We've been spending so much money with them for so long. Any rules-based type of technology used to detect fraud is being outsmarted on the daily. And if you think about it, rules by technology in its mass is a cookie-cutter form that supposes through attribution service providers and other services, are not updated fast enough in order to catch them. Like I can reverse engineers CPM flow on a DSP within a couple hours. What do you think some major companies are doing that are just setting out massive amounts of fraud? So there needs to be a shift to smarter solutions and utilizing machine learning is one of those solutions.

Irina Seals:

Thank you Daniel. That totally makes sense, and I think that also opens up well to the next question.

What are the goals that you are trying to achieve when utilizing a tool like Scalarr?

Daniel Lopez:

With my director Jane at DraftKings, we’re a pretty small team, so there's three main things we're getting with Scalarr.

The first is that they're just basically an extension of our team, really providing us with professionals who are well versed with fraud, engineering, data science. Finding those individuals is hard enough and getting them to come to your team is very hard, so that's a great thing that they're providing to us.

Secondarily, they're providing actionable insights. Data is one thing you can have. You can have all the data in the world, but if you don't have it in a readable actual format that actually makes sense to people, that's tough. You can't actually do anything with that. And, unfortunately, when we're dealing with fraud you not only have to go after partners and publishers to recoup those losses, but you also have to tell your managers, your exact team and everybody who are looking at this traffic, and saying, "hey, where's all that great traffic you were getting me before?" You have to be able to give them a nice graph and example of how this was fraud, and how we should have been buying it.

And thirdly, it's a security blanket. Scalarr is saving our time. Time is one of the essences and it's extremely valuable. If we can't vet with our small team every single last publisher source, every single install that comes through and so Scalarr gives us the assurance that all the traffic we're buying is truly incremental and not fraudulent. And that's priceless to us.

Irina Seals:

Thank you for sharing that with us! I have a question for the audience. Could you just give us a sense of how many people here, if you are a marketer, have been utilizing a machine learning technology in the past for your fraud prevention? Okay, three people. It looks like it's not the majority, but I'm not sure if everyone is a marketer obviously.

Daniel, it looks like there's still a lot of people out there that probably haven't tried it and are probably also hesitant to do so because it is a new technology. I wanted to understand from your point of view, what are your ideas when it comes to you like starting with machine learning?

Daniel Lopez:

I don't know why anybody would be hesitant towards exploring a new technology that is based or founded on the premise of helping you do your job more effectively. We as marketers have the mentality of never leaving any stone left unturned, taking every opportunity possible and so you should really be doing that with your technology. You should be reinvesting in yourselves and paying for the best services available to make sure that you can do your job as effectively and efficiently as possible.

I think it also has to do a lot with asking yourselves to be honest with yourselves. We all have looked at the KPI's being delivered by several certain pubs and they're out. You'll have three or four that are just otherworldly and they're just demolishing your KPIs delivering at a tenth of your CPA goal. And every week you kind of look at it and it's well so-and-so didn't catch it which is whether that's a MMP or your standard fraud solution. And you guess it's going to be legitimate. I can guarantee you, 99% of the time it is not legitimate. So if we really need to just suck it up and get to the bottom of these sources, smart technologies will catch up and eliminate fraud. It's our fault that we have a fraud because we keep on buying it.

Irina Seals:

Thanks for being so honest with us and reflecting on that.

Before we started to work together I remember when we chatted you guys were looking into a lot of different tools in the industry to make a decision which one is the best for you. And I was curious to learn what was it that convinced you to start working with us?

Daniel Lopez:

It was a pretty funny story. We had gone live with a test with another anti-fraud solution, who's actually one of the biggest players in the fraud tech space, and I wasn't brought into the conversation till later because we're a small team so I had other priorities. But on a recap call when we were reviewing the results of the test and the dashboard, they had highlighted and flagged this particular install as MTTI (Mean-time-to-install) click spam so I asked the person straight up: okay, this is click spam, right? So it's flagged at MTTI, which is mean time to install, and how many clicks did this install have? And they've asked me, “what?” Okay, I took a step back: how many clicks did this install have? You've had it flagged for click spam but it's under MTTI, here you told me the clicks. And they had no answer for me.

The truth was that they wanted to charge us $15,000 a month for their “cutting edge fraud tech”. But that was what I could do in a simple sql query which takes date diff between install and click timestamp and put it in a bucket. If anybody wants to do top-of-the-line fraud tech right there I just gave you the secret. That is completely worthless. You need to be looking at impressions, clicks, installs, live JSON data streams and analyzing post install data metrics in order to really capture these smart bots and sophisticated fraud techniques.

When I got on the phone with Irina and Inna [Scalarr’s CEO] the first thing I asked them was if they use this. They said yes and I said let’s do this.

Irina Seals:

Daniel, do you have any closing thoughts and any advice for the marketers here when it comes to mobile ad fraud in general?

Daniel Lopez:

I will point out some best practices that could be applied to fraud and in every aspect of the mobile app business.

Firstly, track absolutely everything: track your get live JSON feeds of your impressions, your clicks, your installs, and tie absolutely everything to a device ID network whatever you can. And continue that on the back end. Put every single last bit of information you can in a single source. Your data is worthless unless people has have easy access to it and can read it. And have that be your basis of operations across the entire org. Your data operations team, business operations team, your marketing team, absolutely everybody should be using the same exact data source because it means nothing if you're all speaking the different language.

Secondly, try to keep it simple. Usually, the simplest stuff scales. And you want to be able to grow in scale as much as possible in this ecosystem.

If you want to learn a little bit more about how Scalarr does fraud detection and why we use machine learning for it, please reach out to us here.