Mobile Stats

Another slow news day, the big story seems to be Google’s ability to get the whole world to blog about a 404 page. Can’t wait to see some real info.

One of the most common questions I’ve been hearing recently is about how to record stats and count unique users on mobile devices. David Adams from Mobile Research got a lot of attention at Mobile 2.0 when he talked about the mobile analytics service they’re looking to launch as an online service. Excellent! But I think we also need some public discussion about how internet scale mobile metrics should work.

The “internet scale” there is very important. If you need to setup some monitoring for a deal with Sprint or 3 you can figure out what they do and how to properly measure. But what if your mobile traffic can be coming from any carrier anywhere in the world? What if it could also be coming from special purpose mobile devices like Nintendo DS or Playstation Portable systems? Or machines like the Wii? How do you put together a stats system that reliably figures out if a particular user or device has been seen before?

We’re very quick to say that mobile offers a unique advantage in increased ability to identify and target individual users. While that might be easily true inside the carrier umbrella, extending it out to the Wild Wild Web is what makes it actually interesting. And it’s much harder to determine the right way to track once you cross carrier boundaries. Here are some of the interesting challenges and benefits I’ve run across so far:

• Cookie support is still very non-standard, see the entry about cookies at the Wireless FAQ. There’s everything to worry about from the device not supporting them, to the gateway stripping them, to cookies getting deleted at the end of the session. That stuff can be a pain in the ass when developing an app. But in that environment it means you either have to do URL based session management or maybe the user has to login every time they hit the service. If you’re talking about delivering reliable metrics however that’s killer, you just have no idea how a mobile browser is going to behave.
• The MSISDN is anything but standard. In theory it’s a way to uniquely identify the SIM card used in GSM systems. Fantastic! Except that even across different GSM carriers the use of the MSISDN is inconsistent. My understanding of the evolution of the feature was that originally it was the phone number of the handset sent in the clear, and you can still see it used that way at times (China Mobile for example sends phone number in the clear… but not User Agent, go figure). However it was quickly realized that blasting out a users phone number with every HTTP request sent from their handset was rife with potential for abuse. So then the numbers started getting obfuscated. And put into headers with slightly different names, or jammed into different headers completely. Sometimes combined with session info so that the MSISDN isn’t actually always the same ID for the same handset on the same network. Add that together with not all carriers being cased on GSM.
• Because traffic out to the wider internet from carriers tends to come out through a relatively small set of gateway machines it’s much harder to do identification of unique user agent/IP pairs to count unique users.
• Plus carrier gateways behave weird. Sometimes they send multiple requests when I expect them to send one. Sometimes they cache stuff when I don’t really expect them to. Plus you throw transcoding engines into the mix and it gets really hard to predict what could be happening on a network.

That makes it a pretty difficult environment in general, and I haven’t really seen much that addresses the core issue of getting folks to agree on some common metrics and how they should be calculated.