Mike Rowehl: This is Mobility

The wraps are off the latest project I’ve been working on: Chomp, a social iPhone application recommendation service. I actually started consulting with them to fill in some time, but liked the app and the people so much that I couldn’t resist jumping in full time. Now that the app is out and I can talk a bit more publicly it’s time to start building up a team. Obviously, about the app itself, the reception has been great and folks seem to love it:

• Geek.com - the one I’m most proud of
• Techcrunch - obTechCrunchShoutout
• ABC News - on the TV!! Not really, they’re talking about us, but the guy is just randomly poking around on an iPhone. Poor TV, you just don’t get it do you?

The goals of the business don’t stop at an iPhone app, we have plans to move into other areas over time. But the iPhone app market is the biggest juiciest lowestest hanging piece of fruit in the mobile arena right now. It would be crazy not to take a bite. Ben can answer a lot more of those style questions than I can however. I’m just here to keep the lights on and the servers running.

We have a jobs page up, but there isn’t too much color to them yet. In particular, I need some folks to join me on backend engineering and operations tasks, so I figured I would shout out here. Right now the team is only 5 people, so it’s a chance to get in at the very start for the right folks. We haven’t yet figured out exactly how the jobs break down, but at a startup job titles don’t really mean that much anyway. Here’s a rundown of the kinds of things I’ve been doing for the last two months, to give you a feel for what we have going on:

• Picked up and finished off a spin of the server side software. Standard LAMP stack with the PHP side built on top of CodeIgniter. There was some general cleanup work like reformatting the server responses to be more consistent across all the API calls, some new features like adding in bookmarking and avatar uploads, and generally operationalizing the code with stats logging, database migrations, processes for background tasks and cleanup.
• Prepping the production deployment systems. Which in this case meant getting Puppet up and going, setting up the modules for the bits of software we were planning to use, bits of custom config for things like the database systems (we’ve tried to set them up to be sharding-friendly right off the bat), and generally setting up the parts of the system we didn’t need till we went live (like load balancing and spreading out the front end web systems, and syslog-ng to pull together all the logs to a central system)
• Setting up system and application monitoring and metrics, so that we have alerts that fire if something seems to be going wrong and pretty graphs to show us where that wrong thing might be.
• Doing a few spins of designing a custom crawler that pulls info out of iTunes so that we can make it available in our app. With the number of apps, the number of regions, the number of languages, and the general lack of any kind of off-the-shelf solution for this, it’s been a pretty complex problem to work out. Our data is pretty timely right now, but this is one area in particular where there’s a lot of room for improvement.
• Adding in some very simple strategies for caching and load shedding if we ever hit the kind of load where we were in danger of degrading performance. Fortunately we’ve yet to have to pull any of the escape valve levers (fingers crossed), but with the system growing as quickly as it’s been growing I’m sure that’s only a matter of time.

Generally, it feels great to be back at the controls of a set of servers experiencing the kind of explosive growth we saw during the early days at AdMob. But there’s definitely more to do than I can handle all by myself. And I know there are plenty of folks out there who are better at the bits and pieces of this than I am, so I would love to be able to peel some of this stuff off to the right people and start scaling the human side before scaling the machine side gets out of control. Plus, when the weather gets nicer, I spend way too much time at the track to be the only person available to respond to ops issues :-)

So if you’re interested, particularly you folks who I’ve worked on projects with before, ping me. mike at chompapps dot com. Yes, this means I’m going to start reading my email again now.

After upgrading to Snow Leopard we’ve had only one real issue so far, Zend Server Community Edition didn’t want to start up. Well, technically, parts of it. The apache instance was running, by mysql and the admin interface were dead. I found this post about watchdog errors, but even after putting in the fix for lighthttpd I still wasn’t getting the services starting up. Same deal for Tony. Poking around in the startup files it looked like mysql data was owned by user 103, and the mysql scripts were trying to start as user ‘zend’, however I have no zend user on my system. I wasn’t able to find anything about it, but I figured hell, let me give it a try and create a user Zend. This is the command to run from the terminal (funky huh?):

sudo dscl . -create /Users/zend UniqueID 103

Which creates a zend user.. somewhere.. I’m not familiar with that bit of OS X magic yet. It’s not in /etc/passwd, but if you ‘ls -l /usr/local/zend/mysql’ you should see the data directory owned by zend again. Now just restart and you should be peachy:

sudo /usr/local/zend/bin/zendctl.sh restart

Works for Tony and I at least, so we figured we would share. голова болит секс голова болит секс

There seems to be a lot of “take it for granted” style discussion around the possibility of Android fragmentation. Without strong top down control of the platform, folks seem to think that we’re going to end up with hundreds of versions of Android, all slightly different. This was the nightmare that mobile Java became. There were large engineering teams dedicated to and specialized in taking your write-once-run-anywhere app and actually getting it to work on the volume of handsets you wanted to hit.

It’s true that technically anyone who wants to build a slightly different version of Android can, so in that sense there CAN be fragmentation. However, the ecosystem has also fundamentally changed. Before we had the carriers tightly controlling the channel, dictating what was in and out in terms of hardware, enforcing strict standards on the software they pushed to users, and doing everything they could to keep anyone else from pushing software to users. In that world the burden of dealing with fragmentation was with the developers, and the benefit went to the carriers. The carriers controlled the channel, so fragmentation continued.

The one lesson that everyone in mobile seems to have learned over the last year was that the carriers were really bad at determining the right hardware and managing that application and content catalog. It’s why everyone is jumping on the App Store Bandwagon. One of the side-effects of that is that it breaks the strongly controlled content and application channel. If ATT decides that their Android version is going to do Bluetooth slightly different, sure, go ahead. But how can they strongarm Android developers who produce Bluetooth apps into making an ATT specific version? The control isn’t really there any more. Developers might do it, but only if ATT is offering up enough incremental sales revenue to make the port worthwhile. Right now developers frequently do it cause they’re contractually obligated to if they want ATT to promote their app.

Anyone who’s worked in open source knows that forks are technically possible, but practically uncommon. When the “market” is completely open folks tend to follow a path that serves them best. And it’s a reinforcing path. Even when someone with vested interest tries to keep a fork distinct, normally the burden ends up being more on the controller than anyone else. Forks tend to get folded back or die off pretty quickly. I think the same thing should happen with Android. Sure, people might try at the start to get a leg up by including proprietary features and customizations. But in the long run if Android as a whole works, the only person they’re hurting with a fork is themselves (to the tune of a decreased application catalog). And although it’s possible for them to create a custom application catalog with some differentiation in the short term and attempt to keep that differentiation going, there’s no difference between that and the strongly controlled channel that we currently see failing today. The cost of trying to do so will outweigh the benefit of joining the mainline. толстые проститутки

The dotMobi folks are up for an emerging technology award at CTIA for Instant Mobilizer. Instant Mobilizer is the evolution of Mowser, so of course it’s got a special place in my heart. The dotMobi folks have done an awesome job bringing it to market and adding some innovative mobile-specific features. Check out the info at the CTIA site and click “Mark as Favorite” if you like what they’ve done. I’m pulling for you folks, good luck!

I’m seeing more and more mobile payments systems getting used in actual products. Stuff like Zong, Paymo and Mobillcash. Surprisingly enough, the places I’m hearing about systems like this being used are in social games (for example social applications using Offerpal) instead of in mobile applications. One of the criticisms leveled at mobile advertising pretty frequently is that it doesn’t make much sense to have an advertising system built somewhere that commerce isn’t flowing. Why advertise on mobile phones if few people are buying through mobile phones?

I still hear pretty regularly that trying to deliver a paid application globally on mobile phones (note, I’m talking about systems outside of the app store models that have cropped up recently, they’re definitely an exception to just about everything) is at best a pain. Often it involves enough cost, revenue share, and engineering that the economics frequently don’t work out. So you’re stuck going through an existing channel even if you would technically have a route directly to consumers. I was expecting that to eventually get sorted out, that someone would crack the global mobile payments problem, allow unbanked users to pay using premium SMS, and deliver the platform in a package that developers could integrate without having to bend over backwards. And that has happened. But the unexpected part is that the uptake seems to be on the wired web, as an alternative payment form alongside credit cards. I was expecting the evolution to be that the systems would be used to back transactions originated on the mobile first, and then eventually start creeping onto the web as well.

For anyone who hasn’t poked around with the systems yet, the flow goes something like this when I pay with a mobile payments system on the web:

1. I decide to purchase a weeks supply of heroin for my virtual junkie for $5 in real world money (my virtual junkie is trying to make it to Rock Star status, adding heroin is the fastest way)
2. When asked how I want to pay I select “by mobile phone” and enter in my phone number
3. I get an SMS saying “Do you want to pay Virtual Junkie $5? Reply with Y to pay”
4. I send an SMS response, replying with Y
5. I get a confirmation back saying that I’ve paid the $5
6. My junkie gets his fix in the game. No more bugs crawling under his skin, excellent, everyone is happy

Actually a pretty nice system compared to paying by credit card. With most credit card systems I have a bunch of info to enter in, if I even have the card I want to use on me. And for folks who don’t have credit cards, like those using prepaid accounts where they charge up with cash each month (or kids who have their parents paying their phone bill), it opens them up to online purchases. Cool, nice, makes sense, I’m happy it happened, lots of good going on there.

So why didn’t this end up opening up mobile commerce more? Or did it, and I’m just not hearing about it? Why does Zong list all online properties and no mobile-specific properties in their list of partners? Did the new commerce line just follow the existing line of purchases? Are folks still hesitant to kick off a transaction from a phone, whereas they just “trust” the web more? They’re paying with their phone, so it’s not an aversion to using the mobile itself as a payment mechanism. But maybe an aversion to starting the transaction on the mobile. Or is it a technical/user experience glitch? Does punching out to reply to an SMS often kill the interaction going on in the mobile browser? Leading to a large dropoff or user disappointment when they can’t find their way back to their purchase? Or is it simply that social and online services are performing so much better than mobile services that the incremental gain of opening up unbanked transactions for existing online purchases way outweighs the benefits of opening up commerce completely tied to the mobile?

I’m pretty sure this is indicative of an opportunity of some kind. For instance, Apple is supposedly adding in-application payments options so that users can pay for goods or service updates within an app after purchase. So there’s probably a decent amount of demand for a system of the kind. I would assume that existing developers could probably use something like Zong or Paymo with some pretty minimal infrastructure to provide a system of their own to get in-app payments working today. We’ll have to see how granular Apple gets with their payment system, and what the terms end up being. Having a bunch of options popping up at the same time should lead to a nice healthy market. And help to keep the Apple based system in check should they start using their position as channel controller to try to squeeze developers. порно фото скачать

I posted yesterday about wanting to use my phone to publish my location in realtime, and I got a bunch of fantastic responses both in the comments and as emails (thanks folks!) I fooled around with some of them with varying degrees of success. Nokia Sport Tracker seems like it would be great! However, I can’t figure out how to make the phone app do it’s job. Maybe my fault, I have an E71 and it’s not listed as a supported model. I’ll have to dig out my N95 and try that one. However, on the Android side, I put FirePin on my phone and was able to upload and share out info about my route (course, it only has to points, cause I have the phone on wifi only right now). FirePin supposedly also works with FireEagle based on the comments I got. But I can’t figure out how that’s supposed to work.

Generally I’ve been hearing frequently about how location based services are really starting to happen this time around. Yea, yea, I know. That’s what we’ve said every year for the last decade. Don’t even bother, I’ve heard it all before. However, this time we have some open platforms with GPS built into the handset, which means there’s a way to work around most of the problems inherent in the carrier based system we had before. Any by “problems” I mean “crippling cost structure”. I can kinda understand how that seems like a more mature market for location based services. Because end users can grant permission to an application for it to directly grab location info, there are all kinds of services out there.

However, I would hardly count that as platform maturity. That’s a degree of market maturity, but not platform maturity. Sure, it’s easier to build a location based app. Just plop your application into any one of the silos that folks have built to control the deployment of your application and management of your data, and blammo! New app. Yea, not the way I normally think about these things. Which is not to belittle the efforts like FireEagle, I just think there are a few missing bits.

Here’s the kinds of stuff I was expecting to see:

• A somewhat generic app that records location information. Recording information to a local file seems to be pretty decent, I guess there are standards floating around for that. However, there doesn’t seem to be much of a standardized interface/protocol for streaming location information up to a server. GPSGate has a protocol that seems to operate over UDP and TCP, but that doesn’t seem to be an accepted public format.
• Some server component that I can chuck on one of my machines to accept samples sent up by that little client shim.
• Basic tools to display those samples on a map. Google has provided most of the backbone and made it pretty simple to use. But there’s also Openstreetmap for those who are Google alergic.

That basic set of functionality is a write-once bit of infrastructure. It gets us out of the “Oh yea, FirePin is great!! Wait, no, you can’t use it on your Nokia”. Not that the tools won’t evolve. But there should be an open set of base clients for all the different mobile platforms, and then anyone who wants to build a web app that pulls in location info can just say “Okay, add local.rowehl.com:9900 to your location client to start updating the Mike-o-matic pile of gold finder app with your current location” Instead we have all these different point solution client applications, frequently hardcoded to send location to a given server. Dumb.

I’m planning to go on an extended motorcycle trip in May. Whenever I leave to go somewhere for more than a few hours and my friends know about it I tend to get a lot of calls. They get very concerned and call to make sure I’m not lying on the side of the road somewhere bleeding. Can’t blame them, the precedent has been set. And hey, it feels good to know that next time I actually am lying on the side of the road bleeding there should be someone looking for me pretty quick even if I don’t have a sweeper following me. However, it also means that there are a bunch of worried people out there if I don’t answer my phone for some reason, for which I feel guilty. “I should be able to solve this with technology!” says Mike the geek.

Immediate thought, Google Latitude. I can keep recording info about where I am in the background with both the Android and Symbian phones I have. That way folks can see where I’m going as I’m going, and if I don’t answer the phone they at least know I’m moving. Which should mean I’m okay, unless I’m wedged under a truck getting dragged down the road. I haven’t figured out a technical solution to detecting that, so I’m leaving that corner case off the table for now.

However, the only way I can see to get info out of Latitude is through the iGoogle widget. Not a bad way for it to work, but I want to just post my location to my blog. Seriously, I don’t give a crap about privacy. I wanna publish where I am for everyone. And I’m not going to get my mom and my sisters to sign up for Google accounts, just not gonna happen. Is there a way to do a background update of location info and publish it your own website for example? Or a system where I can suck it back out easily and map the results?

Another option is to geotag photos and publish them on Flickr. That works, but requires that I’m taking photos in order to update folks about where I am. Perhaps a cool addition, but doesn’t get to the root problem I’m trying to solve, which effectively comes down to passively instrumenting myself using my cell phone and publishing that info as realtime as possible online publicly. I’m sure I could knock together an Android app to do it pretty quickly, but I have to assume there’s something out there already.

There’s a package now for installing X11 on the G1, which builds on the Debian for G1 package. I haven’t fooled around with the stuff yet, but what am I interested in running with access to the full set of Debian software? Well that should be obvious. Metaspoit of course. Cellular interface + Wifi interface + handheld device + automated security scanning = fun!

I saw some random references to something called Sexy View, malware aimed at Nokia devices. I was just going to ignore it, but then I realized it appears to be a signed application. Delicious. If nothing else that should allow the response folks to track down where it came from I would assume. The reports out there are vague so far at best, but I’m hoping at some point something will shed some light on how this came about. I’m assuming something happened like some company got careless (or went out of business and just ignored) their signing key for applications, and some malicious party got hold of it. Very curious about this I am.

A friend over at Google provided me with an unlocked Android phone, which I’ve been poking around with here and there doing some development. I hadn’t really poked around with the phone itself all that much, meaning looking at hacks and usage tricks and whatnot. Yesterday I decided I wanted to update to the latest firmware (RC33) however, so the journey began. I have my G1 running on the AT&T network, which is where some of the complexity comes from.

I started out with the normal instructions for manually installing the RC33 update, which are pretty actually slick and simple. However, I was getting a failed check when trying to install the firmware. I assume there’s some internal ID that identifies the phone as a TMobile phone or not, and my dev phone doesn’t have that, so the installer was throwing some assertion failure check. So I figured I would try out one of the jailbroken images if I’m going to have to muck around for a while anyway.

There is an alternative version of the RC33 update available, called the JF33 sometimes, and other times the JF RC33. The JF stands for Jesusfreak, the forum user who repacks the roms and has apparently provided test keys that can be used to downgrade a unit. Once you know what you’re looking for, there are tons of posts in forums and blogs all over the place. The problem is just figuring out what you need to search for.

The process goes something like this:

• downgrade to RC29 so you can get root access
• hack the activation so that you can activate your RC29 phone on ATT
• now you can install the recovery keys and load the JF33 rom

It was a bit of an involved process to get through. And at first when I installed the RC29 firmware and had to re-activate, I was afraid I wasn’t going to be able to do so without a TMobile SIM. Just a bit of APN hackery took care of that however. Phew! Now I have Latitude working in maps, and root access in the terminal. W00t! Back to developing now.