Technology

Information about the plumbing and structure

The Web is an App (Via Diego)

Diego is updating his blog again! Awesome! Check out his “the web is an app” post for a laydown of how generalized web development is shifting. Things were already tilting in the direction of “websites” being basically a static javascript loaded that pulled in resources and data as necessary. Mobile devices have really thrown a few cans of gas on the fire however. Before it was just annoying when logic crept into display code. Now with multiple distinct front-end views it’s gone from annoying to painful.

The role of browser Javascript framework on the mobile end seems to be an area of hot contention right now. With behaviors optimized for touch devices being the headline item in most of the conversations I’m seeing. The Uxebu blog and Wolfram Kriesing’s twitter feed are good places to look if you’re interested in following how it evolves. One of the most significant recent changes is the rollup of the ExtJS, JQTouch, and Raphael projects into a new company and product called Sencha. They seem to be ahead of the curve in pulling everything together to build the mobile specific web interface on top of REST services the way that Diego is talking about. And their recent large investment from Sequoia would seem to indicate that they’re in good shape to extend that lead. Check out their online KitchenSink demo to see some of the interaction styles they support. Still lots of work to be done, but I’m hearing more and more developers saying they’re at least starting to evaluate toolkits like these for doing their mobile web UI.

Removing the Password from a PDF

I have a few ebooks I’ve purchased online that came as password protected PDFs. While mildly annoying while trying to read them on a desktop system, it’s patently absurd when I try to move them over to the iPad to use. There are a bunch of hacks floating around describing how to use conversion software to remove the password. Such as convert to a postscript file and then back to a PDF. Generally you end up with a pretty crappy PDF out the other end. You always loose hyperlinks (table of content or index), and a lot of times the formatting can get screwy. Fortunately I found a few comments mentioning qpdf, which is in the default repos for Ubuntu at least:

  • sudo apt-get install qpdf
  • qpdf --password=******** --decrypt lame_pass_version.pdf happy_version.pdf

Yay! No more crashing iBooks trying to read stuff I paid money for. And there’s an actual preview of the cover on the bookshelf now. Amazing.

WordPress Updated

I did the update to WordPress 3.0 and added a new theme for the blog (you’ll have to actually visit the site to see, most of you read through RSS). Most of my motivation was getting something in there that looks better on mobile devices. The theme I have on now does a completely fluid layout instead of fixed width, so it looks a lot better on iPhone/iPad/Android. Also on the list of stuff to try out is the WordPress Mobile Pack that Andrea and James released.

Definitely recommend doing the upgrade and switching around themes. Especially for viewing on iPhone/Android devices. The Mystique theme is what I’m currently using by the way. On the settings page it has a toggle for fixed/fluid layout. Ahh, nice and simple.

Web Apps in Chomp

We made what I’m hoping is going to be a major change to Chomp today, we’ve put HTML web applications side-by-side with native applications in our discovery service. With all the talk around mobile web applications recently, and then the showcase of web stack applications that Apple released last night, seems like the timing couldn’t have been better. Awesome there, very happy about that. But what I’m more hoping for is that this is the very start of a trend toward increased attention and effort going into mobile web apps. A much longer term goal.

I’ve been having the discussion about the tension between mobile web apps and native mobile apps for years now, and it seems like the whole ecosystem has been “just about to tip” for that entire time. The same issues keep coming up over and over again (distribution issues, on-device presence, access to native APIs and local data, offline usage, and montization models being more abstracted through the web than when direct sales are possible) – but it’s finally looking like enough of them have been taken care of to make the system capable of supporting significant business.

The last major chunk of the problem that I attacked was the monitization side at AdMob, who have done a fantastic job of making it possible to make money writing mobile web apps. Apple and Google have put webkit in hands, which starts to take care of native API access, offline usage, and on-device presence (though we have a ways to go there still, solid progress is being made). So the one remaining large issue, distribution. I think if we can figure out how to start getting people to mobile web apps in decent numbers, most of the other problems with the web stack will find a way to work themselves out.

There are plenty of other longer term issues we’re going to have to take a look at, like should an end user even need to know the difference between a native app and web app? As a discovery service and not a marketplace, we don’t have direct control over that – but the answer to the question and the direction of the market is going to have to inform our model of distribution. And does a direct monetization style of web app (web widget, whatever you want to call it) need to exist to blend out the advertising dollars available and raise the total pot of addressable market for mobile web content? Lots more that we need to discuss, but I’m happy that we were able to take a concrete step today toward making mobile web apps more powerful. Hopefully the first of many steps toward putting the mobile web on equal footing with native applications. Of course, if these are the kinds of questions and issues you’re interested in, come join in the effort at Chomp. We’re hiring! :-) Sorry, couldn’t resist.

Symbian Cross-Platform WRT Tools

Symbian released a new version of their Web Runtime Tools. Not many folks seem to be paying attention cause they’re too busy watching Adobe and Apple go at it. But still, it happened. I was surprised to see releases for Mac and Linux there too, which is awesome. Development tools being Windows only in the past was one of the major issues keeping me away from Symbian as a hobby platform.

I downloaded and installed the release, fooled around with the examples in the quickstart, and it’s working! I installed Chrome from the developer channel (wasn’t a big deal to do) and the emulator is even working… a completely unexpected and welcome change for Symbian development!

WRT widgets can also populate content on the home screen now too. Very interesting. Of course, this all happens at a point where the most recent Nokia device I have doesn’t support any of this. And purchase of a Nokia device isn’t anywhere on my roadmap. So I can’t try any of this stuff out in the really real world as I would like to. This is a really great move in the right direction. I just hope it’s not coming too late.

Tyranny vs Fragmentation

Like it or not, it’s one or the other. Either someone controls the whole system and enforces a standard, like Apple with the iPhone. Or folks are allowed to do what they want, and differences in opinion end up leading to multiple versions in circulation, like Android. There are shades in between, but those are your two extremes. What weirds me out currently however is that the same people seem to complain about both. They complain about Apple exerting control over their own platform, and they also complain about Android being fragmented. You’ve gotta pick one, like it or not.

Actually, that’s not strictly true. There is another option. You could, you know, compete. It takes is a few hundred million dollars, awesome user experience designers, pushing the boundary of what handheld hardware can do, writing a platform from the ground up, capturing the imagination of developers, and delivering that solution via a channel that’s struggling with its own identity crisis. Not many folks seem to be taking that route however. Odd…. Plenty of folks willing to piss and moan about it, but just not enough to get people over that hurdle into action.

Why not? Cause the “horrible tyranny” of the app store is something that people have opted into. If the tyranny were really that horrible people would stop buying iPhones. Period. So even though there are a bunch of vocal people whose agendas aren’t served by the current state of the iPhone ecosystem, the only reason that ecosystem has enjoyed the success it has is that it does hit the sweet spot for users. And even the folks who whine the loudest about how horrible the App Store system is usually can’t stand to walk away from a platform that actually serves their needs. Take a wander to one of the Adobe offices at some point and count the number of iPhones you see. Quite a few it turns out.

On the other side of the fence, yes you always get some degree of fragmentation with an open system. But the severity of that fragmentation is a function of how well any one individual branch serves the needs of end users. Once one branch starts doing a fantastic job of serving users well and attracts more users, that starts to attract more developers. More developers means more applications, which gets more users. And as long as the folks behind that one branch can execute well, you get this nice virtuous feedback cycle going. In theory. A good place to see it actually working is the Linux kernel itself. Anyone can fork off major versions if they want to, but generally no one does cause Linus is fantastically good at his job.

However, thus far, Android is not a fantastic place to see it happening. There’s no clear strong front-runner in terms of device manufacturer and sub-version of Android, so it feels like a lot of fragmentation. If one of those versions had 95% of the market however, no one would really care about the “fragmentation”, they would build to the successful version. That’s not up to the developers and manufacturers to mandate however (cause then we would be back to tyranny, right, which is what we’re explicitly trying to avoid), the success of one particular version needs to be chosen by end users. So how do you get to that point. Well, it’ll probably take a few hundred million dollars of backing, awesome user experience and design, top notch hardware, etc…

Over the last three years we’ve finally managed to drag mobile out of the rut its been stuck in for a decade (Okay, Apple dragged the whole ecosystem out of the rut damnit – I personally had little to do with it, though I would have loved to). But now the conversations seem to be cycling back to the same issues that got this thing gridlocked in the first place – channel control and fragmentation. I hope we’re not falling into yet another rut. Maybe whoever ends up buying the smoking remains of Palm can turn that into yet another viable option?

Although I hate to admit it, it might be that mobile just isn’t really ready for genuine evolution yet. There’s a market evolution theory says that initially technological solutions demand a tightly integrated environment so that the end product can be of high enough quality to serve the general public. Then, eventually, the product is good enough that it actually over-serves most of the public. At that point modular solutions and customization are the name of the game, as users mix and match what they want depending on individual needs.

Everything is setup for the next phase of evolution to be possible. Linux and Symbian providing open operating systems, Modu and Bug Labs providing hardware platforms, and folks like DoubleTwist providing a media and device management tool. All the parts are there. In theory we should be in a great position to see the major changes that happened over the past three years to accelerate and continue to roll along. But it’s not happening. Makes me feel all itchy. Like I’m missing something that I shouldn’t be missing.

Opening Up Mobile Monetization

Interesting post at Mobiletech : Mobile Web vs. Native Apps. Revisited. Obviously, working at Chomp to help prop up the discovery end of the app ecosystem and funneling ourselves some of that sweet sweet app store revenue via affiliate programs, I do a whole bunch of speculating about web apps vs native apps. Particularly wondering if we’re painting ourselves into a corner working through the app stores. What I’ve ended up doing is just assuming that the browser will be a good enough delivery mechanism for applications, and it frees you to think about the really important issues. Will it be with HTML5 that we see native equivalent feel for web apps? Or with HTML6? Or HTML14.8? Ultimately I’ve decided it doesn’t matter. It’s a technical problem, and technical problems have a way of disappearing over time. And disappearing almost immediately when the right motivations are in place.

So what are the real underlying issues? I think John Arne is right to call out business models as the big issue standing in the way. I want to dig a bit deeper on that one however. Why aren’t the monetization models on mobile as diverse and rich as they are out on the web in general? It’s because generally the only thing people buy on their mobile phone is stuff for their mobile phone. If I take a look at what I’ve purchased with my iPhone over the last month, it’s an app here and there, maybe a song or two (okay, I didn’t buy any music last month – so it’s a bad example month.. but you get the idea). On the other hand, what did I pay for online in the last month? A couple of books, bits of furniture for the new apartment, hotel reservations, more electronics that I should probably admit to.

I would argue that’s the actual significant shift which put the web smack in the middle of much of the interesting change we’ve seen over the last 15 years. If all I could get via the web was information and software I think the web would have had a lot less impact. However, once I could get stuff, real life things shipped or waiting for me as a result of activity online, that opened up a whole new world of monetization methods. Why? Cause without the real world stuff the only people willing to advertise are folks selling software or subscriptions. With real world stuff the pool of advertising and promotional dollars skyrockets, and the associated ceiling for earning via indirect monetization gets lifted.

So instead of tracking if it’s going to be technically feasible to develop a native-feeling-enough HTML5 app and using that to determine if mobile development is going to tip to the web, I’ve been watching folks looking to push the business models out. Bringing offline commerce to mobile handsets is probably the reason folks like Gowalla and FourSquare are viewed as hot tickets by the investment community, and not just fads. And I don’t think it’s an accident that lots of folks who have had positive experiences with the app store, such as Flixster, have a component of their app that’s based on in-app purchases of offline goods (movie tickets in this case).

The two common subcases of the discussion that don’t move things offline (in-app purchases of digital goods, and paid distribution for web apps) are effectively just tweeks to the existing model. They don’t really unlock any new value, they just shift around where the customer decision point is and potentially unlock a larger portion of the same pool of revenue. But they don’t fundamentally shift things in a way that give access to previously untapped pools of money. They’re problems worth addressing, but they’re really the blunt end of the stick.

The promising part is that really there’s no technical barrier in the way of opening up mobile services to offline commerce. Much of the same systems that we have for online commerce work just as well on a handset. But just like with the web, having the technology available doesn’t mean you have a system that works for users. Apple has set the expectation at a certainly level with respect to “purchases on an iPhone”, being able to tie into an existing payment method has left users with an expectation for everything else to operate as smoothly. So yes, there’s definitely some work to be done, hard problems to be solved. And I think that’s really the problem to be tracking here with respect to mobile web or native app development.

Starting up a Posse

The wraps are off the latest project I’ve been working on: Chomp, a social iPhone application recommendation service. I actually started consulting with them to fill in some time, but liked the app and the people so much that I couldn’t resist jumping in full time. Now that the app is out and I can talk a bit more publicly it’s time to start building up a team. Obviously, about the app itself, the reception has been great and folks seem to love it:

  • Geek.com – the one I’m most proud of
  • Techcrunch – obTechCrunchShoutout
  • ABC News – on the TV!! Not really, they’re talking about us, but the guy is just randomly poking around on an iPhone. Poor TV, you just don’t get it do you?

The goals of the business don’t stop at an iPhone app, we have plans to move into other areas over time. But the iPhone app market is the biggest juiciest lowestest hanging piece of fruit in the mobile arena right now. It would be crazy not to take a bite. Ben can answer a lot more of those style questions than I can however. I’m just here to keep the lights on and the servers running.

We have a jobs page up, but there isn’t too much color to them yet. In particular, I need some folks to join me on backend engineering and operations tasks, so I figured I would shout out here. Right now the team is only 5 people, so it’s a chance to get in at the very start for the right folks. We haven’t yet figured out exactly how the jobs break down, but at a startup job titles don’t really mean that much anyway. Here’s a rundown of the kinds of things I’ve been doing for the last two months, to give you a feel for what we have going on:

  • Picked up and finished off a spin of the server side software. Standard LAMP stack with the PHP side built on top of CodeIgniter. There was some general cleanup work like reformatting the server responses to be more consistent across all the API calls, some new features like adding in bookmarking and avatar uploads, and generally operationalizing the code with stats logging, database migrations, processes for background tasks and cleanup.
  • Prepping the production deployment systems. Which in this case meant getting Puppet up and going, setting up the modules for the bits of software we were planning to use, bits of custom config for things like the database systems (we’ve tried to set them up to be sharding-friendly right off the bat), and generally setting up the parts of the system we didn’t need till we went live (like load balancing and spreading out the front end web systems, and syslog-ng to pull together all the logs to a central system)
  • Setting up system and application monitoring and metrics, so that we have alerts that fire if something seems to be going wrong and pretty graphs to show us where that wrong thing might be.
  • Doing a few spins of designing a custom crawler that pulls info out of iTunes so that we can make it available in our app. With the number of apps, the number of regions, the number of languages, and the general lack of any kind of off-the-shelf solution for this, it’s been a pretty complex problem to work out. Our data is pretty timely right now, but this is one area in particular where there’s a lot of room for improvement.
  • Adding in some very simple strategies for caching and load shedding if we ever hit the kind of load where we were in danger of degrading performance. Fortunately we’ve yet to have to pull any of the escape valve levers (fingers crossed), but with the system growing as quickly as it’s been growing I’m sure that’s only a matter of time.

Generally, it feels great to be back at the controls of a set of servers experiencing the kind of explosive growth we saw during the early days at AdMob. But there’s definitely more to do than I can handle all by myself. And I know there are plenty of folks out there who are better at the bits and pieces of this than I am, so I would love to be able to peel some of this stuff off to the right people and start scaling the human side before scaling the machine side gets out of control. Plus, when the weather gets nicer, I spend way too much time at the track to be the only person available to respond to ops issues :-)

So if you’re interested, particularly you folks who I’ve worked on projects with before, ping me. mike at chompapps dot com. Yes, this means I’m going to start reading my email again now.

Zend Server Community Edition vs Snow Leopard

After upgrading to Snow Leopard we’ve had only one real issue so far, Zend Server Community Edition didn’t want to start up. Well, technically, parts of it. The apache instance was running, by mysql and the admin interface were dead. I found this post about watchdog errors, but even after putting in the fix for lighthttpd I still wasn’t getting the services starting up. Same deal for Tony. Poking around in the startup files it looked like mysql data was owned by user 103, and the mysql scripts were trying to start as user ‘zend’, however I have no zend user on my system. I wasn’t able to find anything about it, but I figured hell, let me give it a try and create a user Zend. This is the command to run from the terminal (funky huh?):

sudo dscl . -create /Users/zend UniqueID 103

Which creates a zend user.. somewhere.. I’m not familiar with that bit of OS X magic yet. It’s not in /etc/passwd, but if you ‘ls -l /usr/local/zend/mysql’ you should see the data directory owned by zend again. Now just restart and you should be peachy:

sudo /usr/local/zend/bin/zendctl.sh restart

Works for Tony and I at least, so we figured we would share. голова болит секс голова болит секс

Not Sure I Buy the Android Fragmentation Argument

There seems to be a lot of “take it for granted” style discussion around the possibility of Android fragmentation. Without strong top down control of the platform, folks seem to think that we’re going to end up with hundreds of versions of Android, all slightly different. This was the nightmare that mobile Java became. There were large engineering teams dedicated to and specialized in taking your write-once-run-anywhere app and actually getting it to work on the volume of handsets you wanted to hit.

It’s true that technically anyone who wants to build a slightly different version of Android can, so in that sense there CAN be fragmentation. However, the ecosystem has also fundamentally changed. Before we had the carriers tightly controlling the channel, dictating what was in and out in terms of hardware, enforcing strict standards on the software they pushed to users, and doing everything they could to keep anyone else from pushing software to users. In that world the burden of dealing with fragmentation was with the developers, and the benefit went to the carriers. The carriers controlled the channel, so fragmentation continued.

The one lesson that everyone in mobile seems to have learned over the last year was that the carriers were really bad at determining the right hardware and managing that application and content catalog. It’s why everyone is jumping on the App Store Bandwagon. One of the side-effects of that is that it breaks the strongly controlled content and application channel. If ATT decides that their Android version is going to do Bluetooth slightly different, sure, go ahead. But how can they strongarm Android developers who produce Bluetooth apps into making an ATT specific version? The control isn’t really there any more. Developers might do it, but only if ATT is offering up enough incremental sales revenue to make the port worthwhile. Right now developers frequently do it cause they’re contractually obligated to if they want ATT to promote their app.

Anyone who’s worked in open source knows that forks are technically possible, but practically uncommon. When the “market” is completely open folks tend to follow a path that serves them best. And it’s a reinforcing path. Even when someone with vested interest tries to keep a fork distinct, normally the burden ends up being more on the controller than anyone else. Forks tend to get folded back or die off pretty quickly. I think the same thing should happen with Android. Sure, people might try at the start to get a leg up by including proprietary features and customizations. But in the long run if Android as a whole works, the only person they’re hurting with a fork is themselves (to the tune of a decreased application catalog). And although it’s possible for them to create a custom application catalog with some differentiation in the short term and attempt to keep that differentiation going, there’s no difference between that and the strongly controlled channel that we currently see failing today. The cost of trying to do so will outweigh the benefit of joining the mainline. толстые проститутки