I’ve been using the PHP interface for WURFL on a few projects recently and always had some nagging questions about how the thing works in terms of caching and performance. So today I really dug into the stuff that’s there to find out why WURFL seems to take longer than what I expect it to. First I should mention that I might have a non-standard workload going on here. The site that I’m trying to optimize gets traffic from all over the place, and WURFL seems to be optimized for a small working set of maybe two dozen or so active sessions.
My test was realistic however, I grabbed a stream of traffic from the website I wanted to optimize, snagged the user agent from each of 500 requests, and then replayed those user agents in the order they came into the site. My test script just loaded up WURFL and did a GetDeviceCapabilitiesFromAgent(), which is pretty standard behavior from what I was able to tell looking at what other folks do. The starting values hovered around 81 seconds to fulfill 500 requests.
The way the caching works in WURFL the scripts end up including a lot of PHP that’s actually data var_dump()ed by the parse routines. So the obvious place to start was trying to speed up loading of files, and APC seems to be the favored accelerator cache these days. It made a pretty decent difference actually, with the average time dropping down to about 67 seconds after installing APC and doing nothing else.
Matching was the next obvious place to look, so I started checking out the logic in GetDeviceCapabilitiesFromAgent() that does the matching against agent strings. I was surprized to see that the code seemed to be doing a linear scan through the agent strings trying to find one that matched, including an optimization to subgroup by a match against the first four characters of the agent, and relooping with sucessively smaller substrings until one of the agents matches. About a dozen different ideas popped to mind about how to fix that up some, but the quickest to implement was turning the agent array used as the main cache element (cache.php, which creates wurfl_agents) from a flat array indexed by full agent strings into a two leveled array indexed by the first four letters of the agent strings (which is naturally very sparse) and then holding a subarray of agents. That paid off, but only a little bit, the average time was now about 60 seconds.
If a major change like that to the matching logic didn’t improve the performance drastically there must be something in there that I’m missing. So I figured maybe this fast cache agent2id thingie is too small. So I up the number of entries in that cache to 60 from 30 and retry the tests. The benchmark goes UP this time. Interesting, very interesting. So I actually take a look at the file and realize that the thing is hundreds of kilobytes long. And with the frequent switches of user agent forcing rewrites of that file, for my workload at least that cache file is causing tons of thrashing. I turned it off completely and reran my tests. 10 seconds to process the 500 agents. Now that’s more like it!
Going back and fooling around with it some it looks like I can get down to 25 seconds with simple APC and removing the use of the WURFL_AGENT2ID_FILE. If you’re seeing a lot of load from scripts using WURFL I suggest checking that out, playing with the config file if you’re not comfortable hacking the code. It looks like there’s a lot of savings that can be wrung out of that code if your workload is outside the expected.