Comments on: WURFL Lightweight for PHP http://www.thisismobility.com/blog/2006/06/21/wurfl-lightweight-for-php/ Ripping mobility from the clutches of telecom Fri, 21 Nov 2008 13:30:58 +0000 http://wordpress.org/?v=2.6.3 By: Chris Abbott http://www.thisismobility.com/blog/2006/06/21/wurfl-lightweight-for-php/#comment-24874 Chris Abbott Sun, 30 Jul 2006 09:51:32 +0000 http://www.thisismobility.com/blog/?p=141#comment-24874 Apart from all the UAProfiles, there's a new approach to the device detection problem in the "DetectRight" link to have a play with. You might also want to check out what happened at the W3C Device Description Workshop in Madrid in July to see how the industry proposes to address the problem of everyone writing their own device repositories :) Chris Apart from all the UAProfiles, there’s a new approach to the device detection problem in the “DetectRight” link to have a play with. You might also want to check out what happened at the W3C Device Description Workshop in Madrid in July to see how the industry proposes to address the problem of everyone writing their own device repositories :)

Chris

]]> By: miker http://www.thisismobility.com/blog/2006/06/21/wurfl-lightweight-for-php/#comment-17482 miker Mon, 10 Jul 2006 23:17:35 +0000 http://www.thisismobility.com/blog/?p=141#comment-17482 Ooo, I didn't know about the UAProf to WURFL script, I'll have to look that up. Thanks! Ooo, I didn’t know about the UAProf to WURFL script, I’ll have to look that up. Thanks!

]]> By: Strangiato http://www.thisismobility.com/blog/2006/06/21/wurfl-lightweight-for-php/#comment-17278 Strangiato Mon, 10 Jul 2006 16:27:46 +0000 http://www.thisismobility.com/blog/?p=141#comment-17278 Don't have speed figures for you atm. I don't actually use WURFL at my current position, but I am thinking of using it for a small personal project and was debating between using the Java vs PHP for that. I noticed the warnings about the speed of the PHP version and decided to just take a look and hack away for the heck of it. As for an adaptive WURFL, what has worked in the past for me is to record once for each unique user-agent during the day, the full set of HTTP headers. As a nightly process check for unknown user-agents. If a UAProf is given, use that to generate device information if possible, as a last resort use the HTTP accept headers to update device information. A UAProf to WURL entry utility is available on the WURFL site, (I think in ruby) but simple enough to do in PHP/Perl/Python. Then you'd run the wurfl update script if needed. Not realtime...but then again for the small project i'm working on, realtime device updating isn't critical. Don’t have speed figures for you atm. I don’t actually use WURFL at my current position, but I am thinking of using it for a small personal project and was debating between using the Java vs PHP for that. I noticed the warnings about the speed of the PHP version and decided to just take a look and hack away for the heck of it.
As for an adaptive WURFL, what has worked in the past for me is to record once for each unique user-agent during the day, the full set of HTTP headers. As a nightly process check for unknown user-agents. If a UAProf is given, use that to generate device information if possible, as a last resort use the HTTP accept headers to update device information. A UAProf to WURL entry utility is available on the WURFL site, (I think in ruby) but simple enough to do in PHP/Perl/Python. Then you’d run the wurfl update script if needed. Not realtime…but then again for the small project i’m working on, realtime device updating isn’t critical.

]]> By: miker http://www.thisismobility.com/blog/2006/06/21/wurfl-lightweight-for-php/#comment-15781 miker Sat, 08 Jul 2006 15:51:11 +0000 http://www.thisismobility.com/blog/?p=141#comment-15781 I actually had the hash as a two level structure first, and then moved it off to individual files. Even with an accelerator in place to cache a compiled version of the agents files, reading in the entire array structure bloated the runtime some. For the test set I had, keeping the array parts distinct helped out. How much of a difference did #2 make for you? I'm considering doing that, but I'm not sure if the result is just going to be that the block cache layer at the OS level just needs to cache more files. With the existing ssytem there is the benefit of having a smaller on disk working set to cache. I actually ended up using a direct check before scanning, but I didn't bother posting it. Turns out it only hits an exact match 30% of the time and didn't make much of a difference. I'm considering trying to make WURFL adaptive, so that it learns new User Agents as it sees them. That was one of the benefits of agent2id, even though the cache of capabilities detracted from the lookup benefits for me. I'm thinking of putting something of the sort back in there. I actually had the hash as a two level structure first, and then moved it off to individual files. Even with an accelerator in place to cache a compiled version of the agents files, reading in the entire array structure bloated the runtime some. For the test set I had, keeping the array parts distinct helped out.

How much of a difference did #2 make for you? I’m considering doing that, but I’m not sure if the result is just going to be that the block cache layer at the OS level just needs to cache more files. With the existing ssytem there is the benefit of having a smaller on disk working set to cache.

I actually ended up using a direct check before scanning, but I didn’t bother posting it. Turns out it only hits an exact match 30% of the time and didn’t make much of a difference. I’m considering trying to make WURFL adaptive, so that it learns new User Agents as it sees them. That was one of the benefits of agent2id, even though the cache of capabilities detracted from the lookup benefits for me. I’m thinking of putting something of the sort back in there.

]]> By: Strangiato http://www.thisismobility.com/blog/2006/06/21/wurfl-lightweight-for-php/#comment-15528 Strangiato Fri, 07 Jul 2006 23:01:05 +0000 http://www.thisismobility.com/blog/?p=141#comment-15528 I like your ideas for speeding up the WURFL for php. Using only multichache layout for the files and getting rid of the agent2id file really cleans up the code quite a bit. I used your approach with a couple of tweaks. 1. Instead of breaking the user-agent to wurfl id into multiple little cache files based on the first to characters of the user agent, I simply made that hash structure 2 levels deep. The first level contains keys which contain the value of the user agent up to the first slash. That key then points to a hash containing the full user_agent to wurfl ids for agents that start with similar values. This is much simpler as there is only one file to worry about and you don't have to scan the ENTIRE key set of the user agents. You can quickly tell if a similar agent exists, and if it does you only have to scan a handful looking for the best one. (Which brings up another point, first check if there is an exact match using isset() before trying the linear scan looking for the longest match.) 2. I went ahead and stored in each device cache file the entire capability set. Its trivial during parsing to look these up, and when loading a device's capabilities at runtime you only have to load a single file. The resultant files are all around 16k. (An extra benefit to this is that it is easily adaptable to storing these as blobs in a database.) 3. Minor tweaks that probably aren't worth the speedup. 3a - Modified the matchlen() function to scan each user_agent character by character rather than repeatedly calling substr(). Theoretically scanning a string once is faster than scanning a string over and over (1+2+3+4+5...etc) 3b - getDeviceCapability() seemed to be doing something odd, it was linearly scanning all the capabilities looking for a key match. The structure is a hash, why not use isset() and directlly return the value requested if it exists, much faster. I like your ideas for speeding up the WURFL for php.
Using only multichache layout for the files and getting rid of the agent2id file really cleans up the code quite a bit.

I used your approach with a couple of tweaks.

1. Instead of breaking the user-agent to wurfl id into multiple little cache files based on the first to characters of the user agent, I simply made that hash structure 2 levels deep.
The first level contains keys which contain the value of the user agent up to the first slash. That key then points to a hash containing the full user_agent to wurfl ids for agents that start with similar values.
This is much simpler as there is only one file to worry about and you don’t have to scan the ENTIRE key set of the user agents. You can quickly tell if a similar agent exists, and if it does you only have to scan a handful looking for the best one. (Which brings up another point, first check if there is an exact match using isset() before trying the linear scan looking for the longest match.)

2. I went ahead and stored in each device cache file the entire capability set. Its trivial during parsing to look these up, and when loading a device’s capabilities at runtime you only have to load a single file. The resultant files are all around 16k. (An extra benefit to this is that it is easily adaptable to storing these as blobs in a database.)

3. Minor tweaks that probably aren’t worth the speedup.
3a - Modified the matchlen() function to scan each user_agent character by character rather than repeatedly calling substr(). Theoretically scanning a string once is faster than scanning a string over and over (1+2+3+4+5…etc)
3b - getDeviceCapability() seemed to be doing something odd, it was linearly scanning all the capabilities looking for a key match. The structure is a hash, why not use isset() and directlly return the value requested if it exists, much faster.

]]>