Monday, September 21, 2009

RSSCloud

(The below post is relatively technical)

Over at Dave Winer's scripting.com blog there's work being done to determine how to best discover RSS feeds and keep them up-to-date.

Mr. Winer has defined a solution which consists of the following:

- Create a web service with an HTTP-to-DNS API
- Use the above to enter in a unique subdomain a TXT record into a whose sole string is the url for the feed
- Also create an HTTP proxy service that exposes the TXT record to a web browser, when such browser is requests the subdomain

This works, except that TXT records are definitely sub-optimal for this. One should use other types of records more suitable for this, such as NAPTR records. Furthermore, if you have a .tel domain, whether the final implementation uses TXT or NAPTR records, you're already in great shape because you've already got all the tools necessary to support this.

Regarding TXT vs. NAPTR, I suggest using NAPTR records of type 'x-rss' (and also 'x-opml') such that a DNS query to (for example) rss.asseily.tel will look like this:

100 100 "u" "E2U+x-rss:http" "!^.*$!http://rikkles.blogspot.com/feeds/posts/default!" .


The DNS query to get the above is:
dig +short rss.asseily.tel naptr


A NAPTR record has the following advantages over a TXT record:

- It's got an enumservice type ('x-rss:http') which allows clients to understand what it is ('x-rss') and how to access it ('http'). TXT records don't have any of that, which can lead to much confusion unless you structure them accordingly (i.e. have multiple strings, the first one being the type).

- It's got an order and a preference (both 100 in the above case), which allows you to specify multiple ordered URLs for the same feed, and allows you to have multiple feeds in the same subdomain: each unique feed has the same unique order number, and for each unique feed you can have multiple urls ordered by preference. Note that I didn't invent this usage, it's standard for NAPTR records

- The URL that you see in the NAPTR record is actually called a 'replacement', because that's what it is. It 'replaces' the request made, and is in fact a fully qualified regular expression. In the above case, we're saying "please replace the whole query with 'http://rikkles.blogspot.com/feeds/posts/default'. The actual request made was for 'rss.asseily.tel'. That request is now replaced with the give URL. Because this is a regular expression, you could have been a lot more specific with the replacement if you wanted to

Since I'm using a .tel domain, I've got access to an API. The PHP code to create this RSS entry in the DNS is below:


include('Telhosting_Client.php');

$naptr1 = array(
'order' => '100',
'preference' => '100',
'services' => 'E2U+x-rss:http',
'flags' => 'u',
'regexp' => '!^.*$!http://rikkles.blogspot.com/feeds/posts/default!',
'owner' => '@',
'profiles' => '_all_',
);

$config = array();
$config['login'] = '****';
$config['password'] = '****';
$config['wsdl'] = 'my.wsdl';
$domain = 'rss.asseily.tel';

$client = new Telhosting_client($config);
$client->store_record($domain, 'naptr', $naptr1);


Can't be much simpler than that...

3 comments:

Stéphane Bortzmeyer said...

NAPTR have problems, too. For instance, there is zero implementation of NAPTR processing in free software (free as in free speech, not free as in free beer) and it is not obvious to create one, since NAPTR are quite complicated.

rikkles said...

That used to be correct a year ago, but since the advent of .tel NAPTR records have seen a lot more usage. Incidentally, NAPTR isn't very complicated, all it has is a couple more fields than a TXT record.

You can find a lot of relevant links here:
http://dev.telnic.org/pages/downloads.html

Specifically:
- For C/C++ code, look at the kiax softphone
- For Objective-C code (iPhone) look at my open source apps Superbook and My.tel
- For C#/.Net, look at the source code of the Windows Mobile app.
- For Java code, look at the source code of the Blackberry app.
- For Perl code, look in CPAN for Net::Lookup::DotTel

If you need other code, please let me know.
H

sull said...

I reg'd rsscloud.tel about a month ago when we were all first discussing this stuff on scripting.com. this was my suggestion, along with letting users setup subdomains to bootstrap one option for rsscloud centric identities. in effect, an exercise.

i'll prob still move that forward and prefer to use the REST API instead of SOAP. can you ping me? on gmail - sulleleven. thanks.

sull