i’m a lumberjaph

This week, with Alexis’s help, I’ve been working on adding auto-(de)serialization to Dancer’s request. This features will be available in the next Dancer version, the 1.170 (which will be out before April).

The basic idea was to provides to developer a simple way to access data that have been send in a serialized format, and to properly serialize the response.

At the moment, the supported serializers are :

• Dancer::Serialize::JSON
• Dancer::Serialize::YAML
• Dancer::Serialize::XML
• Dancer::Serialize::Mutable

Configuring an application to use the serializer

To activate serialization in your application:

set serializer => 'JSON';

or in your configuration file:

serializer: "JSON"

A simple handler

Let’s create a new dancer application (you can fetch the source at github) :

dancer -a dancerREST
cd dancerREST
vim dancerREST.pm

package dancerREST;
use Dancer ':syntax';
 
my %users = ():
 
# create a new user
post '/api/user/' => sub {
    my $params = request->params;
    if ( $params->{name} && $params->{id} ) {
        if ( exists $users{ $params->{id} } ) {
            return { error => "user already exists" };
        }
        $users{$params->{id}} = {name => $params->{name}};
        return { id => $params->{id}, name => $params->{name} };
    }
    else {
        return { error => "name is missing" };
    }
};
 
true;

We can test if everything works as expected:

plackup app.psgi&
> curl -H "Content-Type: application/json" -X POST http://localhost:5000/api/user/ -d '{"name":"foo","id":1}'
# => {"name":"foo","id":"1"}

Now we add a method to fetch a list of users, and a method to get a specific user:

# return a specific user
get '/api/user/:id' => sub {
    my $params = request->params;
    if ( exists $users{ $params->{id} } ) {
        return $users{$params->{id}};
    }
    else {
        return { error => "unknown user" };
    }
};
 
# return a list of users
get '/api/user/' => sub {
    my @users;
    push @users, { name => $users{$_}->{name}, id => $_ } foreach keys %users;
    return \@users;
};

If we want to fetch the full list:

> curl -H "Content-Type: application/json" http://localhost:5000/api/user/
# => [{"name":"foo","id":"1"}]

and a specific user:

> curl -H "Content-Type: application/json" http://localhost:5000/api/user/1
# => {"name":"foo"}

The mutable serializer

The mutable serializer will try to load an appropriate serializer guessing from the Content-Type and Accept-Type header. You can also overload this by adding a content_type=application/json parameter to your request.

While setting your serializer to mutable, your let your user decide which format they prefer between YAML, JSON and XML.

And the bonus

Dancer provides now a new method to the request object : is_ajax. Now you can write something like

get '/user/:id' => sub {
    my $params = request->params;
    my $user    = $users{ $params->{id} };
    my $result;
    if ( !$user ) {
        _render_user( { error => "unknown user" } );
    }
    else {
        _render_user($user);
    }
};
 
sub _render_user {
    my $result  = shift;
    if ( request->is_ajax ) {
        return $result;
    }
    else {
        template 'user.tt', $result;
    }
}

If we want to simulate an AJAX query:

curl -H "X-Requested-With: XMLHttpRequest" http://localhost:5000/user/1

and we will obtain our result in JSON. But we can also test without the X-Requested-With:

curl http://localhost:5000/user/1

and the template will be rendered.

Hope you like this new features. I’ve also been working on something similar for Tatsumaki.

Net::Twitter

I’ve been asked for $work to write an API client for backtype, as we plan to integrate it in one of our services. A couple of days before I was reading the Net::Twitter source code, and I’ve found interesting how semifor wrote it.

Basically, what Net::Twitter does is this: for each API method, there is a twitter_api_method method, where the only code for this method is an API specification of the method. Let’s look at the public timeline method:

twitter_api_method home_timeline => (
    description => <<'',
Returns the 20 most recent statuses, including retweets, posted by the
authenticating user and that user's friends. This is the equivalent of
/timeline/home on the Web.
 
    path      => 'statuses/home_timeline',
    method    => 'GET',
    params    => [qw/since_id max_id count page/],
    required  => [],
    returns   => 'ArrayRef[Status]',
);

The twitter_api_method method is exported with Moose::Exporter. It generates a sub called home_timeline that is added to the class.

MooseX::Net::API

As I’ve found this approch nice and simple, I thought about writing a little framework to easily write API client this way. I will show how I’ve write a client for the Backtype API using this (I’ve wrote some other client for private API at works too).

Backtype API

First we defined our class:

package Net::Backtweet;        
 
use Moose;
use MooseX::Net::API;

MooseX::Net::API export two methods: net_api_declare and net_api_method. The first method is for all the paramters that are common for each method. For Backtype, I’ll get this:

net_api_declare backtweet => (
    base_url    => 'http://backtweets.com',
    format      => 'json',
    format_mode => 'append',
);

This set

• the base URL for the API
• the format is JSON
• some API use an extension at the name of the method to determine the format. “append” do this.

Right now three formats are supported: xml json and yaml. Two modes are supported: append and content-type.

Now the net_api_method method.

net_api_method backtweet_search => (
    path     => '/search',
    method   => 'GET',
    params   => [qw/q since key/],  
    required => [qw/q key/],
    expected => [qw/200/],
);

• path: path for the method (required)
• method: how to acces this resource (GET POST PUT and DELETE are supported) (required)
• params: list of parameters to access this resource (required)
• required: which keys are required
• expected: list of HTTP code accepted

To use it:

my $backtype = Net::Bactype->new();
my $res = $backtype->backtweet_search(q => "http://lumberjaph.net", key => "foo");
warn Dump $res->{tweets};

MooseX::Net::API implementation

Now, what is done by the framework. The net_api_declare method add various attributes to the class:

• api_base_url: base URL of the API
• api_format: format for the query
• api_format_mode: how the format is used (append or content-type)
• api_authentication: if the API requires authentication
• api_username: the username for accessing the resource
• api_password: the password
• api_authentication: does the resource requires to be authenticated

It will also apply two roles, for serialization and deserialization, unless you provides your own roles for this. You can provides your own method for useragent and authentication too (the module only do basic authentication).

For the net_api_method method, you can overload the authentication (in case some resources requires authentication). You can also overload the default code generated.

In case there is an error, an MooseX::Net::API::Error will be throw.

Conclusion

Right now, this module is not finished. I’m looking for suggestions (what should be added, done better, how I can improve stuff, …). I’m not aiming to handle all possibles API, but at least most of the REST API avaible. I’ve uploaded a first version of MooseX::Net::API and Net::Backtype on CPAN, and the code is available on github.

For testing purpose, i’ve set a dumb REST service here (the code is here). I will update this service to add more tests to MX::Net::API.