i’m a lumberjaph

We have the model, the aggregator (and some tests), now we can do a basic frontend to read our feed. For this I will create a webapp using Catalyst.

Catalyst::Devel is required for developping catalyst application, so we will install it first:

    cpan Catalyst::Devel

Now we can create our catalyst application using the helper:

    catalyst.pl MyFeedReader

This command initialise the framework for our application MyFeedReader. A number of files are created, like the structure of the MVC directory, some tests, helpers, …

We start by creating a view, using TTSite. TTSite generate some templates for us, and the configuration for this template. We will also have a basic CSS, a header, footer, etc.

    cd MyFeedReader
    perl script/myfeedreader_create.pl view TT TTSite

TTSite files are under root/src and root/lib. A MyAggregator/View/TT.pm file is also created. We edit it to make it look like this:

    __PACKAGE__->config({
        INCLUDE_PATH => [
            MyFeedReader->path_to( 'root', 'src' ),
            MyFeedReader->path_to( 'root', 'lib' )
        ],
        PRE_PROCESS  => 'config/main',
        WRAPPER      => 'site/wrapper',
        ERROR        => 'error.tt2',
        TIMER        => 0,
        TEMPLATE_EXTENSION => '.tt2',
    });

Now we create our first template, in root/src/index.tt2

    to <a href="/feed/">your feeds</a>

If you start the application (using perl script/myfeedreader_server.pl) and point your browser on http://localhost:3000/, this template will be rendered.

We need two models, one for KiokuDB and another one for MyModel:

lib/MyFeedReader/Model/KiokuDB.pm

    package MyFeedReader::Model::KiokuDB;
    use Moose;
    BEGIN { extends qw(Catalyst::Model::KiokuDB) }
    1;

we edit the configuration file (myfeedreader.conf), and set the dsn for our kiokudb backend

    <Model KiokuDB>
        dsn dbi:SQLite:../MyAggregator/foo.db
    </Model>

lib/MyFeedReader/Model/MyModel.pm

    package MyFeedReader::Model::MyModel;
    use base qw/Catalyst::Model::DBIC::Schema/;
    1;

and the configuration:

    <Model MyModel>
        connect_info dbi:SQLite:../MyModel/model.db
        schema_class MyModel
    </Model>

We got our view and our model, we can do the code for the controller. We need 2 controller, one for the feed, and one for the entries. The Feed controller will list them and display entries titles for a given feed. The Entry controller will just display them.

lib/MyFeedReader/Controller/Feed.pm

    package MyFeedReader::Controller::Feed;
    use strict;
    use warnings;
    use parent 'Catalyst::Controller';
 
    __PACKAGE__->config->{namespace} = 'feed';
 
    sub index : Path : Args(0) {
        my ( $self, $c ) = @_;
        $c->stash->{feeds}
            = [ $c->model('MyModel')->resultset('Feed')->search() ];
    }
 
    sub view : Chained('/') : PathPart('feed/view') : Args(1) {
        my ( $self, $c, $id ) = @_;
        $c->stash->{feed}
            = $c->model('MyModel')->resultset('Feed')->find($id);
    }
 
    1;

The function index list the feeds, while the function view list the entries for a give feed. We use the chained action mechanism to dispatch this url, so we can have
urls like this /feed/*

We create our 2 templates (for index and view):

root/src/feed/index.tt2

    <ul>
        [% FOREACH feed IN feeds %]
            <li><a href="/feed/view/[% feed.id %]">[% feed.url %]</a></li>
        [% END %]
    </ul>

root/src/feed/vew.tt2

    <h1>[% feed.url %]</h1>
 
    <h3>entries</h3>
    <ul>
        [% FOREACH entry IN feed.entries %]
            <li><a href="/entry/[% entry.id %]">[% entry.permalink %]</a></li>
        [% END %]
    </ul>

If you point your browser to http://localhost:3000/feed/ you will see this:

Now the controller for displaying the entries:

    package MyFeedReader::Controller::Entry;
    use strict;
    use warnings;
    use MyAggregator::Entry;
    use parent 'Catalyst::Controller';
 
    __PACKAGE__->config->{namespace} = 'entry';
 
    sub view : Chained('/') : PathPart('entry') : Args(1) {
        my ( $self, $c, $id ) = @_;
        $c->stash->{entry} = $c->model('KiokuDB')->lookup($id);
    }
 
    1;

The function view fetch an entry from the kiokudb backend, and store it in the stash, so we can use it in our template.

root/src/entry/view.tt2

    <h1><a href="[% entry.permalink %]">[% entry.title %]</a></h1>
    <span>Posted [% entry.date %] by [% entry.author %]</span>
    <div id="content">
        [% entry.content %]
    </div>

If you point your browser to an entry (something like http://localhost:3000/entry/somesha256value), you will see an entry:

Et voila, we are done with a really basic feed reader. You can add methods to add or delete feed, mark an entry as read, …

The code is available on github.

Following matt’s post about people not blogging enough about Perl, I’ve decided to try to post once a week about Perl. So I will start by a series of articles about what we call modern Perl. For this, I will write a simple feed agregator (using Moose, DBIx::Class, KiokuDB), some tests, and a basic frontend (with Catalyst). This article will be split in four parts:

• the first one will explain how to create a schema using DBIx::Class.
• the second will be about the aggregator. I will use Moose and KiokuDB.
• the third one will be about writing tests with Test::Class.
• the last one will focus on Catalyst.

The code of these modules will be available on my github account at the same time each article is published.

disclaimer

I’m not showing you how to write the perfect feed aggregator. The purpose of this series of articles is only to show you how to write a simple aggregator using modern Perl.

The database schema

We will use a database to store a list of feeds and feed entries. As I don’t like, no, wait, I hate SQL, I will use an ORM for accessing the database. For this, my choice is DBIx::Class, the best ORM available in Perl.

If you never have used an ORM before, ORM stands for Object Relational Mapping. It’s a SQL to OO mapper that creates an abstract encapsulation of your databases operations. DBIx::Class‘ purpose is to represent “queries in your code as perl-ish as possible.”

For a basic aggregator we need:

• a table for the list of feeds
• a table for the entries

We will create these two tables using DBIx::Class. For this, we first create a Schema module. I use Module::Setup, but you can use Module::Starter or whatever you want.

module-setup MyModel

Edit lib/MyModel.pm :

package MyModel;
use base qw/DBIx::Class::Schema/;
 
__PACKAGE__->load_classes();
 
1;

So, we have just created a schema class. The load_classes method loads all the classes that reside under the MyModel namespace. We now create the result class MyModel::Feed in lib/MyModel/Feed.pm :

package MyModel::Feed;
use base qw/DBIx::Class/;
 
__PACKAGE__->load_components(qw/Core/);
__PACKAGE__->table('feed');
__PACKAGE__->add_columns(qw/ feedid url /);
__PACKAGE__->set_primary_key('feedid');
__PACKAGE__->has_many(entries => 'MyModel::Entry', 'feedid');
1;

Pretty self explanatory: we declare a result class that uses the table feed, with two columns: feedid and url, feedid being the primary key. The has_many method declares a one-to-many relationship.

Now the result class MyModel::Entry in lib/MyModel/Entry.pm :

package MyModel::Entry;
use base qw/DBIx::Class/;
 
__PACKAGE__->load_components(qw/Core/);
__PACKAGE__->table('entry');
__PACKAGE__->add_columns(qw/ entryid permalink feedid/);
__PACKAGE__->set_primary_key('entryid');
__PACKAGE__->belongs_to(feed => 'MyModel::Feed', 'feedid');
 
1;

Here we declare feed as a foreign key, using the column name feedid.

You can do a more complex declaration of your schema. Let’s say you want to declare the type of your fields, you can do this:

__PACKAGE__->add_columns(
 
    'permalink' => {
        'data_type'         => 'TEXT',
        'is_auto_increment' => 0,
        'default_value'     => undef,
        'is_foreign_key'    => 0,
        'name'              => 'url',
        'is_nullable'       => 1,
        'size'              => '65535'
    },
);

DBIx::Class also provides hooks for the deploy command. If you are using MySQL, you may need a InnoDB table. In your class, you can add this:

sub sqlt_deploy_hook {
    my ( $self, $sqlt_table ) = @_;
    $sqlt_table->extra(
        mysql_table_type => 'InnoDB',
        mysql_charset    => 'utf8'
    );
}

next time you call deploy on this table, the hook will be sent to SQL::Translator::Schema, and force the type of your table to InnoDB, and the charset to utf8.

Now that we have a DBIx::Class schema, we need to deploy it. For this, I always do the same thing: create a bin/deploy_mymodel.pl script with the following code:

#!/usr/bin/perl -w
use strict;
use feature 'say';
use Getopt::Long;
use lib('lib');
use MyModel;
 
GetOptions(
    'dsn=s'      => \my $dsn,
    'user=s'     => \my $user,
    'passwd=s' => \my $passwd
) or die usage();
 
my $schema = MyModel->connect($dsn, $user, $passwd);
say 'deploying schema ...';
$schema->deploy;
 
say 'done';
 
sub usage {
    say 'usage: deploy_mymodel.pl --dsn $dsn --user $user --passwd $passwd';
}

This script will deploy for you the schema (you need to create the database first if using with mysql).

Executing the following command:

perl bin/deploy_mymodel.pl --dsn dbi:SQLite:model.db

generate a model.db database so we can work and test it. Now that we got our (really) simple MyModel schema, we can start to hack on our aggregator.

link to the code.

while using DBIx::Class, you may want to take a look at the generated queries. For this, export DBIC_TRACE=1 in your environment, and the queries will be printed on STDERR.

I’ve decided to rewrite the intention cloud. Still with catalyst, but I’ve replaced prototype with jquery this time. I’ve end up with less code than the previous version. For the moment, only google is available, but I will add overture, and may be more engines.

There is still some bug to fix, some tests to add, and I will be able to restore the intentioncloud.net domain.

It’s really easy to plug a database to a catalyst application using Catalyst::Model::DBIC::Schema. Via the helper, you can tell the model to use DBIx::Class::Schema::Loader, so the table informations will be loaded from the database at runtime. You end up with a code that looks like

package intentioncloud::Model::DB;
use strict;
use base 'Catalyst::Model::DBIC::Schema';
__PACKAGE__->config(schema_class => 'intentioncloud::Schema',);
1;

and the schema:

package intentioncloud::Schema;
use strict;
use base qw/DBIx::Class::Schema::Loader/;
__PACKAGE__->loader_options(relationships => 1);
1;

Now, to do a query:

my $rs = $c->model('DB::TableName')->find(1);

and your done !

The code for the intentioncloud is avaible on github

If you use DBIx::Class and you want to see what the SQL generated looks
like, you can set the environment variable DBIC_TRACE.

    % DBIC_TRACE=1 my_programme.pl

And all the SQL will be printed on STDERR.

If you give a filename to the variable, like this

    DBIC_TRACE="1=/tmp/sql.debug"

all the statements will be printed in this file.