Using Open-Xchange Server 6 as RSS Client

RSS with OX6

RSS is a very popular way to distribute all kinds of data in a multicast-like fashion. Nearly every news website provides RSS feeds to spread their content. This gets very useful if you do not want to crawl 10 websites and look for recent changes but get as many information as you need. Some RSS Clients exist out there, some integrated in fat-clients like Thunderbird, some Stand-alone like Lifera and some on a web interface like iGoogle. To enhance Open-Xchange Server 6 to a central information powerhouse, RSS support would make a lot of sense. One way to achieve that, would be a integration of a native RSS client to the server. This may be the cause at some time, but currently there is no support for this. You can either wait till it is natively supported, or use some casual magic to make it work without native support. This How-To is for those who want the second one ;)

Preparation - Get your bags packed

I imply that you have already installed and configured a fully working Open-Xchange Server 6 and the underlying mail system. This How-To extends the installation guide, which recommends Debian based distributions. For sure it will work on any other Linux, BSD or MacOS distribution, but i'd like to keep the whole process consistent, thats why i've chosen Debian Etch with Cyrus/Postfix as the base.

First of all, install the following debian packages:

  • libsasl2-modules
  • mailutils
  • libcyrus-imap-perl22
  • rss2email

At this point, you'll most likely guess what i have in mind - a rss to imap gateway. In abstract, this will fetch rss feeds, put them to a mailbox and filter them to the corresponding subfolder using sieve. These subfolders can be prepared right now. Just create a "RSS" folder in your users "E-Mail" tree and subfolders named "slashdot.org" and "osnews.com"

Joe Doe
\- E-Mail
 \- INBOX
  - DRAFTS
  - RSS
  \- slashdot.org
   - osnews.com

Configuration - Just some lines...

Sieve is used for mail filtering. This task needs access to the users IMAP folders which requires correct credentials. To keep all authentication stuff at one place, i prefer to let PAM (Pluggable Authentication Modules) do the job. It authenticates against the Open-Xchange user database in the backend - if you configured it like described in the installation manual. You may use the configuration from below, just modify the "<YOUROXADMINPASS>" entry.

/etc/pam.d/sieve

auth optional pam_mysql.so host=/var/run/mysqld/mysqld.sock user=openexchange passwd=<YOUROXADMINPASS> db=open-xchange-db [table=login2user LEFT JOIN user ON login2user.id=user.id AND login2user.cid=user.cid] [where=user.cid=1] usercolumn=login2user.uid passwdcolumn=user.userPassword crypt=1
account required pam_mysql.so host=/var/run/mysqld/mysqld.sock user=openexchange passwd=<YOUROXADMINPASS> db=open-xchange-db [table=login2user LEFT JOIN user ON login2user.id=user.id AND login2user.cid=user.cid]  [where=user.cid=1] usercolumn=login2user.uid passwdcolumn=user.userPassword crypt=1


Make sure timsieved is working properly. To ensure that, customize cyrus services to access it. For Debian Etch, i had to modify the CMD property to point to the correct path of timsieved.

/etc/cyrus.conf:

sieve cmd="/usr/lib/cyrus/bin/timsieved" listen="127.0.0.1:2000" prefork=0 maxchild=100

After this value has been correctly configured, restart the cyrus imap server

/etc/init.d/cyrus2.2 restart

This should bring up sieve filtering services for usage on localhost. Note that you are only able to configure the sieve scripts on the machine timsieved is running. You are able to change this, but for security reasons, i would not prefer that.

$ netstat -a | grep sieve
tcp        0      0 localhost:sieve         *:*                     LISTEN

Sieve is running, lets try to use it. Note that sieve authenticates against pam, which is configured to use the Open-Xchange Server user database.

$ sieveshell -u <user> -a <user> localhost
connecting to localhost
Please enter your password: <user password>
>

Great, sieve authenticates correctly and is ready to go. Lets feed it with some scripts for mail filtering. Here's a example for the slashdot.org and osnews.com newsfeed, save that code to a file named "rss-sieve".

require "fileinto";
if header :is "From" "Slashdot <rss@example.com>" {
  fileinto "RSS/slashdot.org";
  }
if header :is "From" "OSNews * <rss@example.com>" {
  fileinto "RSS/osnews.com";
  }

Enter the sieveshell again and import this script

> put rss-sieve
> activate rss-sieve
> ls
rss-sieve  <- active script

Now sieve knows what it has to do - lets give it some work to do. This is where rss2email comes up. It is a small but handy python script which transforms RSS feeds to readable E-Mail and puts them to the users mailbox using the sendmail compatibility interface to Postfix. rss2email needs to be configured a bit before we can start using it:

$ vim /usr/share/rss2email/rss2email.py

Modify some entries to make it easier for sieve to detect that this is RSS stuff

DEFAULT_FROM = "rss@example.com"
FORCE_FROM = 1
DATE_HEADER = 1

You should now tell rss2email where to get the latest news. This will be done via the "r2e" interface which is a bash script. Note that the configured feeds are stored to ~/.rss2email/feeds.dat of the current system user.

$ r2e new user@example.com
$ r2e add http://rss.slashdot.org/Slashdot/slashdot
$ r2e add http://osnews.com/files/recent.xml
$ r2e run
I: Processing http://rss.slashdot.org/Slashdot/slashdot
I: Processing http://osnews.com/files/recent.xml

Thats it, rss2email fetches the subscribed RSS feed, puts it to the Postfix queue and sieve filters the incoming mail according to the configured script. Note that you'll have to modify the sieve script if you like to subscribe to other feeds as well. Your user@example.com account should now get the latest news directly to his RSS/ subfolder. You'll have to run this script every time you wish to update - or let cron do the job for you.

$ crontab -e
*/5 * * * * r2e run 2> /dev/null

Enhanced use-cases

Shared RSS with OX6

Well, now we've a mailbox collecting all kinds of news feeds. As Open-Xchange Server 6 supports ACL with IMAP, you are able to share E-Mail folders to other users. If you share the newsfeed folders to anybody on the server, you achieve a much more efficient spread of information than it would happen if every user uses his own feed. Set read-only flags for all other users to ensure nobody harms the integrity of the delivered data. You may also want to share special newsfeed with a group of people only - just use ACL. This server-side RSS feature eliminates major drawbacks when using RSS in a larger network. Just imagine the difference if 100 users are polling 10 redundant newsfeeds each in contrast to one server which polls 50 different unique newsfeeds - this will decrease your network load.