Freepops Manual: Written by Enrico Tassi, Nicola Cocchiaro
Freepops Manual: Written by Enrico Tassi, Nicola Cocchiaro
Freepops Manual: Written by Enrico Tassi, Nicola Cocchiaro
Date : 2007/10/2812 : 32 : 22
Revision : 1.41
Contents
1 Introduction 4
1.1 Usage situations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2 Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.3 Plugins . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2 History 9
6 Plugins 19
6.1 Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
6.2 abv.lua . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
6.3 aol.lua . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
6.4 davmail.lua . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
6.5 excite.lua . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
6.6 fastmail.lua . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
6.7 gmail.lua . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
6.8 hotmail.lua . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
6.9 juno.lua . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
6.10 kernel.lua . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
6.11 libero.lua . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
6.12 lycos.lua . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
6.13 mail2world.lua . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
6.14 mailcom.lua . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2
CONTENTS CONTENTS
6.15 monitor.lua . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
6.16 netscape.lua . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
6.17 orange.lua . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
6.18 popforward.lua . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
6.19 softhome.lua . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
6.20 squirrelmail.lua . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
6.21 supereva.lua . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
6.22 tin.lua . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
6.23 tre.lua . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
6.24 updater.lua . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
6.25 yahoo.lua . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
6.26 aggregator.lua . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
6.27 flatnuke.lua . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
7 Creating a plugin 40
7.1 Plugins overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
7.1.1 The interface between the C core and a plugin . . . . . . . 41
7.1.2 The interface between a plugin and the C core . . . . . . . 43
7.2 The art of writing a plugin (plugins tutorial) . . . . . . . . . . . . . 44
7.2.1 (step 1) The skeleton . . . . . . . . . . . . . . . . . . . . . . . 44
7.2.2 (step 2) The login . . . . . . . . . . . . . . . . . . . . . . . . . 45
7.2.3 (step 3) Getting the list of messages . . . . . . . . . . . . . . 50
7.2.4 (step 4) The common functions . . . . . . . . . . . . . . . . . 55
7.2.5 (step 5) Deleting messages . . . . . . . . . . . . . . . . . . . 56
7.2.6 (step 6) Downloading messages . . . . . . . . . . . . . . . . 56
7.2.7 (step 7) Test it . . . . . . . . . . . . . . . . . . . . . . . . . . 58
7.2.8 (step 8) The so mentioned last part of the tutorial . . . . . . 58
8 Submitting a bug 64
9 FAQ 64
10 Authors 67
10.1 Developers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
11 Thanks 68
3
1 INTRODUCTION
1 Introduction
FreePOPs is a POP3 daemon plus a LUA interpreter and some extra libraries for
HTTP and HTML parsing. Its main purpose is translating local POP3 requests to
remote HTTP actions on the supported web-mails, but it is really more flexible:
for example there is a plugin to read news from a website as if they were mails
in a mailbox. You can easily extend FreePOPs on the fly, without even restarting
it; you can add a plugin or modify an existing one simply changing the script
file since the plugins are written in LUA and are interpreted on the fly.
1.2 Features
FreePOPs is the only software I know with these features:
• POP3 server RFC compliant (not full featured but compliant).
• Portable (written in C and LUA that is written in C, so everything is written
in the most portable language around the world).
• Small (in the sense of resources usage) and reasonably fast.
• Extremely extensible on the fly using a simple and powerful language.
• Pretty documented.
• Released under the GNU/GPL license (this means FreePOPs is Free Soft-
ware).
4
1.3 Plugins 1 INTRODUCTION
1.3 Plugins
These are the plugins currently included in FreePOPs:
abv.lua (abv.bg) :
This plugin supports these domains: @abv.bg, @gyuvetch.bg, @gbg.bg
aol.lua (aol.com) :
This plugin supports these domains: @aol.com, @aol.com.ar, @aol.fr, @aol.com.mx,
@aol.com.au, @aol.de, @aol.com.pr, @aol.com.br, @jp.aol.com, @aol.com.uk,
@aol.ca, @aola.com, @aim.com
davmail.lua (DAVMAIL) :
This plugin supports these domains: @lycos.co.uk, @lycos.ch, @lycos.de,
@lycos.es, @lycos.it, @lycos.at, @lycos.nl, @spray.se, @jubii.dk
excite.lua (excite) :
This plugin supports these domains: @excite.com, @myway.com
fastmail.lua (fastmail.com) :
This plugin supports these domains: @123mail.org, @150mail.com, @150ml.com,
@16mail.com, @2-mail.com, @4email.net, @50mail.com, @airpost.net, @all-
mail.net, @bestmail.us, @cluemail.com, @elitemail.org, @emailgroups.net,
@emailplus.org, @emailuser.net, @eml.cc, @fastem.com, @fast-email.com,
@fastemail.us, @fastemailer.com, @fastest.cc, @fastimap.com, @fastmail.cn,
@fastmail.com.au, @fastmail.fm, @fastmail.us, @fastmail.co.uk, @fastmail.to,
@fmail.co.uk, @fast-mail.org, @fastmailbox.net, @fastmessaging.com, @fea.st,
@f-m.fm, @fmailbox.com, @fmgirl.com, @fmguy.com, @ftml.net, @hail-
mail.net, @imap.cc, @imap-mail.com, @imapmail.org, @internet-e-mail.com,
@internetemails.net, @internet-mail.org, @internetmailing.net, @jetemail.net,
@justemail.net, @letterboxes.org, @mailandftp.com, @mailas.com, @mail-
bolt.com, @mailc.net, @mailcan.com, @mail-central.com, @mailforce.net,
@mailftp.com, @mailhaven.com, @mailingaddress.org, @mailite.com, @mailmight.com
@mailnew.com, @mail-page.com, @mailsent.net, @mailservice.ms, @mailup.net,
@mailworks.org, @ml1.net, @mm.st, @myfastmail.com, @mymacmail.com,
@nospammail.net, @ownmail.net, @petml.com, @postinbox.com, @post-
pro.net, @proinbox.com, @promessage.com, @realemail.net, @reallyfast.biz,
@reallyfast.info, @rushpost.com, @sent.as, @sent.at, @sent.com, @speed-
post.net, @speedymail.org, @ssl-mail.com, @swift-mail.com, @the-fastest.net,
@theinternetemail.com, @the-quickest.com, @veryfast.biz, @veryspeedy.net,
@warpmail.net, @xsmail.com, @yepmail.net, @your-mail.com
5
1.3 Plugins 1 INTRODUCTION
flatnuke.lua (flatnuke) :
This plugin supports these domains: @flatnuke, @...
gmail.lua (GMail.com) :
This plugin supports these domains: @gmail.com
hotmail.lua (hotmail.com) :
This plugin supports these domains: @hotmail.com, @msn.com, @webtv.com,
@charter.com, @compaq.net, @passport.com, @hotmail.de, @hotmail.it,
@hotmail.co.uk, @hotmail.co.jp, @hotmail.fr, @messengeruser.com, @hot-
mail.com.ar, @hotmail.co.th, @hotmail.com.tr, @milanosemplice.it
juno.lua (juno.com) :
This plugin supports these domains: @netzero.net, @netzero.com, @juno.com
libero.lua (Libero.IT) :
This plugin supports these domains: @libero.it, @inwind.it, @iol.it, @blu.it
mail2world.lua (mail2world.com) :
This plugin supports these domains: @mail2*.com
mailcom.lua (mail.com) :
This plugin supports these domains: @mail.com, @email.com, @iname.com,
@cheerful.com, @consultant.com, @europe.com, @mindless.com, @earth-
ling.net, @myself.com, @post.com, @techie.com, @usa.com, @writeme.com,
@2die4.com, @artlover.com, @bikerider.com, @catlover.com, @cliffhanger.com,
@cutey.com, @doglover.com, @gardener.com, @hot-shot.com, @inorbit.com,
@loveable.com, @mad.scientist.com, @playful.com, @poetic.com, @pop-
star.com, @popstarmail.org, @saintly.com, @seductive.com, @soon.com,
@whoever.com, @winning.com, @witty.com, @yours.com, @africamail.com,
@arcticmail.com, @asia.com, @australiamail.com, @europe.com, @japan.com,
@samerica.com, @usa.com, @berlin.com, @dublin.com, @london.com, @madrid.com,
@moscowmail.com, @munich.com, @nycmail.com, @paris.com, @rome.com,
@sanfranmail.com, @singapore.com, @tokyo.com, @accountant.com, @adexec.com,
@allergist.com, @alumnidirector.com, @archaeologist.com, @chemist.com,
@clerk.com, @columnist.com, @comic.com, @consultant.com, @counsel-
lor.com, @deliveryman.com, @diplomats.com, @doctor.com, @dr.com, @en-
gineer.com, @execs.com, @financier.com, @geologist.com, @graphic-designer.com,
6
1.3 Plugins 1 INTRODUCTION
7
1.3 Plugins 1 INTRODUCTION
monitor.lua (monitor) :
This plugin supports these domains: @monitor
netscape.lua (netscape.net) :
This plugin supports these domains: @netscape.net
popforward.lua (POPforward) :
This plugin supports these domains: @...
softhome.lua (softhome.net) :
This plugin supports these domains: @softhome.net
squirrelmail.lua (SquirrelMail) :
This plugin supports these domains: @...
supereva.lua (Supereva) :
This plugin supports these domains: @supereva.it, @supereva.com, @freemail.it,
@freeweb.org, @mybox.it, @superdada.com, @cicciociccio.com, @mp4.it,
@dadacasa.com, @clarence.com, @concento.it, @dada.net
tin.lua (Tin.IT) :
This plugin supports these domains: @tin.it, @virgilio.it, @alice.it, @tim.it,
@atlantide.it
tre.lua (Tre) :
This plugin supports these domains: @tre.it, @three.com.au
updater.lua (updater) :
This plugin supports these domains: @updater
yahoo.lua (yahoo.com) :
This plugin supports these domains: @yahoo.com, @yahoo.ie, @yahoo.it,
@yahoo.ca, @rocketmail.com, @yahoo.com.ar, @yahoo.co.in, @yahoo.co.id,
8
3 FREEPOPS CONFIGURATION FILE
2 History
FreePOPs was not born from scratch. A similar project (only in the main usage
situation) is LiberoPOPs.
The ancestor of FreePOPs is completely written in C for some uninteresting
reasons. LiberoPOPs supports “plugins” but in a more static and complex way.
The POP3 server frontend could be attached to a backend written in C, this
means you have to recompile and restart LiberoPOPs each time to change a
line in a plugin. Another interesting point is that LiberoPOPs was created from
scratch in a really short time (you have to be Italian and use a @libero.it
mail address to understand why), this means it was born with a lot of bugs and
FIX-ME in the code.
The LiberoPOPs project had a quick success, because everybody needed it,
and this means we had a lot of users. In the opensource (and also Linux)
philosophy you have to release frequently and this was exactly what we did:
we used to release every two days. We were working not with Unix users,
nor hackers, but mostly with Win32 users. Suddenly we realized that they
were lazy/bored of updating the software every 2 days. The ugly Win-world
has taught them that software auto-updates, auto-install and even auto-codes
probably.
We tried to solve this pulling out of the C engine most of the change-prone
code, but this was really hard since C is not thought to do this. After LiberoPOPs
had stabilized we started to think how to solve this.
A scripting/interpreted embedded language seemed to me a nice choice and
after a long search on the net and on the newsgroup of my university I found
LUA.. This is not the place for telling the world how good this small language
is and I won’t talk more about it here. Integrating the LUA interpreter in
LiberoPOPs was not so hard and FreePOPs is the result. Now it is really easier
to write/test a plugin and (even if it is not implemented yet) an auto-update
facility is really easy to code since there is no need to recompile the C core in
most cases.
9
3.1 Simple load balancing 3 FREEPOPS CONFIGURATION FILE
freepops.MODULES_MAP[".*"] = {
name = "popforward.lua",
regex = true,
args = {
host = function(mailaddress)
local name = freepops.get_name(mailaddress)
local slaves = {
{ rex = ’^[0-9]’, host = "localhost", port = 2000 },
{ rex = ’^[a-lA-l]’, host = "localhost", port = 2001 },
{ rex = ’^[m-zM-Z]’, host = "localhost", port = 2002 },
}
local host, port = slaves[1].host, slaves[1].port -- defaults
for _, slave in ipairs(slaves) do
if string.match(name, slave.rex) then
host = slave.host
port = slave.port
break
end
10
4 FREEPOPS COMMAND LINE PARAMETERS
end
return host, port
end
}
}
freepops.MODULES_PREFIX = {
os.getenv("FREEPOPSLUA_PATH_UPDATES") or "./",
os.getenv("FREEPOPSLUA_PATH") or "./",
"./lua/", "./", "./src/lua/", "./modules/include/", "./modules/lib/"}
freepops.MODULES_CPREFIX = {
os.getenv("FREEPOPSLUA_CPATH") or "./",
"./c/", "./", "./updater-ui/fltk/" }
freepops.MODULES_PREFIX_UNOFFICIAL = {
os.getenv("FREEPOPSLUA_PATH_UNOFFICIAL") or "./",
"./src/lua_unofficial/", }
The whole file is reported for completeness, but the last part is the standard
one. The host parameter to the popforward plugin is a function returning dif-
ferent host and port according to the email address taken in input. This simple
balancing mechanism is supported from version 0.2.6. Note that the master
instance has to keep a connection open for every client.
-b addr, - -bind addr Binds over addr instead INADDR_ANY (0.0.0.0). addr
must be a character string containing an IPv4 network address in the
dotted-quad format, “ddd.ddd.ddd.ddd” or a host name.
11
4 FREEPOPS COMMAND LINE PARAMETERS
-n, - -no-pid-file Do not generate the file containing the process’ pid in /ver/run/.
-u name, - -useragent name Use this useragent in http connections. The de-
fault is “Firefox/0.8”. A valid example is mozilla’s “Mozilla/5.0 (X11; U;
Linux i686; en-US; rv:1.5) Gecko/20031024 Debian/1.5-2”.
-k, - -kill Terminates a running FreePOPs program. (Not used under Windows)
-e scriptfile args..., - -execute scriptfile args... This is a full bloated LUA in-
terpreter, the executed script has access to all freepops libraries. The
interpreter calls the main function that must get a table of strings and
return an integer. The arguments passed to freepopsd after the script file
name are put inside the table argument. The return value is returned
from the interpreter.
-c, - -conffile file Users the specified configuration file instead of looking in de-
fault paths like /etc/freepops/config.lua, ./config.lua and /usr/share/frepops/lua/c
- -statistics-all Enable all statistics. Results can be viewed with the monitor
plugin, either with an account like foo@monitor?command=stats or with
freepopsd -e monitor host port password command.
12
5 EMAIL CLIENT CONFIGURATION
13
5.1 Outlook Express tutorial 5 EMAIL CLIENT CONFIGURATION
• From the tools menu choose the Account... item (see figure 1)
Figure 1: main
• In the Server tab type in Incoming mail the name of the computer where
you started FreePOPs, usually localhost. The Account name must be your
complete email address, followed by the domain name your email belongs
to, for example username@domain.com (see figure 3).
• In the Advanced settings tab type in Incoming mail the port number,
that is 2000 if you accepted our settings. De-select This server requires a
secure connection (SSL) (see figure 4).
14
5.1 Outlook Express tutorial 5 EMAIL CLIENT CONFIGURATION
Figure 2: settings
Figure 3: server
15
5.1 Outlook Express tutorial 5 EMAIL CLIENT CONFIGURATION
Figure 4: advanced
16
5.2 Proxy tutorial 5 EMAIL CLIENT CONFIGURATION
17
5.4 LAN tutorial 5 EMAIL CLIENT CONFIGURATION
5.3.4 SpamHilator
Configure your email client with the following parameters: POP3 server (incom-
ing mail): localhost POP3 server port: 110 Username: localhost&email@address&2000
5.3.6 K9
Set up your email client to use 9999 as POP3 server port. Leave localhost as
server name. Then use localhost/2000/email@address as username.
5.3.7 SpamTerminator
Configure your email client with the following parameters: POP3 server (incom-
ing mail): localhost POP3 server port: 8110 Username: email@address#localhost
Then start FreePOPs with the option -p 110.
5.3.8 SpamPal
Configure your email client with the following parameters: POP3 server (incom-
ing mail): localhost POP3 server port: 110 Username: email@address@localhost:2000
18
6 PLUGINS
6 Plugins
Here we give a detailed description of each plugin, but before starting we ex-
plain the general way of passing special arguments to plugins (see the specific
plugin description for a detailed description of the accepted parameters).
6.1 Parameters
Each plugin can receive parameters passed as an addon to username. The fol-
lowing username is for the popforward.lua plugin:
gareuselesinge@mydomain.xx?host=pop.mydomain.xx&port=110
Since you may use some antispam proxy or other program that may handle
your username and may dislike the ? character you may use a space instead
of it. Every following character that is not a letter or a number needs to be
escaped. So they have to be written as %xx, where xx is the hexadecimal code
of the corresponding character (exacly as it happens in URL). The space char-
acter could also be substituted simply by a plus characted +. For example,
if you have to assign the value "My Messages" to the folder parameter, you
need to write folder=My+Messages.Take a look to the Appendix, to find more
informations about hexadecimal codes.
Another way of hacking with the username is the on-the-fly domain-plugin
binding. You may find useful to say: “I want to use plugin X for domain Y
without changing the config.lua file”. In this case you have to use the plugin
name (for example popforward.lua) as the domain name an probably you will
have to pass some arguments to the plugin using the procedure previously
described. This is an example:
gareuselesinge@popforward.lua?host=pop.mymailsite.xx&port=110
Remember that in the case of the use of on-the-fly bindings there will be no
default arguments, thus port=110 can’t be omitted as in the previous example.
6.2 abv.lua
Name: abv.bg
19
6.3 aol.lua 6 PLUGINS
Version: 0.1.6
License: GNU/GPL
Homepage: http://freepops.sourceforge.net/
Domain(regex):
Description: To use this plugin you have to use your full email address as the
username and your real password as the password. For support, please
post a question to the forum instead of emailing the author(s).
6.3 aol.lua
Name: aol.com
Version: 0.1.3
License: GNU/GPL
Homepage: http://freepops.sourceforge.net/
Domain(regex):
Description: To use this plugin you have to use your full email address as the
username and your real password as the password.
20
6.4 davmail.lua 6 PLUGINS
6.4 davmail.lua
Name: DAVMAIL
Version: 0.0.6
License: GNU/GPL
Homepage: http://www.freepops.org/
Author: Enrico Tassi <gareuselesinge (at) users (.) sourceforge (.) net>
Domain(regex):
Parameter:
6.5 excite.lua
Name: excite
Version: 0.0.5a
License: GNU/GPL
Homepage: http://www.freepops.org/en/viewplugins.php
21
6.6 fastmail.lua 6 PLUGINS
Domain(regex):
Description: Excite webmail plugin. Use your full email address as the user-
name and your real password as the password. For support, please post
your questions to the forum.
Parameter:
folder The folder to interact with. Default is Inbox, other values are: Bulk,
Sent, Trash, Drafts or user defined folder.
6.6 fastmail.lua
Name: fastmail.com
Version: 0.0.3c
License: GNU/GPL
Homepage: http://www.freepops.org/
22
6.7 gmail.lua 6 PLUGINS
6.7 gmail.lua
Name: GMail.com
Version: 0.0.54
Requires: FreePOPs 0.2.0
License: GNU/GPL
Available at: http://www.freepops.org/download.php?module=gmail.lua
Homepage: http://www.freepops.org/
Authors: Rami Kattan <rkattan (at) gmail (.) com>, EoinK <eoin.pops (at) gmail
(.) com>
Domain: @gmail.com
Domain(regex):
Description: This is the webmail support for @gmail.com mailboxes.
To use this plugin you have to use your full email address as the user
name and your real password as the password.
Adding some parameters at the end of the username gives the ability to
download email from different folder and/or labels, and export the con-
tacts in CSV format. Check "Supported parameters" for more details about
the available parameters.
23
6.8 hotmail.lua 6 PLUGINS
Note:
When the email client issues the command to delete some messages (be-
cause in its options it is set to delete messages from the server [after x
days]), if checked the inbox folder, the email will be moved to the archive
(folder "all"), while if you checked the spam folder, the email will be moved
to the trash folder, else it will only be marked as read.
Parameters:
folder Used for selecting the folder to operate on (inbox is the default one).
The standard folders are: inbox, starred, sent, all, spam, trash.
Here is an example of a username to get the email from the starred
folder:
foo@gmail.com?folder=starred
If you created custom labels in gmail, you can access them using the
label parameter label=name.
maxmsgs Parameter is used to force the plugin to only download a maxi-
mum number of messages.
enableimap Parameter is used to turn on IMAP for the account. If set to
1, the plugin will enable IMAP access on the user’s account during
the QUIT command.
enableforward Parameter is turn on forwarding for the account. If set
with an email address, the settings will be updated to enable the
user’s settings to forward messages to the supplied address.
label Used for selecting the labels to operate on.
Here is an example of a username to get the email from the label
Friends:
foo@gmail.com?label=Friends
act Possible values:
- export: Exports your gmail contacts into a file called gmail_contacts_export.csv
that will be saved in your home (Unix) or in the My Documents direc-
tory (Windows), that can be imported into your email client.
6.8 hotmail.lua
Name: hotmail.com
Version: 0.1.93
24
6.8 hotmail.lua 6 PLUGINS
License: GNU/GPL
Homepage: http://www.freepops.org/
Authors: Russell Schwager <russell822 (at) yahoo (.) com>, D. Milne <drmilne
(at) safe-mail (.) net>, Peter Collingbourne <pcc03 (at) doc (.) ic (.) ac (.)
uk»
Domain(regex):
Description: This plugin lets you download mail from @hotmail.com and sim-
ilar mailboxes. To use this plugin you have to use your full email address
as the username and your real password as the password. For support,
please post a question to the forum instead of emailing the author(s).
Parameters:
25
6.9 juno.lua 6 PLUGINS
6.9 juno.lua
Name: juno.com
Version: 0.1.1
License: GNU/GPL
Homepage: http://www.freepops.org/
Domain(regex):
Parameters:
folder Parameter is used to select the folder (Inbox is the default) that
you wish to access. The folders that are available are the standard
Yahoo folders, called Inbox, Draft, Sent, Junk Mail and Trash. For
user defined folders, use their name as the value.
emptytrash Parameter is used to force the plugin to empty the trash
when it is done pulling messages.
resetheaders Parameter is used to force the plugin to turn off full headers
when it is done pulling messages.
noattachments Parameter is used to force the plugin to skip attach-
ments. To turn this on, set the value to 1.
6.10 kernel.lua
Name: kernel.org Changelog viewer
Version: 0.0.4
License: GNU/GPL
26
6.11 libero.lua 6 PLUGINS
Homepage: http://www.freepops.org/
Author: Simone Vellei <simone_vellei (at) users (.) sourceforge (.) net>
Domain(regex):
Description: This plugin helps in staying up to date with the Linux kernel
releases. http://kernel.org is the official page with the Linux kernel re-
leases, each with its ChangeLog. You should use something@kernel.org to
receive news about every tree, something@kernel.24 or something@kernel.org.26
for a specific tree. Password is not used, type a random string.
6.11 libero.lua
Name: Libero.IT
Version: 0.2.18
License: GNU/GPL
Homepage: http://www.freepops.org/
Author: Enrico Tassi <gareuselesinge (at) users (.) sourceforge (.) net>
Domain(regex):
Parameter:
folder
27
6.12 lycos.lua 6 PLUGINS
6.12 lycos.lua
Name: Lycos.IT (this plugin doesn’t work!)
Version: 0.0.5
License: GNU/GPL
Homepage: ———
Author: — <—–>
Domain: @...
Domain(regex):
Description: this plugin cant’ do a delete and retr/top. this is half done since
now the lycos webmail is served by the owa.lua plugin.
Parameter:
–name– –desc–
6.13 mail2world.lua
Name: mail2world.com
Version: 0.0.2g
License: GNU/GPL
Homepage: http://freepops.sourceforge.net/
Domain:
Domain(regex): @mail2*.com
Description: To use this plugin you have to use your full email address as the
username and your real password as the password. For support, please
post a question to the forum instead of emailing the author(s).
28
6.14 mailcom.lua 6 PLUGINS
Parameters:
6.14 mailcom.lua
Name: mail.com
Version: 0.1.07b
License: GNU/GPL
Homepage: http://www.freepops.org/
29
6.14 mailcom.lua 6 PLUGINS
30
6.15 monitor.lua 6 PLUGINS
Domain(regex):
Description: This is the webmail support for @mail.com and all its other do-
main mailboxes. To use this plugin you have to use your full email address
as the user name and your real password as the password.
Parameters:
folder Parameter is used to select the folder (Inbox is the default) that you
wish to access. The folders that are available are the standard folders,
called INBOX, Drafts, SENT, and Trash. For user defined folders, use
their name as the value.
emptytrash Parameter is used to force the plugin to empty the trash
when it is done pulling messages.
setoptionoverride Parameter is used to tell the plugin not to change the
mail options on the mail.com website. If you use this option, you
must have full headers enabled in your options. If the value is 1, the
behavior is turned on.
usemailcomloginpage Parameter is used to tell the plugin to use the lo-
gin page on mail.com instead of trying to figure it out by the domain.
If the value is 1, the behavior is turned on.
loginpage Parameter is used to tell the plugin which login page to use.
6.15 monitor.lua
Name: monitor
Version: 0.0.1
License: GNU/GPL
Homepage: http://www.freepops.org/download.php?module=monitor.lua
Domain: @monitor
Domain(regex):
31
6.16 netscape.lua 6 PLUGINS
Parameter:
6.16 netscape.lua
Name: netscape.net
Version: 0.0.3c
License: GNU/GPL
Homepage: http://freepops.sourceforge.net/
Domain: @netscape.net
Domain(regex):
Description: This plugin lets you download mail from @netscape.net mail-
boxes. To use this plugin you have to use your full email address as
the username and your real password as the password.
6.17 orange.lua
Name: Orange (ex Wanadoo)
Version: 0.0.8g
License: GNU/GPL
Homepage: http://www.freepops.org/en/viewplugins.php
Authors: TheMarco <themarco (at) fsmail (.) net>, Ernst Vaarties <evaarties
(at) xs4all (.) nl>
Domain(regex):
32
6.18 popforward.lua 6 PLUGINS
Description: Orange ( ex Wanadoo ) webmail plugin. Use your full email ad-
dress as the username and your real password as the password. For
support, please post your questions to the forum.
Parameters:
folder The folder to interact with. Default is inbox, other values are: junk,
sent, trash, draft or user defined folder.
emptytrash Not recommended. Forces the plugin to empty the trash
when it is done pulling messages. Set it to 1 to activate it.
6.18 popforward.lua
Name: POPforward
Version: 0.0.5
License: GNU/GPL
Homepage: http://www.freepops.org/
Author: Enrico Tassi <gareuselesinge (at) users (.) sourceforge (.) net>
Domain: @...
Domain(regex):
Description: This is a POP3 proxy. Read the parameters to know the addi-
tional features
Parameters:
33
6.19 softhome.lua 6 PLUGINS
pipe_limit Limit the maximum size of piped messages, the default (0) is
to pipe all of them.
6.19 softhome.lua
Name: softhome.net
Version: 0.0.2b
License: GNU/GPL
Homepage: http://freepops.sourceforge.net/
Domain: @softhome.net
Domain(regex):
Description: To use this plugin you have to use your full email address as the
username and your real password as the password. For support, please
post a question to the forum instead of emailing the author(s).
Parameter:
6.20 squirrelmail.lua
Name: SquirrelMail
Version: 0.0.4
License: GNU/GPL
Homepage: http://www.freepops.org/
Author: Eddi De Pieri <dpeddi (at) users (.) sourceforge (.) net>
Domain: @...
34
6.21 supereva.lua 6 PLUGINS
Domain(regex):
Description: This plugin supports webmails made with squirrelmail. You have
to hack by hand the plugin to make it work with your website. since now
it supports only version 1.2.
6.21 supereva.lua
Name: Supereva
Version: 0.2.7a
License: GNU/GPL
Homepage: http://www.freepops.org
Authors: Andrea Dalle Molle <Tund3r (at) fastwebnet (dot) it>, Enrico Tassi
<gareuselesinge (at) users (dot) sourceforge (dot) net>, Visioning <unknown>,
Viruzzo <unknown>
Domain(regex):
6.22 tin.lua
Name: Tin.IT
Version: 0.2.11h
License: GNU/GPL
Homepage: http://www.freepops.org/
Author: Enrico Tassi <gareuselesinge (at) users (.) sourceforge (.) net>
35
6.23 tre.lua 6 PLUGINS
Domain(regex):
Parameters:
folder
limit
6.23 tre.lua
Name: Tre
Version: 0.0.5
License: GNU/GPL
Homepage: http://www.freepops.org/
Author: Eddi De Pieri <dpeddi (at) users (.) sourceforge (.) net>
Domain(regex):
Description: To use this plugin you have to configure your mail client using as
username your phone number formatted as 393921234567@three.com.XX
and as password the usim’s original pin code provided by three. PS: this
plugin is tested only whith "three italy", please report me if it works with
the three webmail of your country!.
Parameters:
36
6.24 updater.lua 6 PLUGINS
6.24 updater.lua
Name: updater
Version: 0.2.4
Requires: FreePOPs 0.2.3
License: GNU/GPL
Available at: http://freepops.sourceforge.net/download.php?module=updater.lua
Homepage: http://www.freepops.org/
Author: Russell Schwager <russells (at) despammed (.) com>
Domain: @updater
Domain(regex):
Description: This plugin is used to retrieve updated lua modules for freepops.
To use this plugin correctly, you need to set the settings for the account
created for this to ’leave mail on server’. The first time, you use this
account, all the plugins will be retrieved.
Parameter:
modlist The list of modules to update, separated by ’,’. Example: aaa@updater?modli
6.25 yahoo.lua
Name: yahoo.com
Version: 0.2.1h
Requires: FreePOPs 0.2.8
License: GNU/GPL
Available at: http://www.freepops.org/download.php?module=yahoo.lua
Homepage: http://www.freepops.org/
Authors: Russell Schwager <russell822 (at) yahoo (.) com>, Nicola Cocchiaro
<ncocchiaro (at) users (.) sourceforge (.) net>
Domains: @yahoo.com, @yahoo.ie, @yahoo.it, @yahoo.ca, @rocketmail.com,
@yahoo.com.ar, @yahoo.co.in, @yahoo.co.id, @yahoo.com.tw, @yahoo.co.uk,
@yahoo.com.cn, @yahoo.es, @yahoo.de, @talk21.com, @btinternet.com,
@yahoo.com.au, @yahoo.co.nz, @ymail.com, @yahoo.in
37
6.26 aggregator.lua 6 PLUGINS
Domain(regex):
Parameters:
folder Parameter is used to select the folder (Inbox is the default) that
you wish to access. The folders that are available are the standard
Yahoo folders, called Inbox, Draft, Sent, Bulk and Trash (for yahoo.it
domains you may use the same folder names or the corresponding
names in Italian: InArrivo, Bozza, Inviati,Anti-spam, Cestino). For
user defined folders, use their name as the value.
view Parameter is used when getting the list of messages to pull. It de-
termines what messages to be pulled. Possible values are All, Unread
and Flag.
markunread Parameter is used to have the plugin mark all messages that
it pulls as unread. If the value is 1, the behavior is turned on.
nossl Parameter is used to force the module to login through plain HTTP
and not HTTPS with SSL. If the value is 1, the SSL is not used.
emptytrash Parameter is used to force the plugin to empty the trash
folder when it is done pulling messages. Set the value to 1.
emptybulk Parameter is used to force the plugin to empty the bulk folder
when it is done pulling messages. Set the value to 1.
maxmsgs Parameter is used to force the plugin to only download a maxi-
mum number of messages.
keepmsgstatus Parameter is used to maintain the status of the message
in the state it was before being pulling. If the value is 1, the behavior
is turned on and will override the markunread flag.
domain Parameter is used to override the domain in the email address.
This is used so that users don’t need to add a mapping to config.lua
for a hosted hotmail account.
6.26 aggregator.lua
Name: RSS/RDF aggregator
Version: 0.2.8
38
6.26 aggregator.lua 6 PLUGINS
License: GNU/GPL
Homepage: http://www.freepops.org/
Authors: Simone Vellei <simone_vellei (at) users (.) sourceforge (.) net>, Fer-
nando Lucas Rodriguez <info (at) fernandolucas (.) es>
Domain(regex):
Description: Usually you can benefit from the W3C’s RSS for mat when you
read some website news. The RSS file indexes the news, providing a link to
them. This plugin can make your mail client see the RSS file as a mailbox
from which you can download each news as if it was a mail message. The
only limitation is that this plugin can fetch only a news summary plus the
news link. To use this plugin you have to use a casual user name with the
@aggregator suffix (ex: foo@aggregator) and as the password the URL of
the RSS file(ex: http://www.securityfocus.com/rss/vulnerabilities.xml).
For your commodity we added some alias for you. This means you have
not to search by hand the URL of the RSS file. We added some domain, for
example @securityfocus.com, that can be used to directly use the aggre-
gator plugin with these website. To use these alias you have to use a user
name in the form something@aggregatordomain and a casual password.
aggregatordomain description
freepops.rss.en http://www.freepops.org/ news (English)
freepops.rss.it http://www.freepops.org/ news (Italian)
flatnuke.sf.net http://flatnuke.sourceforge.net/ news (Italian)
ziobudda.net http://ziobudda.net/ news (both Italian and English)
punto-informatico.it http://punto-informatico.it/ news (Italian)
linuxdevices.com http://linuxdevices.com/ news (English)
gaim.sf.net http://gaim.sourceforge.net/ news (English)
securityfocus.com http://www.securityfocus.com/ new vulnerabilities (English)
games.gamespot.com http://www.gamespot.com/ computer games news (English)
news.gamespot.com http://www.gamespot.com/ GameSpot news (English)
kerneltrap.org http://kerneltrap.org news (English)
mozillaitalia.org http://www.mozillaitalia.org news (Italian)
linux.kerneltrap.org http://linux.kerneltrap.org news (English)
linuxgazette.net http://linuxgazette.net news (English)
39
6.27 flatnuke.lua 7 CREATING A PLUGIN
6.27 flatnuke.lua
Name: flatnuke
Version: 0.0.6
License: GNU/GPL
Homepage: http://www.freepops.org/
Author: Simone Vellei <simone_vellei (at) users (.) sourceforge (.) net>
Domain(regex):
There are some alias for FlatNuke sites, see the aggregator plugin documen-
tation to know what this means:
aggregatordomain description
freepops.en http://www.freepops.org/ full news (English)
freepops.it http://www.freepops.org/ full news (Italian)
flatnuke.it http://flatnuke.sourceforge.net/ full news (Italian)
7 Creating a plugin
Two sections follow, the first is a quick overview of what a plugin has to do, the
latter is a more detailed tutorial. Before proceeding I suggest you read some
stuff that is at the base of plugin writing:
40
7.1 Plugins overview 7 CREATING A PLUGIN
1. Since plugins are written in LUA you must read at least the LUA tuto-
rial (HTTP://lua-users.org/wiki/LuaTutorial); many thanks to the guys
who wrote it. LUA is a quite simple scripting language, easy to learn,
and easy to read. If you are interested in this language you should read
THE book about LUA (“Programming in Lua” by Roberto Ierusalimschy
HTTP://www.inf.puc-rio.br/~roberto/book/). It is a really good book, be-
lieve me. Today I’ve seen that the book is completely available online here.
HTTP://www.lua.org/pil/
3. Read carefully this tutorial, it is hardly a good tutorial, but is better than
nothing.
5. After creating a prototype, you should read a full featured plugin. The
libero.lua plugin is really well commented, you may start there.
7. FreePOPs is licensed under the GNU/GPL license, therefore any and all
software that makes use of its code must be released under the same li-
cense. This includes plugins. For more information read the enclosed
COPYING file or see the license text at HTTP://www.gnu.org/licenses/gpl.html.
41
7.1 Plugins overview 7 CREATING A PLUGIN
of the source distribution) is really short and easy to read. As your intuition
should suggest the POP3 client may ask the pop3 server to know something
about the mail that is in the mailbox and eventually retrieve/delete a message.
And this is exactly what it does.
The backend must implement all the POP3 commands (like USER, PASS,
RETR, DELE, QUIT, LIST, ...) and must give back to the frontend the result.
Let us give a simple example of a POP3 session taken from the RFC:
1 S: <wait for connection on TCP port 110>
2 C: <open connection>
3 S: +OK POP3 server
4 C: USER linux@kernel.org
5 S: +OK now insert the password
6 C: PASS gpl
7 S: +OK linux’s maildrop has 2 messages (320 octets)
8 C: STAT
9 S: +OK 1 320
10 C: LIST
11 S: +OK 2 messages (320 octets)
12 S: 1 320
13 S: .
14 C: RETR 1
15 S: +OK 120 octets
16 S: <the POP3 server sends message 1>
17 S: .
18 C: DELE 1
19 S: +OK message 1 deleted
20 C: QUIT
21 S: +OK dewey POP3 server signing off (maildrop empty)
22 C: <close connection>
23 S: <wait for next connection>
In this session the backend will be called for lines 4, 6, 8, 10, 14, 18, 20 (all
the C: lines) and respectively the functions implementing the POP3 commands
will be called this way
user(p,"linux@kernel.org")
pass(p,"gpl")
stat(p)
list_all(p)
retr(p,1)
dele(p,1)
quit_update(p)
Later I will make clear what p is. I hope we’ll remove it making it implicit for
complete transparency. It is easy to understand that there is a 1-1 mapping
42
7.1 Plugins overview 7 CREATING A PLUGIN
between POP3 commands and plugin function calls. You can view a plugin as
the implementation of the POP3 interface.
Code Meaning
POPSERVER_ERR_OK No error
POPSERVER_ERR_NETWORK An error concerning the network
POPSERVER_ERR_AUTH Authorization failed
POPSERVER_ERR_INTERNAL Internal error, please report the bug
POPSERVER_ERR_NOMSG The message number is out of range
POPSERVER_ERR_LOCKED Mailbox is locked by another session
POPSERVER_ERR_EOF End of transmission, used in the popserver_callback
POPSERVER_ERR_TOOFAST You are not allowed to reconnect to the
server now, wait a bit and retry
POPSERVER_ERR_UNKNOWN No idea of what error I’ve encountered
In our case the most appropriate error codes are POPSERVER_ERR_AUTH and
POPSERVER_ERR_OK. This is a simple case, in which an error code is enough.
Now we analyze the more complex case of the call to list_all(p). Here we
have to return an error code as before, but we also have to inform the C core
of the size of all messages in the mailbox. We need the p parameter passed
to each plugin function (note that that parameter may became implicit in the
future). p stands for the data structure that the C core expects us to fill calling
appropriate functions like set_mailmessage_size(p,num,size) where num
is the message number and size is the size in bytes. Usually it is really common
to put more functions all together. For example when you get the message list
page in a webmail you know the number of the messages, their size and uidl so
you can fill the p data structure with all the informations for LIST, STAT, UIDL.
The last case that we examine is retr(p,num,data). Since a mail message
can be really big, there is no pretty way of downloading the entire message
without making the mail client complain about the server death. The solution
is to use a callback. Whenever the plugin has some data to send to the client
he should call the popserver_callback(buffer,data). data is an opaque
structure the popserver needs to accomplish its work (note that this parameter
may be removed for simplicity). In some cases, for example if you know the
message is small or you are working on a fast network, you can fetch the whole
43
7.2 The art of writing a plugin (plugins tutorial) 7 CREATING A PLUGIN
message and send it, but remember that this is more memory consuming.
-- ************************************************************************** --
-- FreePOPs @--put here domain-- webmail interface
--
-- $Id: manual.tex,v 1.41 2007/10/28 12:32:22 gareuselesinge Exp $
--
-- Released under the GNU/GPL license
-- Written by --put Name here-- <--put email here-->
-- ************************************************************************** --
Now we have an empty plugin, but it is not enough to start hacking on it. We
need to open the config.lua file (in the win32 distribution it is placed in the
main directory, while in the Unix distribution it is in /etc/freepops/; other
copies of this file may be included in the distributions, but they are backup
copies) and add a line like this
-- foo plugin
freepops.MODULES_MAP["foo.xx"] = {name="foo.lua"}
at the beginning of the file. Before ending the first step you should try if the
plugin is correctly activated by FreePOPs when needed. To do this we have to
add few lines to foo.lua, in particular we have to add an error return value to
user().
44
7.2 The art of writing a plugin (plugins tutorial) 7 CREATING A PLUGIN
-- -------------------------------------------------------------------------- --
-- Must save the mailbox name
function user(pstate,username)
return POPSERVER_ERR_AUTH
end
Now the user function always fails, returning an authentication error. Now
you have to start FreePOPs (if it is already running you don’t have to restart
it) and start telnet (under win32 you should open a DOS prompt, under Unix
you have the shell) and type telnet localhost 2000 and then type user
test@foo.xx.
tassi@garfield:~$ telnet localhost 2000
Trying 127.0.0.1...
Connected to garfield.
Escape character is ’^]’.
+OK FreePOPs/0.0.10 pop3 server ready
user test@foo.xx
-ERR AUTH FAILED
Connection closed by foreign host.
If you start FreePOPs with the -w switch you should read this on standard
error/standard output:
45
7.2 The art of writing a plugin (plugins tutorial) 7 CREATING A PLUGIN
and the plugin has been changed a bit to store the user login and print some
debug info. This is the plugin that gave this output:
foo_globals= {
username="nothing",
password="nothing"
}
-- -------------------------------------------------------------------------- --
-- Must save the mailbox name
function user(pstate,username)
foo_globals.username = username
print("*** the user wants to login as ’"..username.."’")
return POPSERVER_ERR_OK
end
-- -------------------------------------------------------------------------- --
-- Must login
function pass(pstate,password)
foo_globals.password = password
print("*** the user inserted ’"..password..
"’ as the password for ’"..foo_globals.username.."’")
return POPSERVER_ERR_AUTH end
-- -------------------------------------------------------------------------- --
-- Must quit without updating
function quit(pstate)
return POPSERVER_ERR_OK
end
Here we have some important news. First the foo_globals table that will
contain all the globals (values that should be available to successive function
calls) we need. So far we have the username and password there. The user()
46
7.2 The art of writing a plugin (plugins tutorial) 7 CREATING A PLUGIN
function now stores the passed username in the foo_globals table and prints
something on standard output. The pass() function likewise stores the pass-
word in the global table and prints some stuff. The quit() function simply
returns POPSERVER_ERR_OK to make FreePOPs happy.
Now that we know how FreePOPs will act during the login we have to imple-
ment the login in the webmail, but first uncomment the few lines in the init()
function (that is called when the plugin is started), that loads the browser.lua
module (the module we will use to login in the webmail). Here is the webmail
login page viewed with Mozilla and the source of the page (you can see it with
Mozilla with Ctrl-U, figure 5).
Figure 5: login
<html>
<head>
<title>foo.xx webmail login</title>
</head>
<body style="background-color : grey; color : white">
<h1>Webmail login</h1>
<form name="webmail" method="post" action="http://localhost:3000/">
login: <input type="text" size="10" name="username"> <br>
password: <input type="password" size="10" name="password"> <br>
<input type="submit" value="login">
</form>
</body>
</html>
Here we have 2 input fields, one called username and one called password.
When the user clicks login the web browser will POST to HTTP://localhost:3000/
the form contents (I used a local address for comfort, but it should be some-
thing like HTTP://webmail.foo.xx/login.php). This is what the browser
sends:
47
7.2 The art of writing a plugin (plugins tutorial) 7 CREATING A PLUGIN
POST / HTTP/1.1
Host: localhost:3000
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040614 Firefox/0.8 A
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Content-Type: application/x-www-form-urlencoded
Content-Length: 37
username=test%40foo.xx&password=hello
We are not interested in the first part (the HTTP header, since the browser
module will take care of it) but in the last part, the posted data. Since the fields
of the form were username and password, the posted data is
username=test%40foo.xx&password=hello. Now we want to reproduce the
same HTTP request with our plugin. This is the simple code that will do just
that.
-- -------------------------------------------------------------------------- --
-- Must login
function pass(pstate,password)
foo_globals.password = password
-- post it
local file,err = nil, nil
file,err = b:post_uri(post_uri,post_data)
48
7.2 The art of writing a plugin (plugins tutorial) 7 CREATING A PLUGIN
return POPSERVER_ERR_AUTH
end
First we create a browser object, then we build the post_uri and post_data
using a simple string.format (printf-like function). And this is the resulting
request
POST / HTTP/1.1
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040322 Firefox/0.8
Pragma: no-cache
Accept: */*
Host: localhost
Content-Length: 35
Content-Type: application/x-www-form-urlencoded
username=test@foo.xx&password=hello
that is essentially the same (we should url-encode the post data with curl.escape())
we wanted to do. We saved the browser object to the global table, since we want
to use the same browser all the time.
Now that we have logged in, we want to check the resulting page, and maybe
extract a session ID that will be used later. This is the code to extract the
session id and the HTML page we have received in response to the login request
if id == nil then
return POPSERVER_ERR_AUTH
end
foo_globals.session_id = id
return POPSERVER_ERR_OK
end
<html>
<head>
<title>foo.xx webmail</title>
49
7.2 The art of writing a plugin (plugins tutorial) 7 CREATING A PLUGIN
Figure 6: logindone
</head>
<body style="background-color : grey; color : white">
<h1>Webmail - test@foo.xx</h1>
Login done! click here to view the inbox folder.
<a href="http://localhost:3000/inbox.php?session_id=ABCD1234">inbox</a>
</body>
</html>
50
7.2 The art of writing a plugin (plugins tutorial) 7 CREATING A PLUGIN
Figure 7: inbox
and this is the HTML body (only the first 2 messages are reported)
51
7.2 The art of writing a plugin (plugins tutorial) 7 CREATING A PLUGIN
We have retrieved the HTML using the browser and the get_uri() method
(remember the URI for the inbox was in the login page). As you can see the
messages are in a table and this table has the same structure for each message.
This is the place in which you may use mlex. Just take all the stuff between
<tr> and </tr> of a message row and delete all but the tags name. Then
replace every empty space (we call space the string between two tags) with a
“.*”. This is what we have obtained (it should be all in the same line, here is
wrapped for lack of space) from the first message.
.*<tr>.*<td>.*<b>.*</b>.*</td>.*<td>.*<b>.*<a>.*</a>.*</b>.*</td>.*
<td>.*<b>.*</b>.*</td>.*<td>.*<b>.*</b>.*</td>.*
<td>.*<input>.*</td>.*</tr>
This expression is used to match the table row containing info about the
message. Now cut and paste the line and replace every space and every tag
with O (the letter, not the digit 0) or X. Put an X in the interesting fields (in our
example the size and the input tag, that contains the message uidl).
O<O>O<O>O<O>O<O>O<O>O<O>O<O>O<O>O<O>O<O>O<O>O
<O>O<O>X<O>O<O>O<O>O<O>O<O>O<O>O
<O>O<X>O<O>O<O>
While the first expression will be used to match the table row, this one will
be used to extract the important fields. This is the code that starts mlex on the
HTML and fills the popstate data structure with the captured data.
-- -------------------------------------------------------------------------- --
-- Fill the number of messages and their size
function stat(pstate)
local file,err = nil, nil
local b = foo_globals.browser
file,err = b:get_uri("http://localhost:3000/inbox.php?session_id="..
foo_globals.session_id)
local e = ".*<tr>.*<td>.*<b>.*</b>.*</td>.*<td>.*<b>.*<a>"..
".*</a>.*</b>.*</td>.*<td>.*<b>.*</b>.*</td>.*<td>.*"..
"<b>.*</b>.*</td>.*<td>.*<input>.*</td>.*</tr>"
local g = "O<O>O<O>O<O>O<O>O<O>O<O>O<O>O<O>O<O>O<O>O<O>O"..
"<O>O<O>X<O>O<O>O<O>O<O>O<O>O<O>O<O>O<X>O<O>O<O>"
local x = mlex.match(file,e,g)
--debug print
x:print()
52
7.2 The art of writing a plugin (plugins tutorial) 7 CREATING A PLUGIN
set_popstate_nummesg(pstate,x:count())
for i=1,x:count() do
local _,_,size = string.find(x:get(0,i-1),"(%d+)")
local _,_,size_mult_k = string.find(x:get(0,i-1),"([Kk][Bb])")
local _,_,size_mult_m = string.find(x:get(0,i-1),"([Mm][Bb])")
local _,_,uidl = string.find(x:get(1,i-1),"check_(%d+)")
set_mailmessage_size(pstate,i,size)
set_mailmessage_uidl(pstate,i,uidl)
end
return POPSERVER_ERR_OK
end
We have not listed here how we added the dummy return POPSERVER_ERR_OK
line to the quit() function. The source code listed before uses mlex to extract
the two interesting strings, then parses them searching for the size and the size
multiplier and the uidl. Then sets the mail message attributes. But here you
can see that we just matched the first message. To match the other messages
we have to inform the mlex module that the <b> tag is optional (you can see
that only the first message is in bold). So we change the expressions to
53
7.2 The art of writing a plugin (plugins tutorial) 7 CREATING A PLUGIN
.*<tr>.*<td>[.*]{b}.*{/b}[.*]</td>.*<td>[.*]{b}.*<a>.*</a>.*{/b}[.*]</td>.*
<td>[.*]{b}.*{/b}[.*]</td>.*<td>[.*]{b}.*{/b}[.*]</td>.*
<td>.*<input>.*</td>.*</tr>
and
O<O>O<O>[O]{O}O{O}[O]<O>O<O>[O]{O}O<O>O<O>O{O}[O]<O>O
<O>[O]{O}X{O}[O]<O>O<O>[O]{O}O{O}[O]<O>O
<O>O<X>O<O>O<O>
Now the stat command responds with +OK 4 45056 and the debug print is
Now we have a proper function stat that fill the popstate data structure with
the info the popserver needs to respond to a stat request. Since the list, uidl,
list_all and uidl_all requests can be satisfied with the same data we will use the
standard function provided by the common.lua module. It will be explained in
the next step, but we have to add 2 important lines to the stat() function, to
avoid a double call.
function stat(pstate)
if foo_globals.stat_done == true then return POPSERVER_ERR_OK end
foo_globals.stat_done = true
return POPSERVER_ERR_OK
end
The most important function is done, but a lot of notes must be written here.
First, mlex is really comfortable sometimes, but you may find more helpful
using the lua string library or the regularexp library (posix extended regular
expressions) to reach the same point. Second, this implementation stops at
the first inbox page. You should visit all the inbox pages maybe using the
do_until() function in the support.lua library (that will be briefly described
at the end of this tutorial). Third we make no error checking. For example the
file variable may be nil and we must check these things to make a good plugin.
54
7.2 The art of writing a plugin (plugins tutorial) 7 CREATING A PLUGIN
-- -------------------------------------------------------------------------- --
-- Fill msg uidl field
function uidl(pstate,msg) return common.uidl(pstate,msg) end
-- -------------------------------------------------------------------------- --
-- Fill all messages uidl field
function uidl_all(pstate) return common.uidl_all(pstate) end
-- -------------------------------------------------------------------------- --
-- Fill msg size
function list(pstate,msg) return common.list(pstate,msg) end
-- -------------------------------------------------------------------------- --
-- Fill all messages size
function list_all(pstate) return common.list_all(pstate) end
-- -------------------------------------------------------------------------- --
-- Unflag each message marked for deletion
function rset(pstate) return common.rset(pstate) end
-- -------------------------------------------------------------------------- --
-- Mark msg for deletion
function dele(pstate,msg) return common.dele(pstate,msg) end
-- -------------------------------------------------------------------------- --
-- Do nothing
function noop(pstate) return common.noop(pstate) end
but first add the common module loading code to your init() function
-- checks on globals
freepops.set_sanity_checks()
return POPSERVER_ERR_OK
end
55
7.2 The art of writing a plugin (plugins tutorial) 7 CREATING A PLUGIN
-- -------------------------------------------------------------------------- --
-- Update the mailbox status and quit
function quit_update(pstate)
-- we need the stat
local st = stat(pstate)
if st ~= POPSERVER_ERR_OK then return st end
if delete_something then
b:post_uri(post_uri,post_data)
end
return POPSERVER_ERR_OK
end
Consider we do the post only if at least one message is marked for deletion.
Another important think to keep in mind is that making only one post for all
messages is better than making a single post for each message. When it is
possible you should reduce the number of HTTP requests as much as you can
since it is here we move FreePOPs from a rabbit to a tortoise.
56
7.2 The art of writing a plugin (plugins tutorial) 7 CREATING A PLUGIN
is usually simple. It really depends on the webmail, but here we will talk of
the simple case, while at the end of the tutorial you will see how to deal with
complex webmails. The simple case is the one in which the webmail has a save
message button. And the saved message is a plain text file containing both the
header and the body of the message. There are only two interesting points in
this case, firstly big messages, secondly the dot issue.
Big messages are a cause of timeout. Yes, the most simple way of down-
loading a message is calling b:get_uri() and store the message in a variable,
and then send it to the mail client with popserver_callback(). But think
that a 5MB mail, downloaded with a 640Kbps DSL connection, at full 80KBps
speed, takes 64 seconds to download. This means your plugin will not send
any data to the mail client for more than one minute and this will make the
mail client to disconnect from FreePOPs thinking the POP3 server is dead. So
we must send the data to the mail client as soon as we can. For this we have
the b:pipe_uri() function that calls a callback whenever it has some fresh
data. The following code is the callback factory function, that creates a new
callback to pass to the pipe_uri browser method.
--------------------------------------------------------------------------------
-- The callback factory for retr
--
function retr_cb(data)
local a = stringhack.new()
return function(s,len)
s = a:dothack(s).."\0"
popserver_callback(s,data)
return len,nil
end
end
Here you see that the callback simply uses popserver_callback() to pass
the data to the mail client, but before doing this it mangles the data with the
stringhack. But this is the second interesting point.
The POP3 protocol should end the retr command answer with a line that
contains only 3 bytes, “.\r\n”. But what if a line, inside the mail body, is
a simple point? We have to escape it to “..\r\n”. This is not so hard, a
string.gsub(s,”\r\n.\r\n”,”\r\n..\r\n”) is all we need... but not in the
case of callbacks. The send callback will be called with some fresh data, and
called more than once if the mail is big. And if the searched pattern is trun-
cated between two calls the string.gsub() method will fail. This is why the
stringhack module helps us. The a object lives as long as the callback function
will be called (see the closure page of the lua tutorial) and will keep in mind
that the searched pattern may be truncated.
Finally the retr() code
57
7.2 The art of writing a plugin (plugins tutorial) 7 CREATING A PLUGIN
-- -------------------------------------------------------------------------- --
-- Get message msg, must call
-- popserver_callback to send the data
function retr(pstate,msg,pdata)
-- we need the stat
local st = stat(pstate)
if st ~= POPSERVER_ERR_OK then return st end
-- the callback
local cb = retr_cb(data)
The multi-page stat is the real good implementation for stat(). We men-
tioned before that our implementation lists only the messages in the first
page. The code for parsing and extracting interesting info from a page is
already written, we simply need a function that checks if we are in the last
page and if not it changes the value of a uri variable. The uri variable
will be used by the fetch function. In this case you should use the support
module with the do_until cycle. This is a simple example of do_until()
58
7.2 The art of writing a plugin (plugins tutorial) 7 CREATING A PLUGIN
-- -------------------------------------------------------------------------- --
-- Fill the number of messages and their size
function stat(pstate)
... some code as before ...
-- gets all the results and puts them in the popstate structure
... some code as before ...
set_mailmessage_size(pstate,i+nmesg_old,size)
set_mailmessage_uidl(pstate,i+nmesg_old,uidl)
end
return true,nil
end
59
7.2 The art of writing a plugin (plugins tutorial) 7 CREATING A PLUGIN
return false
else
return true
end
end
return f,err
end
-- do it
if not support.do_until(retrieve_f,check_f,action_f) then
log.error_print("Stat failed\n")
session.remove(key())
return POPSERVER_ERR_UNKNOWN
end
The only strange things are the retrieve function and the session saving
stuff. Since webmail sometimes timeout you should check if the retrieved
page is valid or not, and eventually retry the login. The session saving is
60
7.2 The art of writing a plugin (plugins tutorial) 7 CREATING A PLUGIN
Saving the session is the way to make FreePOPs really similar to a browser.
This means the next time you check the mail FreePOPs will simply reload
the inbox page and won’t login again. To do this you need a key() function
that gives a unique ID for each session
--------------------------------------------------------------------------------
-- The key used to store session info
--
-- This key must be unique for all webmails, since the session pool is one
-- for all the webmails
--
function key()
return foo_globals.username .. foo_globals.password
end
--------------------------------------------------------------------------------
-- Serialize the internal state
--
-- serial.serialize is not enough powerful to correctly serialize the
-- internal state. The field b is the problem. b is an object. This means
-- that it is a table (and no problem for this) that has some field that are
-- pointers to functions. this is the problem. there is no easy way for the
-- serial module to know how to serialize this. so we call b:serialize
-- method by hand hacking a bit on names
--
function serialize_state()
internal_state.stat_done = false;
return serial.serialize("foo_globals",foo_globals) ..
internal_state.b:serialize("foo_globals.b")
end
Now you have to tell FreePOPs to save the state in the quit_update()
function and load it back in the pass() one. This is the new pass()
structure
function pass(pstate,password)
-- save the password
internal_state.password = password
61
7.2 The art of writing a plugin (plugins tutorial) 7 CREATING A PLUGIN
local s = session.load_lock(key())
return POPSERVER_ERR_OK
else
-- call the login procedure
return foo_login()
end
end
where foo_login() is the old pass() function with minor changes. Don’t
forget to call session.unlock(key()) in the quit() function, since you
have to release the session in case of failure (and quit() is called here)
and to save the session in quit_update()
62
7.2 The art of writing a plugin (plugins tutorial) 7 CREATING A PLUGIN
sends you the message source supports the “Range:” HTTP request field,
or the tin.lua plugin if the server needs to be interrupted in a bad way.
Remember that the top() needs someone that counts the lines and here
we have again the stringhack module that counts and may purge some
lines.
The javascript is the hell of webmails. Javascripts can do anything and you
have to read them to emulate what they do. For example they may add
some cookies (and you’ll have to do this by hand with the b:add_cookie()
as in tin.lua) or they may change some form fields (like in the libero.lua
login load balancing code).
The cookies are sweet enough for us, since the browser module will handle
them for us.
The standard files are really system dependent. Under Windows you’ll have
to constantly look at the stderr.txt and stdout.txt, while under Unix
you will just have to start it with the -w switch and look at the console.
The brute force is called Ethereal. Sometimes things don’t work in the right
way and the only way to debug them is to activate curl debugging to see
what FreePOPs does (b.curl:setopt(curl.OPT_VERBOSE,1)) and sniff
what a real browser does with a tool like Ethereal.
The open source way is the best way of having a good quality piece of soft-
ware. This means you’ll have to release really often your plugin in the de-
velopment phase and interact much with your testers. Trust me it works,
or read “The cathedral and the bazaar” by Eric Raymond.
The mimer module is really beta at the time of this tutorial, but is what you
need if you are in the unlucky case of a webmail that has no save message
button. The lycos.lua plugin is an example of what it can do. The main
interesting function is mimer.pipe_msg() that takes a message header,
a text body (in html o plain text format) and some attachments URIs,
that are downloaded on the fly, composed into a proper mail message and
piped to the mail client.
Parameters to modules may be passed from the config.lua file or on the fly
using user@domain?param1=value1&...¶mN=valueN as described
in the plugins chapter. For the plugin writer there is no difference be-
tween the two passing mechanisms. The parameters are available to the
plugin in the table freepops.MODULE_ARGS.
Regex to define handled domains are allowed since version 0.0.29. Official
plugins can have a config.lua line like
63
9 FAQ
freepops.MODULES_MAP["foo2.*"] = {
name="foo.lua",
regex = true -- enables the regex processing
}
while unofficial plugins can declare a regexp list in the PLUGIN_REGEXES
field. For example
PLUGIN_REGEXES = {"@foo2.*", "@foo3.*", "@foo4[A-Z]*"}
8 Submitting a bug
When you have problems or you think you have found a bug, please follow
strictly this iter:
1. Update to the most recent version of FreePOPs.
2. Try to reproduce the bug, if the bug is not easily reproducible we are out
of luck. Something can still be tried: if the software crashed you could
compile it from the sources, install valgrind, run freepopsd with valgrind
and hope the error messages are interesting.
3. Clean the log files
4. Start FreePOPs with the -w switch
5. Reproduce the bug
6. Send to the developers the log, plus any other info like your system type
and how to reproduce this bug.
9 FAQ
How do I configure FreePOPs?
In normal use conditions you don’t need to configure FreePOPs, you just have
to change your mail client settings as described in the tutorial. For other use
cases we provided specific tutorials. Remember to set the POP3 server address
to localhost (if you’ve installed FreePOPs on the same computer where you read
your mail, otherwise use the IP address of the computer where FreePOPs is
installed), the server port to 2000 (or whatever you chose if you ran FreePOPs
with the -p option) and to set the username to your full email address (in the
form username@webmail.domain).
If you continue having problems after reading the manual and the tutorial
you can post on the forum at HTTP://freepops.diludovico.it (usually in Italian,
but feel free to post in English).
64
9 FAQ
• Most importantly the log of FreePOPs, generated with the -w option (read
further if you don’t know how to add command line parameters). Make
sure that the log doesn’t contain any sensitive info you wouldn’t like to
disclose (to the authors). But don’t worry, your password is never recorded
in the log, it’s substituted by a sequence of nine (9) asterisks, no matter
what is its real length.
65
9 FAQ
66
10 AUTHORS
10 Authors
This manual has been written by Enrico Tassi <gareuselesinge [at] users.sourcefor
and revised and translated by Nicola Cocchiaro <ncocchiaro [at] users.sourceforge.
10.1 Developers
FreePOPs is developed by:
• Enrico Tassi <gareuselesinge [at] users.sourceforge.net>
• Alessio Caprari <alessiofender [at] users.sourceforge.net>
• Nicola Cocchiaro <ncocchiaro [at] users.sourceforge.net>
• Simone Vellei <simone_vellei [at] users.sourceforge.net>
yahoo.lua, hotmail.lua, aol.lua, netscape.lua, mailcom.lua, juno.lua, mail2world.lua,
criticalpath.lua, fastmail.lua are developed by:
• Russell Schwager <russells [at] despammed.com>
67
11 THANKS
11 Thanks
Special thanks go to the users who tested the software, to the hackers who
made it possible to have a free and reliable development environment as the
Debian GNU/Linux system.
Appendix
Since the ASCII table is really common, we have not included one here. You
may ask google.
68