summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authordrebs <drebs@leap.se>2015-04-29 15:46:29 -0300
committerdrebs <drebs@leap.se>2015-04-29 15:46:29 -0300
commit76937040c20dd786825261e6ee2fc3766ef3d8e0 (patch)
treeb4cf68efe89d4dd5bdf4362a388be6b51c403a9d
parentb5ed3e4db16c8e3856b5c45409807d8f58cd957a (diff)
parent799703cf884191d097eb5d5316fa964e421683fd (diff)
Merge tag '0.6.2'
Tag leap.mx version 0.6.2
-rw-r--r--CHANGELOG7
-rw-r--r--README.md93
-rw-r--r--doc/DESIGN.md186
-rw-r--r--doc/NOTES.md29
-rw-r--r--doc/leap-commit-template7
-rw-r--r--doc/leap-commit-template.README47
-rwxr-xr-xpkg/mx.tac4
-rw-r--r--src/leap/mx/alias_resolver.py91
-rw-r--r--src/leap/mx/bounce.py526
-rw-r--r--src/leap/mx/check_recipient_access.py68
-rw-r--r--src/leap/mx/couchdbhelper.py148
-rw-r--r--src/leap/mx/mail_receiver.py151
-rw-r--r--src/leap/mx/tcp_map.py72
13 files changed, 939 insertions, 490 deletions
diff --git a/CHANGELOG b/CHANGELOG
index 6f58b5e..c2f3662 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -1,3 +1,10 @@
+0.6.2 Apr 21, 2015:
+ o Add PGP key lookup on access check server and reject mail if no PGP key
+ was found for the user. Closes #6795.
+ o Fix bounce message recipient. Closes #6854.
+ o Implement bouncing as per RFCs 6522, 3834 and 3464. Closes #6858.
+ o Correctly return async bouncer deferred.
+
0.6.1 Feb 11, 2015:
o Process unprocessed mail when MX starts (closes #2591).
o Log to syslog. Closes: #6307
diff --git a/README.md b/README.md
index 03b0ade..d0b5018 100644
--- a/README.md
+++ b/README.md
@@ -1,83 +1,56 @@
-leap_mx
+Leap MX
=======
-**Note:** Currently in development. Feel free to test, and please [report
- bugs on our tracker](https://we.riseup.net/leap/mx) or [by email](mailto:isis@leap.se).
+
+**Note:** Currently in development. Feel free to test, and please [report bugs
+on our tracker](https://we.riseup.net/leap/mx) or [by
+email](mailto:discuss@leap.se).
An asynchronous, transparently-encrypting remailer for the LEAP platform,
using BigCouch/CouchDB and PGP/GnuPG, written in Twisted Python.
-## [install](#install) ##
+## Installing
-### [virtualenv](#virtualenv) ###
-=================================
-Impatient? Don't like virtualenvs? [tl;dr](#tl;dr)
+ * Leap MX is available as a debian package in [Leap
+ repository](http://deb.leap.se/repository/).
+ * A python package is available in
+ [pypi](https://pypi.python.org/pypi/leap.mx). Use ./pkg/requirements.pip
+ to install requirements.
+ * Source code is available in [github](https://github.com/leapcode/leap_mx).
-Virtualenv is somewhat equivalent to fakeroot for python packages, and -- due
-to being packaged with copies of pip and python -- can be used to bootstrap
-its own install process, allowing pip and python to be used with sudo.
+## Configuring
-#### installing without sudo ####
+A sample config file can be found in pkg/mx.conf.sample
-To install without using sudo, a bootstrap script to handle the setup process
-is provided. It does the following:
+## Running
- 1. Download, over SSL, the latest tarballs for virtualenv and
- virtualenvwrapper from pypi.
- 2. Unpack the tarballs, use the system python interpreter to call the
- virtualenv.py script to setup a bootstrap virtual environment.
- 3. Use the pip installed in the bootstrap virtualenv to install
- virtualenvwrapper in the bootstrap virtualenv.
- 4. Obtain a copy of leap_mx with git clone.
- 5. Use ```mkvirtualenv``` included in the virtualenvwrapper inside the
- bootstrap virtualenv to install a project virtualenv for leap_mx.
+The debian package contains an initscript for the service. If you want to run
+from source or from the python package, maybe setup a virtualenv and do:
-To use the bootstrap script, do:
~~~
-$ wget -O bootstrap https://raw.github.com/isislovecruft/leap_mx/fix/no-suid-for-virtualenv/bootstrap
-$ ./bootstrap
-$ workon leap_mx
+# git clone or unzip the python package, change into the dir, and do:
+$ python setup.py install
+# copy ./pkg/mx.conf.sample to /etc/leap/mx.conf and edit that file, then run:
+$ twistd -ny pkg/mx.tac
~~~
-#### installing in a regular virtualenv ###
-To install python, virtualenv, and get started, do:
+## Hacking
-~~~
-$ sudo apt-get install python2.7 python-dev python-virtualenv virtualenvwrapper
-$ git clone https://github.com/leapcode/leap_mx.git leap_mx
-$ export WORKON_LEAPMX=${PWD}/leap_mx
-$ source /usr/local/bin/virtualenvwrapper.sh
-$ mkvirtualenv -a $WORKON_LEAPMX -r ${WORKON_LEAPMX}/pkg/requirements.pip \
- --no-site-packages --setuptools --unzip-setuptools leap_mx
-~~~
+Please see the doc/DESIGN docs.
-### [tl;dr](#tl;dr) ###
-To get started quickly, without virtualenv, do:
-~~~
-$ sudo apt-get install python git
-$ git clone https://github.com/leapcode/leap_mx.git
-# pip install -r ./leap_mx/pkg/requirements.pip
-~~~
-Although, **it is advised** to install inside a python virtualenv.
+Our bugtracker is [here](https://leap.se/code/projects/mx).
-## [configuration](#configuration) ##
-A sample config file can be found in pkg/mx.conf.sample
+Please use that for bug reports and feature requests instead of github's
+tracker. We're using github for code commenting and review between
+collaborators.
-## [running](#running) ##
-=========================
+## Issues
-To get running, clone this repo, and (assuming you've already set up your
-virtualenv and obtained all the requirements) do:
+* see the [Changelog](./CHANGELOG) for details of all major changes in the different versions
-~~~
-$ twistd -ny mx.tac
-~~~
+### 0.6.1
-## [hacking](#hacking) ##
-=========================
-Please see the HACKING and DESIGN docs.
+* Bouncing messages can get into a bouncing loop (#6858)
-Our bugtracker is [here](https://leap.se/code/projects/eip/issue/new).
+### 0.6.0
-Please use that for bug reports and feature requests instead of github's
-tracker. We're using github for code commenting and review between
-collaborators.
+* leap-mx needs to get restarted after the first incoming mail is delivered (#6687)
diff --git a/doc/DESIGN.md b/doc/DESIGN.md
index 2d9fe82..e98976d 100644
--- a/doc/DESIGN.md
+++ b/doc/DESIGN.md
@@ -1,7 +1,7 @@
-# design #
+# design
+
+## overview
-## overview #
-----------------------
This page pertains to the incoming mail exchange servers of the provider.
General overview of how incoming email will work:
@@ -27,39 +27,36 @@ General overview of how incoming email will work:
9. Soledad, in the background, will then re-encrypt this email (now a soledad
document), and sync to the cloud.
-## postfix pipeline ##
----------------------------
+## postfix pipeline
+
incoming mx servers will run postfix, configured in a particular way:
1. postscreen: before accepting an incoming message, checks RBLs, checks RFC
validity, checks for spam pipelining.
- (pass) proceed to next step.
- (fail) return SMTP error, which bounces email.
+ * (pass) proceed to next step.
+ * (fail) return SMTP error, which bounces email.
2. more SMTP checks: valid hostnames, etc.
- (pass) accepted, proceed to next step.
- (fail) return SMTP error, which bounces email.
+ * (pass) accepted, proceed to next step.
+ * (fail) return SMTP error, which bounces email.
3. check_recipient_access -- look up each recipient and ensure they are
allowed to receive messages.
- (pass) empty result, proceed to next step.
- (fail) return SMTP error code and error comment, bounce message.
+ * (pass) empty result, proceed to next step.
+ * (fail) return SMTP error code and error comment, bounce message.
4. milter processessing (spamassassin & clamav)
- (pass) continue
- (fail) bounce message, flag as spam, or silently kill.
+ * (pass) continue
+ * (fail) bounce message, flag as spam, or silently kill.
5. virtual_alias_maps -- map user defined aliases and forwards
- (local address) continue if new address is for this mx
- (remote address) continue. normally, postfix would relay to the remote domain, but we don't want that.
+ * (local address) continue if new address is for this mx
+ * (remote address) continue. normally, postfix would relay to the remote domain, but we don't want that.
6. deliver message to spool
- (write) save the message to disk on the mx.
+ * (write) save the message to disk on the mx.
7. postfix's job is done, mail_receiver picks up email from spool directory
Questions:
- * what is the best way to have postfix write a message to a spool directory?
- There is a built-in facility for saving to a maildir, so we could just
- specify a common maildir for everyone. alternately, we could pipe to a
- simple command that was responsible for safely saving the file to disk. a
- third possibility would be to have a local long running daemon that spoke
- lmtp that postfix forward the message on to for delivery.
+ * postfix uses a built-in facility and saves all messages to a common
+ maildir. as an alternative, we could have a local long running daemon that
+ spoke lmtp that postfix forward the message on to for delivery.
* if virtual_alias_maps comes after check_recipient_access, then a user with
aliases set but who is over quota will not be able to forward email. i think
this is fine.
@@ -80,41 +77,42 @@ Considerations:
somewhere, and that copy should not be deleted until there is confirmation
that the next stage has succeeded.
-## alias_resolver ##
-------------------------------
-The alias_resolver will be a daemon running on MX servers that handles lookups
-in the user database of email aliases, forwards, quota, and account status.
+## TCP Maps
+
+MX runs TCP maps that handle lookups in the user database of email aliases,
+forwards, quota, and account status.
Communication with:
- 1. postfix:: alias_resolver will be bound to localhost and speak postfix's
- very simple [tcp map protocol -> http://www.postfix.org/tcp_table.5.html].
+ 1. postfix: bind to localhost and speak postfix's very simple [tcp map
+ protocol -> http://www.postfix.org/tcp_table.5.html].
- 2. couchdb:: alias_resolver will make couchdb queries to a local http load
- balancer that connects to a couchdb/bigcouch
- cluster. [directly accessing the couch->https://we.riseup.net/leap+platform/querying-the-couchdb]
- might help getting started.
+ 2. couchdb: make couchdb queries to a local http load balancer that connects
+ to a couchdb/bigcouch cluster.
-### Discussion: ###
+### Discussion
1. we want the lookups to be fast. using views in couchdb, these should be
- very fast. when using bigcouch, we can make it faster by specifying a read
- quorum of 1 (instead of the default 2). this will make it so that only a
- single couchdb needs to be queried to find the result. i don't know if this
- would cause problems, but aliases don't change very often.
+ very fast. when using bigcouch, we can make it faster by specifying a read
+ quorum of 1 (instead of the default 2). this will make it so that only a
+ single couchdb needs to be queried to find the result. i don't know if
+ this would cause problems, but aliases don't change very often.
+
+TCP map is responsible for two map lookups in postfix: ```check_recipient```
+and ```virtual_alias_map```.
-alias_resolver will be responsible for two map lookups in postfix:
+#### check_recipient
-#### check_recipient ####
--------------------------
postfix config:
-@check_recipient_access tcp:localhost:1000@
+```
+check_recipient_access tcp:localhost:2244
+```
-postfix will send "get username@domain.org" and alias_resolver should return an
-empty result ("200 \n", i think) if postfix should deliver email to the
-user. otherwise, it should return an error. here is example response, verbatim,
-that can be used to bounce over quota users:
+postfix sends "get username@domain.org" and alias_resolver returns an empty
+result ("200 \n", i think) if postfix should deliver email to the user.
+otherwise, it returns an error. here is example response, verbatim, that
+can be used to bounce over quota users:
```
200 DEFER_IF_PERMIT Sorry, your message cannot be delivered because the
@@ -126,34 +124,33 @@ to tell them of this problem.
that they should try again soon. Typically, an MX will try repeatedly, at
longer and longer intervals, for four days before giving up.
-#### virtual alias map ####
----------------------------
+#### virtual_alias_map
+
postfix config:
-@virtual_alias_map tcp:localhost:1001@
+```
+virtual_alias_map tcp:localhost:4242
+```
-postfix will send "get alias-address@domain.org" and alias_resolver should
-return "200 id_123456\n", where 123456 is the unique id of the user that has
+postfix sends "get alias-address@domain.org" and alias_resolver returns "200
+123456\n", where 123456 is the unique id of the user that has
alias-address@domain.org.
-couchdb should have a view that will let us query on an (alias) address and
-return the user id.
+couchdb has a view that lets us query on an (alias) address and return the
+user id.
-note: if the result of the alias map (e.g. id_123456) does not have a domain
-suffix, i think postfix will use the 'default transport'. if we want it to use
-the virtual transport instead, we should append the domain (eg
-id_123456@example.org). see
+note: if the result of the alias map (e.g. 123456) does not have a domain
+suffix, postfix will use the 'default transport'. if we want it to use the
+virtual transport instead, we should append the domain (eg
+123456@example.org). see
http://www.postfix.org/ADDRESS_REWRITING_README.html#resolve
-### Current status: ###
+### Current status
+
The current implementation of alias_resolver is in
leap-mx/src/leap/mx/alias_resolver.py.
-The class ```alias_resolver.StatusCodes``` deals with creating SMTP-like
-response messages for Postfix, speaking Postfix's TCP Map protocol (from item
-#1).
-
As for Discussion item #1:
It might be possible to use
@@ -165,37 +162,8 @@ handling Memcached servers, this is in ```twisted.protocols.memcache```. This
should be prioritised for later, if it is decided that querying the CouchDB is
too expensive or time-consuming.
-Thus far, to speed up alias lookup, an in-memory mapping of alias<->resolution
-pairs is created by ```alias_resolver.AliasResolverFactory()```, which can be
-optionally seeded with a dictionary of ```{ 'alias': 'resolution' }``` pairs
-by doing:
-~~~~~~
->>> from leap.mx import alias_resolver
->>> aliasResolverFactory = alias_resolver.AliasResolverFactory(
-... addr='1.2.3.4', port=4242, data={'isis': 'isis@leap.se',
-... 'drebs': 'drebs@leap.se'})
->>> aliasResolver = aliasResolverFactory.buildProtocol()
->>> aliasResolver.check_recipient_access('isis')
-200 OK Others might say 'HELLA AWESOME'...but we're not convinced.
-~~~~~~
-
-TODO:
- 1. The AliasResolverFactory needs to be connected to the CouchDB. The
- classmethod in which this should occur is ```AliasResolverFactory.get()```.
-
- 2. I am not sure where to get the user's UUID from (Soledad?). Wherever we get
- it from, it will need to be returned in
- ```AliasResolver.virtual_alias_map()```, and if we want Postfix to hear about
- it, then that response will need to be fed into ```AliasResolver.sendCode```.
-
- 3. Other than those two things, I think everything is done. The only potential
- other thing I can think of is that the codes in
- ```alias_resolver.StatusCodes``` might need to be urlencoded for Postfix to
- accept them, but this is like two lines of code from urllib.
-
-
-## mail_receiver ##
+## mail_receiver
the mail_receiver is a daemon that runs on incoming MX servers and is
responsible for encrypting incoming email to the user's public key and saving
@@ -203,36 +171,36 @@ the email to an incoming queue database for that user.
communicates with:
- * message spool directory:: mail_reciever sits and waits for new email to be
+ * message spool directory: mail_reciever sits and waits for new email to be
written to the spool directory (maybe using this
https://github.com/seb-m/pyinotify, i think it is better than FAM). when a
new file is dumped into the spool, mail_receiver reads the file, encrypts
the entire thing using the public key of the recipient, and saves to
couchdb.
- * couchdb get:: mail_receiver does a query on user id to get back user's
+ * couchdb get: mail_receiver does a query on user id to get back user's
public openpgp key. read quorum of 1 is probably ok.
- * couchdb put:: mail_receiver communicates with couchdb for storing encrypted
+ * couchdb put: mail_receiver communicates with couchdb for storing encrypted
email for each user (eventually, mail_receiver will communicate with a local
http proxy, that communicates with a bigcouch cluster, but the api is
identical)
-discussion:
- * i am not sure if postfix adds a header to indicate to whom a message was
- actually delivered. if not, this is a problem, because then how do we know
- what db to put it in or what public key to use? this is perhaps a good
- reason to not let postfix handle writing the message to disk, but instead
- pipe it to another command (because postfix sets env variables for stuff
- like recipient).
+### Current Status
- * should the incoming message queue be a separate database or should it be
- just documents in the user's main database with special flags?
+ * Postfix adds a "Delivered-To" header to indicate to whom a message was
+ actually delivered. There may be more than one header of this kind, and we
+ should use the topmost one. That value of that header is "<uuid>@<domain>",
+ and we can extract the user's uid from there to fetch the pgp public key to
+ which encrypt the message.
+
+ * currently, the incoming message is put into the user's database with some
+ special flags. should we have a different way to deal with that?
* whenever possible, we should refer to the user by a fixed id, not their
username, because we want to support the ability to change usernames. so,
- for example, database names should not be based on usernames.
-
-### Current Status: ###
-None of this is done, although having it be a separate daemon sound weird.
+ for example, database names should not be based on usernames. The alias
+ resolver then resolves to the user's uuid and from that point on we can
+ deal only with the user's uid.
-You would probably want to use ```twisted.mail.mail.FileMonitoringService``` to
-watch the mailbox (is the mailbox virtual or a maildir or mbox or?)
+ * We currently use ```twisted.internet.inotify.INotiry()``` to watch the
+ maildir for creation of new files. Alternatelly, we could possibly use
+ ```twisted.mail.mail.FileMonitoringService```.
diff --git a/doc/NOTES.md b/doc/NOTES.md
index a53f49d..337aa96 100644
--- a/doc/NOTES.md
+++ b/doc/NOTES.md
@@ -28,32 +28,3 @@ a viable protocol for this, and how would it interact with CouchDB?
4. What lib should we use for Python + Twisted + GPG/PGP ?
4.a. It looks like most people are using python-gnupg...
-
-
-## Tickets ##
--------------
-
-'''To be created:'''
-
-ticket for feature-alias_resolver_couchdb_support:
-
- o The alias resolver needs to speak to a couchdb/bigcouch
- instance(s). Currently, it merely creates an in-memory dictionary
- mapping. It seems like paisley is the best library for this.
-
-ticket for feature-check_recipient:
-
- o Need various errors for anything that could go wrong, e.g. the recipient
- address is malformed, sender doesn't have permissions to send to such
- address, etc.
- o These errcodes need to follow the SMTP server transport code spec.
-
-ticket for feature-virtual_alias_map:
-
- o Get the recipient's userid from couchdb.
-
-ticket for feature-evaluate_python_gnupg:
-
- o Briefly audit library in order to assess if it has the necessary
- features, as well as its general code quality.
-
diff --git a/doc/leap-commit-template b/doc/leap-commit-template
new file mode 100644
index 0000000..8a5c7cd
--- /dev/null
+++ b/doc/leap-commit-template
@@ -0,0 +1,7 @@
+[bug|feat|docs|style|refactor|test|pkg|i18n] ...
+...
+
+- Resolves: #XYZ
+- Related: #XYZ
+- Documentation: #XYZ
+- Releases: XYZ
diff --git a/doc/leap-commit-template.README b/doc/leap-commit-template.README
new file mode 100644
index 0000000..ce8809e
--- /dev/null
+++ b/doc/leap-commit-template.README
@@ -0,0 +1,47 @@
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+HOW TO USE THIS TEMPLATE:
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Run `git config commit.template docs/leap-commit-template` or
+edit the .git/config for this project and add
+`template = docs/leap-commit-template`
+under the [commit] block
+
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+COMMIT TEMPLATE FORMAT EXPLAINED
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+[type] <subject>
+
+<body>
+<footer>
+
+Type should be one of the following:
+- bug (bug fix)
+- feat (new feature)
+- docs (changes to documentation)
+- style (formatting, pep8 violations, etc; no code change)
+- refactor (refactoring production code)
+- test (adding missing tests, refactoring tests; no production code change)
+- pkg (packaging related changes; no production code change)
+- i18n translation related changes
+
+Subject should use imperative tone and say what you did.
+For example, use 'change', NOT 'changed' or 'changes'.
+
+The body should go into detail about changes made.
+
+The footer should contain any issue references or actions.
+You can use one or several of the following:
+
+- Resolves: #XYZ
+- Related: #XYZ
+- Documentation: #XYZ
+- Releases: XYZ
+
+The Documentation field should be included in every new feature commit, and it
+should link to an issue in the bug tracker where the new feature is analyzed
+and documented.
+
+For a full example of how to write a good commit message, check out
+https://github.com/sparkbox/how_to/tree/master/style/git
diff --git a/pkg/mx.tac b/pkg/mx.tac
index 75d2405..7da59cf 100755
--- a/pkg/mx.tac
+++ b/pkg/mx.tac
@@ -46,8 +46,8 @@ password = config.get("couchdb", "password")
server = config.get("couchdb", "server")
port = config.get("couchdb", "port")
-bounce_from = "bounce"
-bounce_subject = "Delivery failure"
+bounce_from = "Mail Delivery Subsystem <MAILER-DAEMON>"
+bounce_subject = "Undelivered Mail Returned to Sender"
try:
bounce_from = config.get("bounce", "from")
diff --git a/src/leap/mx/alias_resolver.py b/src/leap/mx/alias_resolver.py
index 45a3ed2..c6f2acc 100644
--- a/src/leap/mx/alias_resolver.py
+++ b/src/leap/mx/alias_resolver.py
@@ -19,73 +19,62 @@
"""
Classes for resolving postfix aliases.
+The resolver is queried by the mail server before delivery to the mail spool
+directory, and should return the user uuid. This way, we get rid from the user
+address early and the mail server will delivery the message to
+"<uuid>@<domain>". Later, the mail receiver part of MX will parse the
+"Delivered-To" header to extract the uuid and fetch the user's pgp public key.
+
Test this with postmap -v -q "foo" tcp:localhost:4242
TODO:
o Look into using twisted.protocols.postfix.policies classes for
controlling concurrent connections and throttling resource consumption.
+ o We should probably use twisted.mail.alias somehow.
"""
-try:
- # TODO: we should probably use the system alias somehow
- # from twisted.mail import alias
- from twisted.protocols import postfix
- from twisted.python import log
- from twisted.internet import defer
-except ImportError:
- print "This software requires Twisted. Please see the README file"
- print "for instructions on getting required dependencies."
-
-class LEAPPostFixTCPMapserver(postfix.PostfixTCPMapServer):
- def _cbGot(self, value):
- if value is None:
- self.sendCode(500, postfix.quote("NOT FOUND SRY"))
- else:
- self.sendCode(200, postfix.quote(value))
+from twisted.protocols import postfix
+from leap.mx.tcp_map import LEAPPostfixTCPMapServerFactory
+from leap.mx.tcp_map import TCP_MAP_CODE_SUCCESS
+from leap.mx.tcp_map import TCP_MAP_CODE_PERMANENT_FAILURE
-class AliasResolverFactory(postfix.PostfixTCPMapDeferringDictServerFactory):
- protocol = LEAPPostFixTCPMapserver
+class LEAPPostfixTCPMapAliasServer(postfix.PostfixTCPMapServer):
+ """
+ A postfix tcp map alias resolver server.
+ """
- def __init__(self, couchdb, *args, **kwargs):
- postfix.PostfixTCPMapDeferringDictServerFactory.__init__(
- self, *args, **kwargs)
- self._cdb = couchdb
-
- def _to_str(self, result):
- """
- Properly encodes the result string if any.
+ def _cbGot(self, user_data):
"""
- if isinstance(result, unicode):
- result = result.encode("utf8")
- if result is None:
- log.msg("Result not found")
- return result
+ Return a code and message depending on the result of the factory's
+ get().
- def spit_result(self, result):
+ :param user_data: The user's uuid and pgp public key.
+ :type user_data: list
"""
- Formats the return codes in a postfix friendly format.
- """
- if result is None:
- return None
+ uuid, _ = user_data
+ if uuid is None:
+ self.sendCode(
+ TCP_MAP_CODE_PERMANENT_FAILURE,
+ postfix.quote("NOT FOUND SRY"))
else:
- return defer.succeed(result)
+ # properly encode uuid, otherwise twisted complains when replying
+ if isinstance(uuid, unicode):
+ uuid = uuid.encode("utf8")
+ self.sendCode(
+ TCP_MAP_CODE_SUCCESS,
+ postfix.quote(uuid))
- def get(self, key):
- """
- Looks up the passed key, but only up to the username id of the key.
- At some point we will have to consider the domain part too.
- """
- try:
- log.msg("Query key: %s" % (key,))
- d = self._cdb.queryByAddress(key)
+class AliasResolverFactory(LEAPPostfixTCPMapServerFactory):
+ """
+ A factory for postfix tcp map alias resolver servers.
+ """
+
+ protocol = LEAPPostfixTCPMapAliasServer
- d.addCallback(self._to_str)
- d.addCallback(self.spit_result)
- d.addErrback(log.err)
- return d
- except Exception as e:
- log.err('exception in get: %r' % e)
+ @property
+ def _query_message(self):
+ return "Resolving alias for"
diff --git a/src/leap/mx/bounce.py b/src/leap/mx/bounce.py
new file mode 100644
index 0000000..2ece6df
--- /dev/null
+++ b/src/leap/mx/bounce.py
@@ -0,0 +1,526 @@
+#!/usr/bin/env python
+# -*- encoding: utf-8 -*-
+# bounce.py
+# Copyright (C) 2015 LEAP
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+
+"""
+Everything you need to correctly bounce a message!
+
+This is built from the following RFCs:
+
+ * The Multipart/Report Media Type for the Reporting of Mail System
+ Administrative Messages
+ https://tools.ietf.org/html/rfc6522
+
+ * Recommendations for Automatic Responses to Electronic Mail
+ https://tools.ietf.org/html/rfc3834
+
+ * An Extensible Message Format for Delivery Status Notifications
+ https://tools.ietf.org/html/rfc3464
+"""
+
+
+import re
+import socket
+
+from StringIO import StringIO
+from textwrap import wrap
+
+from email.errors import MessageError
+from email.message import Message
+from email.utils import formatdate
+from email.utils import parseaddr
+from email.mime.base import MIMEBase
+from email.mime.multipart import MIMEMultipart
+from email.mime.text import MIMEText
+from email.generator import Generator
+from email.generator import NL
+
+from twisted.internet import defer
+from twisted.internet import protocol
+from twisted.internet import reactor
+from twisted.internet.error import ProcessDone
+from twisted.python import log
+
+
+EMAIL_ADDRESS_REGEXP = re.compile("[^@]+@[^@]+\.[^@]+")
+HOSTNAME = socket.gethostbyaddr(socket.gethostname())[0]
+
+
+def _valid_address(address):
+ """
+ Return whether address is a valid email address.
+
+ :param address: An email address candidate.
+ :type address: str
+
+ :return: Whether address is valid.
+ :rtype: bool
+ """
+ return bool(EMAIL_ADDRESS_REGEXP.match(address))
+
+
+def bounce_message(bounce_from, bounce_subject, orig_msg, reason):
+ """
+ Bounce a message.
+
+ :param bounce_from: The sender of the bounce message.
+ :type bounce_from: str
+ :param bounce_subject: The subject of the bounce message.
+ :type bounce_subject: str
+ :param orig_msg: The original message that will be bounced.
+ :type orig_msg: email.message.Message
+ :param reason: The reason for bouncing the message.
+ :type reason: str
+
+ :return: A deferred that will fire with the output of the sendmail process
+ if it was successful or with a failure containing the reason for
+ the end of the process if it failed.
+ :rtype: Deferred
+ """
+ orig_rpath = orig_msg.get("Return-Path")
+
+ # do not bounce if sender address is invalid
+ _, addr = parseaddr(orig_rpath)
+ if not _valid_address(addr):
+ log.msg(
+ "Will not send a bounce message to an invalid address: %s"
+ % orig_rpath)
+ return
+
+ msg = _build_bounce_message(
+ bounce_from, bounce_subject, orig_msg, reason)
+ return _async_check_output(["/usr/sbin/sendmail", "-t"], msg.as_string())
+
+
+def _check_valid_return_path(return_path):
+ """
+ Check if a certain return path is valid.
+
+ From RFC 3834:
+
+ Responders MUST NOT generate any response for which the
+ destination of that response would be a null address (e.g., an
+ address for which SMTP MAIL FROM or Return-Path is <>), since the
+ response would not be delivered to a useful destination.
+ Responders MAY refuse to generate responses for addresses commonly
+ used as return addresses by responders - e.g., those with local-
+ parts matching "owner-*", "*-request", "MAILER-DAEMON", etc.
+ Responders are encouraged to check the destination address for
+ validity before generating the response, to avoid generating
+ responses that cannot be delivered or are unlikely to be useful.
+
+ :return: Whether the return_path is valid.
+ :rtype: bool
+ """
+ _, addr = parseaddr(return_path)
+
+ # check null address
+ if not addr:
+ return False
+
+ # check addresses commonly used as return addresses by responders
+ local, _ = addr.split("@", 1)
+ if local.startswith("owner-") \
+ or local.endswith("-request") \
+ or local.startswith("MAILER-DAEMON"):
+ return False
+
+ return True
+
+
+class DeliveryStatusNotificationMessage(MIMEBase):
+ """
+ A delivery status message, as per RFC 3464.
+ """
+
+ def __init__(self, orig_msg):
+ """
+ Initialize the DSN.
+ """
+ MIMEBase.__init__(self, "message", "delivery-status")
+ self.__delitem__("MIME-Version")
+ self._build_dsn(orig_msg)
+
+ def _build_dsn(self, orig_msg):
+ """
+ Build an RFC 3464 compliant delivery status message.
+
+ :param orig_msg: The original bouncing message.
+ :type orig_msg: email.message.Message
+ """
+ content = []
+
+ # Per-Message DSN fields
+ # ======================
+
+ # Original-Envelope-Id (optional)
+ envelope_id = orig_msg.get("Envelope-Id")
+ if envelope_id:
+ content.append("Original-Envelope-Id: %s" % envelope_id)
+
+ # Reporting-MTA (required)
+ content.append("Reporting-MTA: dns; %s" % HOSTNAME)
+
+ # XXX add Arrival-Date DSN field? (optional).
+
+ content.append("")
+
+ # Per-Recipient DSN fields
+ # ========================
+
+ # Original-Recipient (optional)
+ orig_to = orig_msg.get("X-Original-To") # added by postfix
+ _, orig_addr = parseaddr(orig_to)
+ if orig_addr:
+ content.append("Original-Recipient: rfc822; %s" % orig_addr)
+
+ # Final-Recipient (required)
+ delivered_to = orig_msg.get("Delivered-To")
+ content.append("Final-Recipient: rfc822; %s" % delivered_to)
+
+ # Action (required)
+ content.append("Action: failed")
+
+ # Status (required)
+ content.append("Status: 5.0.0") # permanent failure
+
+ # XXX add other optional fields? (Remote-MTA, Diagnostic-Code,
+ # Last-Attempt-Date, Final-Log-ID, Will-Retry-Until)
+
+ # return a "message/delivery-status" message
+ msg = Message()
+ msg.set_payload("\n".join(content))
+ self.attach(msg)
+
+
+class RFC822Headers(MIMEText):
+ """
+ A text/rfc822-headers mime message as defined in RFC 6522.
+ """
+
+ def __init__(self, _text, **kwargs):
+ """
+ Initialize the message.
+
+ :param _text: The contents of the message.
+ :type _text: str
+ """
+ MIMEText.__init__(
+ self, _text,
+ # set "text/rfc822-headers" mime type
+ _subtype='rfc822-headers',
+ **kwargs)
+
+
+BOUNCE_TEMPLATE = """
+This is the mail system at {0}.
+
+I'm sorry to have to inform you that your message could not
+be delivered to one or more recipients. It's attached below.
+
+For further assistance, please send mail to postmaster.
+
+If you do so, please include this problem report. You can
+delete your own text from the attached returned message.
+
+ The mail system
+
+{1}
+""".strip()
+
+
+class InvalidReturnPathError(MessageError):
+ """
+ Exception raised when the return path is invalid.
+ """
+
+
+def _build_bounce_message(bounce_from, bounce_subject, orig_msg, reason):
+ """
+ Build a bounce message.
+
+ :param bounce_from: The sender address of the bounce message.
+ :type bounce_from: str
+ :param bounce_subject: The subject of the bounce message.
+ :type bounce_subject: str
+ :param orig_msg: The original bouncing message.
+ :type orig_msg: email.message.Message
+ :param reason: The reason for the bounce.
+ :type reason: str
+
+ :return: The bounce message.
+ :rtype: MIMEMultipartReport
+
+ :raise InvalidReturnPathError: Raised when the "Return-Path" header of the
+ original message is invalid for creating a
+ bounce message.
+ """
+ # abort creation if "Return-Path" header is invalid
+ orig_rpath = orig_msg.get("Return-Path")
+ if not _check_valid_return_path(orig_rpath):
+ raise InvalidReturnPathError
+
+ msg = MIMEMultipartReport()
+ msg['From'] = bounce_from
+ msg['To'] = orig_rpath
+ msg['Date'] = formatdate(localtime=True)
+ msg['Subject'] = bounce_subject
+ msg['Return-Path'] = "<>" # prevent bounce message loop, see RFC 3834
+
+ # create and attach first required part
+ orig_to = orig_msg.get("X-Original-To") # added by postfix
+ wrapped_reason = wrap(("<%s>: " % orig_to) + reason, 74)
+ for i in xrange(1, len(wrapped_reason)):
+ wrapped_reason[i] = " " + wrapped_reason[i]
+ wrapped_reason = "\n".join(wrapped_reason)
+ text = BOUNCE_TEMPLATE.format(HOSTNAME, wrapped_reason)
+ msg.attach(MIMEText(text))
+
+ # create and attach second required part
+ msg.attach(DeliveryStatusNotificationMessage(orig_msg))
+
+ # attach third (optional) part.
+ #
+ # XXX From RFC 6522:
+ #
+ # When 8-bit or binary data not encoded in a 7-bit form is to be
+ # returned, and the return path is not guaranteed to be 8-bit or
+ # binary capable, two options are available. The original message
+ # MAY be re-encoded into a legal 7-bit MIME message or the
+ # text/rfc822-headers media type MAY be used to return only the
+ # original message headers.
+ #
+ # This is not implemented yet, we should detect if content is 7bit and
+ # use the class RFC822Headers if it is not.
+# try:
+# payload = orig_msg.get_payload()
+# payload.encode("ascii")
+# except UnicodeError:
+# headers = []
+# for k in orig_msg.keys():
+# headers.append("%s: %s" % (k, orig_msg[k]))
+# orig_msg = RFC822Headers("\n".join(headers))
+ msg.attach(orig_msg)
+
+ return msg
+
+
+class BouncerSubprocessProtocol(protocol.ProcessProtocol):
+ """
+ Bouncer subprocess protocol that will feed the msg contents to be
+ bounced through stdin
+ """
+
+ def __init__(self, msg):
+ """
+ Constructor for the BouncerSubprocessProtocol
+
+ :param msg: Message to send to stdin when the process has
+ launched
+ :type msg: str
+ """
+ self._msg = msg
+ self._outBuffer = ""
+ self._errBuffer = ""
+ self._d = defer.Deferred()
+
+ @property
+ def deferred(self):
+ return self._d
+
+ def connectionMade(self):
+ self.transport.write(self._msg)
+ self.transport.closeStdin()
+
+ def outReceived(self, data):
+ self._outBuffer += data
+
+ def errReceived(self, data):
+ self._errBuffer += data
+
+ def processEnded(self, reason):
+ if reason.check(ProcessDone):
+ self._d.callback(self._outBuffer)
+ else:
+ self._d.errback(reason)
+
+
+def _async_check_output(args, msg):
+ """
+ Async spawn a process and return a defer to be able to check the
+ output with a callback/errback
+
+ :param args: the command to execute along with the params for it
+ :type args: list of str
+ :param msg: string that will be send to stdin of the process once
+ it's spawned
+ :type msg: str
+
+ :rtype: defer.Deferred
+ """
+ pprotocol = BouncerSubprocessProtocol(msg)
+ reactor.spawnProcess(pprotocol, args[0], args)
+ return pprotocol.deferred
+
+
+class DSNGenerator(Generator):
+ """
+ A slightly modified generator to correctly parse delivery status
+ notifications.
+ """
+
+ def _handle_message_delivery_status(self, msg):
+ """
+ Handle a message of type "message/delivery-status".
+
+ This is modified from upstream version in that it also removes empty
+ lines in the beginning of each part.
+
+ :param msg: The message to be handled.
+ :type msg: Message
+ """
+ # We can't just write the headers directly to self's file object
+ # because this will leave an extra newline between the last header
+ # block and the boundary. Sigh.
+ blocks = []
+ for part in msg.get_payload():
+ s = StringIO()
+ g = self.clone(s)
+ g.flatten(part, unixfrom=False)
+ text = s.getvalue()
+ lines = text.split('\n')
+ # Strip off the unnecessary trailing empty line
+ if lines:
+ if lines[0] == '':
+ lines.pop(0)
+ if lines[-1] == '':
+ lines.pop()
+ blocks.append(NL.join(lines))
+ else:
+ blocks.append(text)
+ # Now join all the blocks with an empty line. This has the lovely
+ # effect of separating each block with an empty line, but not adding
+ # an extra one after the last one.
+ self._fp.write(NL.join(blocks))
+
+
+class MIMEMultipartReport(MIMEMultipart):
+ """
+ Implement multipart/report MIME type as defined in RFC 6522.
+
+ The syntax of multipart/report is identical to the multipart/mixed
+ content type defined in https://tools.ietf.org/html/rfc2045.
+
+ The multipart/report media type contains either two or three sub-
+ parts, in the following order:
+
+ 1. (REQUIRED) A human-readable message.
+ 2. (REQUIRED) A machine-parsable body part containing an account of
+ the reported message handling event.
+ 3. (OPTIONAL) A body part containing the returned message or a
+ portion thereof.
+ """
+
+ def __init__(
+ self, report_type="message/delivery-status", boundary=None,
+ _subparts=None):
+ """
+ Initialize the message.
+
+ As per RFC 6522, boundary and report_type are required parameters.
+
+ :param report_type: The type of report. This is set as a
+ "Content-Type" parameter, and should match the
+ MIME subtype of the second body part.
+ :type report_type: str
+
+ """
+ MIMEMultipart.__init__(
+ self,
+ # set mime type to "multipart/report"
+ _subtype="report",
+ boundary=boundary,
+ _subparts=_subparts,
+ # add "report-type" as a "Content-Type" parameter
+ report_type=report_type)
+ self._report_type = report_type
+
+ def attach(self, payload):
+ """
+ Add the given payload to the current payload, but first verify if it's
+ valid according to RFC6522.
+
+ :param payload: The payload to be attached.
+ :type payload: Message
+
+ :raise MessageError: Raised if the payload is invalid.
+ """
+ idx = len(self.get_payload()) + 1
+ self._check_valid_payload(idx, payload)
+ MIMEMultipart.attach(self, payload)
+
+ def _check_valid_payload(self, idx, payload):
+ """
+ Check that an attachment is valid according to RFC6522.
+
+ :param payload: The payload to be attached.
+ :type payload: Message
+
+ :raise MessageError: Raised if the payload is invalid.
+ """
+ if idx == 1:
+ # The text in the first section can use any IANA-registered MIME
+ # media type, charset, or language.
+ cond = lambda payload: isinstance(payload, MIMEBase)
+ error_msg = "The first attachment must be a MIME message."
+ elif idx == 2:
+ # RFC 6522 requires that the report-type parameter is equal to the
+ # MIME subtype of the second body type of the multipart/report.
+ cond = lambda payload: \
+ payload.get_content_type() == self._report_type
+ error_msg = "The second attachment's subtype must be %s." \
+ % self._report_type
+ elif idx == 3:
+ # A body part containing the returned message or a portion thereof.
+ cond = lambda payload: isinstance(payload, Message)
+ error_msg = "The third attachment must be a message."
+ else:
+ # The multipart/report media type contains either two or three sub-
+ # parts.
+ cond = lambda _: False
+ error_msg = "The multipart/report media type contains either " \
+ "two or three sub-parts."
+ if not cond(payload):
+ raise MessageError("Invalid attachment: %s" % error_msg)
+
+ def as_string(self, unixfrom=False):
+ """
+ Return the entire formatted message as string.
+
+ This is modified from upstream to use our own generator.
+
+ :param as_string: Whether to include the Unix From envelope heder.
+ :type as_string: bool
+
+ :return: The entire formatted message.
+ :rtype: str
+ """
+ fp = StringIO()
+ g = DSNGenerator(fp)
+ g.flatten(self, unixfrom=unixfrom)
+ return fp.getvalue()
diff --git a/src/leap/mx/check_recipient_access.py b/src/leap/mx/check_recipient_access.py
index b80ccfd..55460a6 100644
--- a/src/leap/mx/check_recipient_access.py
+++ b/src/leap/mx/check_recipient_access.py
@@ -17,26 +17,72 @@
# along with this program. If not, see <http://www.gnu.org/licenses/>.
"""
-Classes for resolving postfix recipient access
+Classes for resolving postfix recipient access.
+
+The resolver is queried by the mail server before delivery to the mail spool
+directory, and should check if the address is able to receive messages.
+Examples of reasons for denying delivery would be that the user is out of
+quota, is user, or have no pgp public key in the server.
Test this with postmap -v -q "foo" tcp:localhost:2244
"""
from twisted.protocols import postfix
-from leap.mx.alias_resolver import AliasResolverFactory
+from leap.mx.tcp_map import LEAPPostfixTCPMapServerFactory
+from leap.mx.tcp_map import TCP_MAP_CODE_SUCCESS
+from leap.mx.tcp_map import TCP_MAP_CODE_TEMPORARY_FAILURE
+from leap.mx.tcp_map import TCP_MAP_CODE_PERMANENT_FAILURE
+
+
+class LEAPPostFixTCPMapAccessServer(postfix.PostfixTCPMapServer):
+ """
+ A postfix tcp map recipient access checker server.
+ The server potentially receives the uuid and a PGP key for the user, which
+ are looked up by the factory, and will return a permanent or a temporary
+ failure in case either the user or the key don't exist, respectivelly.
+ """
-class LEAPPostFixTCPMapserverAccess(postfix.PostfixTCPMapServer):
def _cbGot(self, value):
- # For more info, see:
- # http://www.postfix.org/tcp_table.5.html
- # http://www.postfix.org/access.5.html
- if value is None:
- self.sendCode(500, postfix.quote("REJECT"))
+ """
+ Return a code and message depending on the result of the factory's
+ get().
+
+ If there's no pgp public key for the user, we currently return a
+ temporary failure saying that the user account is disabled.
+
+ For more info, see: http://www.postfix.org/access.5.html
+
+ :param value: The uuid and public key.
+ :type value: list
+ """
+ uuid, pubkey = value
+ if uuid is None:
+ self.sendCode(
+ TCP_MAP_CODE_PERMANENT_FAILURE,
+ postfix.quote("REJECT"))
+ elif pubkey is None:
+ self.sendCode(
+ TCP_MAP_CODE_TEMPORARY_FAILURE,
+ postfix.quote("4.7.13 USER ACCOUNT DISABLED"))
else:
- self.sendCode(200, postfix.quote("OK"))
+ self.sendCode(
+ TCP_MAP_CODE_SUCCESS,
+ postfix.quote("OK"))
+
+
+class CheckRecipientAccessFactory(LEAPPostfixTCPMapServerFactory):
+ """
+ A factory for the recipient access checker.
+
+ When queried, the factory looks up the user's uuid and a PGP key for that
+ user and returns the result to the server's _cbGot() method.
+ """
+
+ protocol = LEAPPostFixTCPMapAccessServer
+ @property
+ def _query_message(self):
+ return "Checking recipient access for"
-class CheckRecipientAccessFactory(AliasResolverFactory):
- protocol = LEAPPostFixTCPMapserverAccess
diff --git a/src/leap/mx/couchdbhelper.py b/src/leap/mx/couchdbhelper.py
index f20f1dd..1752b4e 100644
--- a/src/leap/mx/couchdbhelper.py
+++ b/src/leap/mx/couchdbhelper.py
@@ -15,24 +15,15 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
"""
Classes for working with CouchDB or BigCouch instances which store email alias
maps, user UUIDs, and GPG keyIDs.
"""
-from functools import partial
-
-try:
- from paisley import client
-except ImportError:
- print "This software requires paisley. Please see the README file"
- print "for instructions on getting required dependencies."
-try:
- from twisted.python import log
-except ImportError:
- print "This software requires Twisted. Please see the README file"
- print "for instructions on getting required dependencies."
+from paisley import client
+from twisted.python import log
class ConnectedCouchDB(client.CouchDB):
@@ -66,24 +57,8 @@ class ConnectedCouchDB(client.CouchDB):
username=username,
password=password,
*args, **kwargs)
-
self._cache = {}
- if dbName is None:
- databases = self.listDB()
- databases.addCallback(self._print_databases)
-
- def _print_databases(self, data):
- """
- Callback for listDB that prints the available databases
-
- :param data: response from the listDB command
- :type data: array
- """
- log.msg("Available databases:")
- for database in data:
- log.msg(" * %s" % (database,))
-
def createDB(self, dbName):
"""
Overrides ``paisley.client.CouchDB.createDB``.
@@ -96,110 +71,63 @@ class ConnectedCouchDB(client.CouchDB):
"""
pass
- def queryByAddress(self, address):
+ def getUuidAndPubkey(self, address):
"""
- Check to see if a particular email or alias exists.
+ Query couch and return a deferred that will fire with the uuid and pgp
+ public key for address.
- :param alias: A string representing the email or alias to check.
- :type alias: str
- :return: a deferred for this query
+ :param address: A string representing the email or alias to check.
+ :type address: str
+ :return: A deferred that will fire with the user's uuid and pgp public
+ key.
:rtype twisted.defer.Deferred
"""
- assert isinstance(address, (str, unicode)), "Email or alias queries must be string"
-
# TODO: Cache results
-
d = self.openView(docId="Identity",
viewId="by_address/",
key=address,
reduce=False,
include_docs=True)
- d.addCallbacks(partial(self._get_uuid, address), log.err)
-
+ def _get_uuid_and_pubkey_cbk(result):
+ uuid = None
+ pubkey = None
+ if result["rows"]:
+ doc = result["rows"][0]["doc"]
+ uuid = doc["user_id"]
+ if "keys" in doc:
+ pubkey = doc["keys"]["pgp"]
+ return uuid, pubkey
+
+ d.addCallback(_get_uuid_and_pubkey_cbk)
return d
- def _get_uuid(self, address, result):
- """
- Parses the result of the by_address query and gets the uuid
-
- :param address: alias looked up
- :type address: string
- :param result: result dictionary
- :type result: dict
- :return: The uuid for alias if available
- :rtype: str
- """
- for row in result["rows"]:
- if row["key"] == address:
- uuid = row["doc"].get("user_id", None)
- if uuid is None:
- log.msg("ERROR: Found doc for %s but there's not user_id!"
- % (address,))
- return uuid
- return None
-
- def getPubKey(self, uuid):
+ def getPubkey(self, uuid):
"""
- Returns a deferred that will return the pubkey for the uuid provided
+ Query couch and return a deferred that will fire with the pgp public
+ key for user with given uuid.
- :param uuid: uuid for the user to query
+ :param uuid: The uuid of a user
:type uuid: str
+ :return: A deferred that will fire with the pgp public key for
+ the user.
:rtype: Deferred
"""
d = self.openView(docId="Identity",
- viewId="pgp_key_by_email/",
- user_id=uuid,
+ viewId="by_user_id/",
+ key=uuid,
reduce=False,
include_docs=True)
- d.addCallbacks(partial(self._get_pgp_key, uuid), log.err)
+ def _get_pubkey_cbk(result):
+ pubkey = None
+ try:
+ doc = result["rows"][0]["doc"]
+ pubkey = doc["keys"]["pgp"]
+ except (KeyError, IndexError):
+ pass
+ return pubkey
+ d.addCallbacks(_get_pubkey_cbk, log.err)
return d
-
- def _get_pgp_key(self, uuid, result):
- """
- Callback used to filter the correct pubkey from the result of
- the query to the couchdb
-
- :param uuid: uuid for the user that was queried
- :type uuid: str
- :param result: result dictionary for the db query
- :type result: dict
-
- :rtype: str or None
- """
- for row in result["rows"]:
- user_id = row["doc"].get("user_id")
- if not user_id:
- print("User %s is in an inconsistent state")
- continue
- if user_id == uuid:
- return row["value"]
- return None
-
-if __name__ == "__main__":
- from twisted.internet import reactor
- cdb = ConnectedCouchDB("localhost",
- port=6666,
- dbName="users",
- username="",
- password="")
-
- d = cdb.queryByLoginOrAlias("test1")
-
- @d.addCallback
- def right(result):
- print "Should be an actual uuid:", result
- print "Public Key:"
- print cdb.getPubKey(result)
-
- d2 = cdb.queryByLoginOrAlias("asdjaoisdjoiqwjeoi")
-
- @d2.addCallback
- def wrong(result):
- print "Should be None:", result
-
- reactor.callLater(5, reactor.stop)
- reactor.run()
diff --git a/src/leap/mx/mail_receiver.py b/src/leap/mx/mail_receiver.py
index 630c982..446fd38 100644
--- a/src/leap/mx/mail_receiver.py
+++ b/src/leap/mx/mail_receiver.py
@@ -39,13 +39,8 @@ import signal
import json
import email.utils
-import socket
from email import message_from_string
-from email.MIMEMultipart import MIMEMultipart
-from email.MIMEText import MIMEText
-from email.Utils import formatdate
-from email.header import decode_header
from twisted.application.service import Service, IService
from twisted.internet import inotify, defer, task, reactor
@@ -53,84 +48,15 @@ from twisted.python import filepath, log
from zope.interface import implements
-from leap.soledad.common.crypto import (
- EncryptionSchemes,
- ENC_JSON_KEY,
- ENC_SCHEME_KEY,
-)
+from leap.soledad.common.crypto import EncryptionSchemes
+from leap.soledad.common.crypto import ENC_JSON_KEY
+from leap.soledad.common.crypto import ENC_SCHEME_KEY
from leap.soledad.common.couch import CouchDatabase, CouchDocument
-from leap.keymanager import openpgp
-
-BOUNCE_TEMPLATE = """
-Delivery to the following recipient failed:
- {0}
-
-Reasons:
- {1}
-
-Original message:
- {2}
-""".strip()
-
-
-from twisted.internet import protocol
-from twisted.internet.error import ProcessDone
-
-
-class BouncerSubprocessProtocol(protocol.ProcessProtocol):
- """
- Bouncer subprocess protocol that will feed the msg contents to be
- bounced through stdin
- """
-
- def __init__(self, msg):
- """
- Constructor for the BouncerSubprocessProtocol
-
- :param msg: Message to send to stdin when the process has
- launched
- :type msg: str
- """
- self._msg = msg
- self._outBuffer = ""
- self._errBuffer = ""
- self._d = None
-
- def connectionMade(self):
- self._d = defer.Deferred()
-
- self.transport.write(self._msg)
- self.transport.closeStdin()
-
- def outReceived(self, data):
- self._outBuffer += data
- def errReceived(self, data):
- self._errBuffer += data
-
- def processEnded(self, reason):
- if reason.check(ProcessDone):
- self._d.callback(self._outBuffer)
- else:
- self._d.errback(reason)
-
-
-def async_check_output(args, msg):
- """
- Async spawn a process and return a defer to be able to check the
- output with a callback/errback
-
- :param args: the command to execute along with the params for it
- :type args: list of str
- :param msg: string that will be send to stdin of the process once
- it's spawned
- :type msg: str
+from leap.keymanager import openpgp
- :rtype: defer.Deferred
- """
- pprotocol = BouncerSubprocessProtocol(msg)
- reactor.spawnProcess(pprotocol, args[0], args)
- return pprotocol.d
+from leap.mx.bounce import bounce_message
+from leap.mx.bounce import InvalidReturnPathError
class MailReceiver(Service):
@@ -177,8 +103,6 @@ class MailReceiver(Service):
self._directories = directories
self._bounce_from = bounce_from
self._bounce_subject = bounce_subject
-
- self._domain = socket.gethostbyaddr(socket.gethostname())[0]
self._processing_skipped = False
def startService(self):
@@ -353,7 +277,7 @@ class MailReceiver(Service):
def _get_owner(self, mail):
"""
- Given an email, returns the uuid of the owner.
+ Given an email, return the uuid of the owner.
:param mail: mail to analyze
:type mail: email.message.Message
@@ -361,25 +285,24 @@ class MailReceiver(Service):
:returns: uuid
:rtype: str or None
"""
- uuid = None
-
+ # we expect the topmost "Delivered-To" header to indicate the correct
+ # final delivery address. It should consist of <uuid>@<domain>, as the
+ # earlier alias resolver query should have translated the username to
+ # the user id. See https://leap.se/code/issues/6858 for more info.
delivereds = mail.get_all("Delivered-To")
if delivereds is None:
+ # XXX this should not happen! see the comment above
return None
- for to in delivereds:
- name, addr = email.utils.parseaddr(to)
- parts = addr.split("@")
- if len(parts) > 1 and parts[1] == self._domain:
- uuid = parts[0]
- break
-
+ final_address = delivereds.pop(0)
+ _, addr = email.utils.parseaddr(final_address)
+ uuid, _ = addr.split("@")
return uuid
@defer.inlineCallbacks
- def _bounce_mail(self, orig_msg, filepath, reason):
+ def _bounce_message(self, orig_msg, filepath, reason):
"""
- Bounces the email contained in orig_msg to it's sender and
- removes it from the queue.
+ Bounce the message contained in orig_msg to it's sender and
+ remove it from the queue.
:param orig_msg: Message that is going to be bounced
:type orig_msg: email.message.Message
@@ -388,22 +311,12 @@ class MailReceiver(Service):
:param reason: Brief explanation about why it's being bounced
:type reason: str
"""
- to = orig_msg.get("From")
-
- msg = MIMEMultipart()
- msg['From'] = self._bounce_from
- msg['To'] = to
- msg['Date'] = formatdate(localtime=True)
- msg['Subject'] = self._bounce_subject
-
- decoded_to = " ".join([x[0] for x in decode_header(to)])
- text = BOUNCE_TEMPLATE.format(decoded_to,
- reason,
- orig_msg.as_string())
-
- msg.attach(MIMEText(text))
-
- yield async_check_output(["/usr/sbin/sendmail", "-t"], msg.as_string())
+ try:
+ yield bounce_message(
+ self._bounce_from, self._bounce_subject, orig_msg, reason)
+ except InvalidReturnPathError:
+ # give up bouncing this message!
+ log.msg("Will not bounce message because of invalid return path.")
yield self._conditional_remove(True, filepath)
def sleep(self, secs):
@@ -478,7 +391,7 @@ class MailReceiver(Service):
(filepath.path,))
bounce_reason = "Missing UUID: There was a problem " \
"locating the user in our database."
- yield self._bounce_mail(msg, filepath, bounce_reason)
+ yield self._bounce_message(msg, filepath, bounce_reason)
defer.returnValue(None)
log.msg("Mail owner: %s" % (uuid,))
@@ -486,13 +399,15 @@ class MailReceiver(Service):
log.msg("BUG: There was no uuid!")
defer.returnValue(None)
- pubkey = yield self._users_cdb.getPubKey(uuid)
+ pubkey = yield self._users_cdb.getPubkey(uuid)
if pubkey is None or len(pubkey) == 0:
- log.msg("No public key, stopping the processing chain")
- bounce_reason = "Missing PubKey: There was a problem " \
- "locating the user's public key in our " \
- "database."
- yield self._bounce_mail(msg, filepath, bounce_reason)
+ log.msg(
+ "No public key for %s, stopping the processing chain."
+ % uuid)
+ bounce_reason = "Missing PGP public key: There was a " \
+ "problem locating the user's public key in " \
+ "our database."
+ yield self._bounce_message(msg, filepath, bounce_reason)
defer.returnValue(None)
log.msg("Encrypting message to %s's pubkey" % (uuid,))
diff --git a/src/leap/mx/tcp_map.py b/src/leap/mx/tcp_map.py
new file mode 100644
index 0000000..597c830
--- /dev/null
+++ b/src/leap/mx/tcp_map.py
@@ -0,0 +1,72 @@
+#!/usr/bin/env python
+# -*- encoding: utf-8 -*-
+# tcpmap.py
+# Copyright (C) 2015 LEAP
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+
+from abc import ABCMeta
+from abc import abstractproperty
+
+from twisted.internet.protocol import ServerFactory
+from twisted.python import log
+
+
+# For info on codes, see: http://www.postfix.org/tcp_table.5.html
+TCP_MAP_CODE_SUCCESS = 200
+TCP_MAP_CODE_TEMPORARY_FAILURE = 400
+TCP_MAP_CODE_PERMANENT_FAILURE = 500
+
+
+# we have to also extend from object here to make the class a new-style class.
+# If we don't, we get a TypeError because "new-style classes can't have only
+# classic bases". This has to do with the way abc.ABCMeta works and the old
+# and new style of python classes.
+class LEAPPostfixTCPMapServerFactory(ServerFactory, object):
+ """
+ A factory for postfix tcp map servers.
+ """
+
+ __metaclass__ = ABCMeta
+
+
+ def __init__(self, couchdb):
+ """
+ Initialize the factory.
+
+ :param couchdb: A CouchDB client.
+ :type couchdb: leap.mx.couchdbhelper.ConnectedCouchDB
+ """
+ self._cdb = couchdb
+
+ @abstractproperty
+ def _query_message(self):
+ pass
+
+ def get(self, lookup_key):
+ """
+ Look up user based on lookup_key.
+
+ :param lookup_key: The lookup key.
+ :type lookup_key: str
+
+ :return: A deferred that will be fired with the user's address, uuid
+ and pgp key.
+ :rtype: Deferred
+ """
+ log.msg("%s %s" % (self._query_message, lookup_key,))
+ d = self._cdb.getUuidAndPubkey(lookup_key)
+ d.addErrback(log.err)
+ return d