[bug] add extra CRLF to avoid bad mime parsing in Thunderbird

Thunderbird (as of 37.0b1) will display a blank body (with no
attachments) if some conditions are met:

* disk synchronization is disabled
* mime_part_on_demand = true
* msg size is bigger than the parts_on_demand threshold (30000 by
  default).

Comparing the logs with a well behaved imap server (dovecot, on this
case), it's easy to see that twisted implementation is lacking an extra
line separator at the end of each group of headers that is rendered in
response to each of the `BODY.PEEK[X.MIME]` command that the
mime_parts_on_demand will issue after getting the BODYSTRUCTURE.

This change patches the spew_body command on the body server.
We still would have to see if this is a bad behaviour in the thunderbird
side. The most similar bug I've found is:

https://bugzilla.mozilla.org/show_bug.cgi?id=149771

Which apparently was happening with exchange server.

We should send the patch to upstream twisted as well.

Note that this fix is not enough: the following commit, about fixing the
case of the boundary passed in the BODYSTRUCTURE response is also needed
to fix the bug (since a bad parsing happens all the same).

Resolves: #6773, #5010
Documentation: #6773
Releases: 0.4.0

1 files changed, 61 insertions, 0 deletions

diff --git a/mail/src/leap/mail/imap/server.py b/mail/src/leap/mail/imap/server.py
index 3e10171..26d2001 100644
--- a/mail/src/leap/mail/imap/server.py
+++ b/mail/src/leap/mail/imap/server.py
@@ -60,6 +60,67 @@ class LEAPIMAPServer(imap4.IMAP4Server):
         # populate the test account properly (and only once
         # per session)
 
+    #############################################################
+    #
+    # Twisted imap4 patch to workaround bad mime rendering  in TB.
+    # See https://leap.se/code/issues/6773
+    # and https://bugzilla.mozilla.org/show_bug.cgi?id=149771
+    # Still unclear if this is a thunderbird bug.
+    # TODO send this patch upstream
+    #
+    #############################################################
+
+    def spew_body(self, part, id, msg, _w=None, _f=None):
+        if _w is None:
+            _w = self.transport.write
+        for p in part.part:
+            if msg.isMultipart():
+                msg = msg.getSubPart(p)
+            elif p > 0:
+                # Non-multipart messages have an implicit first part but no
+                # other parts - reject any request for any other part.
+                raise TypeError("Requested subpart of non-multipart message")
+
+        if part.header:
+            hdrs = msg.getHeaders(part.header.negate, *part.header.fields)
+            hdrs = imap4._formatHeaders(hdrs)
+            _w(str(part) + ' ' + imap4._literal(hdrs))
+        elif part.text:
+            _w(str(part) + ' ')
+            _f()
+            return imap4.FileProducer(msg.getBodyFile()
+                ).beginProducing(self.transport
+                )
+        elif part.mime:
+            hdrs = imap4._formatHeaders(msg.getHeaders(True))
+
+            ###### PATCHED #####################################
+            _w(str(part) + ' ' + imap4._literal(hdrs + "\r\n"))
+            ###### END PATCHED #################################
+
+        elif part.empty:
+            _w(str(part) + ' ')
+            _f()
+            if part.part:
+                return imap4.FileProducer(msg.getBodyFile()
+                    ).beginProducing(self.transport
+                    )
+            else:
+                mf = imap4.IMessageFile(msg, None)
+                if mf is not None:
+                    return imap4.FileProducer(mf.open()).beginProducing(self.transport)
+                return imap4.MessageProducer(msg, None, self._scheduler).beginProducing(self.transport)
+
+        else:
+            _w('BODY ' + imap4.collapseNestedLists([imap4.getBodyStructure(msg)]))
+
+    ##################################################################
+    #
+    # END Twisted imap4 patch to workaround bad mime rendering  in TB.
+    # #6773
+    #
+    ##################################################################
+
     def lineReceived(self, line):
         """
         Attempt to parse a single line from the server.