Transparent PGP

[emersion]

No description provided.

[foxcpp]

I was always curious about the whole PGP+email idea you had when the writing PGP backends for go-imap and go-smtp. What's the point? If a person uses PGP - okay, he cares about security, then I really doubt he will also like the idea of sharing private key with the mail server.

[emersion]

Yeah, I'm not sure whether we'll want to do this or not.

The first idea is to encrypt all e-mails that aren't in the SMTP backend. This makes it so they're stored encrypted on disk. This first part only requires the public key.

The second idea is to decrypt in the IMAP backend. This would make it unnecessary for clients to support OpenPGP (many don't). This requires the private key and only works if:

• The server is trusted (aka. it's my server in my basement I only use for myself)
• Clients use disk encryption

The first idea doesn't bring that much value (since the e-mail comes in unecnrypted), and the second idea is maybe not that useful (especially if you're using a good client). In the end I'm not sure all of this is worth it.

[foxcpp]

Both ideas require trusted mail server and so are questionable. PGP wants to have only one source of trust: your key.

[emersion]

The first idea doesn't require a trusted mail server.

[foxcpp]

The first idea raises the complicated question about which key to use and how to obtain it. This is definitely the question about trust and you have to trust the server to pick the right key.

For non-local recipients, you can't automatically obtain the key at all - you can't trust anyone to do it. For local recipients, you can allow users to link their keys to the email account. But this doesn't protect from malicious servers so you have to trust the server anyway.

[emersion]

Well, the first idea's goal is to make it difficult for an attacker who gains access to a server to read past e-mails. Of course once the server is compromised all future conversations are unsafe.

Yeah, this wouldn't work well with non-local recipients.

[foxcpp]

I guess it might make sense to implement server-side encryption for mail storage to protect past messages in case of database compromise. PGP is a ready-to-use public key[1] infrastructure[2], though we might want to use keys generated by the server for each user to avoid all trust issues.

The symmetric key used to protect private key can be derived from user password using the computation-intensive hash algorithm such as bcrypt.

[1] Public key encryption is necessary to ensure that we can deliver new messages without knowledge of the decryption key.
[2] Tho I think we might want to pick something simpler for this purpose, ideas?

There are also some problems that need to be solved:

• Can it be implemented as a storage-agnostic overlay? We need to design something to store and retrieve generated keys. Users of go-imap-sql may expect everything to be in a single database.
• We should not try to decrypt messages that are already PGP-encrypted by. Underlying storage may provide some indication of whether the message is encrypted so it boils down to the first problem.
• We should try to encrypt the meta-data too, this includes envelope and body structure.

[foxcpp]

Throwing ideas into air

Module interface:

type Keyring interface {
  UnlockPrivateKey(username string, secretKey []byte) PrivateKey
  GetPublicKey(username string) PublicKey
}

May be implemented by the underlying mail storage or external storage (simple keyring file?).

We can store the "encrypted" flag using IMAP keywords (custom flags). Like $Maddy_Encrypted (prefixed with maddy to avoid possible clashes with other keywords that may be in use).

Regarding [2] - https://github.com/danielhavir/go-hpke

[emersion]

we might want to use keys generated by the server for each user to avoid all trust issues

Ugh. Then the server can generate keys with weak parameters.

The symmetric key used to protect private key can be derived from user password using the computation-intensive hash algorithm such as bcrypt.

The best option would be to use SRP so that the client doesn't ever send the password to the server. This won't play well with regular IMAP/SMTP authentication.

But why do we need server-side access to the password anyway? We just need to public key.

We should not try to decrypt messages that are already PGP-encrypted by. Underlying storage may provide some indication of whether the message is encrypted so it boils down to the first problem.

Encrypted messages can be detected with the BODYSTRUCTURE. PGP/MIME should be used, because inline PGP is gross.

We should try to encrypt the meta-data too, this includes envelope and body structure.

This is a harder issue, and not possible without a client-side bridge.

[emersion]

Hmm. I think we're not talking about the same feature in the end. I'm only considering the case where the server doesn't know the private key.

[foxcpp]

Hmm. I think we're not talking about the same feature in the end. I'm only considering the case where the server doesn't know the private key.

Well, my whole idea is to make provide server-side encryption for storage without involving the client at all. Should I open a separate issue for it?

[emersion]

Hmm, yeah, I think it would make sense to create a separate issue. Though I doubt there would be a real use-case for the trusted-but-not-trusted server case. It's adding a lot of complexity for little gain.

[Avamander]

Theoretically one thing maddy could do is PGP sign (or encrypt) the e-mails sent by the machine itself (e.g. cron e-mails) to someone. That would protect against otherwise less-than-ideal mail servers on the receiving end.

[alexanderadam]

So this issue mentions different aspects of encryption, but if I understand the current state correctly, this should be about encrypting received emails, right?

In that case you might consider an implementation compatible to pretty Easy privacy/pEp/p≡p.
To simplify this: it's basically GnuPG with some useful tweaks.

One of the most important differences: "traditional" email encryption won't encrypt subjects or metadata. pEp solves this.

You can have a look at this blog post or this implementation if you're looking for an example implementation.

Click to see the Python implementation.

# Copyright 2019 Julian Andres Klode <[email protected]>
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""Encrypt/Decrypt GPG/MIME messages.
This tool can encrypt and decrypt emails using PGP/MIME. Decryption only
works well for emails created with this tool. When encrypting, the tool
preserves all headers in the original email in the encrypted part, and
copies relevant headers to the output. When decrypting, any headers are
ignored, and only the encrypted headers are restored.
"""

import argparse
import sys
import email.encoders
import email.message
import email.mime.application
import email.mime.multipart
import email.mime.message
import typing

import gpg  # type: ignore


def encrypt(message: email.message.Message, recipients: typing.List[str]) -> str:
    """Encrypt given message"""
    with gpg.Context(armor=True) as c:
        keys = []
        for r in recipients:
            keys += list(c.keylist(r))

        encrypted_content, _res, _ = c.encrypt(message.as_bytes(), keys, sign=False)
    if not encrypted_content:
        raise ValueError(encrypted_content.status)

    # Build the parts
    enc = email.mime.application.MIMEApplication(
        _data=encrypted_content,
        _subtype="octet-stream",
        _encoder=email.encoders.encode_7or8bit,
    )

    control = email.mime.application.MIMEApplication(
        _data=b"Version: 1\n",
        _subtype='pgp-encrypted; name="msg.asc"',
        _encoder=email.encoders.encode_7or8bit,
    )
    control["Content-Disposition"] = 'inline; filename="msg.asc"'

    # Put the parts together
    encmsg = email.mime.multipart.MIMEMultipart(
        "encrypted", protocol="application/pgp-encrypted"
    )
    encmsg.attach(control)
    encmsg.attach(enc)

    # Copy headers
    headers_not_to_override = {key.lower() for key in encmsg.keys()}

    for key, value in message.items():
        if key.lower() not in headers_not_to_override:
            encmsg[key] = value

    return encmsg.as_bytes()


def decrypt(message: email.message.Message) -> str:
    """Decrypt the given message"""
    with gpg.Context(armor=True) as c:
        content, _decrypt_res, _verify_res = c.decrypt(message.as_bytes())
        return content


def main() -> None:
    """Program entry"""
    parser = argparse.ArgumentParser(description="Encrypt/Decrypt mail using GPG/MIME")
    parser.add_argument(
        "-d", "--decrypt", action="store_true", help="Decrypt rather than encrypt"
    )
    parser.add_argument(
        "recipient", nargs="*", help="key id or email of keys to encrypt for"
    )
    args = parser.parse_args()
    msg = email.message_from_binary_file(sys.stdin.buffer)

    if args.decrypt:
        sys.stdout.buffer.write(decrypt(msg))
    else:
        sys.stdout.buffer.write(encrypt(msg, args.recipient))


if __name__ == "__main__":
    main()

There are also implementations on Android (F-Droid, k9), iOS, Mozilla Thunderbird since 2018 (they also seem to have a dedicated extension) and commercial(?) implementations for MS Outlook and pEp also had at least one code audit.

PS: Maybe ProtonMail and MailVelope will be compatible as well at one point.