Criterions for a crypto app

Following the previous article, people have asked me what I would consider as a good secure system, and others asked me to review their app, so I think it will be interesting to expose my process when studying those projects.

Threat modeling

The most important point I look for in a project is the threat model. This is the document that will explain for whom the project was created, who are the adversaries, what they are trying to obtain, and which of these threats you are addressing.

Without that document, I cannot know if you considered all the possible actors, and I must infer it from the protocol, which is relatively easy, but my view of the threat model might not correspond to what you expected.

With a good threat model, I can know right away what is your target market (ex: sexting for teens, or secure reporting for journalists in war environments), see if your users will understand the implications, if it will need training, and more importantly, if your system can be safe for that context.

You cannot create a project and say that it will solve all of the privacy problems with some magical crypto algorithm, against all adversaries, even the state actors. I would prefer a useful tool for a niche with real and well defined needs.

Prior art

As you have probably seen, the secure messaging space is already very crowded. If you come up with a new solution to an already solved problem, you need to justify it. Why didn’t you improve an existing project? Couldn’t you adapt someone else’s code, add a better UI?

the NIH syndrome is at the heart of innovation, so I am not against it. But in the case of crypto applications, it might be a good idea to employ already existing (and already audited) code, instead of writing a whole new protocol or algorithm from scratch.

Otherwise, if you are working on an unsolved problem, or improving on current solutions, be prepared to justify it, and a lot, if you employ unusual systems. I am not telling you to avoid funny stuff like Pailler’s cryptosystem, PIR or pairing based cryptography. Just be aware that people will ask you about these.

Publications

That part is fundamental: if you are providing a new protocol or algorithm, you should publish it and ask for review before you start coding and get users. I am not advising you to start up LaTeX and write a paper in ACM format. Just explaining your system on a webpage is fine. The crypto community is full of nice people that will be able to point out if there is any problem (and if you use the academic way of publishing, you might even profit from other people’s funding to get reviews :p).

Some said that the crypto community is full of bitter people eager to hit any new project, following the whole Telegram debacle. That tends to happen when you make a big announcement to get users, telling that it will solve any security problem, and dismiss the opinions of experts, without having asked for review previously.

Note that some of those experts have worked for years on a project before even thinking of communicating about it. As examples, check out Briar, Pond or Cryptosphere: those are quiet but interesting projects. They are not trying to get a lot of users quickly or profit from the post Snowden panic. They have been at it for a long time.

So, publish, ask for review, fix flaws, publish again, fix stuff, and repeat again and again. That is the smartest way to spend your time and money on your project. Once everything is developed and deployed, you will have a hard time trying to plug the holes.

Protocol design

Once we get in the technical stuff, the protocol design is interesting to get a high level view of what you want to achieve. I’ll ask questions like:

  • Is it server centric or P2P? (note: a network of server introduces routing, but is not P2P)
  • Does it include authentication?
  • Is it encrypted end to end?
  • How do you protect against DoS?
  • Is it versioned? Do you allow for protocol version negotiation? Are the algorithms negotiated?
  • Can you revoke keys or identities?

Often, the protocol show what you want to achieve with your system, and it is often answering more threats than the crypto algorithms themselves. A good way to present your protocols is to use diagrams and present the message contents.

Do not insist on algorithms at this point: use general words to describe the primitive you need, like authenticated cipher, public key, key derivation function, MAC. You might change the algorithms later, so stating the properties you need will help reviewers understand what you want to achieve.

A specific note on server VS peer to peer: it is a very understandable feeling for geeks that P2P architectures look better, because they’re decentralizing everything, etc. But they can introduce other problems (like hole punching or sybil attacks), and in some case, you will not be able to avoid servers (for message routing and retries, for mobile systems, etc). Both types of systems are fine, just be aware of their shortcomings.

Cryptographic constructs

Cryptographic algorithms are not enough, you need to apply them correctly. I will have no pity if you say you use “military grade AES 256 encryption” but do not know what is a block cipher mode or Encrypt-Then-MAC. A lot of ugly details can hide here, so do not try to be clever, use battle tested systems:

  • add a separate authentication layer to Diffie-Hellman key exchanges
  • use an authenticated encryption mode
  • use RSA-OAEP instead of PKCS1 padding
  • know well if you need a nonce, an unpredictable number or a time based ID
  • etc.

This is one of the parts where crypto experts will ask annoying questions, because a lot of bugs come from there. They can also propose better solutions (safer, more performant, etc), so listen to them.

If you are employing an unusual scheme here, be prepared to justify it. It might be ok for you, but if the design looks weird to cryptographers, that will raise alarms. Your scheme could be safe, but if it has never been proven right, you are taking a risk, and your users will take that risk too. Is it worth it? Hint: your weird design should provide a unique property that no other algorithm has.

Choice of algorithms

Yes, I do not worry about algorithms until I am already deep in the system. It is not that hard to make correct choices there. Just listen to the recent attacks (ie, avoid RC4) choose large enough keys, choose correct elliptic curves.

Every algorithm has parameters that you need to get right, so be sure to document yourself on your algorithm choices:

  • AES-CBC needs an initialization vector, but AES-CTR uses an incremented nonce
  • RSA needs a good exponent
  • Some elliptic curves work better for some operations

Even if you choose dubious algorithms, if your protocol was well designed, you will be able to move to better algorithm. Be careful with algorithm negotiation, though, a lot of smart people were bitten before.

The implementation

This is probably the part that I will skip, because I do not have the time nor the funding to audit thoroughly the code of every new projects. I will often grep a bit through the code, look for some important points, but this is not something that should be done quickly. This is where the protocol review shows its limits.

Even with a good design, a lot of vulnerabilities can be present in a flawed implementation. Crypto projects should undergo a careful audit like the one Least Authority performed recently on Cryptocat. And that is why you should not communicate about your project before it has been reviewed.

There are things you should always look for in your software projects:

  • encrypting data at rest: if you worry about stolen data, know that a mobile phone or laptop can be stolen
  • random number generation: you should use a CSPRNG, with a good source, and probably some user or device specific data
  • data backup: is it possible? is it safe?
  • software updates: are they downloaded from a secure source? Are the updates verified?
  • Do you use public key pinning?
  • How long are they private keys stored as plaintext in memory?

The implementation details are as important as the whole protocol. You can have a good protocol, but a small error in the code could greatly affect your users. Nevertheless, specifying your protocol is useful, because people can provide better implementations, or make it interoperate with other software. Having other implementations is a good thing: you will not control those versions, but they will be able to construct cool stuff around your system, and make a part of your PR.

User interface

this part is more and more important, because we have been able to create safe systems for years, but often at the price of usability. The user experience of crypto apps needs a lot of innovation, and I’ll follow closely any interesting idea in that space: onboarding experience, useful alerts, user decision making, etc. People should be able to understand when there is a security problem.

I’ll state it once more: if you create a new crypto software, you HAVE to make it easy to use and understand. Some complexity is acceptable, but it must be compensated by documentation (with screenshots, etc) or training.

Other criterions

There are two others that I could think of, but they do not matter that much.

The first is the team. I have been accused of making fun of Telegram for waving around their team of PhDs, but the truth is that I was hopeful: a team full of smart people can come up with interesting design and solve complex problems. If they do not deliver on that, I could be less indulgent. That does not mean I will think less of people without big diplomas. I know too many smart people that dropped out of school to make that mistake. Ultimately, the important thing to judge is the design.

The last parameter is attitude. It is normal to be defensive when someone else reviews your work, but that does not justify denial and dishonesty. People are often taking time off of their job to study your system, so they will be quick and get to the point. If you do not answer or refuse to explain your decisions, it will smell fishy. Even more if you did not ask for a review before communicating about your project. But it does not matter that much. If you are humble and quick to answer, people may help you out of good will, but if you anger cryptographers, you may just have won a free thorough audit 😀

 

Advertisements

Telegram, AKA “Stand back, we have Math PhDs!”

Disclaimer: this post is now very old and may not reflect the current state of Telegram’s protocol. There has been other research in the meantime, and this post should not be used for your choice of secure messaging app. That said, on a personal note, I still think Telegram’s cryptosystem is weird, and its justifications are fallacious. If you want a recommendation on secure messaging apps: use a system based on the Axolotl/Signal protocol. It is well designed and has been audited. Signal and WhatsApp are both using that protocol, and there are others.

Here is the second entry in our serie about weird encryption apps, about Telegram, which got some press recently.

According to their website, Telegram is “cloud based and heavily encrypted”. How secure is it?

Very secure. We are based on a new protocol, MTProto, built by our own specialists, employing time-tested security algorithms. At this moment, the biggest security threat to your Telegram messages is your mother reading over your shoulder. We took care of the rest.

(from their FAQ)

Yup. Very secure, they said it.

So, let’s take a look around.

Available technical information

Their website details the protocol. They could have added some diagrams, instead of text-only, but that’s still readable. There is also an open source Java implementation of their protocol. That’s a good point.

About the team (yes, I know, I said I would not do ad hominem attacks, but they insist on that point):

The team behind Telegram, led by Nikolai Durov, consists of six ACM champions, half of them Ph.Ds in math. It took them about two years to roll out the current version of MTProto. Names and degrees may indeed not mean as much in some fields as they do in others, but this protocol is the result of thougtful and prolonged work of professionals

(Seen on Hacker News)

They are not cryptographers, but they have some background in maths. Great!

So, what is the system’s architecture? Basically, a few servers everywhere in the world, routing messages between clients. Authentication is only done between the client and the server, not between clients communicating with each other. Encryption happens between the client and the server, but not using TLS (some home made protocol instead). Encryption can happen end to end between clients, but there is no authentication, so the server can perform a MITM attack.

Basically, their threat model is a simple “trust the server”. What goes around the network may be safely encrypted, although we don’t know anything about their server to server communication, nor about their data storage system. But whatever goes through the server is available in clear. By today’s standards, that’s boring, unsafe and careless. For equivalent systems, see Lavabit or iMessage. They will not protect your messages against law enforcement eavesdropping or server compromise. Worse: you cannot detect MITM between you and your peers.

I could stop there, but that would not be fun. The juicy bits are in the crypto design. The ideas are not wrong per se, but the algorithm choices are weird and unsafe, and they take the most complicated route for everything.

Network protocol

The protocol has two phases: the key exchange and the communication.

The key exchange registers a device to the server. They wrote a custom protocol for that, because TLS was too slow and complicated. That’s true, TLS needs two roundtrips between the client and the server to exchange a key. It also needs x509 certificates, and a combination of a public key algorithm like RSA or DSA, and eventually a key exchange algorithm like Diffie-Hellman.

Telegram greatly simplified the exchange by requiring three roundtrips, using RSA, AES-IGE (some weird mode that nobody uses), and Diffie-Hellman, along with a proof of work (the client has to factor a number, probably a DoS protection). Also, they employ some home made function to generate the AES key and IV from nonces generated by the server and the client (server_nonce appears in plaintext during the communication):

  • key = SHA1(new_nonce + server_nonce) + substr (SHA1(server_nonce + new_nonce), 0, 12);
  • IV = substr (SHA1(server_nonce + new_nonce), 12, 8) + SHA1(new_nonce + new_nonce) + substr (new_nonce, 0, 4);

Note that AES-IGE is not an authenticated encryption mode. So they verify the integrity. By using plain SHA1 (nope, not a real MAC) on the plaintext. And encrypting the hash along with the plaintext (yup, pseudoMAC-Then-Encrypt).

The final DH exchange creates the authorization key that will be stored (probably in plaintext) on the client and the server.

I really don’t understand why they needed such a complicated protocol. They could have made something like: the client generates a key pair, encrypts the public key with the server’s public key, sends it to the server with a nonce, and the server sends back the nonce encrypted with the client’s public key. Simple and easy. And this would have provided public keys for the clients, for end-to-end authentication.

About the communication phase: they use some combination of server salt, message id and message sequence number to prevent replay attacks. Interestingly, they have a message key, made of the 128 lower order bits of the SHA1 of the message. That message key transits in plaintext, so if you know the message headers, there is probably some nice info leak there.

The AES key (still in IGE mode) used for message encryption is generated like this:

The algorithm for computing aes_key and aes_iv from auth_key and msg_key is as follows:

  • sha1_a = SHA1 (msg_key + substr (auth_key, x, 32));
  • sha1_b = SHA1 (substr (auth_key, 32+x, 16) + msg_key + substr (auth_key, 48+x, 16));
  • sha1_с = SHA1 (substr (auth_key, 64+x, 32) + msg_key);
  • sha1_d = SHA1 (msg_key + substr (auth_key, 96+x, 32));
  • aes_key = substr (sha1_a, 0, 8) + substr (sha1_b, 8, 12) + substr (sha1_c, 4, 12);
  • aes_iv = substr (sha1_a, 8, 12) + substr (sha1_b, 0, 8) + substr (sha1_c, 16, 4) + substr (sha1_d, 0, 8);

where x = 0 for messages from client to server and x = 8 for those from server to client.

Since the auth_key is permanent, and the message key only depends on the server salt (living 24h), the session (probably permanent, can be forgotten by the server) and the beginning of the message, the message key may be the same for a potentially large number of messages. Yes, a lot of messages will probably share the same AES key and IV.

Edit: Following Telegram’s comment, the AES key and IV will be different for every message. Still, they depend on the content of the message, and that is a very bad design. Keys and initialization vectors should always be generated from a CSPRNG, independent from the encrypted content.

Edit 2: the new protocol diagram makes it clear that the key is generated by a weak KDF from the auth key and some data transmitted as plaintext. There should be some nice statistical analysis to do there.

Edit 3: Well, if you send the same message twice (in a day, since the server salt lives 24h), the key and IV will be the same, and the ciphertext will be the same too. This is a real flaw, that is usually fixed by changing IVs regularly (even broken protocols like WEP do it) and changing keys regularly (cf Forward Secrecy in TLS or OTR). The unencrypted message contains a (time-dependent) message ID and sequence number that are incremented, and the client won’t accept replayed messages, or too old message IDs.

Edit 4: Someone found a flaw in the end to end secret chat. The key generated from the Diffie-Hellman exchange was combined with a server-provided nonce: key = (pow(g_a, b) mod dh_prime) xor nonce. With that, the server can perform a MITM on the connection and generate the same key for both peers by manipulating the nonce, thus defeating the key verification. Telegram has updated their protocol description and will fix the flaw. (That nonce was introduced to fix RNG issues on mobile devices).

Seriously, I have never seen anyone use the MAC to generate the encryption key. Even if I wanted to put a backdoor in a protocol, I would not make it so evident…

To sum it up: avoid at all costs. There are no new ideas, and they add their flawed homegrown mix of RSA, AES-IGE, plain SHA1 integrity verification, MAC-Then-Encrypt, and a custom KDF. Instead of Telegram, you should use well known and audited protocols, like OTR (usable in IRC, Jabber) or the Axolotl key ratcheting of TextSecure.

Theoretical definitions for crypto wannabes

Every week, I hear about a new secure software designed to protect your privacy, thwart the NSA/GCHQ and save kittens. Most of the time, though, they’re started by people that are very enthusiastic yet unskilled.

They tend to concentrate directly on choosing algorithms and writing code, instead of stepping back and thinking a bit about what they want to develop.

Sure, they probably spent some time saying things like:

  • that piece of data should absolutely be encrypted
  • users will all have key pairs to authenticate themselves
  • we should use AES, that’s the safest choice (what are theses “modes” you’re talking about?)

That is not how you design a protocol. That is not how you design a software using encryption. And that is not how you will design the next secure distributed social network.

To design your system, you need three things:

  • a good threat model
  • theoretical tools addressing the threats
  • algorithms implementing these theoretical tools

As you see, most of the projects only have the third item, and that’s insufficient to design a correct system. If you don’t have a good threat model, you don’t have a good mental model of your users and attackers, their means and their objectives. If you don’t have the theoretical tools, you will try to shoehorn your favorite algorithm on the problem without knowing if it really fits (example: using hash algorithms to store passwords :p).

So, in this post, I’ll provide those (simplified) theoretical definitions. You will probably recognize some of them.

High level view

First, you need to forget notions like “privacy”, use any of these terms to describe the properties you want to achieve:

  • authentication: you can recognize which entity you are communicating with
  • authorization: the entity cannot get access to data it has no permission on (note that it is different from authentication)
  • confidentiality: the data should not be readable by an entity that has no permission on it (it can be protected by crypto, but also by policies in the code)
  • integrity: unauthorized modification of the data can be detected and marked as invalid
  • non repudiation: an entity cannot deny it has executed an action
  • deniability: an entity _can_ deny it has executed an action

Ok, now that we have some basic properties, let’s apply them: think for a long time about the actors of the system (users, malicious users, admins, sysadmins, random attacker on the network, etc), what authorizations they have, what they should not get access to, what data moves on the network and between whom.

You should now have a very basic threat model and a rough overview of your system or protocol: you know what part of the network communications should be confidential, you know where you would need to authenticate.

You will now need some ideas about the type of attacks that could happen to your system, because you probably did not think of everything. Separate your systems in logical parts (like “client”, and “server”, etc), observe them, and observe how they communicate.

Common attacks

Security properties

Here are some security properties that will be useful when you will try to choose algorithms later:

  • Random oracle: a system that answers deterministically every question with a random answer from its answer space. You cannot predict what it will answer, but if you send the same question twice, it will answer the same both times.
  • Perfect secrecy: for any two plaintext messages of same size, an attacker cannot distinguish which plaintext maps to which ciphertext. Basically, the adversary learns nothing from the ciphertext only.
  • Semantic security: same thing as perfect secrecy, except what the adversary learns on the plaintext is negligible. example: One Time Pad

Security tests

those properties can be tested by creating a “game”, where the attacker tries to guess information on the data:

  • IND-CPA (indistinguishability under the chosen plaintext attack) test: the adversary can generate as many ciphertexts as he wants. Then, he chooses two messages m0 and m1 of same length, those messages are encrypted, and one of the ciphertexts is sent back to him. The adversary should not be able to guess which message was used to generate that ciphertext (note that this is just one way of testing for CPA, there are many other schemes, some with stronger properties)
  • IND-CCA (indistinguishability under the chosen ciphertext attack): the attacker can get the decryption of arbitrary ciphertexts, but should not, from this, be able to decrypt any other ciphertext.

Attack patterns

Here are some common attack types that can be applied to crypto protocols. The list is not exhaustive, and covers only crypto attacks: there are many more ways to attack a system.

  • Replay attacks: the attacker has observed some valid (encrypted or not) data going on the wire, and tries to send it again. Obviously, it should not be accepted
  • MITM (Man In The Middle): the attacker can observe and modify live data running between two actors of the system. In the worst case, the attacker should not be able to forge valid data, decrypt data, or impersonate one of the users. In the best case, it should be detectable.
  • Oracle attack: when an algorithm or protocol has a part that can act as an oracle (can be asked something and give answer, like a server), an attacker could exploit flaws in the algorithm to get useful information on the data (then the oracle is not a random oracle). Timing attacks are part of this type of attack. See also padding oracle attacks, or the recent BREACH attack on TLS.
  • Offline attack: the attacker got access to some encrypted data (on the wire, or by accessing a disk somewhere), stored it, and tries to decrypt it for an amount of time

You should now have a better view of the system: what are the parts of the system that need protection, what attacks they must resist, and what properties they should have.

That means we can go to the next part: choosing the tools to implement the solution.

General cryptographic functions

No, we will not choose algorithms right now. That would be too easy 😀

We will choose from a list of cryptographic constructions that implement some of the security properties of the system, and combine them to meet all the needed properties:

  • Secure Pseudo Random Function: function from spaces K (keys) and X (message) to Y (other message space). The basic definition is that if you choose randomly one of these functions (like choosing randomly a k from K), its output will appear totally random (testable with IND-CPA).
  • Pseudo Random Permutation: this is a PRF where X and Y are the same space. It is bijective (every y from Y maps to exactly one x from X, and every y is an output of the function), and there exists an efficient inversion function for it (from Y to X). Example: AES (in ECB, which is unsafe for common use)
  • Message Authentication Code: defines a pair of algorithms. One of them takes a key k and a message m and outputs a code c. The other algorithm takes k, m, c and outputs True or False. An attacker should not be able to forge a valid c without knowing k. A PRF could be used to construct a MAC system. A hash function too. Example: HMAC
  • Authenticated encryption: an encryption function (with semantic security under the CPA) where the attacker cannot forge new ciphertexts that decrypt correctly. Example: AES-GCM.
  • Hash function: collision resistance (cannot find different messages m1, m2 so that H(m1) == H(m2), with different collision levels). Not easily invertible. Usually, they are fast. Example: SHA2.
  • Trapdoor function: a function that is easy to compute, for which finding its inverse is hard, unless you have specific information. (secure under IND-CCA). examples: RSA, DSA.
  • Zero knowledge proof: a way to prove something to the other party in a communication, without giving her any info, except the proof.

There are a lot of other constructions, depending on your needs, from low level algorithms like the Diffie Hellman key exchange to higher level protocols like OTR. Again, the constructions will depend on the security properties.

Choosing the algorithms

Can we do it now? YES! But there are rules. You must not choose an algorithm because it’s hype or because someone said so in an old book. Basically, you choose algorithms that implement the properties you need (like authenticated encryption), and you choose the parameters of the algorithm (key size, exponent, elliptic curve) depending on the strength you need. Basically, a key size can define how much time encrypted data should remain impossible to decrypt. Those parameters also define the performance of the algorithm. Don’t choose them without consulting experts, or you will face problems similar to those encountered by the projects that used low RSA exponents (it looks good from a performance standpoint, but it introduces very bad security).

Am I done now?

Nope. We have only define some very high level parts. Creating a protocol implies a lot of thoughts on:

  • how you establish the communications
  • protocol negotiation (version, algorithms, etc)
  • key exchange
  • authentication
  • nonce usage
  • storing sessions
  • handling lost connections
  • renegotiation
  • closing the connection
  • etc.

As you can see, designing a protocol involves a lot more than choosing a few algorithms. Note that this was only a very rough overview of what you would need to create a safe system. And we did not even start coding!

So, if you want to build the next privacy protecting system, please talk to experts. They don’t necessarily want to make you feel bad. They just have a lot of formal tools and the experience needed to see what will not work.

Group messaging crypto and the CAP theorem

I often play with group messaging ideas, and recently, an interesting perspective came to me, about the relation between these messaging systems and the constraints of the CAP theorem.

What is the CAP theorem?

Small theoretical background here, feel free to skip if you already know what it is about

Otherwise known as Brewer’s theorem, indicates that three properties are important in distributed systems:

  • Consistency (all nodes see the same data)
  • Availability (every request receives a response)
  • Partition tolerance (the system still works through network splits or node failure)

The CAP theorem tells us that a distributed cannot have the three properties at the same time. It is not really a “two out of three” like most people tend to say, but more of a compromise you have to make. Some examples:

  • in traditional databases systems (with a master-slave model), consistency and availability are high (all nodes can answer with th same data), but partition tolerance is weak because the master is a SPOF.
  • in a fully distributed database (no master model), availability is high (all the nodes can answer), partition tolerance is high (a subset of the cluster could act as the whole database), but consistency is weak (data must be replicated to all the relevant nodes, and that can take time, so the nodes may not see the same data at the same time)
  • in Bitcoin, the consistency is good (all nodes must agree to the same block chain) and partition tolerance is good (for downloading the block chain), but availability is weak for writing, because a new transaction must first propagates to a large enough set of nodes, then a block must be calculated for the transaction to be stored

As you can see, you can shape those properties depending on how you want your system to behave. Maybe you want fast reads, or fast writes, or very strong replication, etc.

Group messaging constraints

This has always been a challenge. Traditionally, chat systems follow a client-server model, where the server redistributes all the messages. As we saw previously, that is bad against partition problems. The usual solution is to have multiple servers talking with each other, as we can see in IRC or XMPP.

For a fully distributed messaging system (if we forget for a moment all the nasty NAT problems),  communication becomes a routing problem: making sure the nodes get all the messages fast enough and in the right order. If you’re building a distributed Twitter, it’s not really a problem, but for an interactive chat system, this becomes really hard. You can try to send your messages directly to all the users in the chat room, but as more people join, sending and receiving all the messages takes more and more time, and so, you sacrifice availability and a bit of consistency (you will not necessarily receive all the messages).

Group messaging crypto

The group messaging problems seemed hairy? Let’s add crypto in the mix, just for fun! What security properties would we want to add to a messaging system?

  • Authentication: every user knows with whom he is talking
  • Confidentiality: an external observer cannot see the content of a message
  • Tampering proof: users can detect when a message has been modified and reject them
  • Perfect forward secrecy: compromising one key does not compromise the whole conversation or past conversations

There are others that we could want, but let’s just concentrate on these ones for now. We can separate the messaging in two phases: the discovery, and the transport. The discovery is when nodes begin talking to each other, authenticating each other, establishing session keys, managing key rotation, etc. Note that discovery can happen at any time, as a new node can appear after a long time. The transport is about sending messages safely and verifying messages.

I think we can assume that, once all the nodes have authenticated themselves and agreed on session keys, the transport is the easy part. Whatever the underlying transport system (broadcast, XMPP, SMTP, etc), once you can send, receive and verify messages, everything is easy.

The interesting part is the discovery. That is where the similarity with a distributed system is evident. You want a lot of nodes to start communicating with each other, you want to propagate information to all the nodes (like session keys), you must handle nodes that go up or down, and clusters splitting.

That’s where the CAP theorem is useful to understand the constraints of the system.

Group authentication is an availability and partition problem: if you cannot start the communication until all the nodes have authenticated each other, you are vulnerable to slow or failing nodes.

Tampering proof is a consistency problem: all the nodes must know the same signature keys to agree on whether a message is correct or not.

Confidentiality is a consistency and partition problem: if you must reach consensus on a group-wide session key, you have to wait for all the nodes to get the new key (consistency). That happens a lot if you want perfect forward secrecy, where you must change keys regularly. Moreover, in case of partition, the different groups will agree to different keys, and a conflict will appear once the split is over.

As you can see, a security model can help you choose which tools (cryptographic or not) you will use to ensure the safety of the system, but with the distributed system theory, you will be able to predict and recognize the behaviour of the system.

Some examples

Let’s see how some more or less known system handle that:

GPG and email

GPG+email is, at its heart, a secure group messaging system. If we analyze its properties, wen can see that it is very good against partition: SMTP handles splits quite well, it can resend messages if they did not pass, or store them for a few days until the next server is up.

For availability, it doesn’t fare very well at high loads if you want to encrypt messages, because you need to encrypt for every host. You cannot take advantage of SMTP’s architecture to reduce the load.

It also has very bad consistency. If you want every node to agree on the keys of each user, you have to do one to one offline meetings between each of the participants. In practice, users make a tradeoff here, so the security assumption is not completely true.

Multi party off the record messaging

There is a paper you can read about MP OTR, which builds on the previous OTR algorithm to provide secure multi party communication. That protocol relies on a heavy setup phase where all the nodes authenticate each other and generate a group encryption key.

This model will fare quite well once consensus has been reached: all the nodes know the ephemeral signature keys and the group encryption key.

Unfortunately, in case of a user joining or leaving the chatroom, the whole shutdown and setup process must happen, making the chatroom unavailable in the meantime.

What about yours?

I see a lot of new projects appearing lately, with the intention to “fix” chat systems or social networks by building a “secure” distributed system. Unfortunately, most of them do not have a serious background in security or distributed systems. The subject of this article, the CAP theorem, is a very small part of the way we think about distributed systems: people have been researching on the subject for years. Similarly, cryptography and protocols have improved a lot lately.

So, if you are building one of these, take your time, forget the hype, read up a bit about the theory, and think about your model before you make your technological choices. Please don’t repeat the worst mistakes.

Your data is precious

Following LinkedIn’s large password leak, I have seen a dangerous thought spread to friends and colleagues:
“so what if my LinkedIn password has leaked? What can they do? Look for a job for me?”

That is based on wrong assumptions about what an attacker wants and can do. And it is mistaking the low value you get from a service with the value of your data. Your data is PRECIOUS. Maybe not to you. But everything can be sold, and you’ll always find someone interested to buy it. Let’s see a few creative uses of your Linkedin account:

Analyze your data

You might think that what you share is of no use to anyone except potential recruiters, but by mixing your resume, shared links, private messages, all the data you put on the website, I could build a nice profile and sell it to advertisers. Did you put your address and phone number somewhere in your profile? Awesome! I have a lot of targeted advertisements for you!

Obtain access to your other accounts(email, Facebook, Twitter, Viadeo…)

With your email address and your password, I could probably guess the password for other services. Almost nobody has strong and different passwords for every service. Would you like to see your Facebook or Twitter account compromised? I don’t think so.

Oh, remember to use a strong password, or even two factor authenticatiob for email. A lot of password recovery systems sues emails, so if your mailbox is compromised, your accounts will be compromised.

Spam/SEO

Nothing ca be done with your account? oh, you have contacts. And maybe, a well referenced profile. I’d be able to send spam links to all your contacts with the user feed, and put them in your profile, to improve the ranking of my websites. Sure, there’s no harm to you, if you don’t care about losing credibility or annoying your contacts.

Using the contact list

Oh, yes, I could sell your contact list, that’s easy money!

While I’m at it, I could have fun with your friends and colleagues:

  • ask them for money, nude pictures, confidential information, etc.
  • tell them that your email account has been compromised, and that they must address their emails to another address controlled by me
  • obtain access to their accounts with social engineering
You may be insignificant, but that’s not necessary true of your contacts. In social networks, your network has a value, and you must protect it. It is your responsibility to make sure your friends and colleagues don’t get compromised through your account.
It reminds me of the 90s, when I often had this dialogue:
Me-You should put an antivirus and firewall on your computer.
You-Why should I? There’s nothing interesting on my computer, why would anyone want to infect it?
Me-I receive from you 10 emails a day, and all of them contain a virus.”

Worst Case Messaging Protocol

We already have lots of tools to communicate safely, and in some cases, anonymously. These tools work well for their purpose, but spying and law enforcement tools have also improved.

People are forced to decrypt their hard drives in airports. Rootkits are developed by police forces. DPI systems are installed on a country level. Using the graph of who communicates with whom, it is possible to map the whole hierarchy of an organisation, even if all communications are encrypted. How can we defend against that?

I believe that the analysis of organisations is one of the biggest threats right now. The ability to recognize an organization is the ability to destroy it by seizing a few members. That may be useful against terrorists, but these tools are too powerful, and they may be used by the wrong hands. Terrorists already know how to defend against such systems, but what of common people? Any company, association or union could be broken anytime.

I think we need tools to mount resilient organizations. A compromised node shouldn’t bring down the whole system. Intercepted messages shouldn’t give too much information on the different parties. People can still find ways to send their messages safely (PGP, dead drops, giving USB keys hand to hand), but we need a framework to build decentralized human systems.

Here is the solution I propose: a messaging protocol designed to protect identities and route around interception. This protocol is designed to work in the worst case (hence its name), when any communication could be intercepted, and even when we can only use offline communication.

Every message is encrypted, signed and nearly anonymized (if you already know the emitter or receiver, you can recognize their presence, but this can be mitigated). To introduce redundant routes in the network, a simple broadcast is used (the protocol is not designed for IP networks, but for human networks, with limited P2P communication). Any node can be used to transmit all the messages. The fundamental idea is that if you’re compromised, the whole organization can still work safely without you.

Prerequisites

Assumptions

  • Every communication can be intercepted
  • if a node is compromised, every message it can read or write is compromised
  • People can still create a P2P (IRL) network to communicate (aka giving an USB key to someone else)

Goals

  • Confidentiality
  • Anonymity
  • Authentication
  • Deniability? (I am just a messenger)
  • Untraceability
  • Can work offline (worst case)

For this protocol, we need a RNG, a HMAC function, a symmetric cipher and an asymmetric cipher. I have not yet decided what algorithms must be used. They could be specified in the message format, to permit future changes in algorithms.

Message format

  • nonce1
  • HMAC( sender’s public key, nonce1 )
  • expiration date
  • symcrypt( symmetric key, message )
  • nonce2
  • For each receiver:
    • asymcrypt( receiver1’s public key, symmetric key )
    • HMAC( receiver1’s public key, nonce2 )
    • asymcrypt( receiver2’s public key, symmetric key )
    • HMAC( receiver2’s public key, nonce2 )
  • asymsign( sender’s private key, whole message )

Here is how it works:
An identity is represented by one or more public keys (you can have a different public key for each of your contacts, it makes messages difficult to link to one person). They are not publicly linked to a name or email address, but the software layer could handle the complexity with a good enough UI. I do not cover key exchange: users must design their own way to meet or establish communications.

Every user is a hub: all messages are broadcasted by all users. To prevent data explosion, expiration dates are used. Users recognize the message they receive and the identity of the emitter, using the HMAC of the public keys. All communication are encrypted, but I use the convergent encryption ideas to have multirecipient messages.

Handling the messages

Parsing

  • verify if one of my public keys is in the receivers by calculating the HMAC with nonce2
  • validate the signature (if I already know the emitter’s public key)
  • decrypt the symmetric key that was encrypted with my public key
  • decrypt the message

Creating messages

  • write the message
  • choose the emitter’s identity and public key
  • choose the receivers and the public keys used
  • choose an expiration date
  • generate two random nonces
  • generate a symmetric key
  • insert nonce 1 and the HMAC of nonce 1 and my key
  • insert the expiration date
  • encrypt the message with the symmetric key and insert it
  • insert nonce 2
  • for each receiver:
    • encrypt the symmetric key with the receiver’s public key and insert the result
    • insert the HMAC of nonce 2 and the receiver’s public key
  • sign the whole message with my private key and append the signature

Sending messages

  • Take all the messages you received
  • filter them according to some rules:
    • remove expired messages
    • remove messages too large?
    • etc
  • add your messages to the list
  • give the message list to the next node in the network

This protocol is voluntarily simple and extensible. New techniques can be devised based on this protocol, like dummy receivers in a message or dummy messages to complicate analysis. It is independent of the type of data transmitted, so any type of client could use this protocol (it is at the same level as IP). Most of the logic will be in the client and the contact manager, to present an interface easy to use.

I truly hope we will never need this to communicate, but recent events have convinced me that I need to release it, just in case.

I am open to any critique or suggestion about this protocol. I will soon open a repository to host some code implementing it, and it will of course be open to contributions 🙂

Yet another authentication scheme

Recently, I was asked to design a new authentication protocol for a web service. I know that I shouldn’t do reinvent the wheel, so I immediatly proposed OAUTH. It turns out that it can’t be used in this situation. Here are the constraints:

-calls to the webservice must be authenticated: I can keep the tokens and signature from OAUTH here. The problem is: how do I get that token?

-calls are made from devices or applications without access to a webbrowser (embedded devices, phones, etc.). The redirection dance of OAUTH is not acceptable here

-communications are done over an untrusted network, without SSL.

-I can’t use application keys and secrets to encrypt and sign the authentication process: clients include open source software and smartphone applications. You can’t hide a secret key in these.

-the protocol has to be simple to implement, on a lot of languages

-the server must not store the password in cleartext (I shouldn’t have to precise this…), the client must not store the password

Summing it up: no preshared keys, no browser, no SSL, untrusted networks, no passwords stored, and an OAUTH-like environment once the client is authenticated (tokens, authorizations, revoking, etc)

Apparently, I should just give up. But I like to play, so I’ll try!

First, I must say that I am not an expert in security nor cryptography. But I’m really enthusiastic about these subjects, and my day job is at a company providing strong authentication solutions (no, this protocol is not related to my day job). So, I know a bit about the subject, and I know that I should ask for reviews, hence this post.

Rough ideas

We need a safe communication over an untrusted network. TLS immediatly comes to mind, but the targeted applications might not have access to a TLS implementation. I’d like to use SRP, but I don’t think I’m able to implement it correctly (and it has to be SIMPLE). Using Diffie-Hellman to establish a shared key is another idea, but it is not safe against MITM.

Here’s my idea: we don’t need to generate a shared secret, we already have it. It’s the password!

But how can I use the password if the server doesn’t store it in cleartext?

The trick: key derivation functions

Decveoplers are finally understanding that they should not use MD5 nor SHA1 to store their passwords, even with a salt, because computing power is so cheap these days that anyone could crack easily a lot of passwords.

It is now recommended to use other functrions tro store passwords. The key derivation functions are a class of functions that create a key from a password. Basically, they do it by interating a lot of times. That makes them very slow, which is an interesting property if you want to stpre passwords: it is too expensive to “crack” the password. PBKDF2, bcrypt and scrypt are well known key derivation functions. They’re simple to use and available in a lot of languages.

With these functions, I can safely store the passwords, and generate a key shared with the client.

In short: if I store kdf(password, N) with N the number of iterations, I can send any M > N to the client and ask him to compute the key, without compromising what I store.

Designing the protocol

Now that we have a way to use a shared key, we can look at what will go over the wire to establish it. If I use directly kdf(pass, M), anybody getting access to the client storage will be able to obtain the key for any L > M. So, the key establishment has to use a nonce. That way, the client will only use the password once and forget it, and store the derivated key.

I would rather use a truly random key that has no relation with the password. It could be given to the client, encrypted with the derivated key. The derivated key could then be thrown away. But I still do not know if it is really necesary.

The server still needs to authenticate the client. The client will make a second call to the web service, signing it with HMAC and the key.

That’s it! It is really simple, so if there are flaws I did not see, you will surely catch them.

TL; DR

The protocol is based on key derivation functions, like PBKDF or bcrypt.

  • The server stores login and H = kdf(pass, N), with N integer
  • The client wants to authenticate and makes a call to the server with the login as argument
  • The server replies with M > N and i nonce
  • The client calculates k1 = kdf(kdf(pass, M)+i, 1)
  • The server calculates k2 = kdf(kdf(H, M-N)+i, 1)
  • The client calls the server with args “user=login&sign=”.HMAC(“user=login”, k2)
  • If k1=k2. The signature matches and the client is authenticated.