“Should you encrypt or compress first?”

There&#x27;s no compress or encrypt _first_.<p>It&#x27;s just compress or not, before encrypting. If security is important, the answer to that is no, unless you&#x27;re an expert and familiar with CRIME and related attacks.<p>Compression after encryption is useless, as there should be NO recognizable patterns to exploit after the encryption.

A more interesting question is whether to compress or <i>sign</i> first.<p>There&#x27;s an interesting article on that topic by Ted Unangst:<p>&quot;preauthenticated decryption considered harmful&quot;<p><a href="http:&#x2F;&#x2F;www.tedunangst.com&#x2F;flak&#x2F;post&#x2F;preauthenticated-decryption-considered-harmful" rel="nofollow">http:&#x2F;&#x2F;www.tedunangst.com&#x2F;flak&#x2F;post&#x2F;preauthenticated-decrypt...</a><p>EDIT: Although the article talks about encrypt+sign versus sign+encrypt, the same argument goes for compress+sign versus sign+compress. You shouldn&#x27;t do anything with untrusted data before having checked the signature - neither uncompress nor decrypt nor anything else.

Where everyone seems to be getting confused is handling a live flow versus handling a finalized flow (a file).<p>* Always pad to combat plain-text attacks, padding in theory shouldn&#x27;t compress well so there&#x27;s no point making the compression less effective by processing it.<p>* Always compress a &#x27;file&#x27; first to reduce entropy.<p>* Always pad-up a live stream, maybe this data is useful in some other way, but you want interactive messages to be of similar size.<p>* At some place in the above also include a recipient identifier; this should be counted as part of the overhead not part of the padding.<p>* The signature should be on everything above here (recipients, pad, compressed message, extra pad).<p>. It might be useful to include the recipients in the un-encrypted portion of the message, but there are also contexts where someone might choose otherwise; an interactive flow would assume both parties knew a key to communicate with each other on and is one such case.<p>* The pad, message, extra-pad, and signature &#x2F;must&#x2F; be encrypted. The recipients &#x2F;may&#x2F; be encrypted.<p>I did have to look up the sign &#x2F; encrypt first question as I didn&#x27;t have reason to think about it before. In general I&#x27;ve looked to experts in this field for existing solutions, such as OpenPGP (GnuPG being the main implementation). Getting this stuff right is DIFFICULT.

This is why military voice encryption sends at a constant bitrate even when you&#x27;re not talking. For serious security applications where fixed links are used, data is transmitted at a constant rate 24&#x2F;7, even if the link is mostly idle.

Wow, what a trainwreck. So many comments in here talking about whether it would be possible to compress data which looks like uniformly random data, for all the tests you would throw at it. Spoiler alert, you can&#x27;t compress encrypted data. This isn&#x27;t a question of whether we know it&#x27;s possible, rather, it&#x27;s a fact that we know it&#x27;s impossible.<p>In fact, if you successfully compress data after encryption, then the only logical conclusion is that you&#x27;ve found a flaw in the encryption algorithm.

Also interesting is <i>which</i> compression algorithm you&#x27;re using. HPACK Header compression in HTTP 2.0 is an attempt to mitigate this problem:<p><a href="https:&#x2F;&#x2F;http2.github.io&#x2F;http2-spec&#x2F;compression.html#Security" rel="nofollow">https:&#x2F;&#x2F;http2.github.io&#x2F;http2-spec&#x2F;compression.html#Security</a>

The paper cited in this article (<i>Phonotactic Reconstruction of Encrypted VoIP Conversations</i>) really deserves to be highlighted, so I submitted it separately:<p><a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=11995298" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=11995298</a><p><a href="http:&#x2F;&#x2F;www.cs.unc.edu&#x2F;~fabian&#x2F;papers&#x2F;foniks-oak11.pdf" rel="nofollow">http:&#x2F;&#x2F;www.cs.unc.edu&#x2F;~fabian&#x2F;papers&#x2F;foniks-oak11.pdf</a>

I don&#x27;t understand... Why couldn&#x27;t you do CRIME with no compression as well? Assuming you can control (parts of) the plaintext, surely plaintext+encrypt gives you more information than plaintext+compress+encrypt?

I picked up on the reference to Stockfighter, but does anyone know if the walking machine learning game mentioned at the end of the article exists? Sounds like a fun game.

Would adding some tiny random size help? Based on my poorly understanding, if after compress, but before encrypt we add random 0 to 16 bytes or 1% of size that could defeat quite a lot of attacks (like CRIME).

Despite the question being flawed. The correct answer is a series of questions: Who is the attacker? What are you guarding? What assumptions are there about the operating environment? What invariants (regulations, compliance, etc) exist?<p>There may be compensating controls that invalidate the perceived needs for encryption or compression, for example. i.e. don&#x27;t design in the dark.<p>Of course, the interviewer may just want a canned scripted answer - but the interview is your chance to shine, showing how you can discuss all the angles.

That was a fun read. Do I detect a nod to tptacek&#x27;s &quot;If You’re Typing the Letters A-E-S Into Your Code You’re Doing It Wrong&quot;?<p><a href="https:&#x2F;&#x2F;www.nccgroup.trust&#x2F;us&#x2F;about-us&#x2F;newsroom-and-events&#x2F;blog&#x2F;2009&#x2F;july&#x2F;if-youre-typing-the-letters-a-e-s-into-your-code-youre-doing-it-wrong&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.nccgroup.trust&#x2F;us&#x2F;about-us&#x2F;newsroom-and-events&#x2F;b...</a>

If you&#x27;re compressing audio, the simple solution is to compress using constant bitrate.

Would be great if Apple understood this and compressed IPA contents before encrypting.<p>Instead, when you submit something to the AppStore, you end up with a much bigger app than the one you uploaded.<p>To add insult to injury, if you ask Apple about this fuck up you get an esoteric support email about removing &quot;contiguous zeros.&quot; As in, &quot;make your app less compressible so it won&#x27;t be obvious we&#x27;re doing this wrong.&quot;

if your compression can compress your encrypted data, you should change your encryption mechanism to something that actually works...

What if you compress and then only send data at regular periods and regular packet sizes? That way no information can be gleaned. E.g. after compressing you pad the data if it is unusually short, or you include other compressed data too, or you only use constant bit-rate compression algorithm.

That quoted voip paper isn&#x27;t actually as damaging as it sounds. IIRC that 0.6 rating was for less than half of the words so if you&#x27;re trying to listen to a conversation to get something meaningful, it&#x27;s probably not going to happen.

Has there been any research into compression that&#x27;s generally safe to use before encryption? E.g., matching only common substrings longer than the key length would (I think?) defeat CRIME at the cost of compression ratio.

Maybe we need encryption that also plays with the length of the message &#x2F; or randomly pad our date before encryption ? I am however no expert, so I have no clue how feasible, or full of holes this method would be .

I am always thinking, if the compression scheme is known, you would need some good noonce to avoid known plaintext (for example, compression format&#x27;s header is always the same), and also by CRIME, which is to remover the dictionary of the compression.<p>I think it is best to use built-in compression scheme by the compression program to do the encryption first, as those often take these into account (and the header is not leaked, since only the content is encrypted).

Can&#x27;t you just add some random length data at the end. You are defeating compression a little bit, but are also making the length non deterministic. I thought pgp did that.

So what does this mean if I am using an encrypted SSL connection that is correctly configured?<p>Is this kind of problem not already dealt with for me by the secure transport layer? It would be a shame if the abstraction were leaky. My understanding of the contract is that whatever bits I supply will be securely transported within the limits of the configuration I have selected.<p>If I pick a bad configuration then yes shame on me, but a good configuration won&#x27;t care if I compress right?

Logically speaking, an encrypted file should have a high entropy set of bits within it. Compressing it would be low return, but higher security since the input file contained more &quot;random&quot; bits.<p>Compressing the source material will yield smaller results but will be more predictable as the file will always contain ZIP headers and other metadata that would possibly make decryption of your file much easier.

If I compress each component (ie: attacker-influenced vs secret) separately, concatenate the results (with message lengths of course), then encrypt the whole message, is that secure?<p>It seems like it should be, but I&#x27;m not an encryption expert. The compression should be pretty good, though.

&gt; The paper Phonotactic Reconstruction of Encrypted VoIP Conversations gives a technique for reconstructing speach from an encrypted VoIP call.<p>The technique to reconstructing speech clearly had its limitations.

This blog is an interesting way to advertise to their target market: us.

The OP should take <a href="https:&#x2F;&#x2F;www.coursera.org&#x2F;learn&#x2F;crypto" rel="nofollow">https:&#x2F;&#x2F;www.coursera.org&#x2F;learn&#x2F;crypto</a>

So if the length of the resulting message is leaking information, salt it by adding some extra random bits to the end to increase the length by a random amount.

None of this seems to apply to documents you generate to supply to someone else you trust. Compress and encrypt seems perfectly fine.

Given the real risk of CRIME attacks, are there &quot;compression aware&quot; encryption algorithms?

tl;dr

A lot of comments here suggesting that encryption increases entropy. While true, it only adds the key&#x27;s entropy to the plaintext&#x27;s entropy. In most real-world cases, len(m) &gt;&gt; len(k), so this is usually an insignificant increase of entropy. Compression <i>also</i> adds a trivial amount of entropy (specifically, the information encoding the algorithm used to compress, even if that information is out of band).

I thought this was common sense. Compress then encrypt. Encryption leads to higher entropy, therefore less effective compression.