Today I realized something. Humans compress their communication by knowing things mutually. For example, if you know a joke and if your friend knows the same joke too, you can remind him/her that joke any second by just telling a word of it.<p>Today our compression algorithms compress stuff by only thinking that nobody knows the joke we are gonna tell and tries to choose shorter words instead of just outputting a word of the joke.<p>What if, we would analyze many compressed files and find the most common binary patterns and create a new compression algorithm which uses these patterns to tell the joke with only a word? And the person who wants to decompress the file would need to have the database as well.
Great in concept until you run into the pigeonhole principle. Such compression schemes are fundamentally flawed and you can spend years wasting time on them until some corner edge case will throw you. Nonetheless, I'd encourage you to satisfy yourself by trying to devise such an elaborate scheme... Pied Piper may have a job opening for you.