Does anyone know of an open source library for validating email headers a la Posterous? I think their model strikes a great balance between usability and security, and wonder if there's anything out there that would facilitate building a similar feature into a homebrew web app.
pyspf (Google it) will do SPF checking for you. If you'd rather do it yourself, SPF is really, really simple to validate in your language of choice. However, not everybody uses SPF.<p>As for "validating" the rest of the email headers, well... I want to strike a balance between "sure you can do that, good luck!" and "the entire anti-spam community has tried this and it is basically impossible, which is why we rely heavily on IP reputation and Bayes-based approaches which do not treat the contents of the headers as semantically meaningful, since they are in the hands of the enemy".
Here's a sketch for checking the validity of the sender. It does not handle all cases and I am sure it has some holes. I am interested in feedback on this. Are there other things to check? Are these checks "safe" for some definition of safe?<p>Step 1: If DKIM header present, then use result of DKIM validation.<p>Step 2: If sending domain has SPF record, then use result of SPF validation.<p>Step 3: If message passes SPF check using a conservatively guessed SPF record, then treat the message as valid.<p>Step 4: If message came from same IP address as other messages for user and some headers match headers from previous messages (fuzzy match on message id?), then treat the message as valid.<p>Step 5: What next? Messages will make it past the previous steps.
Zed Shaw's Lamson project (<a href="http://lamsonproject.com" rel="nofollow">http://lamsonproject.com</a>) has some solid code for handling most of the messiest parts of dealing with email - bounces, unicode, etc.<p>It's structured in a way that makes it very easy to snip out the parts you want to use without necessarily using all the rest.
Wasn't it shown yesterday that posterous has basically no security at all?<p><a href="http://news.ycombinator.com/item?id=1441997" rel="nofollow">http://news.ycombinator.com/item?id=1441997</a>
I'm still wondering what Posterous plans to do when they reach enough of a critical mass that spammers will actively try to impersonate existing accounts. Generalized, non-sender-server-enforced sender authentication does not exist. That's why SPF and DKIM came along ... I'm sure they've had to pour over this. Anyone have a link on design/discussion?
Validating with headers is like securing a webpage by keeping the URL a secret or browser user agent and ip address. It gives a false sense of security and is very vulnerable to cracks.<p>If you're going to validate with headers then feel free to call it usable but don't call it security.
Is there any degree of "free" validation if you route all the emails through another service that probably does some of this. E.g. a gmail account that forwards all incoming mail onto your servers.
Am I missing something obvious against backtracking the headers to the server immediately before your own? If the IP of that machine differs in future emails, ask for confirmations?