Hi HN,<p>I've been working on a project where I need to programmatically create and manage email addresses under my own domain for each user onboarded. I currently use Resend for sending emails (e.g., welcome emails), and my domain's DNS is managed on Cloudflare. However, Resend doesn't seem to support programmatically creating email addresses.<p>My goal is to programatically create emails, monitor email and parse them to LLM etc<p>I've done some research and considered setting up a custom SMTP server to handle this. Has anyone here implemented a similar system? What are the best practices for setting up a custom SMTP server for this purpose? Any recommendations on tools or libraries to make this easier?<p>I'm looking for advice on the best way forward. Appreciate any insights!
You don't need your own SMTP server for this. I'm working on a project with a similar need at the moment and Mailgun's Email Routing feature is exactly what you need.<p><a href="https://www.mailgun.com/products/send/inbound-routing/" rel="nofollow">https://www.mailgun.com/products/send/inbound-routing/</a>
If you want to implement this yourself just realize that email is a text format. HTTP (2616) is actually based on RFC822 for format. That is important because receiving email requires a full understanding of the routing mechanisms defined in 821 and successive documents. Sending, though, just requires that you get 822 right because sending an email message only requires writing the contents correctly over a socket and DNS. Knowing that is how spammers can blast out millions of emails at almost no cost, because you don’t have to be an email server or client to send emails.
Lots of managed providers allow you to set up catch all inboxes that you can then get webhooks for. SES and Mailgun are the two I've used successfully.
I've implemented an email relay before. We had an email address like relay+<code>@site.com that was delivered to a webhook via Mailgun. The code after the plus gave us all the unique information we needed to handle the email.
You can use something like Mail in a box [0] to setup your own SMTP server.<p>[0] <a href="https://github.com/mail-in-a-box/mailinabox">https://github.com/mail-in-a-box/mailinabox</a>
First, let me start with saying what you are wanting is a bad idea. Parts can be done by using dovecot as the MDA with a backed mysql/mariadb database (where you host your own mailboxes).<p>You will have a number of problems that may or may not be recoverable. This is the front lines.<p>I've worked with email (and related components) for years, and it is easily one of the most automated and targeted set of protocols for bad actors.<p>Any server that contains an MX record is going to be flooded with bots testing your defenses, and they will often be from large botnets; we easily reached 1 million unique IPv4 addresses within a month with fail2ban and ipset doing quite a lot of the work for our public facing server (these were just the repeat offenders that failed graduated response). Fail2ban has problems as well (where it may not work properly if you didn't verify).<p>Parsing to LLM is one of the worst ideas you mention because you are effectively deserializing user input into a trusted zone. Please review OWASP. Any scriptkiddie is going to have a field day with you.<p><a href="https://cheatsheetseries.owasp.org/cheatsheets/Deserialization_Cheat_Sheet.html" rel="nofollow">https://cheatsheetseries.owasp.org/cheatsheets/Deserializati...</a><p>Additionally, as a result of spam, mail delivery is almost more important than setting up a barebones server. You can set up a server but not have the mail received by your intended recipients.<p>This is because mail delivery includes both the sending, and the receiving portions. Email service providers (ESP) will reject mail by default in many cases depending on various reputational factors they do not publicly disclose.<p>There are processes you must follow to build up a reputation which is tied to both your domain records, and your IP history. These requirements change arbitrarily depending on the ESP recipient. For example, google at one point required that any email sent include duplicates of the message as both a plain, and a html mimetype. This may no longer be the case, but its an example of arbitrary requirements.<p>The general business processes required to get a good reputation score involves parsing the many whitepapers located at M3AAWG.org. There are a lot of them, but these are where the general consensus requirements for email to be delivered are being posted.<p>If you get on a block list because you didn't properly secure your system, or failed to follow the posted whitepaper guidelines, you may not be removed from those private lists for up to a year or more, and there is no technical contact to reach out for re-evaluation even then even when it is received, it may go straight to the SPAM folder (each one lowering your reputation score).<p>Most ESPs have postmaster services available (if you sign up) which are crucial in discovering and heading these potential problems off.<p>There are a number of metrics beyond the sending process that also go into reputation. For example, Google lowers reputation if the emails being received in a mailbox are never opened, read by the recipient, or interacted with through various widgets in their web GUI. Microsoft and other companies do the same.<p>Needless to say, this is a very nuanced subject area, with the downsides of doing something wrong, often requiring a start from scratch/back to formula approach.