Introduction to HTTP Multipart

The article talks about multipart&#x2F;form-data in particular.<p>Another thing one might run across is multipart&#x2F;x-mixed-replace. I wrote a crate for that. [1] I didn&#x27;t see a spec for it, but someone since pointed out to me that it&#x27;s probably identical to multipart&#x2F;mixed, and now seeing an example in the multer README it clicks that I should have looked at RFC 2046 section 5.1.1, [2] which says this:<p>&gt; This section defines a common syntax for subtypes of &quot;multipart&quot;. All subtypes of &quot;multipart&quot; must use this syntax.<p>...and written a crate general enough for all of them. Maybe I&#x27;ll update my crate for that sometime. My crate currently assumes there&#x27;s a Content-Length: for each part, which isn&#x27;t specified there but makes sense in the context I use it. It wouldn&#x27;t be hard to also support just the boundary delimiters. And then maybe add a form-data parser on top of that.<p>btw, the article also talks specifically about proxying the body. I don&#x27;t get why they&#x27;re parsing the multipart data at all. I presume they have a reason, but I don&#x27;t see it explained. I&#x27;d expect that a body is a body is a body. You can stream it along, and perhaps also buffer it in case you want to support retrying the backhaul request, probably stopping the buffering at some byte limit at which you give up on the possibility of retries, because keeping arbitrarily large bodies around (in RAM or even spilling to SSD&#x2F;disk) doesn&#x27;t sound fun.<p>[1] <a href="https:&#x2F;&#x2F;crates.io&#x2F;crates&#x2F;multipart-stream" rel="nofollow">https:&#x2F;&#x2F;crates.io&#x2F;crates&#x2F;multipart-stream</a><p>[2] <a href="https:&#x2F;&#x2F;datatracker.ietf.org&#x2F;doc&#x2F;html&#x2F;rfc2046#section-5.1.1" rel="nofollow">https:&#x2F;&#x2F;datatracker.ietf.org&#x2F;doc&#x2F;html&#x2F;rfc2046#section-5.1.1</a>

&gt; Another advantage of multipart is that the server can stream each part separately. For example, say you&#x27;re uploading 5 files by encoding them into a JSON object. Your server will have to buffer the entire JSON object into memory, decode it, and examine each file.<p>That&#x27;s not true. You can stream JSON, too. You just have to do something fancier than JSON.stringify().

I wonder why we didn&#x27;t use a framed representation rather than delimiters that have to be searched for (which isn&#x27;t so simple). It makes writing a streaming MIME parser much harder. With a compulsory Content-Length things would be much easier.<p>At least with multipart&#x2F;form-data we get to avoid transfer encodings, which are also quite annoying to handle (especially as they can be nested, which is probably the worst aspect of RFC2046).

Shameless plug for my multipart crate: <a href="https:&#x2F;&#x2F;github.com&#x2F;cetra3&#x2F;mpart-async">https:&#x2F;&#x2F;github.com&#x2F;cetra3&#x2F;mpart-async</a> which I&#x27;ve been using happily in production for a long time now

One thing the article doesn&#x27;t mention, but that&#x27;s also interesting is that multipart is also heavily used for email. With multipart, you can send an HTML version along with a plan text version of your email. Email attachments are also solved via multipart. So I think this explains some of the decisions that were made when specifying HTTP Multipart.

In just the past week I used multipart&#x2F;form-data to proxy an incoming file upload stream to another backend server which then streamed it to a cloud bucket. Working great so far and was fairly simple to set up!

Does HTTP&#x2F;2 have support for parallel uploads? As an end-user if I happen to select 1 huge file first, followed by a bunch of tiny files, having to wait for the entire huge file to be streamed to the server before it even knows about any of the others seems less than ideal. It also seems odd there&#x27;s no way to provide a hint to the server about individual file-sizes up-front (only Content-Length which is your upper bound for the total size of all files). I&#x27;m also curious how clients are expected to generate boundary strings and what is supposed to happen if the uploaded file is discovered to have an instance of the boundary string in it. Or are we just relying on that being sufficiently improbable not to worry about it (a la guid uniqueness...)?

&gt; You can gzip the entire Multipart response, but you cannot pick and choose compression for particular parts.<p>Sure you can. I designed a system where the client uploads a multipart-form that&#x27;s three parts, the first part is JSON (&quot;meta&quot;) and the next two parts are gzipped blobs (&quot;raw.gz&quot; and &quot;log.gz&quot;). The server reads the first part which is metadata that tells it how to handle the next two parts.<p>I happen to be using Falcon and streaming-form-data on the server side.<p><a href="https:&#x2F;&#x2F;falcon.readthedocs.io&#x2F;en&#x2F;stable&#x2F;" rel="nofollow">https:&#x2F;&#x2F;falcon.readthedocs.io&#x2F;en&#x2F;stable&#x2F;</a><p><a href="https:&#x2F;&#x2F;streaming-form-data.readthedocs.io&#x2F;" rel="nofollow">https:&#x2F;&#x2F;streaming-form-data.readthedocs.io&#x2F;</a>