Types for Python HTTP APIs

In our startup we use marshmallow [1] to validate and make REST API type aware. So marshmallow models validates the data types based on validation rules for request&#x2F;response.<p>In place where we need client to be able to define the validation the request&#x2F;response will consist of data, validation json schema. In this case use jsonschema validator [2] [3].<p>So for every json request&#x2F;request data, it&#x27;s passed through a marshmallow model which validates the json. In some cases we use jsonschema validator when we have dynamic json data with schema definition for validations.<p>For response the function return value pass through a marshmallow model. We are moving towards making all of internal functions&#x2F;methods with type annotations to generate documentation using Sphinx [4] plugin.<p>We are not Instagram but are very happy with it and can replace flask library with bottle or other wsgi compatible framework and it will still work.<p>[1] <a href="https:&#x2F;&#x2F;marshmallow.readthedocs.io&#x2F;en&#x2F;stable&#x2F;" rel="nofollow">https:&#x2F;&#x2F;marshmallow.readthedocs.io&#x2F;en&#x2F;stable&#x2F;</a><p>[2] <a href="https:&#x2F;&#x2F;python-jsonschema.readthedocs.io&#x2F;" rel="nofollow">https:&#x2F;&#x2F;python-jsonschema.readthedocs.io&#x2F;</a><p>[3] <a href="https:&#x2F;&#x2F;pydantic-docs.helpmanual.io&#x2F;" rel="nofollow">https:&#x2F;&#x2F;pydantic-docs.helpmanual.io&#x2F;</a><p>[4] <a href="https:&#x2F;&#x2F;pypi.org&#x2F;project&#x2F;sphinx-autodoc-typehints&#x2F;#data" rel="nofollow">https:&#x2F;&#x2F;pypi.org&#x2F;project&#x2F;sphinx-autodoc-typehints&#x2F;#data</a>

Been doing this before the typing module was released, using hug: <a href="http:&#x2F;&#x2F;www.hug.rest&#x2F;" rel="nofollow">http:&#x2F;&#x2F;www.hug.rest&#x2F;</a><p>FastAPI has a lot of advantages at this point for someone looking to do the same: <a href="https:&#x2F;&#x2F;github.com&#x2F;tiangolo&#x2F;fastapi" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;tiangolo&#x2F;fastapi</a><p>hug has some catching up to do, some of it just because it got in so early (It needs to be updated to be compatible with mypy and the types defined typing.py!) in any case using typing on API endpoints - independent of how you feel about Dynamic vs Static typing in general, just makes a lot of sense IMHO.

Looks like FastAPI framework with handler type anotations

Nice writeup - I&#x27;ve been doing this type of thing for 10 years before type annotations by passing the types to the decorator and then using kwargs to supply defaults.<p><pre><code> @annotate(foo=int, bar=str) def view_func(foo=0, bar=&#x27;hi there&#x27;): ... </code></pre> The types could also be arbitrary callables to parse things like datetime and what not. I&#x27;d parse the params from either the get params or post data (json, urlencoded, etc).<p>But now I&#x27;m using graphene and graphql to handle all this -- it&#x27;s a better way to do all of it imho. Of course this all came along after instagram, so you didn&#x27;t have that choice back then.

Rather than reimplementing this stuff, and assuming you are using Django, you could just use Django Rest Framework to get OpenAPI, typed serializers, etc.<p>In that framework your serializers are by default auto-generated from your model classes; this is convenient to get started, just like Django itself.

I&#x27;m always amazed that, in majority of cases, we use formal specifications -- like OpenAPI -- for documentation instead to guide our implementations. The common pattern is to build an API first, and then extract routing and validation information into an OpenAPI spec, which is then used to set up a test server for clients to develop on top on.<p>But if you have a relatively clear idea of what your API should look like, there are great benefits to be gained by providing the specification first. This way, you don&#x27;t need for the API to be implemented to start developing the clients, even by completely unrelated teams. Second, your spec will already include your routing and validation rules, and there is no need to manually specify e.g. Pydantic models.<p>I recently wrote a PoC of a framework [0] that uses the OpenAPI spec to easily implement a REST(ful) API; in a nutshell, you need to implement endpoint functions that correspond to the spec&#x27;s `operationId` names, and it will automatically route a request to the right endpoint. It is fully ASGI compliant and as has a bonus client module, which allows you to do the requests.<p>[0] <a href="https:&#x2F;&#x2F;github.com&#x2F;berislavlopac&#x2F;pyotr" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;berislavlopac&#x2F;pyotr</a>

I&#x27;d love to see some more details on the tooling they created to generate OpenAPI schemas by extracting types from their code.<p>Related: are there any good Python libs for doing request&#x2F;response validation based on OpenAPI v3 schemas?

No one is mentioning that Instagram app which doesn’t have that many features, and is quite poorly designed, somehow requires over 2000 API calls to function.<p>I would dare anyone come up with even 100 for the Instagram app.

I love types and I’m stoked about them coming back into style. For awhile, it was a major trend to avoid static types. I think most people’s problem ended up not being with types themselves- but poor type systems. For myself, I am particularly impressed with what TypeScript has done with its types, really amazing how expressive&#x2F;flexible&#x2F;and inspective it is. Anyways, just happy with the trend of embracing types and hopes it continues.

This is a bit lower level as it&#x27;s only concerned with serializing, but a package I wrote called json_syntax[1] will take @dataclass or @attr.s classes decorated with type annotations and build encoders and decoders for them.<p>It also handles Union types reasonably well and lets you put in hooks to handle ugly cases. It&#x27;s used in production on a system with a big complicated payload, and I designed it to be easily extensible if the standard rules don&#x27;t work for you.<p>[1]: <a href="https:&#x2F;&#x2F;pypi.org&#x2F;project&#x2F;json-syntax&#x2F;" rel="nofollow">https:&#x2F;&#x2F;pypi.org&#x2F;project&#x2F;json-syntax&#x2F;</a>

I’ve been working on a similar solution for Flask in K Health. Currently we have great serialisation &#x2F; deserialisation from types to JSON, next step will be creating OpenAPI documents

I&#x27;ve been using type hints with my bottle projects for some time now. Makes it a lot easier to write JSON based apis.<p><a href="https:&#x2F;&#x2F;github.com&#x2F;theSage21&#x2F;bottle-tools" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;theSage21&#x2F;bottle-tools</a>

When I read the headline and saw the source, I assumed this would be about GraphQL. I know Instagram utilizes GraphQL, for example on the web client, so now I&#x27;m wondering how that fits in.

Flask Swagger already has a way to annotate it quite well.

I find it really impressive that such a large organization runs anything on Python. Isn&#x27;t the speed optimization potential immense? Does the amount of Python involved just not matter compared to image data?<p>Secondly I find it really impressive that such a large company with so many smart people can produce an application so mediocre and make the experience extra terrible and me wonder what the absolute fuck is up with that company by trying to block desktop browsers from the perfectly useable on desktop web app (the one in which you can upload things).<p>Especially for a photo centric application (that has since begun to be used for original video production, of course hampered by the insane lack of any options, starting with the aspect ratio) one could expect a normal work flow to include transferring photos from a camera to a desktop computer. Making your browser pretend to be a mobile phone seems like a step that could maybe, if you really tried (to take out the arbitrary restriction that you must have explicitly added in the first place), be made unnecessary.<p>So that&#x27;s my (condensed) rant about Instagram as a whole.

The whole Type thing in Python and all the workarounds to make Python type-safish looks and feels absolutely pathetic. I have no idea why a company like Facebook takes so much time to apologize for the inadequacy of Python as a programming language in the broader sense. Especially since they are utilizing a service based architecture they easily could make the switch to a language that actually supports types. Also the obvious performance gains...