Check out apache arrow in rust for transferring large amounts data between rust and python. With it you can get zero copy transfers between the languages. Afaik there is some performance penalty involved when converting python types to native rust types.<p>It used to be hard to do such transfers between rust and python using arrow but from v3 or v4, arrow implementation in rust started supporting the c data interface. Creating the array in rust and sending it to python might involve leaking a boxed object though<p>Check this out if you want to see how
<a href="https://github.com/jhoekx/python-rust-arrow-interop-example/blob/master/src/lib.rs" rel="nofollow">https://github.com/jhoekx/python-rust-arrow-interop-example/...</a>
<a href="https://pypi.org/project/blake3" rel="nofollow">https://pypi.org/project/blake3</a> has been based on PyO3 for about a year and a half now, and it's been smooth sailing. One of the reasons we reach for compiled code from Python is to get multithreading working for CPU-bound tasks, and the combination of PyO3 plus Rayon on the Rust side is especially nice.
I recently wanted to use some rust crates from Python and decided to give PyO3 a go and I must say it works really well.<p>Took about one evening to plug it all together and get it running. The guide at pyo3.rs was really helpful and maturin to build binary packages just works. You pass in python types and they arrive as rust-native types. Makes writing code feel very native.<p>The result: <a href="https://github.com/Grollicus/pyttfwrap" rel="nofollow">https://github.com/Grollicus/pyttfwrap</a> takes a string and splits it as it would wrap when rendered with a given TTF font. Most of the code is about keeping a reference to the loaded font as that's an expensive operation.<p>Was some great fun and I'm sure I'll use it some more.
This is smart thinking. Python is the world's best glue language and Rust is on track to become the world's best compiled, safe language.<p>It makes sense to combine the two and replace C in this.
I hope someone creates a rule set for Bazel to automate this stuff. This seems like a really nice way to start rewriting performance critical code in a safe language!
We used PyO3 at $work to expose a Rust implementation of a compute-intensive algorithm to an existing Python codebase.<p>The teammate who did it had been using Rust only for a couple months and none of us had ever used PyO3. He got it done in just a couple days. I consider that an endorsement of the API they've built.<p>It's heavily macro-based, which does cause some confusion. But if you spend some time with their examples, finding the fast path isn't too tricky.
We've been using PyO3 and Maturin at Spring for a while now, and happily. The smooth Python interop means we can call out to Rust without much pain for performance-critical codepaths. But the other side-effect is that we can use Rust across the org more broadly, even when Python interop isn't a consideration -- e.g., for isolated services, or applications that need to compile to Windows, or whatever else -- since we're building up the cultural knowledge and shared libraries to do so.
For Julia there's pycall: <a href="https://github.com/JuliaPy/PyCall.jl" rel="nofollow">https://github.com/JuliaPy/PyCall.jl</a> I believe there's one for the opposite direction as well.
I'm curious about any rough edges.<p>1. What happens if the Rust code is misbehaving, panicking, etc?<p>2. What's build support like for this?<p>I'd love to gut what little Python code I have left and use Rust.
<a href="https://pypi.org/project/dhall/" rel="nofollow">https://pypi.org/project/dhall/</a> uses PyO3 to reuse Rust's implementation to allow Python to load and dump Dhall configs, since no good and performant native python implementations exist. It's been quite pleasant for this purpose.
This is actually a deserializer which I wrote. I would have done it in C but I stumbled on PyO3 and gave it a go: <a href="https://github.com/m3047/pyo3-deserializer" rel="nofollow">https://github.com/m3047/pyo3-deserializer</a>
I've been waiting a while for a good article on integrating rust into python for a while. I was inspired by the amazing performance of the orjson python lib.