I've been feeling FOMO (for lack of a better term) about recent AI & ML/GPT progression.<p>It feels like ML/AI it might be the beginning of the end for a large class of things (if I wanted to be alarmist I'd say "everything") -- and the fact that Fabrice Bellard has jumped in and done the absolutely obvious rising-tide thing (building an API that abstracts the technologies) speaks volumes.<p>Releasing something like this fits to Fabrice's pattern of work -- he built Qemu and that served as a similar enabling fabric for people to run virtual machines. QuickJS quietly powers some JS-on-another-platform functionality.<p>Simon was right. The Stable Diffusion moment[0] is already here. It's going to accelerate. It was already moving at a speed that was hard to follow, and it's about to get even faster.<p>There are too many world-changing things moving forward at the same time, and I'm only looking at such a small cut of the tech sphere. I don't know what to do with myself, I feel so thoroughly unprepared.<p>[0]: <a href="https://simonwillison.net/2023/Mar/11/llama" rel="nofollow">https://simonwillison.net/2023/Mar/11/llama</a>
> All is included in a single binary. Very few external dependencies (Python is not needed) so installation is easy on most Linux distributions.<p>I have to disagree. The combination of being closed-source and dynamically linked makes a program a hassle to run on Linux. Even if it isn't at the moment of release, it soon becomes one. While <i>ts_server</i> is better than most, it already requires an old version of <i>libjpeg-turbo</i> not available in my distribution's repositories. I had to run it in a Rocky Linux container:<p><pre><code> docker run \
--rm \
--mount type=bind,source="$(pwd)",target=/app/ \
--publish 127.0.0.1:8080:8080 \
rockylinux:9 \
sh -c 'dnf install -y libjpeg libmicrohttpd && cd /app/ && ./ts_server ts_server.cfg'
</code></pre>
The solutions to this problem that I am aware of that do not involve releasing the source code are: 1) static linking; 2) containers; 3) shipping a Windows binary :-) ("Win32 is the only stable ABI on Linux" -- <a href="https://blog.hiler.eu/win32-the-only-stable-abi/" rel="nofollow">https://blog.hiler.eu/win32-the-only-stable-abi/</a>).
The Fabrice Bellard web page must be one of the most underselling ones on the entire web. So many amazing projects. Not a word that really emphasize the importance, coolness. Just a simple list with short factual descriptions.
Very interesting as usual from Fabrice Bellard, but I'm a little bit disappointed this time, because libnc is a closed source DLL. Nevertheless it will be interesting to compare it to the amazing work of Georgi Gerganov: GGML Tensor Library. Both are heavily optimized, supports AVX intrinsics and are plain C/C++ implementation without dependencies.
Anyone know what format the models have to be in for use with textsynth? I looked at the gpt2 example binary (gpt2_117M.bin) and it seems like the "normal" params.json is embedded as a header for the binary and then some ascii string like "attn/c_attn/" and then the binary weights.<p>I tried just using the Stanford Alpaca fine-tuned version of the llama 7B weights that work with llama.cpp with textsynth but it didn't like that (ggml-alpaca-7b-q4.bin: invalid file header). Having a textsynth HTTP API would save me a lot of hassle . I'm currently wrapping the stdin/out of a execution of a modified llama.cpp binary and that's extremely messy.
In comments on this post, and elsewhere on other posts about AI, I see a lot of people referring to worries around the potential for lots of types of jobs to be heavily impacted by this technology.<p>I feel like people are often referring to 'coding' when they express these worries. You know, actually writing code, having been given a spec to do so, and perhaps also participating in code review, writing tests, all the usual engineer stuff.<p>My question is, amongst the HN crowd, what kinds of roles or areas do we think might be somewhat immune to this effect? The first thing that occurs to me are security, infrastructure & ops, networking. And of course the requirements gathering stage of software development. It is already the case that a lot of senior devs probably don't write much code and spend more time on communication between different stakeholders and overseeing whoever (or whatever) is writing the code.<p>Anyone else been thinking about this? What tech roles might thrive in the face of AI.
For me, the most interesting part is the statistics on all the models. These show that 8 bit quantization is basically as good as the full model and 4 bit is very close. This is the first time I see such table across a large number of models in one place.
"<i>The CPU version is released as binary code under the MIT license</i>"<p>This gives off the surreal sci-fi vibe that the binary <i>is</i> the source. And who knows... true wizards work in mysterious ways.