Embed is in C23

I represent Sweden in the ISO WG14, and I voted for the inclusion of Embed in to C23. Its a good feature. But its not a necessary feature and I think JeanHeyd is wrong in his criticism of the pace of wg14 work. I have found everyone in wg14 to be very hardworking and serious about their work.<p>Cs main strengthen is its portability and simplicity. Therefore we should be very conservative, and not add anything quickly. There are plenty of languages to choose form if you want a &quot;modern&quot; language with lots of conveniences. If you want a truly portable language there is really only C. And when I say truly, I mean for platforms without file systems, or operating systems or where bytes aren&#x27;t 8 bits, that doesn&#x27;t use ASCI or Unicode, where NULL isn&#x27;t on address 0 and so on.<p>We are the stewards of this, and the work we put in, while large, is tiny compared to the impact we have. Any change we makes, needs to be addressed by every compiler maintainer. There are millions of lines of code that depend on every part of the standard. A 1% performance loss is millions of tons of CO2 released, and billions in added hardware and energy costs.<p>In this privileged position, we have to be very mindful of the concerns of our users, and take the time too look at every corner case in detail before adding any new features. If we add something, then people will depend on its behavior, no matter how bad, and we therefor will have great difficulty in fixing it in the future without breaking our users work, so we have to get it right the first time.

Ha I suggested this on the C++ proposals mailing list 7 years ago:<p><a href="https:&#x2F;&#x2F;groups.google.com&#x2F;a&#x2F;isocpp.org&#x2F;g&#x2F;std-proposals&#x2F;c&#x2F;b6ncBojU8wI&#x2F;m&#x2F;blx1GlLqUAUJ" rel="nofollow">https:&#x2F;&#x2F;groups.google.com&#x2F;a&#x2F;isocpp.org&#x2F;g&#x2F;std-proposals&#x2F;c&#x2F;b6n...</a><p>Enjoy the naysayers if you like! I&#x27;m glad someone spent the time and effort to push past them. Bit too late for me - I have moved on to Rust which had support for this from version 1.0.0.<p>&gt; There&#x27;s also the standard *nix&#x2F;BSD utility &quot;xxd&quot;.<p>&gt; Seems like the niche is filled. Or, at least, if you want to claim that<p>&gt; (A) XPM<p>&gt; (B) incbin<p>&gt; (C) &quot;xxd -i&quot;<p>&gt; (D) various ad-hoc scripts given in <a href="http:&#x2F;&#x2F;stackoverflow.com&#x2F;questions&#x2F;8707183&#x2F;script-tool-to-convert-file-to-c-c-source-code-array" rel="nofollow">http:&#x2F;&#x2F;stackoverflow.com&#x2F;questions&#x2F;8707183&#x2F;script-tool-to-co...</a><p>&gt;...do NOT completely fill this evolutionary niche<p>&gt; This ultimately would encourage a weird sort of resource management philosophy that I think might be damaging in the long run.<p>&gt; Speaking from experience, it is a tremendously bad idea to bake any resource into a binary.<p>&gt; I&#x27;ll point out that this is a non-issue for Qt applications that can simply use Qt&#x27;s resources for this sort of business.<p>(Though credit to Matthew Woehlke, he did point out a solution which is basically identical to #embed)<p>&gt; I find this useless specially in embedded environments since there should be some processing of the binary data anyway, either before building the application<p>In fairness there was a decent amount of support. But given the insane amount of negativity around an obviously useful feature I gave up.<p>I wonder if there was a similar response to the proposal to include `string::starts_with()`...

<i>&gt; told me this form was non-ideal and it was worth voting against (and that they’d want the pure, beautiful C++ version only[1])</i><p>I heard about #embed, but I didn&#x27;t hear about std::embed before. After looking at the proposal, to me it does look a lot better than #embed, because reading binary data and converting it to text, only to then convert it to binary again seems needlessly complex and wasteful. I also don&#x27;t like that it extends the preprocessor, when IMHO the preprocessor should at worst be left as is, and at best be slowly deprecated in favour of features which compose well with C proper.<p>Going beyond the gut reaction and moving on to hard data, as you can expect from this design, std::embed of course is faster during compilation than #embed for bigger files (comparable for moderately-sized files, and a bit slower for tiny files).<p>I&#x27;m not a huge fan of C++, but the fact that C++ removed trigraphs in C++17 and that it&#x27;s generally adding features replacing the preprocessor scores a point with me.<p>[1]: &lt;<a href="https:&#x2F;&#x2F;www.open-std.org&#x2F;jtc1&#x2F;sc22&#x2F;wg21&#x2F;docs&#x2F;papers&#x2F;2020&#x2F;p1040r6.html" rel="nofollow">https:&#x2F;&#x2F;www.open-std.org&#x2F;jtc1&#x2F;sc22&#x2F;wg21&#x2F;docs&#x2F;papers&#x2F;2020&#x2F;p10...</a>&gt;

This serves the same use as Rust&#x27;s `include_bytes!` macro, right? Presumably most people just use this feature as a way to avoid having to stuff binary data into a massive array literal, but in our case it&#x27;s essential because we&#x27;re actually using it to stuff binaries from earlier in our build step into a binary built later in the build step. Not something you often need, but very handy when you do.

&gt; Even among people who control all the cards, they are in many respects fundamentally incapable of imagining a better world or seizing on that opportunity to try and create one, let alone doing so in a timely fashion.<p>That does sound soul-crushing. Congrats on this achievement!

This reminds me, I&#x27;d argue that the explosion of JS frameworks can be mainly blamed on one thing: the lack of an &lt;include src=&quot;somemodule.html&quot;&gt; tag. If you have that you basically have 80% of vue.js already natively supported. No clue why this was never added in any fashion. Change my mind.

Honestly I&#x27;m usually very wary of additions to C, as one of its greatest strengths (to me) is how rather straightforward it is as a language in terms of conceptual simplicity. There just aren&#x27;t that many big concepts to understand in the language. (On the other hand there&#x27;s _many_ footguns but that&#x27;s another issue.)<p>That said, to me this seems like a great addition to the language. It&#x27;s very single-purpose in its usage (so it doesn&#x27;t seem to add much conceptual complexity to the language) and it replaces something genuinely painful (arcane linker hacks). I&#x27;m very much looking forward to using this as I often make single-executable programs in C. The only thing that&#x27;s unfortunate is I&#x27;m sure it&#x27;ll take decades before proprietary embedded toolchains add support for this.

There is way too much in C already.<p>The first commandment of C is: &#x27;writing a naive C compiler should be &quot;reasonable&quot; for a small team or even one individual&#x27;. That&#x27;s getting harder and harder, longer and longer.<p>I did move from C being &quot;the best compromise&quot; to &quot;the less worse compromise&quot;.<p>I wish we had a &quot;C-like&quot; language, which would kind of be a high-level assembler which: has no integer promotion or implicit casts, has compile-time&#x2F;runtime casts (without the horrible c++ syntax), has sized primitive types (u64&#x2F;s64,f32&#x2F;f64,etc) at its core, has sized literals (42b,12w,123dw,2qw,etc), has no typedef&#x2F;generic&#x2F;volatile&#x2F;restrict&#x2F;etc well that sort of horrible things, has compile-time and runtime &quot;const&quot;s, and I am forgetting a lot.<p>From the main issues: the kernel gcc C dialect (roughly speaking, each linux release uses more gcc extensions). Aggressive optimizations can break some code (while programing some hardware for instance).<p>Maybe I should write assembly, expect RISC-V to be a success, and forget about all of this.

I’m really amazed at how divisive this one is, and the number of comments here questioning what seems to me a really useful and well-thought-out feature, something I’d have loved to have used many many times over the years.<p>I guess the heated arguments here help me understand how it could have taken so long to get this standardised, though, so that’s something!<p>Congratulations and thank you to the OP for doing this, and thanks also for this really interesting (if depressing) view of the process.

This is a really, really good feature and I am so glad it is finally getting standardized. C23 is shaping up to be a very good revision to the C standard. I’m hoping the proposal to allow redeclaration of identical structs gets in as well as you would finally be able to write code using common types without having to coordinate which would allow interoperability between independently written libraries.

Congratulations to the author. Things like this are why I hope Carbon exists. Evolving c++ seems like a dumpster fire, despite whatever compelling arguments about comparability you are going to drop on me.

Re: #embed &lt;&#x2F;dev&#x2F;urandom&gt;<p>Just a random thought, but I&#x27;d expect a compiler to do exactly what&#x27;s described if I tell it:<p><pre><code> static char foo[123] = { #embed &lt;&#x2F;dev&#x2F;urandom&gt; }; </code></pre> This would address the most common case with infinity files, and then just let the compiler error out if the array size is not specified.

Interesting. I look forward to this. What I&#x27;ve been doing now to embed a source.png file is something like this, where I generate source code from a file&#x27;s data:<p>in embed_dump.cpp:<p><pre><code> #include &lt;fstream&gt; #include &lt;iostream&gt; int main(){ std::ifstream f; f.open(&quot;.&#x2F;source.png&quot;); std::cout &lt;&lt; &quot;&#x2F;&#x2F;automatically generated by embed_dump from project files:&quot; &lt;&lt; std::endl &lt;&lt; &quot;const char embedded_tex[] = {&quot;; char a; while(f.good()){ f.read(&amp;a,1); std::cout &lt;&lt; int(a) &lt;&lt; &quot;,&quot;; } std::cout &lt;&lt; &quot;};&quot; &lt;&lt; std::endl; f.close(); } </code></pre> Then I set up my makefile like this (main_stuff.cpp #includes embedded_files.h):<p><pre><code> main_stuff: main_stuff.cpp embedded_files.h c++ main_stuff.cpp embeded_files.h: embed_dump source.png .&#x2F;embed_dump &gt; embeded_files.h embed_dump: embed_dump.cpp c++ embed_dump.cpp -o embed_dump</code></pre>

What about creating object files from raw binary files and then linking against them? That&#x27;s what I (and of course many others) do for linking textures and shaders into the program. It&#x27;s a bit ugly though that with this approach you can&#x27;t generate custom symbol names, at least with the GNU linker.<p>This #embed feature might be a nice alternative for small files. Well for large files you usually don&#x27;t even want to store them inside the binary, so the compilation overhead might be miniscule, since the files are, by intention, small.<p>When I read the introduction of the article - about allowing us to cram anything we want into the binary - I was hoping to see a standard way to disable optimizations (When the compiler deletes your code and you don&#x27;t even notice).

Off the top of my head, I think there&#x27;s some niche use in embedding shaders so that they don&#x27;t need to be stored as strings (no IDE support) or read at runtime (slower performance).

Other stuff:<p><a href="https:&#x2F;&#x2F;twitter.com&#x2F;rcs&#x2F;status&#x2F;1550526425211584512" rel="nofollow">https:&#x2F;&#x2F;twitter.com&#x2F;rcs&#x2F;status&#x2F;1550526425211584512</a><p>nullptr! auto! constexpr!

If you want to start playing with this now, my C preprocessor Cedro (<a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=28166125" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=28166125</a>, <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=31193138" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=31193138</a>) will insert the byte literals for the compiler:<p><a href="https:&#x2F;&#x2F;sentido-labs.com&#x2F;en&#x2F;library&#x2F;cedro&#x2F;202106171400&#x2F;#binary-include" rel="nofollow">https:&#x2F;&#x2F;sentido-labs.com&#x2F;en&#x2F;library&#x2F;cedro&#x2F;202106171400&#x2F;#bina...</a><p>That’s the same that `xxd` does, for instance, and it works with C89&#x2F;C99&#x2F;C11.<p>The advantage is that since it uses the same syntax as C23, it is easier to switch to using the compiler: just remove the Cedro pragma `#pragma Cedro 1.0 #embed` and it will compile as C23.<p>The source code (Apache 2.0) and GitHub link are at: <a href="https:&#x2F;&#x2F;sentido-labs.com&#x2F;en&#x2F;library&#x2F;" rel="nofollow">https:&#x2F;&#x2F;sentido-labs.com&#x2F;en&#x2F;library&#x2F;</a>

Looks great. I&#x27;ve been writing cmake hacks to include assets in executables for too long.

Can’t you just add binary data into a custom section of your ELF executable?

5 years ago I wrote a small python script [1] to help me solve &quot;the same problem&quot;. It reads files in a folder and generates an header file containing the files&#x27; data and filenames. Is very simple and was to helping me on a job. It has limitations, don&#x27;t be too hard on me :)<p>[1] <a href="https:&#x2F;&#x2F;github.com&#x2F;daxliar&#x2F;pyker" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;daxliar&#x2F;pyker</a>

This will simplify a lot of build pipelines for sure.<p>One thing that isn&#x27;t clear from skimming the article, how do you refer to the embedded data again?

Everlasting glory to JeanHeyd Meneide, @thephantomderp, for getting this feature into C.<p>I am wondering, though - where does this stand in C++?

This is a cool feature and I&#x27;ll likely be using it in the years to come. However, the posix standard command xxd and its -i option can achieve this capability portably today.<p>It will be useful to achieve it directly in the preprocessor however. I wonder how quickly can it be added to cpp?

I&#x27;ve always used xxd -i for embedding, doesn&#x27;t have the mentioned problems and works everywhere, as it simply outputs a header file with byte array.

It is something that I had wanted in C too, for a while, so I am glad that they added this #embed command.

How to read unsigned data? Is there a stadardized parameter, or does this require a vendor extension?

What situations is this useful for?

&gt; “Touch grass”, some people liked to tell me. “Go outside”, they said (like I would in the middle of Yet Another Gotdang COVID-19 Spike). My dude, I’ve literally gotten company snail mail, and it wasn’t a legal notice or some shenanigans like that! Holy cow, real paper in a real envelope, shipped through German Speed Mail!! This letter alone probably increases my Boomer Cred™ by at least 50; who needs Outside anymore after something like this?<p>Touch grass indeed. Sure, #embed is a nice feature, but this self-indulgent writing style I can’t stand.

This is wrong.<p>It belongs in the linker and you can pull the symbol it creates in with extern. I’ve been doing this for about 25 years.

C89 is where C should&#x27;ve stayed at. If you need to convert a file to a buffer and stick that somewhere in your translation unit, use a build system. Don&#x27;t fuck with C.

It takes literally 5 minutes to write a python script that does this.<p>It took a long time to get this adopted because people are most likely busy with things that cannot be already solved trivially.

Haven&#x27;t the people making the standards other things to do, like, integrating <i>useful features</i> instead of duplicating incbin.h [0] years after that feature worked?<p><a href="https:&#x2F;&#x2F;github.com&#x2F;graphitemaster&#x2F;incbin&#x2F;blob&#x2F;main&#x2F;incbin.h" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;graphitemaster&#x2F;incbin&#x2F;blob&#x2F;main&#x2F;incbin.h</a>

&gt; The directive is well-specified, currently, in all cases to generate a comma-delimited list of integers.<p>While a noble act, this is nearly as inefficient as using a code generator tool to convert binary data into intermediate C source. Other routes to embed binary data don&#x27;t force the compiler to churn through text bloat.<p>It would be much better if a new keyword were introduced that could let the backend fill in the data at link time.

C++ keeps kicking ass!<p>Feel sorry for crab people.

&gt; vendor extensions ... were now a legal part of the syntax. If your implementation does not support a thing, it can issue a diagnostic for the parameters it does not understand. This was <i>great</i> stuff.<p>I can’t be the only one who thinks magic comment is already an ugly escape hatch, adding a mini DSL to it that can mean anything to anyone just makes it ten times worse. It’s neither beautiful nor great.<p>&gt; do extensions on #embed to support different file modes, potentially <i>reading from the network</i> (with a timeout), and other shenanigans.<p>(Emphasis mine.) My god.

Personally I feel the C committee should&#x27;ve disbanded after the first standard (the C++ one, after the 2003 technical corrigendum). I didn&#x27;t mind C99 much, but it looks like C(++)reeping featuritis is a nasty habit.<p>These gratuitous standards prompt newbies to use the new features (it&#x27;s &quot;modern&quot;) and puzzled veterans to keep up and reinternalize understanding of new variants of languages they&#x27;ve been using for decades. There&#x27;s no real improvement, just churn. Possibly it&#x27;s one of the instruments of ageism. More incompatibility with existing software and existing programmers.

This is a cool feature, but the author doesn&#x27;t do himself any favors with his style of writing that greatly overestimates the importance of his own feature in the great scheme of things. Remarks like &quot;an extension that should’ve existed 40-50 years ago&quot; make me think, if we should&#x27;ve really bothered all compiler vendors with implementing this 40-50 years ago. After all, you can already a) directly put your binary data in the source file like shown after the preprocessor step and b) read a file at runtime. I&#x27;m not saying this isn&#x27;t useful, but it&#x27;s a rather niché performance improvement than a core language feature.