There's a much simpler way to do this:<p>If you want your library to operate on bytes, then rather than taking in an io.Reader and trying to figure out how to get bytes out of it the most efficient way, why not just have the library taken in []byte rather than io.Reader?<p>If someone has a complex reader and needs to extract to a temporary buffer, they can do that. But if like in the author's case you already have []byte, then just pass that it rather than trying to wrap it.<p>I think the issue here is that the author is adding more complexity to the interface than needed.<p>If you need a []byte, take in a []byte. Your callers should be able to figure out how to get you that when they need to.<p>With go, the answer is usually "just do the simple thing and you will have a good time".
Instead of focusing on bytes.Reader, the author could maybe propose that bytes.Buffer (which already has a Bytes method) implements Peek as well.<p>bytes.Buffer already has many methods exposing the internal buffer, so one more is not insurmountable.
The biggest issue here IMO is the interaction between two things:<p>- "Upcasting" either to a concrete type or to an interface that implements a specific additional function; e.g. in this case Bytes() would probably be useful<p>- Wrapper types, like bufio.Reader, that wrap an underlying type.<p>In isolation, either practice works great and I think they're nice ideas. However, over and over, they're proving to work together poorly. A wrapper type can't easily forward the type it is wrapping for the sake of accessing upcasts, and even if it did, depending on the type of wrapper it might be bad to expose the underlying type, so it has to be done carefully.<p>So instead this winds up needing to be handled basically for each type hierarchy that needs it, leading to awkward constructions like the Unwrap function for error types (which is very effective but weirder than it sounds, especially because there are two Unwraps) and the ResponseController for ResponseWriter wrappers.<p>Seems like the language or standard library needs a way to express this situation so that a wrapper can choose to be opaque or transparent and there can be an idiomatic way of exposing this.
> The bytes.Reader should really implement Peek. I’m pretty sure the reason it doesn’t is because this is the only way of creating read only views of slices. And a naughty user could peek at the bytes and then modify them. Sigh. People hate const poisoning, but I hate this more.<p>When I was a Google, a team adjacent to ours was onboarding a new client with performance demands that they could not realistically meet with anything resembling their current hardware footprint. Their service was a stateless Java service, so they elected to rewrite in C++. Now, Java has some overhead because of garbage collection and the JVM, and they hoped that this might move the needle, but what happened was they went from 300qps/core to 1200, with lower tail latency. Literally 3x improvement.<p>Why? Probably a lot of reasons, but the general consensus was: Java has no const, so many of Google’s internal libraries make defensive copies in many places, to guarantee immutability (which is valuable in a highly concurrent service, which everything there is). This generates a huge amount of garbage that, in theory, is short-lived, rarely escapes its GC generation, and can all be cleaned up after the request is finished. But their experience was that it’s just much faster to not copy and delete things all over the place. Which you can often avoid by using const effectively. I came to believe that this was Java’s biggest performance bottleneck, and when I saw that Go had GC with no const, I figured it would have the exact same problem
> Now, why doesn’t bytes.Reader implement Peek? It’s just a byte slice, it’s definitely possible to peek ahead without altering stream state. But it was overlooked, and instead this workaround is applied.<p>When I first looked at Go, it seemed to have far too many layers of abstraction on top of one another. Which is so ironic, considering that's one of the main things it was trying to fix about Java. It ended up becoming the thing it fought against.
This really feels like trying to use Go for a purpose that it is inherently not designed for: absolute performance.<p>Go is a fantastic case study in "good enough" practical engineering. And sometimes that means you can't wring out the absolute max performance and that's okay.<p>It's frustrating, but it is fulfilling it's goals. It's goals just aren't yours.
Type inspection is the flaw of Go's interface system. Try to make a type that delegates to another object, and the type inspection breaks. It's especially noticeable with the net/http types, which would be great to intercept, but then breaks things like Flusher or Hijacker.
The reason interface smuggling exists as a pattern in the Go standard library and others is because the Go team (and those who agree with its philosophy) take breaking API changes really seriously.<p>It is no small feat that Go is still on major version 1.
Interesting, I don't know Go (yet) but I was messing with it the other weekend.<p>Can someone please summarize this []byte vs somethingReader thing for me? Assume I can program, just not familiar with Go.<p>I was reading off sockets and it looked to me that the example code (i randomly ran into) had too many Reader something or other.<p>Edit: Ok, I know what a streaming class does, they're available in many frameworks. I'm more interested in why you'd get forced to use them in the context of the Go standard library.<p>Are they mandatory for sockets? Or for interacting with other common functions that I'd use to process the data out of my sockets?<p>I just wanted to read up to a new line or a hard coded size limit from a socket... ;) Without getting accidentally quadratic in either cpu use or memory use...
It seems that go library is ok with you paying the performance price when using io.Reader/io.Writer on memory structures.<p>You can write clean idiomatic code, but it won’t be the fastest. So for maximum results you should always do everything manually for your use case: i.e. don’t use additional readers/writers and operate on []byte directly if that is what you are working with.<p>I think it is mostly a good thing - you can quickly write simple but slower code and refactor everything later when needed.
I am once again begging for this to be implemented: <a href="https://github.com/golang/go/issues/4146">https://github.com/golang/go/issues/4146</a>
I am not a gopher, so this may be a dumb question: when an io.Reader produces a buffer of its contents, does it not have the option of just returning the buffer it wraps if it does in fact wrap a buffer? Something like (pseudocode) `if self isa BufferedReader { self.takeBuffer() } else { let buffer = newBuffer(); self.fill(buffer); buffer }`.
> What I would like is for my image decoding function to notice that the io.Reader it has been given is in fact a bytes.Reader so we can skip the copy.<p>What a terrible idea. If you want bytes.reader, then use that in the function signature, or better yet just a byte slice. It should have been a red flag when your solution involves the unsafe package