It should be noted that the reason this is wrong under Docker is that /proc/cpuinfo is not cgroup-aware. LXC works around this by creating a fake /proc/cpuinfo (and /proc/meminfo)[1] which matches the values set by cgroup limits.<p>[1]: <a href="https://github.com/lxc/lxcfs" rel="nofollow">https://github.com/lxc/lxcfs</a>
IIRC this utility is a wrapper around glibc get_nprocs(), which on Linux will ultimately read from /sys/devices/system/cpu/online. If you need this information from a compiled language, you should probably use the glibc wrapper.
Even this I feel doesn't work all the time. At least from memory, I remember some problems where on kubernetes if you look at nproc or cpuinfo, that will give you the number of processors on the host machine, and not necessarily the CPU limit which is set per container. Although maybe I'm mis-remembering. I guess nproc could look at something else as well to filter based on that, and I agree having a nice portable commandline is way better than looking at /proc.
> So, what this will end up doing is just increase the number of context switches, possibly also adding a performance degradation.<p>I think this is a jumping to conclusions a bit. You’d need to be reading /proc/cpuinfo in a fairly tight loop in order for the time spend doing any context switching to dominate the runtime cost of a program. Is this a common practice in some types of computation? Or am I misunderstanding this quote from the article?
It's amazing how much knowledge is being lost with the advent of container revolutions and orchestrator Os'es. Cgroups is _still_ opt in not opt out...though those who never have worked in HPC on bare metal with parallel libraries and approaches for cpu like openmp are loudly complaining about control groups that don't control or provide consistent view of userland like there isn't a host system underneath. It's like this (<a href="https://www.gnu.org/software/libc/manual/html_node/CPU-Affinity.html" rel="nofollow">https://www.gnu.org/software/libc/manual/html_node/CPU-Affin...</a>) wasn't the way things were done (and still are to this day) in everything but the container world. I know it's hard to grasp but not everyone is containerizing everything.
I see suggestions to use ‘make -j$(nproc)’. However, I was taught when learning MPI programming 20 years ago, you want think at least “one to load, one to run”. The idea being that processes and threads waiting on IO will sleep/yield while another process or thread can continue executing. An especially useful strategy if you can afford the RAM (trading RAM for CPU). Consequently, I tend to run twice as many ‘make’ jobs as threads on my systems.
Interesting, but how does nproc get the right information ?<p>OMP_NUM_THREADS and OMP_THREAD_LIMIT are in principle extra limitations, not the main source of information, right ?