>With no more “cheap tricks” of switching the run-time again, how can we scale the calculation engine 100x, from millions to billions of cells?<p>>In summary: by moving from maps to arrays. That may seem like an awfully pedestrian observation, but it certainly wasn’t obvious to us at the outset that this was the crux of the problem!<p>starting about 20 years ago programming has moved to "map-based" programming, coinciding with the hardware availability and the rise of of Java, JavaScript, Python and the likes where maps are much more convenient. Instead of various data structures everything started to become a map. Very convenient. Set/get by name. Good enough fit pretty much anywhere (i remember one of my particular creation using Perl in 1999 where i did let myself to have a go at it and had multiple levels of maps of references to other maps
- the code happened to be widely copied, yet nobody successfully extended/modified it as far as i know :). Comes with the price though, so sometimes people would have to dust off and use the old ones, like say the arrays above.