> In total the VMs use 182 GB of RAM and 94 CPU cores. The total storage capacity is 620 GB, but that’s not all used.<p>That level of hardware/cores seems a bit over the top given what TPB does.<p>When I was a boy we had this thing called 'Alta Vista'. It was <i>the</i> search engine before Bing! came along. Processors did not run at gigahertz speeds back then and a large disk was 2Gb. Nonetheless most offices had the internet and when people went searching 'Alta Vista' was the first port of call for many.<p>TPB has an index of a selective part of the internets, i.e. movies, software, music, that sort of thing. Meanwhile, back in the 1990's, AltaVista indexed everything, as in the entire known internets, with everything stored away in less than the 620Gb used by TPB for their collection of 'stolen' material.<p>From <a href="http://en.wikipedia.org/wiki/AltaVista" rel="nofollow">http://en.wikipedia.org/wiki/AltaVista</a><p>Alta Vista is a very large project, requiring the cooperation of at least 5 servers, configured for searching huge indices and handling
a huge Internet traffic load. The initial hardware configuration for Alta Vista is as follows:<p>Alta Vista -- AlphaStation 250 4/266
4 GB disk
196 MB memory
Primary web server for gotcha.com
Queries directed to WebIndexer or NewsIndexer<p>NewsServer -- AlphaStation 400 4/233
24 GB of RAID disks
160 MB memory
News spool from which news index is generated
Serves articles (via http) to those without news server<p>NewsIndexer -- AlphaStation 250 4/266
13 GB disk
196 MB memory
Builds news index using articles from NewsServer
Answers news index queries from Alta Vista<p>Spider -- DEC 3000 Model 900 (replacement for Model 500)
30 GB of RAID disk
1GB memory
Collects pages from the web for WebIndexer<p>WebIndexer -- Alpha Server 8400 5/300
210 GB RAID disk (expandable)
4 GB memory (expandable)
4 processors (expandable)
Builds the web index using pages sent by Spider.
Answers web index queries from Alta Vista