I am recently learning the basics of computer architecture with a slight remark on parallel computing. The whole design seems so terrible complex to me that I hardly believe in straight performance gains in the near future. Especially regarding GPUs / parallel computing. Is my impression wrong ? Are there any good, complete and up-to-date sources to this topic ?
The short answer is that clock rates are plateauing. Once you pass 2GHz heat, electrical characteristics, etc start becoming very big issues. As far as I can recall there are hardly any CPUs around that go much past 4GHz.<p>So if you want to increase throughput you start running multiple CPUs in parallel to increase the throughput. If you were to have discrete CPUs then the wiring between them, memory and I/O becomes another serious issue, so they try to integrate as many CPUs onto a single die in order to reduce the electrical delays, etc.<p>People often forget that in 1 ns (i.e. at 1GHz clock rate) light travels about 1 foot and less than that in electrical conductors. So keeping conductor lengths is a very big issue.