Book: Superintelligence

Getting back into this dense book about how superhuman intelligence might be created and the dangers we need to address before we do. I bought it in 2016 when it was released but was intimidated into not finishing it.

In some ways the book parallels “The Age of EM” — both authors are Oxford lecturers— but this one is sober and well grounded. We’ll see how long I can plow this hard ground.

Nick Bostrom wrote:

...an artificial intelligence need not much resemble a human mind.  AIs could be--indeed, it is likely that most will be--extremely alien.  We should expect that they will have very different cognitive architectures than biological intelligences, and in their early stages of development they will have very different profiles of cognitive strengths and weaknesses.  Furthermore the goals of AIs could diverge radically from those of human beings.  There is no reason to expect a generic AI to be motivated by love or hate or pride or other such common human sentiments: these complex adaptations would require deliberate expensive effort to recreate in AIs.

What a refreshing change from The Age of EMs! The author cites twelve articles by Robin Hanson in his bibliography, but obviously vehemently disagrees with him.

Chapter two describes five possible paths to superintelligence:

• artificial intelligence
  
• whole brain emulation
  
• biological (via eugenics)
  
• brain-computer hybrids
  
• networks and a self-aware internet
  

That last one is fun to think about.  At what point, how many computers need to be on the internet, how many connections need to be between them before they start to "talk among themselves"?

The third one the author discusses at length, without once acknowledging the huge political resistance to such a move.

"Superintelligence," the author points out, is a relative term.  The establishment of language and speech, writing and printing, standardized education, the population explosion and modern day connectivity have combined to make us "superintelligent" compared to our genetically-identical Pleistocene forebears.

"Superintelligence" also does not imply wisdom, the author points out.