Book: Army of None

Subject: Book: Army of None Sun Mar 24, 2019 8:43 am

Starting the third of Bill Gates' five best books of 2018. This one, by Paul Scharre, is about automation in warfare, or "your tank will be programmed to kill you. Deal with it."

At present unmanned drones patrol the sky and rain death on the hapless targets, but they're still flown (via remote control) by human operators. Life-and-death decisions on who to kill are still made by human soldiers.

In the near future (or earlier) this could be fully automated.

The author illustrates the dilemma with a couple real-life scenarios. In one, he was a sniper on patrol in Afghanistan, keeping track of "insurgents." A little girl tending goats brought her flock up to their position. They realized she was radioing their position to the insurgents. Theoretically that makes her an "enemy combatant" and legal to shoot to kill. But morally none of the soldiers felt right about killing a 9-year old shepherd.

Would a computer have any moral qualms?

In another example, sunlight reflecting off clouds gave several false positives to Soviet missile tracking systems, telling them a US nuclear strike was underway in 1983. The general monitoring the system overruled the system, on a gut reaction, and aborted a counter-strike. His own common sense prevented WWIII.

Would a computer have considered the devastating consequences of being wrong?

There's no doubt automated systems can react faster, can process more variables, can be more "expert" than humans making the same decisions. But sometimes the result of relying on them will have devastating consequences.

Human error caused Chernobyl, Fukushima, Three Mile Island, Exxon Valdez, Hiroshima, and any number of other disasters. We accept disasters caused by Mother Nature. We accept disasters caused by human error.

Our experiences in disasters caused by AI are considerably less accepted as inevitable.

Subject: Re: Book: Army of None Sat Apr 06, 2019 10:38 am

Paul Scharre wrote:: Only twenty-two nations have said they say support a ban on lethal autonomous weapons: Pakistan, Ecuador, Egypt, the Holy See, Cuba, Ghana, Bolivia, Palestine, Zimbabwe, Algeria, Costa Rica, Mexico, Chile, Nicaragua, Panama, Peru, Argentina, Venezuela, Guatemala, Brazil, Iraq and Uganda (as of November 2017). None of these states are major military powers and some, such as Costa Rica or the Holy See, lack a military entirely.

Subject: Re: Book: Army of None Sat Apr 06, 2019 11:05 am

Paul Scharre wrote:: [A] controversial 15-second video clip released in the summer of 2015 was taken by a Connecticut teenager of a drone he armed himself. A gunshot cuts through the low buzz of the drone's rotors. The camera jerks backward from the recoil. Red and yellow wires snake over the drone and into the gun's firing mechanism, allowing the human controller to remotely pull the trigger.

Law enforcement and the FAA investigated, but no laws were broken. The teenager used the drone on his family's property in the New England woods. There are no laws against firing weapons from a drone, provided it is done on private property.

When I asked the Pentagon's chief weapons buyer Frank Kendall what he feared [most] it wasn't Russian war bots, it was cheap commercial drones. If autonomous weapons could be built by virtually anyone in their garage, bottling up the technology and enforcing a ban would be extremely difficult. I wanted to know, could someone leverage commercially available drones to make a do-it-yourself autonomous weapon? I was terrified by what I found.

Subject: Re: Book: Army of None Sun Apr 28, 2019 4:10 pm

The author talks briefly about machines (AI) killing people, about warfare where one AI fights another AI, about instances in war (like the Christmas truce) where human soldiers refused to kill each other.

Perhaps the paradigm of war will completely change some day.

General Curtis LeMay wrote:: All war is immoral and if you let that bother you, you're not a good soldier.

If soldiers were not involved in combat -- if it was one robot army facing another robot army -- would countries be more willing to wage wars? Would nations be unwilling to surrender, as long as they could keep building robots?

Subject: Re: Book: Army of None Sun Apr 28, 2019 6:30 pm

Paul Scharre wrote:: When I visited her lab at Duke, [Missy Cummings] showed me a van they were using to test how pedestrians interact with self-driving cars. The secret, Cummings told me, was that the car wasn't self-driving at all. There was a person behind the wheel. The experiment was to see if pedestrians would change their behavior if they thought the car was self-driving. And they did. "We see some really dangerous behaviors," she said. People would carelessly walk in front of the van, assuming it would stop. Pedestrians perceived the automation as more reliable than a human driver and changed their behavior as a result, acting more recklessly themselves.

» Book: The Book of Lost Books
» Book: The Sum of Us
» Book: What If?
» Book: Superintelligence
» Book: On The Rez