Autonomous weapons are not just science fiction

Autonomous weapons (aka "killer robots") were the basis for the Terminator movies and uncounted spinoffs and copycats.  But the concept is achievable, and the potential consequences are unthinkable:

A very, very small quadcopter, one inch in diameter can carry a one- or two-gram shaped charge. You can order them from a drone manufacturer in China. You can program the code to say: “Here are thousands of photographs of the kinds of things I want to target.” A one-gram shaped charge can punch a hole in nine millimeters of steel, so presumably you can also punch a hole in someone’s head. You can fit about three million of those in a semi-tractor-trailer. You can drive up I-95 with three trucks and have 10 million weapons attacking New York City. They don’t have to be very effective, only 5 or 10% of them have to find the target.

There will be manufacturers producing millions of these weapons that people will be able to buy just like you can buy guns now, except millions of guns don’t matter unless you have a million soldiers. You need only three guys to write the program and launch them. So you can just imagine that in many parts of the world humans will be hunted. They will be cowering underground in shelters and devising techniques so that they don’t get detected. This is the ever-present cloud of lethal autonomous weapons.

They could be here in two to three years.

              — Stuart Russell, professor of computer science and engineering at the University of California Berkeley

That's the intro to a frankly unsettling article.

...lethal autonomous weapons systems (LAWS): weapons that have the ability to independently select and engage targets... humans out of the loop — where the human releases the machine to perform a task and that’s it — no supervision, no recall, no stop function.

One of the very real problems with attempting to preemptively ban LAWS is that they kind of already exist. Many countries have defensive systems with autonomous modes that can select and attack targets without human intervention — they recognize incoming fire and act to neutralize it... Meanwhile, offensive systems already exist, too: Take Israel’s Harpy and second-generation Harop, which enter an area, hunt for enemy radar, and kamikaze into it, regardless of where they are set up. The Harpy is fully autonomous...

Among the lauded new technologies is swarms — weapons moving in large formations with one controller somewhere far away on the ground clicking computer keys. Think hundreds of small drones moving as one, like a lethal flock of birds...

I worry it will breed way more terrorist activities. You can call them insurgents, you can call them terrorists, I don’t care, when you realize that you can’t ever fight the state mano-a-mano anymore, if people are pissed off, they’ll find a way to vent that frustration, and they will probably take it out on people who are defenseless. 

Much more in the longread.