Blog Archives
2016: A year for action
We’re almost a month into 2016 and autonomous weapons systems have already been in the news thanks to a strong panel discussion at the World Economic Forum in Davos. The Campaign to Stop Killer Robots was pleased to see the panel agree that the world needs a diplomatic process to pre-emptively ban autonomous weapons systems started soon. You can read the whole analysis by the Campaign’s coordinator here.
Yes 2016 is starting on a high note for the campaign but this is not the time to be complacent. We need to keep that momentum going internationally and here in Canada. The new government has yet to share a national policy on autonomous weapons systems. Before the election, the Liberal Party of Canada wrote that:
“Emerging technologies such as Lethal Autonomous Weapon Systems pose new and serious ethical questions that must be studied and understood. The Liberal Party of Canada will work with experts and civil society to ensure that the Canadian Government develops appropriate policies to address the use and proliferation of autonomous weapon systems.”
Now that the Liberals form the government, they will have to develop “appropriate policies” soon because the international community is moving forward, albeit verrrrrry slowly. States are meeting in April 2016 for a third (and hopefully final) informal experts meeting on autonomous weapons systems under the United Nations’ Convention on Conventional Weapons and then at the end of the year, states will have the opportunity to start negotiations on a pre-emptive ban. The UN process has been called “glacial” and that it “shows no sense of urgency” but there’s time for states to pick up the pace and Canada can take a leadership role.
Canadian industry, academics and NGOs have already taken a leadership role on banning autonomous weapon systems so now it’s the government’s turn. The Canadian government and Prime Minister Trudeau made a big impression at the World Economic Forum so we hope that they will take that energy forward to act on one of newist issues discussed there. Let’s make 2016 a year of action on autonomous weapons systems.
Mind Over Machine: Why Human Soldiers are (and Will Remain) Better than Killer Robots
Guest post by MAC Research Associate, Andrew Luth
This summer, movie-goers are flocking to theatres to see tales of superheroes, dinosaurs, and plucky college singing groups. Two of the season’s biggest movies, Avengers: Age of Ultron and Terminator Genisys have more in common than an over-reliance on computer-generated visual effects. Both feature killer robots: advanced weapons systems capable of fighting and killing independent of human command. Killer robots have been a staple of popcorn flicks for decades, but these days movies aren’t the only place we can expect to see them turning up. Many of the world’s most advanced militaries are getting closer and closer to producing killer robots of their own. Killer robots or autonomous weapons systems (AWS) are machines capable of identifying and attacking targets without human intervention. Despite the moral and legal concerns about such weapons, leading scientists and engineers are warning that AWS may be only a few years away from reality. The few who support the development of AWS tend to view them as inherently superior to human soldiers. Robots, they argue, don’t get tired or emotional, and are more expendable than human soldiers. As University of Massachusetts-Amherst Professor Charli Carpenter explains, some supporters have even gone so far as to say that “robots won’t rape,” overlooking the reality that rape and other war crimes are often ordered military tactics. All such arguments assume AWS will make better soldiers than humans. However, they fail to fully consider how human soldiers are actually superior to AWS. Several attributes of human physiology and behaviour give human soldiers the edge over autonomous weapons systems not just now, but for the foreseeable future.
According to the international legal principle of distinction, belligerent parties must distinguish between civilians and combatants when using force in combat. Human soldiers have a significant advantage over artificial systems in meeting this requirement. The human brain and eye work in tandem to process complex visual information incredibly quickly and efficiently. This skill is invaluable on the battlefield, enabling soldiers to pick out subtle distinctions in shape, colour, texture, and movement from long distances and use that information to their advantage. Technology is developing quickly and it is conceivable that computers will someday rival our visual processing powers, but no computer program has yet come close to human abilities to pick out patterns and identify objects even in motionless two dimensional images. Even further out of the realm of possibility for robotics is the brain’s aptitude for reading human behaviour. The human mind is particularly attuned to reading tiny changes in expression and body language even subconsciously. This is immensely important in combat scenarios, where soldiers need to determine an unknown party’s intent almost instantly, with fractions of a second making the difference between life and death. The science of computer vision is advancing rapidly, but it is likely to be decades before AWS can even approach the visual acuity of human soldiers, if ever.
Even if scientists eventually develop autonomous weapons systems with visual processing skills superior to our own, a human soldier would still have many advantages over killer robots. The highly flexible and adaptive nature of the human mind is perhaps the most distinct advantage. This flexibility allows us to receive and process information both from our natural senses and external sources. In addition to acquiring information by communicating with other soldiers, humans can quickly learn to integrate data from radar, night vision, infrared, and other technologies. Furthermore, to analyze this information human soldiers draw on a wealth of learning and experience from all areas of life. Robots, however, are generally designed to analyze specific information sources using pre-determined metrics, making it impossible for them to evaluate or even to detect unanticipated information. In many situations, the success of a mission could balance on the ability to respond to such information.
The human mind’s flexibility also means soldiers can perform any number of activities a situation requires. This is invaluable during military conflict. In his famous work The Art of War, Chinese military strategist Sun Tzu explains “just as water retains no constant shape, so in warfare there are no constant conditions.” Truly successful military tactics, he writes, are “regulated by the infinite variety of circumstances.” Humans are well-equipped to respond to this infinite variety. A modern infantry soldier can fire a rifle accurately, provide emergency medical aid, accept a prisoner’s surrender, operate a vehicle, assess enemy tactics, and perform any number of other necessary tasks. Robots however are specialists, designed to respond to a specific scenario or perform a single task, often in controlled environments. In his recent piece on killer robots for Just Security, retired Canadian military officer John MacBride quotes famed German military theorist Helmuth von Moltke’s observation that “no operation extends with any certainty beyond the first encounter with the main body of the enemy.” When a mission’s parameters change quickly, human minds learn and adapt, developing creative solutions to novel problems. However, when robots meet unanticipated challenges, they often fail spectacularly, necessitating significant human intervention. As MacBride explains, this is distinct cause for concern. There are bound to be programming flaws and oversights when a machine developed years in advance under controlled conditions makes its debut on a battlefield. IBM’s famed computing system Watson illustrated this perfectly during its star turn on the television game show Jeopardy!. Despite its dominant win over two human champions, in response to a question in the Final Jeopardy category of US Cities, Watson answered ‘Toronto’. Such failure is humourous in a game show setting, but the consequences of a similar error on the battlefield could be deadly.
In spite of Watson’s amazing performance, its failures demonstrate that neither human beings nor technological systems can be perfect. Whether out of fatigue, emotion, prejudice, or simple lack of information, human soldiers can and do make poor decisions. When these mistakes result in the deaths of fellow soldiers or innocent civilians, judicial systems are in place to hold military personnel accountable for their unethical behaviour or poor judgement. If AWS are deployed it is inevitable they too will perpetrate atrocities, whether from programming error, technical failure, or unpredictable variables. However, our society has no recourse for crimes committed by robots. Our justice system rests upon punishing immoral acts, but an autonomous weapons system has about as much sense of right and wrong as a toaster. Robots lack the capacity to make ethical decisions, acting only as their programming dictates. Nonetheless, a crime perpetrated by a robot is still a crime. Should society therefore pursue justice with the programmer? The commander? Or would leaders deem certain levels of ‘collateral damage’ acceptable and overlook any atrocities perpetrated by an AWS?
Our respect for the capacity of others to make moral choices is one among many reasons we value human life so highly. As such, the supporters of autonomous weapons systems often claim the best argument for AWS adoption is the potential they have to reduce human casualties. This assertion is tenuous at best. Given that autonomous weapons systems would already require remote oversight and operation capabilities, it would be a simple matter of procedure to give human operators final approval over the use of lethal force on a given target. It is unlikely fully ceding authority over weapons systems to computers would do anything to make military personnel safer. In fact, AWS might actually increase the likelihood of military engagement. Operating an AWS is far cheaper than training and deploying a human soldier, making them relatively expendable. Having access to relatively cheap and easily-replaced military assets significantly lowers the political and financial costs of military action, making states more likely to wage war in the first place. We have already witnessed the advent of this trend with the proliferation of unmanned military drones. Drone technology now allows leaders to conduct military campaigns abroad while their citizens pay little attention. Autonomous weapons systems could take this trend to its extreme, with robots conducting foreign bombing raids or assassinations with little human involvement. Protecting military personnel is a worthy goal, but our aversion to the human cost of war is the reason we place such high value on peace in the first place. Each tragic loss of life compels a society to consider the worthiness of its cause. Sending robots to do the killing externalizes the horrific consequences of war, making governments more willing to wage wars and less concerned with ending them.
We live in a world that sometimes forces us to take human lives. For thousands of years, some of humanity’s greatest minds have worked to develop philosophical and ethical frameworks to guide our decisions in war. Recently however, it has been difficult for us to keep pace with technology’s rapid proliferation. As technology revolutionizes all aspects of society, we can scarcely consider the social and ethical consequences of each new development before it arrives. The advent of nuclear weapons, the internet and countless other scientific advances all bear witness to our ethical tardiness. Although scientists are now making huge breakthroughs in robotics and artificial intelligence, no matter how skilled robots become at distinguishing between targets, we owe it to ourselves and all of humanity to fully consider each decision to use deadly force. Passing this choice off to an amoral machine would be unethical by definition. We currently live in a world where killer robots appear only in movies and other works of fiction, but it may not be long before they make the jump from movie screens to the real world. The international community must take action and ban these immoral weapons before they become a reality.
After graduating from Calvin College in Grand Rapids, Michigan, Andrew Luth spent two years living and working in China. He is currently pursuing his master’s degree at Carleton University’s Norman Paterson School of International Affairs in Ottawa, Canada. His academic interests include disarmament, conflict analysis and resolution, and the Asia-Pacific region.
Guest Post – Killer Robots in Geneva: Through the Ottawa Looking Glass
After the last informal meeting of experts in Geneva on killer robots (or as they prefer to call them “lethal autonomous weapon systems”) wrapped up it is an appropriate time to take stock of what we learned from the conference. A lot of ground was covered in Geneva, too much to cover in one short blog post, but there were a few ideas that received a lot of attention that are worth mentioning here.
First and foremost the idea of ‘meaningful human control’ got a lot of attention from all sides in the debate. So what is meaningful human control and how does that impact the debate on killer robots? Simply put, meaningful human control means that a human will always be the one that makes the decision whether or not to use force. There are three ways in which these systems are often described: human ‘in the loop’, human ‘on the loop’ and human ‘out of the loop’. A system with humans ‘out of the loop’ is the type of system that can target and use force without any human control and is the type of system that the Campaign to Stop Killer Robots seeks to ban. Systems with humans ‘on the loop’ give humans the ability to monitor the activity of the weapon and stop it if necessary. However, these systems may not furnish the decision maker with enough time to assess the information reported by the weapon. Finally, systems with humans ‘in the loop’ are more akin to traditional weapon systems, where the decision to use force rests firmly with a human operator.
The discussion of meaningful human control was linked to discussions about whether or not it was ethical or moral to delegate life and death decisions to machines. Some criticize this approach on the basis that meaningful human control isn’t a legal standard, or is too vague, but that criticism misses the point. This moral and ethical consideration is at the heart of the debate on killer robots; if only strict legal standards were applied then the ability and function of the technology would begin to determine how it is used. Strictly applying legal standards may approve the use of killer robots in areas that seemingly have no impact on civilians such as in outer space. Once such a precedent was set it would be difficult to stop the full use of killer robots.
After meaningful human control, the arguments made against a pre-emptive ban on killer robots formed a consistent theme throughout the conference, no matter the specific subject at hand. The refrain goes something like this, “We don’t know how this technology will evolve, so a pre-emptive ban could deprive the world of potentially useful technologies”. There is a concrete example of this not happening (the ban on blinding laser weapons), and various other treaties with dual-use implications have proven that banning a class of weapon does not adversely impact commercial or industrial activity. The Chemical Weapons Convention, which was discussed, provides a good example of how an export-control regime and competent verification can stop the spread of chemical weapons, while maintaining the ability of states to develop chemical industries.
Clearly then, neither of these two things should stop us from a pre-emptive ban on killer robots. As a co-founder of the Campaign to Stop Killer Robots, Mines Action Canada encourages all of you to engage with the issue and to advocate for a ban with your friends, family, local politician and anyone else who wants to listen. An easy way to start would be signing and sharing our petition to Keep Killer Robots Fiction here: /.
Michael Binnington is a M.A. Candidate at Norman Paterson School of International Affairs and a Research Associate at Mines Action Canada.
Video Contest Launched
**CONTEST IS NOW INTERNATIONAL – STUDENTS FROM AROUND THE WORLD WELCOME TO ENTER**
Mines Action Canada is launching a Keep Killer Robots Fiction video contest for students. We are inviting students from across Canada around the world to make and submit 2 minute video on the theme of “Keep Killer Robots Fiction“.
What is the purpose? The purpose of this competition is to find new, compelling and provocative ways to start a conversation in the public about autonomous weapons systems. Autonomous weapons systems or killer robots are future weapons that can select and fire upon targets without human control.
Killer robots have been a staple trope in fiction and entertainment for years. Over the past decade, the possibility of fully autonomous weapons is becoming closer to reality. Recently we have seen a dramatic rise in unmanned weapons that has changed the face of warfare. New technology is permitting serious efforts to develop fully autonomous weapons. These robotic weapons would be able to choose and fire on targets on their own, without any human intervention. This capability would pose a fundamental challenge to the protection of civilians and to compliance with international human rights and humanitarian law. For clarity it is necessary to note that fully autonomous weapons are not drones; drones have a human pilot in a remote location. Fully autonomous weapons are a large step beyond armed drones. You can learn more about autonomous weapons systems online at: www.stopkillerrobots.ca.
Your submission should illustrate one of the major problems with autonomous weapons systems or ask a question about handing over life and death decisions to a machine:
- A lack of accountability – who is responsible if an autonomous weapon kills the wrong person or malfunctions?
- Inability to distinguish between legitimate and legal targets and others – human soldiers must be able to tell the difference between soldiers and civilians, could a robot ever make that distinction?
- The moral issues surrounding outsourcing life and death decisions to machines – is it right to allow machines to choose to end a human life?
Please don’t limit yourself to these example questions about autonomous weapons, they are intended to inspire you to create some questions of your own to guide your project.
What do I win? The winner or winning team will receive a Go-Pro HERO 4 Silver camera and a 3-way mount.
Who can participate? Submissions will be accepted from any contestant between the ages of 18 and 30 who is currently enrolled in post-secondary education.
How do I enter the competition? Submitting your entry to the video contest is easy! Simply complete these three steps by March 15, 2015:
- Visit the contest entry form on our website, and fill in all of the required information.
- Upload your video to Vimeo and specify the location (URL) on the entry form. Memberships to Vimeo are free.
- Submit your online entry form to the Mines Action Canada team.
The Contest Rules and other information can be found in the Video Contest Announcement. Please read the announcement carefully to ensure that your project is eligible for consideration by our panel of expert judges. The contest entry form is available online at: http://goo.gl/forms/0VOGD6mgTp.
New Video on Proportionality
Proportionality is a key term for international humanitarian law that means any collateral damage must be proportional to the military gain from any action. We are very concerned that fully autonomous weapons systems or killer robots won’t be able to weigh proportionality.
Canadian Robotics Firm Takes Stand against Killer Robots
Great news! Today, Clearpath Robotics, a robotics firm based in Kitchener, Ontario, announced a world leading policy to “not manufacture weaponized robots that remove humans from the loop” and pledged their support for the Campaign to Stop Killer Robots. In an open letter, Ryan Gariepy, Co-Founder and CTO, writes that “[d]espite our continued involvement with Canadian and international military research and development, Clearpath Robotics believes that the development of killer robots is unwise, unethical, and should be banned on an international scale.”
As a co-founder and the Canadian representatives of the Campaign to Stop Killer Robots, Mines Action Canada welcomes Clearpath Robotics’ decision and applauds their staff for their thoughtful and courageous stance on this issue. “Clearpath Robotics has set the ethical standard for robotics companies around the world. Their pledge to not manufacture autonomous weapons systems demonstrates clearly that research and development into autonomous robots and military robots does not require the creation of ‘killer robots’ and that there are many applications of autonomous robotics that can benefit humanity,” said Paul Hannon, Executive Director, Mines Action Canada. “As Canadian, I am proud that a Canadian company was the first in the world to pledge to not manufacture killer robots.”
As the international community is scheduled to discuss autonomous weapons systems at the United Nations again this fall, Mines Action Canada strongly supports Clearpath Robotics’ pledge and we join them in encouraging “those who might see business opportunities in this technology to seek other ways to apply their skills and resources for the betterment of humankind.” We look forward to similar statements from other robotics companies in Canada and around the world. Members of the public who share Clearpath Robotics’ views can sign the Keep Killer Robots Fiction petition at /KRpetition while individual roboticists and scientists can join the International Committee for Robot Arms Control’s Scientists’ Call online at: http://icrac.net/call/.
Great Campaign Events in Canada!
Mines Action Canada and the Campaign to Stop Killer Robots hosted a number of events in Ottawa over the past two days to begin the discussion in Canada about autonomous weapons. We were pleased to have Mary Wareham, Global Coordinator of the Campaign to Stop Killer Robots and Advocacy Director, Arms Division at Human Rights Watch along with Peter Asaro, Professor at the New School and Vice-Chair of the International Committee for Robot Arms Control (ICRAC) join us and local expert Ian Kerr, in Ottawa for the events. Be sure to check out the great summary of the events on the Campaign’s website.
The two days started with an op-ed by Ian Kerr who holds the Canada Research Chair in Ethics, Law and Technology at the University of Ottawa and is a member of ICRAC.
On April 28th, we met with other peace, disarmament and development organizations to talk about the campaign and to begin to build a stronger civil society presence in Canada on this issue. There was a lot of a interest from our non-profit colleagues so we look forward to hearing more voices on this issue in the near future.
Later that day, we hosted a public event at Ottawa City Hall. There was a panel discussion with Peter, Paul, Mary and Ian followed by a rather lively Question and Answer session with the audience. The audience was generally quite supportive of the Campaign and our efforts to achieve a pre-emptive ban on autonomous weapons. Audience members with backgrounds in engineering, law, the military and politics all expressed concern about the development of killer robots.
The following morning, MAC hosted a breakfast briefing for parliamentarians and their staff, other NGOs and decision makers in Ottawa. The Bagels and ‘Bots breakfast was the first time some of these decision makers had heard of the issue and it seemed to strike a chord with many in the room. After breakfast, the team was off to Parliament Hill for a press conference. At the press conference and in MAC’s press release, campaigners called for Canadian leadership on this issue internationally and for Canada to be the first country in the world to declare a moratorium on the development and production of killer robots.
The media in Ottawa and across the country have taken quite an interest in these events. The Canadian Press story was picked up in newspapers across the country as well as national media outlets and there was an associated list of facts about killer robots. The Sun News Network, and Ottawa Citizen also covered the Campaign while MAC has received a number of radio interview requests. Paul Hannon, Executive Director, was on CKNW Morning News with Philip Till.
One very exciting result of these activities is that The Globe and Mail’s editorial team has come out in support of the Campaign to Stop Killer Robots and our call:
The world has a long banned some weapons deemed dangerous, indiscriminate or inhumane, including chemical weapons and land mines. Autonomous robot weapons carry all such risks, and add new ones to the list. They are not wielded remotely by humans, but are intended to operate without supervision. They’re about turning life and death decisions over to software. Canada should be a leading voice advocating for a global protocol limiting their development and use.
Also Jian Ghomenshi on CBC Radio’s Q called for Canadian leadership on killer robots, he says that leadership on this issue is something Canadians could be proud of and that it could be a legacy issue for Prime Minister Stephen Harper.
The Keep Killer Robots Fiction initiative is off to a great start. You can get involved by signing and sharing the petition at: /KRpetition.
Launch Event for Campaign to Stop Killer Robots in Canada
In Ottawa?
Come join us to launch the Campaign to Stop Killer Robots in Canada.
Join us at Ottawa City Hall for a panel discussion on fully autonomous weapons, led by Mines Action Canada and including guest speakers:
- Ian Kerr – Professor, University of Ottawa, and Canada Research Chair for Ethics, Law and Technology (Ottawa, ON)
- Mary Wareham – Advocacy Director – Arms Division, Human Rights Watch, and Global Coordinator for the Campaign to Stop Killer Robots (Washington, DC)
- Peter Asaro – The New School for Public Engagement and Co-Founder and Co-Chair of the International Committee for Robot Arms Control (New York, NY)
- Paul Hannon – Executive Director, Mines Action Canada (Ottawa, ON)
Where? : Ottawa City Hall – the Colonel By room
When? : April 28th, 7:00 pm (Doors at 6:45 pm)
Check out the Public Event Flyer for all the details.
Guest Post: Killer Robots in Canadian Context
By Matthew Taylor
There is nothing Canadian about machines that kill people without human control. Machines that have no conscience. Machines that have no compassion. Machines without the ability to distinguish between someone who is a genuine threat and someone in the wrong place at the wrong time.
We, as a people, have for many years sought to build a safer and more peaceful world. Former Prime Minister Brian Mulroney made Nelson Mandela and the end of apartheid in South Africa “the highest priority of the government of Canada in our foreign affairs.” Former Prime Minister Lester Pearson brought about modern peacekeeping in 1956. Former Foreign Affairs Minister Lloyd Axworthy gathered states in our nation’s capital to end the use of anti-personnel landmines around the world. These men understood that a desire for peace and justice is a basic Canadian value. That is not something a machine can ever understand.
This issue presents us as Canadians with an opportunity to share our values, and our vision for a safer world. Killer Robots are perhaps the most important international arms control issue to emerge since nuclear weapons were dropped on Hiroshima and Nagasaki. Nuclear weapons redefined how we understood and approached warfare. That is why it is so absolutely necessary for the world to confront the problem of killer robots before and not after they see action on the battlefield.
The costs of playing catch up are far too evident. Once weapons are employed, most countries will scramble to re-adjust for the change in balance in power. During World War I chemical weapons were used against Canadian soldiers causing blindness, death and unspeakable suffering. Nearly one hundred years later chemical weapons were being used in Syria causing death and significant harm to civilians. With thousands of casualties of chemical weapons in between, the difficulty of banning weapons once they have been put into use is quite evident.
History has shown that the support and leadership of our nation can bring about international change. We have a duty as moral entrepreneurs to prevent the horror of autonomous killing machines from ever becoming a reality.
In November 2013, states agreed to discuss the question of lethal autonomous robots in meetings of the Convention on Conventional Weapons in May, 2014. This umbrella agreement allows for 117 member states to consider issues of arms control.
But at the moment, the official Canadian government position on Killer Robots is unclear. A government statement in the February 2014 edition of L’actualite offers little insight. In the article, a Canadian Foreign Affairs spokesman indicated that Canada does not ban weapons that do not yet exist. But in fact, Canada has participated in a pre-emptive ban of weapons before.
In 1995, Canada was one of the original parties to Protocol IV of the Convention to Conventional Weapons. This international agreement banning blinding lasers was made in the very same forum in whichkiller robots are set to be discussed in May. This not only represents a step in the right direction but a precedent upon which to build.
If a pre-emptive ban has been done before, it can be done again. Whether a weapon exists yet or not should have no bearing on whether the technology should be illegal under international humanitarian law. What should matter is whether we as a people believe that these weapons can ever be considered to be humane. To me, and to many others, the answer to that question is clearly no.
If you feel that as Canadians we must take a stand, please join me in signing our petition to Keep Killer Robots Fiction.
Matthew Taylor is an intern at Mines Action Canada and is a Master of the Arts Candidate at the Norman Paterson School of International Affairs at Carleton University specializing in Intelligence and National Security.
States to work on Killer Robots
Today the States Parties to the Convention on Conventional Weapons (CCW) agreed to convene a meeting to discuss fully autonomous weapons or killer robots in May 2014. Mines Action Canada (MAC), a co-founder of the Campaign to Stop Killer Robots, welcomes this historic decision to begin to address this issue. MAC encourages all states to pursue an international ban on these weapons to ensure there will always be meaningful human control over targeting decisions and the use of violent force.
We are pleased that Canada made its first public statements on this topic during the CCW joining the other 43 nations who have spoken out on fully autonomous weapons since May. MAC looks forward to working with the Government of Canada to develop national policies on fully autonomous weapons. Along with our colleagues from the Campaign to Stop Killer Robots we hope to see Canada actively participate in the CCW discussions. Mines Action Canada encourages Canada to take on a leadership role in international efforts to ban fully autonomous weapons and ensure that humans will always have meaningful control over life and death decisions in conflict.
“If we have learned anything from the Canadian led efforts to ban landmines, it is that the world cannot afford to wait until there is a humanitarian crisis to act. We need a preemptive ban on fully autonomous weapons before they can cause a humanitarian disaster,” said Paul Hannon, Executive Director, Mines Action Canada.
For more details on the CCW mandate, please see the international web site for the Campaign to Stop Killer Robots.