2016: A year for action
We’re almost a month into 2016 and autonomous weapons systems have already been in the news thanks to a strong panel discussion at the World Economic Forum in Davos. The Campaign to Stop Killer Robots was pleased to see the panel agree that the world needs a diplomatic process to pre-emptively ban autonomous weapons systems started soon. You can read the whole analysis by the Campaign’s coordinator here.
Yes 2016 is starting on a high note for the campaign but this is not the time to be complacent. We need to keep that momentum going internationally and here in Canada. The new government has yet to share a national policy on autonomous weapons systems. Before the election, the Liberal Party of Canada wrote that:
“Emerging technologies such as Lethal Autonomous Weapon Systems pose new and serious ethical questions that must be studied and understood. The Liberal Party of Canada will work with experts and civil society to ensure that the Canadian Government develops appropriate policies to address the use and proliferation of autonomous weapon systems.”
Now that the Liberals form the government, they will have to develop “appropriate policies” soon because the international community is moving forward, albeit verrrrrry slowly. States are meeting in April 2016 for a third (and hopefully final) informal experts meeting on autonomous weapons systems under the United Nations’ Convention on Conventional Weapons and then at the end of the year, states will have the opportunity to start negotiations on a pre-emptive ban. The UN process has been called “glacial” and that it “shows no sense of urgency” but there’s time for states to pick up the pace and Canada can take a leadership role.
Canadian industry, academics and NGOs have already taken a leadership role on banning autonomous weapon systems so now it’s the government’s turn. The Canadian government and Prime Minister Trudeau made a big impression at the World Economic Forum so we hope that they will take that energy forward to act on one of newist issues discussed there. Let’s make 2016 a year of action on autonomous weapons systems.
Video Contest Winner Announced
Next week, states will decide if and how they will continue international talks on autonomous weapons systems at the UN`s Convention on Conventional Weapons in Geneva. We and the whole Campaign to Stop Killer Robots are calling on states to take the next step towards a ban by agreeing to a Group of Governmental Experts (GGE) in 2016. A GGE will allow states to explore the issues surrounding autonomous weapons systems in depth.
With such an important decision looming over states, we are launching the winners of our youth video contest. Last week, we shared the runner-up video.
Today, we are pleased to announce that Steven Hause of Florida State University won the video contest. Steven’s video covers a number of the key concerns the Campaign has about autonomous weapons systems. We hope that this video will remind governments of the need to take action at CCW next week.
Video Contest – Runner Up Announced
In less than two weeks, states will decide if and how they will continue international talks on autonomous weapons systems at the UN`s Convention on Conventional Weapons in Geneva. We and the whole Campaign to Stop Killer Robots are calling on states to take the next step towards a ban by agreeing to a Group of Governmental Experts.
With such an important decision looming over states, we are launching the winners of our youth video contest. This week, we are pleased to present the runner-up video (and top high school video) by Daryl, Henry, Joseph and Anders at Petersburg High School.
Please feel free to share widely!
We thank all those who submitted videos to the contest and congratulate Daryl, Henry, Joseph and Anders on their excellent video. Come back next week to see the winning entry.
Mind Over Machine: Why Human Soldiers are (and Will Remain) Better than Killer Robots
Guest post by MAC Research Associate, Andrew Luth
This summer, movie-goers are flocking to theatres to see tales of superheroes, dinosaurs, and plucky college singing groups. Two of the season’s biggest movies, Avengers: Age of Ultron and Terminator Genisys have more in common than an over-reliance on computer-generated visual effects. Both feature killer robots: advanced weapons systems capable of fighting and killing independent of human command. Killer robots have been a staple of popcorn flicks for decades, but these days movies aren’t the only place we can expect to see them turning up. Many of the world’s most advanced militaries are getting closer and closer to producing killer robots of their own. Killer robots or autonomous weapons systems (AWS) are machines capable of identifying and attacking targets without human intervention. Despite the moral and legal concerns about such weapons, leading scientists and engineers are warning that AWS may be only a few years away from reality. The few who support the development of AWS tend to view them as inherently superior to human soldiers. Robots, they argue, don’t get tired or emotional, and are more expendable than human soldiers. As University of Massachusetts-Amherst Professor Charli Carpenter explains, some supporters have even gone so far as to say that “robots won’t rape,” overlooking the reality that rape and other war crimes are often ordered military tactics. All such arguments assume AWS will make better soldiers than humans. However, they fail to fully consider how human soldiers are actually superior to AWS. Several attributes of human physiology and behaviour give human soldiers the edge over autonomous weapons systems not just now, but for the foreseeable future.
According to the international legal principle of distinction, belligerent parties must distinguish between civilians and combatants when using force in combat. Human soldiers have a significant advantage over artificial systems in meeting this requirement. The human brain and eye work in tandem to process complex visual information incredibly quickly and efficiently. This skill is invaluable on the battlefield, enabling soldiers to pick out subtle distinctions in shape, colour, texture, and movement from long distances and use that information to their advantage. Technology is developing quickly and it is conceivable that computers will someday rival our visual processing powers, but no computer program has yet come close to human abilities to pick out patterns and identify objects even in motionless two dimensional images. Even further out of the realm of possibility for robotics is the brain’s aptitude for reading human behaviour. The human mind is particularly attuned to reading tiny changes in expression and body language even subconsciously. This is immensely important in combat scenarios, where soldiers need to determine an unknown party’s intent almost instantly, with fractions of a second making the difference between life and death. The science of computer vision is advancing rapidly, but it is likely to be decades before AWS can even approach the visual acuity of human soldiers, if ever.
Even if scientists eventually develop autonomous weapons systems with visual processing skills superior to our own, a human soldier would still have many advantages over killer robots. The highly flexible and adaptive nature of the human mind is perhaps the most distinct advantage. This flexibility allows us to receive and process information both from our natural senses and external sources. In addition to acquiring information by communicating with other soldiers, humans can quickly learn to integrate data from radar, night vision, infrared, and other technologies. Furthermore, to analyze this information human soldiers draw on a wealth of learning and experience from all areas of life. Robots, however, are generally designed to analyze specific information sources using pre-determined metrics, making it impossible for them to evaluate or even to detect unanticipated information. In many situations, the success of a mission could balance on the ability to respond to such information.
The human mind’s flexibility also means soldiers can perform any number of activities a situation requires. This is invaluable during military conflict. In his famous work The Art of War, Chinese military strategist Sun Tzu explains “just as water retains no constant shape, so in warfare there are no constant conditions.” Truly successful military tactics, he writes, are “regulated by the infinite variety of circumstances.” Humans are well-equipped to respond to this infinite variety. A modern infantry soldier can fire a rifle accurately, provide emergency medical aid, accept a prisoner’s surrender, operate a vehicle, assess enemy tactics, and perform any number of other necessary tasks. Robots however are specialists, designed to respond to a specific scenario or perform a single task, often in controlled environments. In his recent piece on killer robots for Just Security, retired Canadian military officer John MacBride quotes famed German military theorist Helmuth von Moltke’s observation that “no operation extends with any certainty beyond the first encounter with the main body of the enemy.” When a mission’s parameters change quickly, human minds learn and adapt, developing creative solutions to novel problems. However, when robots meet unanticipated challenges, they often fail spectacularly, necessitating significant human intervention. As MacBride explains, this is distinct cause for concern. There are bound to be programming flaws and oversights when a machine developed years in advance under controlled conditions makes its debut on a battlefield. IBM’s famed computing system Watson illustrated this perfectly during its star turn on the television game show Jeopardy!. Despite its dominant win over two human champions, in response to a question in the Final Jeopardy category of US Cities, Watson answered ‘Toronto’. Such failure is humourous in a game show setting, but the consequences of a similar error on the battlefield could be deadly.
In spite of Watson’s amazing performance, its failures demonstrate that neither human beings nor technological systems can be perfect. Whether out of fatigue, emotion, prejudice, or simple lack of information, human soldiers can and do make poor decisions. When these mistakes result in the deaths of fellow soldiers or innocent civilians, judicial systems are in place to hold military personnel accountable for their unethical behaviour or poor judgement. If AWS are deployed it is inevitable they too will perpetrate atrocities, whether from programming error, technical failure, or unpredictable variables. However, our society has no recourse for crimes committed by robots. Our justice system rests upon punishing immoral acts, but an autonomous weapons system has about as much sense of right and wrong as a toaster. Robots lack the capacity to make ethical decisions, acting only as their programming dictates. Nonetheless, a crime perpetrated by a robot is still a crime. Should society therefore pursue justice with the programmer? The commander? Or would leaders deem certain levels of ‘collateral damage’ acceptable and overlook any atrocities perpetrated by an AWS?
Our respect for the capacity of others to make moral choices is one among many reasons we value human life so highly. As such, the supporters of autonomous weapons systems often claim the best argument for AWS adoption is the potential they have to reduce human casualties. This assertion is tenuous at best. Given that autonomous weapons systems would already require remote oversight and operation capabilities, it would be a simple matter of procedure to give human operators final approval over the use of lethal force on a given target. It is unlikely fully ceding authority over weapons systems to computers would do anything to make military personnel safer. In fact, AWS might actually increase the likelihood of military engagement. Operating an AWS is far cheaper than training and deploying a human soldier, making them relatively expendable. Having access to relatively cheap and easily-replaced military assets significantly lowers the political and financial costs of military action, making states more likely to wage war in the first place. We have already witnessed the advent of this trend with the proliferation of unmanned military drones. Drone technology now allows leaders to conduct military campaigns abroad while their citizens pay little attention. Autonomous weapons systems could take this trend to its extreme, with robots conducting foreign bombing raids or assassinations with little human involvement. Protecting military personnel is a worthy goal, but our aversion to the human cost of war is the reason we place such high value on peace in the first place. Each tragic loss of life compels a society to consider the worthiness of its cause. Sending robots to do the killing externalizes the horrific consequences of war, making governments more willing to wage wars and less concerned with ending them.
We live in a world that sometimes forces us to take human lives. For thousands of years, some of humanity’s greatest minds have worked to develop philosophical and ethical frameworks to guide our decisions in war. Recently however, it has been difficult for us to keep pace with technology’s rapid proliferation. As technology revolutionizes all aspects of society, we can scarcely consider the social and ethical consequences of each new development before it arrives. The advent of nuclear weapons, the internet and countless other scientific advances all bear witness to our ethical tardiness. Although scientists are now making huge breakthroughs in robotics and artificial intelligence, no matter how skilled robots become at distinguishing between targets, we owe it to ourselves and all of humanity to fully consider each decision to use deadly force. Passing this choice off to an amoral machine would be unethical by definition. We currently live in a world where killer robots appear only in movies and other works of fiction, but it may not be long before they make the jump from movie screens to the real world. The international community must take action and ban these immoral weapons before they become a reality.
After graduating from Calvin College in Grand Rapids, Michigan, Andrew Luth spent two years living and working in China. He is currently pursuing his master’s degree at Carleton University’s Norman Paterson School of International Affairs in Ottawa, Canada. His academic interests include disarmament, conflict analysis and resolution, and the Asia-Pacific region.
Guest Post – Killer Robots in Geneva: Through the Ottawa Looking Glass
After the last informal meeting of experts in Geneva on killer robots (or as they prefer to call them “lethal autonomous weapon systems”) wrapped up it is an appropriate time to take stock of what we learned from the conference. A lot of ground was covered in Geneva, too much to cover in one short blog post, but there were a few ideas that received a lot of attention that are worth mentioning here.
First and foremost the idea of ‘meaningful human control’ got a lot of attention from all sides in the debate. So what is meaningful human control and how does that impact the debate on killer robots? Simply put, meaningful human control means that a human will always be the one that makes the decision whether or not to use force. There are three ways in which these systems are often described: human ‘in the loop’, human ‘on the loop’ and human ‘out of the loop’. A system with humans ‘out of the loop’ is the type of system that can target and use force without any human control and is the type of system that the Campaign to Stop Killer Robots seeks to ban. Systems with humans ‘on the loop’ give humans the ability to monitor the activity of the weapon and stop it if necessary. However, these systems may not furnish the decision maker with enough time to assess the information reported by the weapon. Finally, systems with humans ‘in the loop’ are more akin to traditional weapon systems, where the decision to use force rests firmly with a human operator.
The discussion of meaningful human control was linked to discussions about whether or not it was ethical or moral to delegate life and death decisions to machines. Some criticize this approach on the basis that meaningful human control isn’t a legal standard, or is too vague, but that criticism misses the point. This moral and ethical consideration is at the heart of the debate on killer robots; if only strict legal standards were applied then the ability and function of the technology would begin to determine how it is used. Strictly applying legal standards may approve the use of killer robots in areas that seemingly have no impact on civilians such as in outer space. Once such a precedent was set it would be difficult to stop the full use of killer robots.
After meaningful human control, the arguments made against a pre-emptive ban on killer robots formed a consistent theme throughout the conference, no matter the specific subject at hand. The refrain goes something like this, “We don’t know how this technology will evolve, so a pre-emptive ban could deprive the world of potentially useful technologies”. There is a concrete example of this not happening (the ban on blinding laser weapons), and various other treaties with dual-use implications have proven that banning a class of weapon does not adversely impact commercial or industrial activity. The Chemical Weapons Convention, which was discussed, provides a good example of how an export-control regime and competent verification can stop the spread of chemical weapons, while maintaining the ability of states to develop chemical industries.
Clearly then, neither of these two things should stop us from a pre-emptive ban on killer robots. As a co-founder of the Campaign to Stop Killer Robots, Mines Action Canada encourages all of you to engage with the issue and to advocate for a ban with your friends, family, local politician and anyone else who wants to listen. An easy way to start would be signing and sharing our petition to Keep Killer Robots Fiction here: /.
Michael Binnington is a M.A. Candidate at Norman Paterson School of International Affairs and a Research Associate at Mines Action Canada.
CCW Closing Statement
Executive Director Paul Hannon delivered our closing statement at the Convention on Conventional Weapons today. Download the statement here or read it below.
The Way Forward
Thank you Mr. Chair and your team for the strong foundation to move forward with the urgency and focus this issue requires. This week we have seen wide-ranging discussions on autonomous weapons systems. The CCW does not often enough deal with issues of morality, human rights and ethics. We welcome all states who have asserted the necessity of maintaining meaningful human control over the use of force. These conversations should continue and deepen.
There is one issue we would like to raise as food for thought. At times during the week, we have felt that some have underestimated the skills, knowledge, intelligence, training, experience, humanity and morality that men and women in uniform combine with situational awareness and IHL to make decisions during conflict. We work closely with roboticists, engineers, and technical experts and despite their expertise and the high quality of their work we do not believe an algorithm could replicate this complex decision making process. Robotics should only be used to inform and supplement human decision making. To go further than that risks “dehumanizing those we expose to harm” as RCW’s CCW Report’s editorial stated yesterday.
Allow me to conclude with the assertion that the international response to the possibility of autonomous weapons systems must not be limited to transparency alone. The expert presentations and the debates this week have strengthened our belief that autonomous weapons systems are not a typical new weapon and our current IHL and weapons review processes will not be sufficient. A mandate for a group of governmental experts next year is an appropriate and obvious next step. We look forward to working with the high contracting parties to ensure that meaningful human control remains at the centre of all decisions to use violent force.
Let’s not throw the baby out with the bathwater!
Today at the Convention on Conventional Weapons meeting about lethal autonomous weapons systems, Mines Action Canada released a new memo to delegates on the impact of autonomous weapons systems on public trust in robotics. In this memo we discuss how the creation and use of autonomous weapons systems could change public perception of robotics more generally. Read the memo here and let us know what you think!
Will the use of killer robots make you more or less likely to want other autonomous robots in your life?
Updated Protocol IV Paper Released for Convention on Conventional Weapons Delegates
With the Convention on Conventional Weapons high contracting parties meeting this week to discuss lethal autonomous weapons systems, Mines Action Canada has released an updated memo to delegates on CCW Protocol IV which pre-emptively banned blinding laser weapons.
Please down load the Updated Protocol IV Memo.
Mines Action Canada’s Opening Statement at Convention on Conventional Weapons
Mines Action Canada delivered an opening statement at the Convention on Conventional this afternoon. The text of the statement is available online here.
Opening Statement – Convention on Conventional Weapons 13 April 2015
Thank you Mr.Chair. I appreciate the opportunity to speak on behalf of Mines Action Canada. Mines Action Canada is a Canadian disarmament organization that has been working to reduce the impact of indiscriminate weapons for over twenty years. For years we have worked with partners around the world including here at the CCW to respond to the global crisis caused by landmines and cluster munitions. We have seen that the international community can come together to respond to a humanitarian catastrophe and can create international humanitarian law to protect civilians often after the fact due to the changing nature of conflict and technological advances. However, we are here today in an attempt to look forward. We are looking at future weapons that will require new legal instruments to prevent future catastrophes. Throughout this week I hope we will keep my grandmother’s advice in mind: an ounce of prevention is worth a pound of cure.
As a co-founder of the Campaign to Stop Killer Robots, Mines Action Canada is very conscious of public opinion concerning autonomous weapons systems. Since last year’s discussions here at the CCW, opposition to autonomous weapons systems has grown in Canada. In addition to our member organizations, academics, parliamentarians, industry, faith communities and members of the general public have expressed concern about the potential humanitarian impacts of autonomous weapons systems. The widespread opposition to this technology indicates that there may be negative consequences for robotics more generally should autonomous weapons systems be used in armed conflict or in other circumstances. The erosion of public trust in robotic systems and autonomy as a result of the use of autonomous weapons systems could severely limit our ability to harness the good that robotics could do for humanity.
In addition to these concerns about the impact on public trust in robotics, we have numerous legal, moral, ethical, technical, military, political and humanitarian concerns about autonomous weapons systems which have led to the conclusion that new international humanitarian law is needed to ensure meaningful human control over these and other weapons. There is a moral imperative to consider the long term effects of the development and deployment of autonomous weapons systems on human society. Proponents of these technologies cite possible battlefield benefits and yet a discussion only dealing with short term or battlefield effects is not enough. We must ask the difficult questions: is it acceptable to cede decisions over life and death in conflict to machines? Who would be accountable for autonomous weapons systems? How can IHL adapt when new technology blurs the line between combatant and weapon?
IHL has demonstrated an ability to adapt and evolve to prevent the development and deployment of new and unnecessarily harmful technology. CCW Protocol IV banning blinding laser weapons is a good example which demonstrates that not only is there a need to add to IHL to address new technology, but also that we can prevent the development and use of weapons before their unacceptable humanitarian consequences create a catastrophe. We have published a memo to delegates which further explores the lessons learned from Protocol IV.
Autonomous weapons systems are not your average new weapon; they have the potential to fundamentally alter the nature of conflict. As a “game-changer” autonomous weapons deserve a serious and in-depth discussion. We hope that this week will see attempts to define meaningful human control and will foster a strong desire to pursue discussions towards a new legal instrument that places meaningful human control at the centre all decisions to use violent force.
Thank you.
Limited Time Only: Get the Keep Killer Robots Fiction T-shirt!
Mines Action Canada is offering the Keep Killer Robots Fiction t-shirt for a limited time only. Campaign supporters can purchase this Keep Killer Robots Fiction t-shirt through TeeSpring until March 30, 2015. The Keep Killer Robots Fiction t-shirt is available in three styles: unisex (S-5XL), women’s fitted (S-2XL) and children’s (S-XL). These t-shirts can be shipped almost anywhere in the world.
The proceeds from the sales of this limited edition t-shirt supports Mines Action Canada’s advocacy work on autonomous weapons systems. You can purchase your t-shirt by visiting TeeSpring’s safe and secure site: /keepkillerrobotsfiction.
The Keep Killer Robots Fiction t-shirt is available for a limited time only. Order before March 30 to ensure you don’t miss out!
Help us keep autonomous weapons systems off the battlefield and keep killer robots fiction!