Blog Archives
2016: A year for action
We’re almost a month into 2016 and autonomous weapons systems have already been in the news thanks to a strong panel discussion at the World Economic Forum in Davos. The Campaign to Stop Killer Robots was pleased to see the panel agree that the world needs a diplomatic process to pre-emptively ban autonomous weapons systems started soon. You can read the whole analysis by the Campaign’s coordinator here.
Yes 2016 is starting on a high note for the campaign but this is not the time to be complacent. We need to keep that momentum going internationally and here in Canada. The new government has yet to share a national policy on autonomous weapons systems. Before the election, the Liberal Party of Canada wrote that:
“Emerging technologies such as Lethal Autonomous Weapon Systems pose new and serious ethical questions that must be studied and understood. The Liberal Party of Canada will work with experts and civil society to ensure that the Canadian Government develops appropriate policies to address the use and proliferation of autonomous weapon systems.”
Now that the Liberals form the government, they will have to develop “appropriate policies” soon because the international community is moving forward, albeit verrrrrry slowly. States are meeting in April 2016 for a third (and hopefully final) informal experts meeting on autonomous weapons systems under the United Nations’ Convention on Conventional Weapons and then at the end of the year, states will have the opportunity to start negotiations on a pre-emptive ban. The UN process has been called “glacial” and that it “shows no sense of urgency” but there’s time for states to pick up the pace and Canada can take a leadership role.
Canadian industry, academics and NGOs have already taken a leadership role on banning autonomous weapon systems so now it’s the government’s turn. The Canadian government and Prime Minister Trudeau made a big impression at the World Economic Forum so we hope that they will take that energy forward to act on one of newist issues discussed there. Let’s make 2016 a year of action on autonomous weapons systems.
Video Contest Winner Announced
Next week, states will decide if and how they will continue international talks on autonomous weapons systems at the UN`s Convention on Conventional Weapons in Geneva. We and the whole Campaign to Stop Killer Robots are calling on states to take the next step towards a ban by agreeing to a Group of Governmental Experts (GGE) in 2016. A GGE will allow states to explore the issues surrounding autonomous weapons systems in depth.
With such an important decision looming over states, we are launching the winners of our youth video contest. Last week, we shared the runner-up video.
Today, we are pleased to announce that Steven Hause of Florida State University won the video contest. Steven’s video covers a number of the key concerns the Campaign has about autonomous weapons systems. We hope that this video will remind governments of the need to take action at CCW next week.
Video Contest – Runner Up Announced
In less than two weeks, states will decide if and how they will continue international talks on autonomous weapons systems at the UN`s Convention on Conventional Weapons in Geneva. We and the whole Campaign to Stop Killer Robots are calling on states to take the next step towards a ban by agreeing to a Group of Governmental Experts.
With such an important decision looming over states, we are launching the winners of our youth video contest. This week, we are pleased to present the runner-up video (and top high school video) by Daryl, Henry, Joseph and Anders at Petersburg High School.
Please feel free to share widely!
We thank all those who submitted videos to the contest and congratulate Daryl, Henry, Joseph and Anders on their excellent video. Come back next week to see the winning entry.
Mind Over Machine: Why Human Soldiers are (and Will Remain) Better than Killer Robots
Guest post by MAC Research Associate, Andrew Luth
This summer, movie-goers are flocking to theatres to see tales of superheroes, dinosaurs, and plucky college singing groups. Two of the season’s biggest movies, Avengers: Age of Ultron and Terminator Genisys have more in common than an over-reliance on computer-generated visual effects. Both feature killer robots: advanced weapons systems capable of fighting and killing independent of human command. Killer robots have been a staple of popcorn flicks for decades, but these days movies aren’t the only place we can expect to see them turning up. Many of the world’s most advanced militaries are getting closer and closer to producing killer robots of their own. Killer robots or autonomous weapons systems (AWS) are machines capable of identifying and attacking targets without human intervention. Despite the moral and legal concerns about such weapons, leading scientists and engineers are warning that AWS may be only a few years away from reality. The few who support the development of AWS tend to view them as inherently superior to human soldiers. Robots, they argue, don’t get tired or emotional, and are more expendable than human soldiers. As University of Massachusetts-Amherst Professor Charli Carpenter explains, some supporters have even gone so far as to say that “robots won’t rape,” overlooking the reality that rape and other war crimes are often ordered military tactics. All such arguments assume AWS will make better soldiers than humans. However, they fail to fully consider how human soldiers are actually superior to AWS. Several attributes of human physiology and behaviour give human soldiers the edge over autonomous weapons systems not just now, but for the foreseeable future.
According to the international legal principle of distinction, belligerent parties must distinguish between civilians and combatants when using force in combat. Human soldiers have a significant advantage over artificial systems in meeting this requirement. The human brain and eye work in tandem to process complex visual information incredibly quickly and efficiently. This skill is invaluable on the battlefield, enabling soldiers to pick out subtle distinctions in shape, colour, texture, and movement from long distances and use that information to their advantage. Technology is developing quickly and it is conceivable that computers will someday rival our visual processing powers, but no computer program has yet come close to human abilities to pick out patterns and identify objects even in motionless two dimensional images. Even further out of the realm of possibility for robotics is the brain’s aptitude for reading human behaviour. The human mind is particularly attuned to reading tiny changes in expression and body language even subconsciously. This is immensely important in combat scenarios, where soldiers need to determine an unknown party’s intent almost instantly, with fractions of a second making the difference between life and death. The science of computer vision is advancing rapidly, but it is likely to be decades before AWS can even approach the visual acuity of human soldiers, if ever.
Even if scientists eventually develop autonomous weapons systems with visual processing skills superior to our own, a human soldier would still have many advantages over killer robots. The highly flexible and adaptive nature of the human mind is perhaps the most distinct advantage. This flexibility allows us to receive and process information both from our natural senses and external sources. In addition to acquiring information by communicating with other soldiers, humans can quickly learn to integrate data from radar, night vision, infrared, and other technologies. Furthermore, to analyze this information human soldiers draw on a wealth of learning and experience from all areas of life. Robots, however, are generally designed to analyze specific information sources using pre-determined metrics, making it impossible for them to evaluate or even to detect unanticipated information. In many situations, the success of a mission could balance on the ability to respond to such information.
The human mind’s flexibility also means soldiers can perform any number of activities a situation requires. This is invaluable during military conflict. In his famous work The Art of War, Chinese military strategist Sun Tzu explains “just as water retains no constant shape, so in warfare there are no constant conditions.” Truly successful military tactics, he writes, are “regulated by the infinite variety of circumstances.” Humans are well-equipped to respond to this infinite variety. A modern infantry soldier can fire a rifle accurately, provide emergency medical aid, accept a prisoner’s surrender, operate a vehicle, assess enemy tactics, and perform any number of other necessary tasks. Robots however are specialists, designed to respond to a specific scenario or perform a single task, often in controlled environments. In his recent piece on killer robots for Just Security, retired Canadian military officer John MacBride quotes famed German military theorist Helmuth von Moltke’s observation that “no operation extends with any certainty beyond the first encounter with the main body of the enemy.” When a mission’s parameters change quickly, human minds learn and adapt, developing creative solutions to novel problems. However, when robots meet unanticipated challenges, they often fail spectacularly, necessitating significant human intervention. As MacBride explains, this is distinct cause for concern. There are bound to be programming flaws and oversights when a machine developed years in advance under controlled conditions makes its debut on a battlefield. IBM’s famed computing system Watson illustrated this perfectly during its star turn on the television game show Jeopardy!. Despite its dominant win over two human champions, in response to a question in the Final Jeopardy category of US Cities, Watson answered ‘Toronto’. Such failure is humourous in a game show setting, but the consequences of a similar error on the battlefield could be deadly.
In spite of Watson’s amazing performance, its failures demonstrate that neither human beings nor technological systems can be perfect. Whether out of fatigue, emotion, prejudice, or simple lack of information, human soldiers can and do make poor decisions. When these mistakes result in the deaths of fellow soldiers or innocent civilians, judicial systems are in place to hold military personnel accountable for their unethical behaviour or poor judgement. If AWS are deployed it is inevitable they too will perpetrate atrocities, whether from programming error, technical failure, or unpredictable variables. However, our society has no recourse for crimes committed by robots. Our justice system rests upon punishing immoral acts, but an autonomous weapons system has about as much sense of right and wrong as a toaster. Robots lack the capacity to make ethical decisions, acting only as their programming dictates. Nonetheless, a crime perpetrated by a robot is still a crime. Should society therefore pursue justice with the programmer? The commander? Or would leaders deem certain levels of ‘collateral damage’ acceptable and overlook any atrocities perpetrated by an AWS?
Our respect for the capacity of others to make moral choices is one among many reasons we value human life so highly. As such, the supporters of autonomous weapons systems often claim the best argument for AWS adoption is the potential they have to reduce human casualties. This assertion is tenuous at best. Given that autonomous weapons systems would already require remote oversight and operation capabilities, it would be a simple matter of procedure to give human operators final approval over the use of lethal force on a given target. It is unlikely fully ceding authority over weapons systems to computers would do anything to make military personnel safer. In fact, AWS might actually increase the likelihood of military engagement. Operating an AWS is far cheaper than training and deploying a human soldier, making them relatively expendable. Having access to relatively cheap and easily-replaced military assets significantly lowers the political and financial costs of military action, making states more likely to wage war in the first place. We have already witnessed the advent of this trend with the proliferation of unmanned military drones. Drone technology now allows leaders to conduct military campaigns abroad while their citizens pay little attention. Autonomous weapons systems could take this trend to its extreme, with robots conducting foreign bombing raids or assassinations with little human involvement. Protecting military personnel is a worthy goal, but our aversion to the human cost of war is the reason we place such high value on peace in the first place. Each tragic loss of life compels a society to consider the worthiness of its cause. Sending robots to do the killing externalizes the horrific consequences of war, making governments more willing to wage wars and less concerned with ending them.
We live in a world that sometimes forces us to take human lives. For thousands of years, some of humanity’s greatest minds have worked to develop philosophical and ethical frameworks to guide our decisions in war. Recently however, it has been difficult for us to keep pace with technology’s rapid proliferation. As technology revolutionizes all aspects of society, we can scarcely consider the social and ethical consequences of each new development before it arrives. The advent of nuclear weapons, the internet and countless other scientific advances all bear witness to our ethical tardiness. Although scientists are now making huge breakthroughs in robotics and artificial intelligence, no matter how skilled robots become at distinguishing between targets, we owe it to ourselves and all of humanity to fully consider each decision to use deadly force. Passing this choice off to an amoral machine would be unethical by definition. We currently live in a world where killer robots appear only in movies and other works of fiction, but it may not be long before they make the jump from movie screens to the real world. The international community must take action and ban these immoral weapons before they become a reality.
After graduating from Calvin College in Grand Rapids, Michigan, Andrew Luth spent two years living and working in China. He is currently pursuing his master’s degree at Carleton University’s Norman Paterson School of International Affairs in Ottawa, Canada. His academic interests include disarmament, conflict analysis and resolution, and the Asia-Pacific region.
Guest Post – Killer Robots in Geneva: Through the Ottawa Looking Glass
After the last informal meeting of experts in Geneva on killer robots (or as they prefer to call them “lethal autonomous weapon systems”) wrapped up it is an appropriate time to take stock of what we learned from the conference. A lot of ground was covered in Geneva, too much to cover in one short blog post, but there were a few ideas that received a lot of attention that are worth mentioning here.
First and foremost the idea of ‘meaningful human control’ got a lot of attention from all sides in the debate. So what is meaningful human control and how does that impact the debate on killer robots? Simply put, meaningful human control means that a human will always be the one that makes the decision whether or not to use force. There are three ways in which these systems are often described: human ‘in the loop’, human ‘on the loop’ and human ‘out of the loop’. A system with humans ‘out of the loop’ is the type of system that can target and use force without any human control and is the type of system that the Campaign to Stop Killer Robots seeks to ban. Systems with humans ‘on the loop’ give humans the ability to monitor the activity of the weapon and stop it if necessary. However, these systems may not furnish the decision maker with enough time to assess the information reported by the weapon. Finally, systems with humans ‘in the loop’ are more akin to traditional weapon systems, where the decision to use force rests firmly with a human operator.
The discussion of meaningful human control was linked to discussions about whether or not it was ethical or moral to delegate life and death decisions to machines. Some criticize this approach on the basis that meaningful human control isn’t a legal standard, or is too vague, but that criticism misses the point. This moral and ethical consideration is at the heart of the debate on killer robots; if only strict legal standards were applied then the ability and function of the technology would begin to determine how it is used. Strictly applying legal standards may approve the use of killer robots in areas that seemingly have no impact on civilians such as in outer space. Once such a precedent was set it would be difficult to stop the full use of killer robots.
After meaningful human control, the arguments made against a pre-emptive ban on killer robots formed a consistent theme throughout the conference, no matter the specific subject at hand. The refrain goes something like this, “We don’t know how this technology will evolve, so a pre-emptive ban could deprive the world of potentially useful technologies”. There is a concrete example of this not happening (the ban on blinding laser weapons), and various other treaties with dual-use implications have proven that banning a class of weapon does not adversely impact commercial or industrial activity. The Chemical Weapons Convention, which was discussed, provides a good example of how an export-control regime and competent verification can stop the spread of chemical weapons, while maintaining the ability of states to develop chemical industries.
Clearly then, neither of these two things should stop us from a pre-emptive ban on killer robots. As a co-founder of the Campaign to Stop Killer Robots, Mines Action Canada encourages all of you to engage with the issue and to advocate for a ban with your friends, family, local politician and anyone else who wants to listen. An easy way to start would be signing and sharing our petition to Keep Killer Robots Fiction here: /.
Michael Binnington is a M.A. Candidate at Norman Paterson School of International Affairs and a Research Associate at Mines Action Canada.
Video Contest Launched
**CONTEST IS NOW INTERNATIONAL – STUDENTS FROM AROUND THE WORLD WELCOME TO ENTER**
Mines Action Canada is launching a Keep Killer Robots Fiction video contest for students. We are inviting students from across Canada around the world to make and submit 2 minute video on the theme of “Keep Killer Robots Fiction“.
What is the purpose? The purpose of this competition is to find new, compelling and provocative ways to start a conversation in the public about autonomous weapons systems. Autonomous weapons systems or killer robots are future weapons that can select and fire upon targets without human control.
Killer robots have been a staple trope in fiction and entertainment for years. Over the past decade, the possibility of fully autonomous weapons is becoming closer to reality. Recently we have seen a dramatic rise in unmanned weapons that has changed the face of warfare. New technology is permitting serious efforts to develop fully autonomous weapons. These robotic weapons would be able to choose and fire on targets on their own, without any human intervention. This capability would pose a fundamental challenge to the protection of civilians and to compliance with international human rights and humanitarian law. For clarity it is necessary to note that fully autonomous weapons are not drones; drones have a human pilot in a remote location. Fully autonomous weapons are a large step beyond armed drones. You can learn more about autonomous weapons systems online at: www.stopkillerrobots.ca.
Your submission should illustrate one of the major problems with autonomous weapons systems or ask a question about handing over life and death decisions to a machine:
- A lack of accountability – who is responsible if an autonomous weapon kills the wrong person or malfunctions?
- Inability to distinguish between legitimate and legal targets and others – human soldiers must be able to tell the difference between soldiers and civilians, could a robot ever make that distinction?
- The moral issues surrounding outsourcing life and death decisions to machines – is it right to allow machines to choose to end a human life?
Please don’t limit yourself to these example questions about autonomous weapons, they are intended to inspire you to create some questions of your own to guide your project.
What do I win? The winner or winning team will receive a Go-Pro HERO 4 Silver camera and a 3-way mount.
Who can participate? Submissions will be accepted from any contestant between the ages of 18 and 30 who is currently enrolled in post-secondary education.
How do I enter the competition? Submitting your entry to the video contest is easy! Simply complete these three steps by March 15, 2015:
- Visit the contest entry form on our website, and fill in all of the required information.
- Upload your video to Vimeo and specify the location (URL) on the entry form. Memberships to Vimeo are free.
- Submit your online entry form to the Mines Action Canada team.
The Contest Rules and other information can be found in the Video Contest Announcement. Please read the announcement carefully to ensure that your project is eligible for consideration by our panel of expert judges. The contest entry form is available online at: http://goo.gl/forms/0VOGD6mgTp.
A New Mandate and a New Video
Today at the UN in Geneva, states approved a new mandate for further discussions about autonomous weapons systems in 2015. To celebrate, we are pleased to share our new video on why we need to Keep Killer Robots Fiction.
Remember, students you can make your own killer robots video in our film contest.
New Video on Human Control
This week states are meeting at the United Nations in Geneva to decide if discussion on lethal autonomous weapons systems will continue at the Convention on Conventional Weapons. States should continue to discuss this issue and to debate key problems with autonomous weapons systems. One of the key problems is the issue of human control. Learn more with this new video.
New Video on Proportionality
Proportionality is a key term for international humanitarian law that means any collateral damage must be proportional to the military gain from any action. We are very concerned that fully autonomous weapons systems or killer robots won’t be able to weigh proportionality.
Canadian Robotics Firm Takes Stand against Killer Robots
Great news! Today, Clearpath Robotics, a robotics firm based in Kitchener, Ontario, announced a world leading policy to “not manufacture weaponized robots that remove humans from the loop” and pledged their support for the Campaign to Stop Killer Robots. In an open letter, Ryan Gariepy, Co-Founder and CTO, writes that “[d]espite our continued involvement with Canadian and international military research and development, Clearpath Robotics believes that the development of killer robots is unwise, unethical, and should be banned on an international scale.”
As a co-founder and the Canadian representatives of the Campaign to Stop Killer Robots, Mines Action Canada welcomes Clearpath Robotics’ decision and applauds their staff for their thoughtful and courageous stance on this issue. “Clearpath Robotics has set the ethical standard for robotics companies around the world. Their pledge to not manufacture autonomous weapons systems demonstrates clearly that research and development into autonomous robots and military robots does not require the creation of ‘killer robots’ and that there are many applications of autonomous robotics that can benefit humanity,” said Paul Hannon, Executive Director, Mines Action Canada. “As Canadian, I am proud that a Canadian company was the first in the world to pledge to not manufacture killer robots.”
As the international community is scheduled to discuss autonomous weapons systems at the United Nations again this fall, Mines Action Canada strongly supports Clearpath Robotics’ pledge and we join them in encouraging “those who might see business opportunities in this technology to seek other ways to apply their skills and resources for the betterment of humankind.” We look forward to similar statements from other robotics companies in Canada and around the world. Members of the public who share Clearpath Robotics’ views can sign the Keep Killer Robots Fiction petition at /KRpetition while individual roboticists and scientists can join the International Committee for Robot Arms Control’s Scientists’ Call online at: http://icrac.net/call/.