Delivered by: Erin Hunt, Programme Coordinator
Thank you Mr. Chair. As a co-founder of the Campaign to Stop Killer Robots, Mines Action Canada is very conscious of public opinion and the public conversation concerning autonomous weapons systems. Recently, autonomous weapon systems have been in the news in Canada. Last week, over 200 Canadian Artificial Intelligence experts released an open letter to Prime Minister Justin Trudeau calling for Canadian leadership on the issue. The letter states [quote] “Lethal autonomous weapons systems that remove meaningful human control from determining the legitimacy of targets and deploying lethal force sit on the wrong side of a clear moral line.” [unquote]
Copies of this letter can be found at the back of the room. It is not only in Canada where the AI research community is speaking out – a similar letter was also released in Australia. As mentioned by my colleague, since the last time the CCW met a letter from over 100 founders of robotics and artificial intelligence companies calling for a preemptive ban on autonomous weapons was also released. Additional national letters are in the works.
These public letters show that concerns about possible negative impacts of a pre-emptive ban are misplaced as ICRAC made clear moments ago and what the research community is calling for is bold and decisive action.
Mines Action Canada appreciates the significant number of expert presentations we have had this week but we hope that states will take time to share their views substantially over the remaining days.
From states who say Article 36 review may be sufficient to deal with our concerns about autonomous weapons systems, we hope to hear how an Article 36 review would be able to assess bias in the data used in machine learning and how comportment with IHL would be ensured by systems that continue to learn after the review.
In light of persistent statements from some delegations that they are uncertain about what we are talking about here, we hope to hear states share their current understanding of autonomous weapons systems. Specific definitions are not needed at this stage but we believe there is more clarity and consensus on these questions than one may think.
We would like to hear more on next steps from states who are calling for a pre-emptive ban. Mines Action Canada would welcome concrete discussions on how to ensure that momentum is not lost on this issue. We lost a week of work in August but as I mentioned at the beginning of my statement, the public conversation about autonomous weapons continues to advance and the people at home expect us to make progress.
This week it is important to continue to build on the work done in the past and to ensure that further discussions take place in 2018. Administrative challenges do not lessen “the need to continue the codification and progressive development of the rules of international law applicable in armed conflict” that is reaffirmed in the Preamble of this Convention. The technology is rapidly advancing and so must our conversations here.
Mines Action Canada welcomes the letter calling for a ban on the weaponization of Artificial Intellegence (AI) from the Canadian AI research community which was sent to Prime Minister Justin Trudeau. This letter follows a number of international letters in recent years (from faith leaders, scientists, Nobel laureates, company founders and others) addressed either to the UN or the global community in support of actions to prevent the development of autonomous weapons.
“This letter is evidence that Canadian AI community wants to see leadership from Canada,” said Paul Hannon, Executive Director, Mines Action Canada. “Clearly Canada should become the 20th country to call for a pre-emptive ban on autonomous weapons and to lead a process to ensure that autonomous weapons systems never arrive on the battlefield.”
More than 200 AI researchers in Canada signed the open letter to the Prime Minister “calling on you and your government to make Canada the 20th country in the world to take a firm global stand against weaponizing AI. Lethal autonomous weapons systems that remove meaningful human control from determining the legitimacy of targets and deploying lethal force sit on the wrong side of a clear moral line.”
The letter goes further asking “Canada to announce its support for the call to ban lethal autonomous weapons systems at the upcoming United Nations Conference on the Convention on Certain Conventional Weapons (CCW). Canada should also commit to working with other states to conclude a new international agreement that achieves this objective.” One of the letter’s authors, Dr. Ian Kerr of the University of Ottawa wrote an op-ed in the Globe and Mail bringing the letter’s message to Canadians from coast to coast to coast.
Dr. Kerr notes, “it is not often that captains of industry, scientists and technologists call for prohibitions on innovation of any sort, let alone an outright ban. But the Canadian AI research community is clear: We must not permit AI to target and kill without meaningful human control. Playing Russian roulette with the lives of others can never be justified. The decision on whether to ban autonomous weapons goes to the core of our humanity.”
This letter has been released one week before the international community meets under the auspices of the CCW to discuss the issue of autonomous weapons systems. Mines Action Canada’s Programme Coordinator, Erin Hunt will be attending the meeting next week in Geneva. She said “in past discussions at the CCW, some states have expressed concern that a prohibition on autonomous weapons systems would have a negative impact on AI research more broadly. This letter and the similar one released by Australian AI experts show that those concerns are misplaced. The AI research community is calling for the opposite – bold and decisive action to prohibit autonomous weapons systems in order to support the development of AI that would benefit humanity.”
The Campaign to Stop Killer Robots is deeply disappointed that the Convention on Conventional Weapons (CCW) has cancelled a crucial week of formal discussions on fully autonomous weapons in August. This step was taken because of the failure of several states, most notably Brazil, to pay their assessed dues for the convention’s meetings.
“The collective failure of countries to find a solution to their financial woes doesn’t mean they can stop addressing concerns over weapons that would select and attack targets without further human intervention” said Mary Wareham of Human Rights Watch, coordinator of the Campaign to Stop Killer Robots. “If the CCW is unable to act, nations must find other ways to maintain the momentum toward a ban,” she said. “Countries that agree with the need to retain human control of weapons systems should move swiftly to adopt national policies and laws and to negotiate a new international treaty prohibiting fully autonomous weapons.”
The call for a preemptive ban on fully autonomous weapons has been endorsed by 19 countries and dozens more states have affirmed the need to retain human control over the selection of targets and use of force. This clearly indicates that they see a need to prevent the development of fully autonomous weapons. Last December, China became the first permanent member of the UN Security Council to find that new international law is required to regulate fully autonomous weapons.
The Campaign calls on Canada and all countries to urgently address the enormous humanitarian challenges posed by these weapons by endorsing the call for a ban. It is vital and urgent that all stakeholders work together to secure a new international treaty before these weapons are unleashed.
“Canada has a long history of taking action when the CCW is unable to move forward,” said Paul Hannon, Executive Director of Mines Action Canada, a co-founder of the Campaign to Stop Killer Robots. “We are calling on Canada to act now to ensure that there is always meaningful human control over weapons. The international community cannot let the work done thus far go to waste.”
The Campaign to Stop Killer Robots fundamentally objects to permitting machines to take a human life on the battlefield or in policing, border control, and other circumstances. It calls for a preemptive ban on fully autonomous weapons through new international law as well as through domestic legislation.
Following the launch of the Campaign to Stop Killer Robots and a debate in the Human Rights Council, countries agreed in November 2013 to begin discussing what they called lethal autonomous weapons systems at the Convention on Conventional Weapons at the United Nations in Geneva. The CCW is a framework treaty that prohibits or restricts certain weapons and its 1995 protocol on blinding lasers is an example of a weapon being preemptively banned before it was acquired or used.
Most of the CCW’s 124 high contracting parties participated in three meetings on lethal autonomous weapons systems in 2014-2016, in addition to UN agencies, the International Committee of the Red Cross, and the Campaign to Stop Killer Robots. Last December at their Fifth Review Conference CCW states decided to formalize and expand those deliberations by establishing a Group of Governmental Experts on lethal autonomous weapons systems to meet in August and November 2017, chaired by Ambassador Amandeep Singh Gill of India.
However, on 30 May, the CCW’s president-designate Ambassador Matthew Rowland of the UK announced that the Group of Governmental Experts meeting scheduled for 21-25 August has been cancelled due to a lack of funds. Previously Rowland issued several warnings that that the lack of payment of assessed financial contributions would mean the likely cancellation of CCW meetings planned for 2017.
Several countries have financial arrears from previous years, but according to the UN’s official summary, Brazil accounts for 86 percent of the outstanding contributions due to four core humanitarian disarmament treaties, including the CCW. Brazil last paid its assessed CCW contributions in 2010. The Campaign to Stop Killer Robots has appealed to Brazil to pay its outstanding contributions without delay and it challenges CCW states to achieve cost saving measures in other ways that do not require the cancellation of key meetings.
Several autonomous weapons systems with various degrees of human control are currently in use by high-tech militaries including CCW states the US, China, Israel, South Korea, Russia, and the UK. The concern is that low-cost sensors and advances in artificial intelligence are making it increasingly practical to design weapons systems that would target and attack without any meaningful human control. If the trend towards autonomy continues, humans will start to fade out of the decision-making loop, first retaining only a limited oversight role, and then no role at all.
Canada, France, UK, and the US supported establishing the CCW Group of Governmental Experts last December, but remain unambitious in their overall goals for the process by proposing a focus on sharing best practices and achieving greater transparency in the conduct of legal reviews of new weapons systems. Russia openly opposed the creation of a Group of Governmental Experts, but did not block multilateral consensus for establishing one.
Next week, states will decide if and how they will continue international talks on autonomous weapons systems at the UN`s Convention on Conventional Weapons in Geneva. We and the whole Campaign to Stop Killer Robots are calling on states to take the next step towards a ban by agreeing to a Group of Governmental Experts (GGE) in 2016. A GGE will allow states to explore the issues surrounding autonomous weapons systems in depth.
With such an important decision looming over states, we are launching the winners of our youth video contest. Last week, we shared the runner-up video.
Today, we are pleased to announce that Steven Hause of Florida State University won the video contest. Steven’s video covers a number of the key concerns the Campaign has about autonomous weapons systems. We hope that this video will remind governments of the need to take action at CCW next week.
Guest post by MAC Research Associate, Andrew Luth
This summer, movie-goers are flocking to theatres to see tales of superheroes, dinosaurs, and plucky college singing groups. Two of the season’s biggest movies, Avengers: Age of Ultron and Terminator Genisys have more in common than an over-reliance on computer-generated visual effects. Both feature killer robots: advanced weapons systems capable of fighting and killing independent of human command. Killer robots have been a staple of popcorn flicks for decades, but these days movies aren’t the only place we can expect to see them turning up. Many of the world’s most advanced militaries are getting closer and closer to producing killer robots of their own. Killer robots or autonomous weapons systems (AWS) are machines capable of identifying and attacking targets without human intervention. Despite the moral and legal concerns about such weapons, leading scientists and engineers are warning that AWS may be only a few years away from reality. The few who support the development of AWS tend to view them as inherently superior to human soldiers. Robots, they argue, don’t get tired or emotional, and are more expendable than human soldiers. As University of Massachusetts-Amherst Professor Charli Carpenter explains, some supporters have even gone so far as to say that “robots won’t rape,” overlooking the reality that rape and other war crimes are often ordered military tactics. All such arguments assume AWS will make better soldiers than humans. However, they fail to fully consider how human soldiers are actually superior to AWS. Several attributes of human physiology and behaviour give human soldiers the edge over autonomous weapons systems not just now, but for the foreseeable future.
According to the international legal principle of distinction, belligerent parties must distinguish between civilians and combatants when using force in combat. Human soldiers have a significant advantage over artificial systems in meeting this requirement. The human brain and eye work in tandem to process complex visual information incredibly quickly and efficiently. This skill is invaluable on the battlefield, enabling soldiers to pick out subtle distinctions in shape, colour, texture, and movement from long distances and use that information to their advantage. Technology is developing quickly and it is conceivable that computers will someday rival our visual processing powers, but no computer program has yet come close to human abilities to pick out patterns and identify objects even in motionless two dimensional images. Even further out of the realm of possibility for robotics is the brain’s aptitude for reading human behaviour. The human mind is particularly attuned to reading tiny changes in expression and body language even subconsciously. This is immensely important in combat scenarios, where soldiers need to determine an unknown party’s intent almost instantly, with fractions of a second making the difference between life and death. The science of computer vision is advancing rapidly, but it is likely to be decades before AWS can even approach the visual acuity of human soldiers, if ever.
Even if scientists eventually develop autonomous weapons systems with visual processing skills superior to our own, a human soldier would still have many advantages over killer robots. The highly flexible and adaptive nature of the human mind is perhaps the most distinct advantage. This flexibility allows us to receive and process information both from our natural senses and external sources. In addition to acquiring information by communicating with other soldiers, humans can quickly learn to integrate data from radar, night vision, infrared, and other technologies. Furthermore, to analyze this information human soldiers draw on a wealth of learning and experience from all areas of life. Robots, however, are generally designed to analyze specific information sources using pre-determined metrics, making it impossible for them to evaluate or even to detect unanticipated information. In many situations, the success of a mission could balance on the ability to respond to such information.
The human mind’s flexibility also means soldiers can perform any number of activities a situation requires. This is invaluable during military conflict. In his famous work The Art of War, Chinese military strategist Sun Tzu explains “just as water retains no constant shape, so in warfare there are no constant conditions.” Truly successful military tactics, he writes, are “regulated by the infinite variety of circumstances.” Humans are well-equipped to respond to this infinite variety. A modern infantry soldier can fire a rifle accurately, provide emergency medical aid, accept a prisoner’s surrender, operate a vehicle, assess enemy tactics, and perform any number of other necessary tasks. Robots however are specialists, designed to respond to a specific scenario or perform a single task, often in controlled environments. In his recent piece on killer robots for Just Security, retired Canadian military officer John MacBride quotes famed German military theorist Helmuth von Moltke’s observation that “no operation extends with any certainty beyond the first encounter with the main body of the enemy.” When a mission’s parameters change quickly, human minds learn and adapt, developing creative solutions to novel problems. However, when robots meet unanticipated challenges, they often fail spectacularly, necessitating significant human intervention. As MacBride explains, this is distinct cause for concern. There are bound to be programming flaws and oversights when a machine developed years in advance under controlled conditions makes its debut on a battlefield. IBM’s famed computing system Watson illustrated this perfectly during its star turn on the television game show Jeopardy!. Despite its dominant win over two human champions, in response to a question in the Final Jeopardy category of US Cities, Watson answered ‘Toronto’. Such failure is humourous in a game show setting, but the consequences of a similar error on the battlefield could be deadly.
In spite of Watson’s amazing performance, its failures demonstrate that neither human beings nor technological systems can be perfect. Whether out of fatigue, emotion, prejudice, or simple lack of information, human soldiers can and do make poor decisions. When these mistakes result in the deaths of fellow soldiers or innocent civilians, judicial systems are in place to hold military personnel accountable for their unethical behaviour or poor judgement. If AWS are deployed it is inevitable they too will perpetrate atrocities, whether from programming error, technical failure, or unpredictable variables. However, our society has no recourse for crimes committed by robots. Our justice system rests upon punishing immoral acts, but an autonomous weapons system has about as much sense of right and wrong as a toaster. Robots lack the capacity to make ethical decisions, acting only as their programming dictates. Nonetheless, a crime perpetrated by a robot is still a crime. Should society therefore pursue justice with the programmer? The commander? Or would leaders deem certain levels of ‘collateral damage’ acceptable and overlook any atrocities perpetrated by an AWS?
Our respect for the capacity of others to make moral choices is one among many reasons we value human life so highly. As such, the supporters of autonomous weapons systems often claim the best argument for AWS adoption is the potential they have to reduce human casualties. This assertion is tenuous at best. Given that autonomous weapons systems would already require remote oversight and operation capabilities, it would be a simple matter of procedure to give human operators final approval over the use of lethal force on a given target. It is unlikely fully ceding authority over weapons systems to computers would do anything to make military personnel safer. In fact, AWS might actually increase the likelihood of military engagement. Operating an AWS is far cheaper than training and deploying a human soldier, making them relatively expendable. Having access to relatively cheap and easily-replaced military assets significantly lowers the political and financial costs of military action, making states more likely to wage war in the first place. We have already witnessed the advent of this trend with the proliferation of unmanned military drones. Drone technology now allows leaders to conduct military campaigns abroad while their citizens pay little attention. Autonomous weapons systems could take this trend to its extreme, with robots conducting foreign bombing raids or assassinations with little human involvement. Protecting military personnel is a worthy goal, but our aversion to the human cost of war is the reason we place such high value on peace in the first place. Each tragic loss of life compels a society to consider the worthiness of its cause. Sending robots to do the killing externalizes the horrific consequences of war, making governments more willing to wage wars and less concerned with ending them.
We live in a world that sometimes forces us to take human lives. For thousands of years, some of humanity’s greatest minds have worked to develop philosophical and ethical frameworks to guide our decisions in war. Recently however, it has been difficult for us to keep pace with technology’s rapid proliferation. As technology revolutionizes all aspects of society, we can scarcely consider the social and ethical consequences of each new development before it arrives. The advent of nuclear weapons, the internet and countless other scientific advances all bear witness to our ethical tardiness. Although scientists are now making huge breakthroughs in robotics and artificial intelligence, no matter how skilled robots become at distinguishing between targets, we owe it to ourselves and all of humanity to fully consider each decision to use deadly force. Passing this choice off to an amoral machine would be unethical by definition. We currently live in a world where killer robots appear only in movies and other works of fiction, but it may not be long before they make the jump from movie screens to the real world. The international community must take action and ban these immoral weapons before they become a reality.
After graduating from Calvin College in Grand Rapids, Michigan, Andrew Luth spent two years living and working in China. He is currently pursuing his master’s degree at Carleton University’s Norman Paterson School of International Affairs in Ottawa, Canada. His academic interests include disarmament, conflict analysis and resolution, and the Asia-Pacific region.