Santa Fe Institute

Evolution and Restraint of Malicious Behavior in Complex Systems

Working Group

February 06, 2017 - February 07, 2017
9:00 AM
Collins Conference Room

Abstract.  Self-optimizing agents in many systems achieve competitive advantage by discovering how to exploit their environment and other agents, often maliciously subverting the operating rules of the system. For example, viruses and cancer exploit the reproductive mechanisms of host cells to replicate and spread; bullies use intimidation to exert power, violating social norms; and cybercriminals exploit software vulnerabilities, eroding trust in networked systems. As robotics and massively embedded computation become increasingly pervasive and autonomous, we can expect that evolution and technology will collide in the physical world. Autonomous vehicles and fleets of drones will revolutionize transportation and warfare, computers will be embedded in our home appliances, and medical implants will restore function to paralyzed limbs and deliver drugs. These systems will be guided by adaptive algorithms encoded in software, ultimately becoming more autonomous, more lifelike, and more likely to cheat.

Biological defense systems are ubiquitous, robust, multi-level and exquisitely complex. Similarly, social animals have evolved numerous forms of restraint to prevent conflicts from escalating unnecessarily. We aim to understand how these could arise and be sustained in cyber physical systems. We will examine the evolution of defense and restraint in nature and apply these strategies to software, robots and cyberwarfare.

Purpose: Research Collaboration

SFI Host: Stephanie Forrest and Melanie Moses

  • * SFI community lectures are free, open, & accessible to the public.
  • * Other SFI events are by invitation only.

Events News

Apply now for SFI's 2017 Complex Systems Summer School Nov. 1, 2016 -

The Santa Fe Institute is accepting applications for its signature education program for graduate students and postdocs: the 2017 Complex Systems Summer School, June 11-July 7, 2017, in Santa Fe ...

Video: Jordan Ellenberg on the power of uncertainty and contradiction Oct. 19, 2016 -

During an October 18 SFI Community Lecture in Santa Fe, mathematician Jordan Ellenberg explored how math can help us think about the seemingly uncertain matters that dominate our lives. Watch ...

Video: Watch Seth Lloyd's 2016 Ulam Lectures: Birth, collapse, and rebirth at The Information Edge Sept. 29, 2016 -

In two lectures, Seth Lloyd explores what happens when one system gains an advantage in collecting and processing information – an advantage he believes underlies all creation and destruction in our ...

Ecologists gather the pieces of a first-principles-derived ecological theory Sept. 21, 2016 -

A team of ecologists met at SFI recently to begin synthesizing an efficient theory that aims toward a more unified understanding of ecology.

Watch the video of Rosalind Picard's SFI Community Lecture Sept. 2, 2016 -

During an SFI Community Lecture in Santa Fe, Rosalind Picard reveals some of the surprises she has discovered at the intersection of human emotion and wearable tech. Watch her talk ...

More News