Issue 5 Evil Winter 2001/02
The Emergence of Social Inequality among Robots
In 1949 the British cyberneticist, Grey Walter, and his technician, "Bunny" Warren, built two electronic tortoises, Elmer and Elsie. The tortoises were capable of avoiding obstacles, moving toward a light source (phototaxis), and parking in a hutch to recharge their batteries. They were built with analog electronics that connected simple sensors with motors. Grey Walter had designed these robots to investigate how a "living brain" manages to exhibit flexible control, learning, and adaptivity vis-à-vis the environment. And indeed, Elmer and Elsie showed rather surprising emergent behaviors that looked remarkably similar to some animals. They would zigzag to the charging station when their batteries were low, avoid obstacles on the way, and perform a kind of dance with each other due to the intermingling of attraction and repulsion. People ascribed all sorts of behaviors to these robots, as well as emotional states like fear and anger.
This situation models one faced by many animals in nature. There is an environment that contains sources of food, but the animals have to do something to get this food. There are typically a lot of competitors for the food, usually animals of the same species, and occasionally cooperation is needed. For example, birds that have a nest with eggs need to take turns feeding themselves, and hunters may need to organize in groups to attack larger animals. Ethological investigations over the past few decades have shown that animals adapt remarkably to their environment. They exploit the available resources very efficiently and will cooperate when needed. We managed to elicit these behaviors in the robots as well, and investigated in particular what kind of learning and adaptation strategies the robots needed to adopt to ensure group survival. But we also witnessed the emergence of a remarkable pattern of behavior that was neither preprogrammed nor expected.
In the ecosystem we constructed, two robots starting as equals needed to coordinate their efforts to survive. When robot one (R1) was in the charging station, the other robot (R2) had to keep pushing against the energy-draining box. When R2 was exhausted, it would go toward the charging station and push R1 out so that it could recharge itself. R1 would then start pushing against the box, and so on. As mentioned earlier, the timing of the cycle was not preprogrammed and indeed it could not be specified in advance because the physical properties of batteries are unknown and change with time. The robots needed to adapt their behavior to be compatible with the situation. They would monitor how much energy was available at the charging station while they were recharging and would work harder next time if there was not enough energy. If there was still plenty of energy in the charging station after they had recharged themselves, they would work a bit less. So each robot would individually optimize its own behavior with the idea that global optimality would emerge from the individual behaviors.
Indeed, we observed that after a while the robots regulated their turn-taking to be almost optimally compatible with the pressures in the environment that we set up for them. Both robots were taking roughly equal amounts of time to work and to recharge. But then we observed something strange and unforeseen. One robot managed to maneuver itself into a situation where it worked much less. The other robot was pushing the boxes twice as long as the first, and was on the verge of utter collapse due to lack of energy. The "master" enjoyed life by roaming around freely in the arena, only occasionally pushing the boxes. How was this possible? How could two robots that started with exactly the same hardware and software nevertheless develop a relationship where one appears to exploit the other?
The answer turns out to be surprisingly simple and is in line with research on game theory in economics. The robots behave in the real world and have only limited "knowledge" about this world. Moreover, the real world introduces various random factors that can disturb the equilibrium. What happens here is that due to chance, one robot, say R1, happens to be a bit less efficient in pushing against the boxes. After all, this is a non-trivial task. It requires detecting the boxes, moving towards them, and hitting them. When R1 is less efficient or less lucky in a particular run, less energy is available for R2, which therefore decides to work a bit more when it is its turn to push the boxes. But this makes more energy available to R1, which therefore decides to work even less. And this causes R2 to work just a bit more again next time to compensate for the lack of energy available to it when entering the charging station. So there is a hidden positive feedback loop in the system. Once the equilibrium is broken, there is a steady evolution towards exploitation. In the parlance of dynamical systems, we would say that there is "symmetry breaking." The system, consisting of the cycles of pushing and recharging, moves from one stable attractor (every robot works an equal amount of time) to another stable attractor (one robot works twice as much as the other one).
What to make of the stunning results of this experiment? Recall that the methodology of Artificial Intelligence research is to examine the consequences of certain assumptions. Here we examine what happens when two agents need to cooperate in order to exploit resources in the environment available for their survival. This sort of situation is very common, at the level of families, workplaces, and nations. The experiment does not imply that inequalities are inevitable or that those who were lucky, like R1, have a kind of birthright to their position. However it warns us that inequalities may develop even if the rules are completely fair and initial conditions unbiased. If one were to apply these results to human societies, it has the disturbing implication that even if every agent within a society attempts a fair distribution of labor and benefit, gross inequalities will nevertheless have a tendency to develop. The experiment does not say what could be done about it. Do we need an external controller that monitors the distribution of labor and occasionally sweeps in to re-arrange matters? Do the agents need a way to keep track of what work others are doing and be given the right to re-equilibrate if they judge the situation to be unfair? Should work be divided equally by a central authority who ensures that no one deviates? What happens when resources expand or diminish or when tasks need to become specialized? These questions have occupied social scientists for centuries and we are now finding new tools in A.I. for their examination.
Scott Canazine, Jean-Louis Deneubourg, Nigel Franks, James Sneyd, Guy Theraulez, & Eric Bonabeau, Self-Organization in Biological Systems (Princeton: Princeton University Press, 2001).
Grey Walter, The Living Brain (New York: W. W. Norton, 1963).
David McFarland, Artificial Ethology (Oxford: Oxford University Press, 2000).
Luc Steels, "The Artificial Life Route to Artificial Intelligence," in Chris Langton, ed., Artificial Life (MIT Press, Cambridge Mass: MIT Press, 1997).
Luc Steels, "Language Games for Autonomous Robots," in IEEE Intelligent Systems, September/October 2001, pp. 16–22.
Luc Steels is professor of artificial intelligence at the University of Brussels and director of the Sony Computer Science laboratory in Paris. He is the author of many books. His work on the origins of language in populations of robots has recently been featured in various exhibitions, including “LAwerp, 1999) and “NO1SE” (Cambridge and London, 2000).
Cabinet is a non-profit organization supported by the Lambent Foundation, the Orphiflamme Foundation, the New York Council on the Arts, the NYC Department of Cultural Affairs, the National Endowment for the Arts, the Andy Warhol Foundation for the Visual Arts, the Katchadourian Family Foundation, Goldman Sachs Gives, the Danielson Foundation, and many generous individuals. Please consider making a tax-deductible donation by visiting here.
© 2002 Cabinet Magazine