Researchers at Carnegie Mellon University developed a robot that can paint an abstract acrylic. Is it art?
One day recently, on a table in Jean Oh’s lab in the Squirrel Hill neighborhood of Pittsburgh, a robot arm was busy at a canvas. Slowly, as if the air were viscous, it dipped a brush into a pool of light gray paint on a palette, swung around and stroked the canvas, leaving an inch-long mark amid a cluster of other brushstrokes. Then it pulled back and paused, as if to assess its work.
The strokes, mostly different shades of gray, suggested something abstract — an anthill, maybe. Dr. Oh, the head of the roBot Intelligence Group at Carnegie Mellon University, dressed in a sweatshirt bearing the words “There Are Artists Among Us,” looked on with approval. Her doctoral student, Peter Schaldenbrand, stood alongside.
The process of moving from language prompts to pixelated images to brushstrokes can be complicated, as the robot must account for “the noise of the real world,” Dr. Oh said. But she, Mr. Schaldenbrand and Jim McCann, a roboticist at Carnegie Mellon who also helped develop FRIDA, believe that the research is worth pursuing for two reasons: It could improve the interface between humans and machines, and it could, through art, help connect people to one another.
“These models are trained based on everybody’s data,” Dr. McCann said, referring to the large language models that power tools like ChatGPT and DALL-E. “And so I still think we’re figuring out how projects like this, that use such models, can deliver value back to people.”
The sim-to-real gap provides a surprisingly tricky problem for roboticists and computer engineers. Some artificial intelligence systems can list the steps involved in walking (tighten your quadriceps and flex your tibialas posterior, tilt your weight back and tense your gluteus maximus) and can make a simulated body walk in a virtual world. So it’s tempting to think that these systems could easily make a physical body walk in the real world.
Not so. In the 1980s, the computer scientist Hans Moravec noted that A.I. was good at engaging in complicated reasoning and parsing vast amounts of data but that it was bad at simple physical activities, like picking up a bottle of water. This is known as Moravec’s paradox. (The physical superiority of humans might be explained by our body’s long evolutionary history; the tasks that are simple for us are supported by millions of years of Darwinian experimentation.
