Stories about a digitised fly brain were getting stranger. People were already running Bad Apple on it, with a fruit fly model twitching alongside. I watched and could not work out what “running” meant here.
A connectome maps connections between neurons. But where had the moving body come from? Were the muscles modelled? What about the halteres? What exactly connected the image going in to the movement on the right?
With a relevant education and some experience in neuroimaging, I found this confusion particularly annoying. I decided to investigate by adapting an existing demo.
Chukovsky instead of Bad Apple
I took the badapple-fly repository and began adapting the demo with Codex. Instead of the original video, I decided to “run” Chukovsky’s poem about a fly “on a fly.” The quotation marks are doing important work.
Letters become a visual signal projected onto a neural representation. Alongside it, movement is produced by an approximate motor decoder. Between the text, the highlighted neurons and the body sit transformations and assumptions.
The fly is not reading Chukovsky or understanding the poem. Projecting letters does not become reading because there is a picture of a brain next to it. And a body model that moves convincingly does not, by itself, establish that the whole chain is biologically accurate.
My part
My contribution was choosing the material and adapting an existing demo into this strange art experiment. Other researchers produced the connectome and body model. The project builds on MaleCNS, NeuroMechFly and badapple-fly; those foundations belong alongside my adaptation.
The project was frankly stupid, but at least I understood what was happening. The phrase “a poem on a fly brain” now points me towards specific transformations, source work and assumptions. Chukovsky had helped debug my understanding.
Open the interactive Fly Poetry demo.