Artist-in-residence and visiting scholar Rebecca Kamen has blended AI and art to produce animated illustrations representing how a dyslexic brain interprets information.
Communicating thoughts with words is considered a uniquely human evolutionary adaptation known as language processing. Fundamentally, it is an information exchange, a lot like data transfer between devices, but one riddled with discrete layers of complexity, as the ways in which our brains interpret and express ideas differ from person to person.
Learning challenges such as dyslexia are underpinned by these differences in language processing and can be characterized by difficulty learning and decoding information from written text.
Artist-in-residence in Penn’s Department of Physics and Astronomy Rebecca Kamen has explored her personal relationship with dyslexia and information exchange to produce works that reflect elements of both her creative process and understanding of language. Kamen unveiled her latest exhibit at Arion Press Gallery in San Francisco, where nine artists with dyslexia were invited to produce imaginative interpretations of learning and experiencing language.
The artists were presented with several prompts in varying formats, including books, words, poems, quotes, articles, and even a single letter, and tasked with creating a dyslexic dictionary: an exploration of the ways in which their dyslexia empowered them to engage in information exchange in unique ways.
Undiagnosed dyslexia
“[For the exhibit], each artist selected a word representing the way they learn, and mine was ‘lens,’” explains Kamen. “It’s a word that captures how being dyslexic provides me with a unique perspective for viewing and interacting with the world.”
From an early age, Kamen enjoyed learning about the natural sciences and was excited about the process of discovery. She struggled, however, with reading at school, which initially presented an obstacle to achieving her dreams of becoming a teacher. “I had a difficult time getting into college,” says Kamen. “When I graduated high school, the word ‘dyslexia’ didn’t really exist, so I assumed everyone struggled with reading.”
RELATED
- Researchers measure different types of curiosity studying ‘hunters and busybodies’
- Exploring what it means to be curious
- Reimagining scientific discovery through the lens of an artist
Kamen was diagnosed with dyslexia well into her tenure as a professor. “Most dyslexic people face challenges that may go unnoticed by others,” she says, “but they usually find creative ways to overcome them.”
This perspective on seeing and experiencing the world through the lens of dyslexia not only informed Kamen’s latest work for the exhibition “Dyslexic Dictionary,” but also showcased her background in merging art and science. For decades, Kamen’s work has investigated the intersection of the two, creating distinct ways of exploring new relationships and similarities.
“Artists and scientists are curious creatures always looking for patterns,” explains Kamen. “And that’s because patterns communicate larger insights about the world around us.”
Creativity and curiosity
This idea of curiosity and the patterns its neural representations could generate motivated “Reveal: The Art of Reimagining Scientific Discovery,” Kamen’s previous exhibit, which was inspired by the work of Penn professor Dani Bassett, assistant professor David Lydon-Staley and American University associate professor Perry Zurn on the psychological and historical-philosophical basis of curiosity.
The researchers studied different information-seeking approaches by monitoring how participants explore Wikipedia pages and categorically related these to two ideas rooted in philosophical understandings of learning: a “busybody,” who typically jumps between diverse ideas and collects loosely connected information; and a more purpose-driven “hunter,” who systematically ties in closely related concepts to fill their knowledge gaps.
They used these classifications to inform their computational model, the knowledge network. This uses text and context to determine the degree of relatedness between the Wikipedia pages and their content—represented by dots connected with lines of varying thickness to illustrate the strength of association.
In an adaption of the knowledge network, Kamen was classified as a dancer, an archetype elaborated on in an accompanying review paper by Dale Zhou, a Ph.D. candidate in Bassett’s Complex Systems Lab, who had also collaborated with Kamen on “Reveal.”
“The dancer can be described as an individual that breaks away from the traditional pathways of investigation,” says Zhou. “Someone who takes leaps of creative imagination and in the process, produces new concepts and radically remodels knowledge networks.”
The artwork Kamen produced for “Dyslexic Dictionary” harkens back to “Reveal.” The knowledge network provided a way to depict Kamen’s distinct way of processing verbal information by analyzing transcripts of researcher interviews to construct a dataset of the keywords Kamen said.
From those interviews, Zhou mapped out the relatedness between each topic and clustered similar words together, graphically representing Kamen’s information-seeking and idea-generation process.
Zhou notes that, “the distance between the dots here shows how Rebecca’s able to bridge dissimilar ideas.” Kamen calls that, “one of the gifts of dyslexia that I’m learning to appreciate.”
Kamen took Zhou’s visualization and affixed several of her artworks to a few nodes, relating the keywords she had used in the interviews to recognizable images.
AI and information processing
One of those artworks was Kamen’s Corona 3, a sculpture she produced during the pandemic that showcases her ability to connect disparate concepts. She relates models of the sun’s corona, the outermost layer of its atmosphere composed of hot plasma, to the spike proteins on the surface of SARS-CoV-2, which play a major role in its ability to infect host cells. Kamen highlights a similarity in how they interact with elements within their systems: The corona interacts with the sun’s powerful magnetic field and plasma, which cause solar winds that can affect bodies in the solar system and the spike proteins interact with the host cells.
Artists and scientists are curious creatures always looking for patterns. And that’s because patterns communicate larger insights about the world around us.Artist-in-residence Rebecca Kamen
“People are fascinated when I show them these types of associations because we all have the ability to channel our curiosity and create new connections, but neurodiversity and dyslexia provide a catalyst for making these connections more apparent,” says Kamen. “What’s exciting here is that Dale was able to take the language I use to explain my process of discovery, and by connecting ideas, create something totally new.”
Zhou used artificial intelligence to transform Corona 3into an animation based on the interview transcripts and keywords from Kamen’s presentations.
https://player.vimeo.com/video/792376037An animation of Kamen’s sculpture, Corona 3. (Image: Dale Zhou)
Zhou created the animation with the open-source AI tool Stable Diffusion. Typically, AI models take in user-defined, labeled data points that train the model to do a specific assigned task. With Stable Diffusion, “the model has the whole internet to source images paired with text,” explains Zhou. “It’s trained by adding randomness to an incredibly large training dataset sequentially. The task is to learn how to reverse this addition by sequentially removing randomness, letting it reliably return new images similar to those in the training dataset.
“But there’s also a way you can make an image resemble text descriptions, which can change the image slowly over time and morph into something that sort of mimics the stream of consciousness that underpins Rebecca’s process.”
For their next piece, “Kamen’s Lens,” Zhou generated another animation, this time from a series of emails between Kamen and poet SJ Fowler that Fowler turned into poems.
https://player.vimeo.com/video/792375967Kamen used fragments of her email correspondence with poet SJ Fowler to inform her illustrations. Fowler then converted these emails into poems, which Zhou used as prompts for the AI-rendered final animation. (Image: Rebecca Kamen and Dale Zhou)
“By having artificial intelligence translate words into images, we appreciate a visual reimagining of language, which echoes some of the aspects of my experience with dyslexia in a profound new way,” says Kamen. “When I lecture and speak with students who struggle with reading the same way I did at their age, it’s nice to share that it can be seen as a gift and a way to create something new.”
Kamen hopes to use what she has learned from this process to inform her next exhibition at the American Philosophical Society, a special collection honoring unrecognized women in science. It will draw from existing knowledge bases, namely the discoveries and scientific breakthroughs these figures inspired, and reimagine them through Kamen’s lens incorporating art to highlight their significance for scientific and non-scientific audiences.
Source: https://penntoday.upenn.edu/news/through-lens-digital-depiction-dyslexia
