Among the many attractions in this year’s Canadian National Exhibition (CNE) in Toronto are the sights and sounds of… proteins.

John Bonnett, an associate professor in the Departments of History and Interactive Arts and Science, will be displaying his DataScapes project as part of an augmented reality exhibit created by Brock University, York University and the University of Toronto.

“In some ways we got a really lucky break with Pokemon Go coming up; augmented reality is very much on peoples’ minds now,” says Bonnett. “What this exhibit will do is to show that there are other ways of using augmented reality beyond gaming, including art.”

Bonnett first unveiled his DataScapes project during Congress 2014 held at Brock.

The project consists of two pieces: The Five Senses, and Emergence. The first work presents music and images derived from proteins involved with the five senses: sight; touch; taste; smell; and hearing.

When viewers point their phone or tablet at the QR code, they will see a hovering floating cube and monoliths surrounding the cube, Bonnett says. In addition to seeing the images, people will hear music based on patterns derived from the protein data.

Bonnett and colleagues obtained data from the National Institutes of Health in the U.S. To generate the music and visual art in The Five Senses, Bonnett and his team started with the amino acid data contained in each protein record.

Each amino acid constituting a referenced letter was represented as a letter. The research team converted the letters into numbers, starting from A equaling 1 all the way to Z equaling 26.

Student composers in the Department of Music aligned those numbers with a musical scale to produce raw music strings. The composers then fashioned those sounds into a musical piece.

To produce the images seen in the cube, colleague Bill Ralph with the Department of Mathematics used the same numbers to create an algorithm that converted the numbers into images.

For the second piece, Emergence, Bonnett and his colleagues used a text dataset as the basis for the work’s sound and images. Bonnett isolated passages in the Holy Bible and gave the text to the student composer, who turned the letters into numbers and then converted those numbers into music strings to make a music composition.

Computer science student Chris Jacques worked under the direction of Michael Winter, who initiated development on the project’s app, which was then completed by Mark Anderson and Joseph Bolton from Edge Hill University in the UK.

DataScapes will be one of three augmented reality projects at the CNE. It will be featured with contributions from the University of Toronto and York University.

“I was invited to participate,” says Bonnett. “The invitation can be taken as an indication that Brock University is seen as one of the leading universities in the province with respect to exploring the potential of augmented reality to support a variety of applications, including art.”

The CNE runs from August 19 to September 5. The project’s app is available for free downloading from both the Apple App and Google Play stores.