Research, sonification, User interfaces

Week 4 Update: Initial Research and Sonifications

This week, my aim is to try and narrow down the scope of the literature that I have been searching through and gain a more of a specific, cohesive idea to work with.

On Wednesday of this past week, one of the library staff gave a talk about how to effectively use search terms to maximise the best possible results. This was definitely a worthwhile talk to attend as it helped me realise the importance of having a clear set of search terms. To obtain these, it is necessary for me to narrow down my project to a set of searchable terms which can be hopefully be moulded into a title.

Refining the research

To do this, I decided to go back through and look at what I have been reading for the past few weeks and see if there had been anything that was standing out for me.  So far, I have read a fair number of introductory papers and highlighted some relevant sections. Grouping these together, I found that the main interests that I have had are:

  1. Interactivity: building a sonification system which has intuitive user interaction- such as that seen in many tangible interfaces.  What is it about a system that makes it intuitive? What research has been done on this and what could still be done?
  2. Multimodal perception in auditory data analysis: how can our sensory processing be used to create greater understanding of sonification.
  3. Spatial sound and spatial data: Using x-y-(z?) or spherical polar co-ords (r, thetaphi) to create user interface which mapped to some form of spatial sound?

Of these interests, I think I would like to pursue the ideas behind user interaction and tangible interfaces the most and so I have decided to limit my search to these articles. I’ve also read some engaging ideas of catering to the needs of both musical novices and experienced computer musicians using interactive technology [1], as this rings true especially useful for sonification systems as it might often be catered towards those with little/no prior musical experience, which I would like to include in my research somehow also. Hopefully now this narrower scope can help me progress quicker with my research.

Creating sonifications 

This week, I also had a go at creating my own sonifications of data, given the research that I had been doing. Given the current political climate I decided to use data provided by the electoral commission which detailed voter turnout in all constituencies in the 2015 UK General Election. To do this I mapped the data to midi note and then using the software “Pure Data” sent this data to a Digital Audio Workstation (Ableton Live) where the MIDI input was then converted to an audio output of a xylophone plug-in.  This sort of sonification is known as parameter mapping [2].

 

This was an interesting exercise in applying the knowledge that I have been learning about over the past few weeks into practice. This short piece of sonification was more useful for me to see how the process worked than to gain much insight from the data. Perhaps it might also be interesting to juxtapose this sonification with another audio track that sonified the voter turnout from the EU membership referendum from last year.

Screen Shot 2017-05-31 at 23.50.55

Screen Shot 2017-06-01 at 01.14.54
Above shows the midi notes generated from the sonification in Ableton Live, which in themselves provide a visualisation of the data and below shows the pure data patch that was used to implement it.

Furthermore, this piece does not provide any user interaction with the data, which is crucial to interactive sonification. Hopefully in the research that I will be undertaking over the next few weeks I will be able to apply both the work

Comment below what you think of my research ideas, the sonification piece and if there are any other data-sets that might be useful to try and sonify.

 

References 

[1] S. Saue,  “A model for interaction in exploratory sonification displays.” Georgia Institute of Technology, 2000.

[2] T. Hermann and A. Hunt, The Sonification Handbook. Berlin: Logos, 2011.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s