On Line Research

Been preparing a Panel for the Canadian Music Week conference in Toronto comparing on-line and traditional phone based research systems. For most of you who have never been to a CMW convention Canadian Music Week is a unique gathering of Canadian artists and broadcasters every March in Toronto. There are many sessions for both the broadcasters and the musicians and lots of showcases - it's a lot more like the Country convention in Nashville than the typical NAB or R and R meeting we have in the U.S.

Our panel for the On-Line/Traditional session includes Jeff Vidler from Solutions Research in Canada, Mark Kassof from Kassof Research, Tom Meyer from Media Score on line research and Julie Adams VP/GM/PD at CHDI in Toronto. I've worked with Jeff, Mark and Tom on various projects in Canada and with Tom and Jeff we have stations where we do all on-line research. Perceptual studies, library music tests (300 plus titles) and monthly call out style testing is all done on-line.

There are a lot of advantages to doing research this way including:

• Sample Size - In most music tests we are lucky to have 80 people in the room, but most on-line projects are at least 300. The much larger sample allows us to slice and dice the data a lot more with confidence in the results.
• Respondents dictate the pace - Since all the respondents are doing the study on their computers at their pace they can stop for dinner, replay the song or audio/video element and take the test at their pace. No more people in a music test finding themselves 3 songs behind or rushing through answers on the phone. People actually have time to think about their responses and it shows in the data.
• Phones - Even though Arbitron and BBM still use phones to gather the sample it's quickly becoming an outdated way to do research. We can't use cell phones and so many people really don't communicate much anymore on their land lines.
• New presentations on the questions - On line we can play audio, video and even use some java/flash tricks to make the questionnaire more interesting and interactive. No more repeating 'on a scale of 1 to 5 with 5 being love it and 1 meaning you hate it...' We can put a little volume control up and tell them to turn it up or down.
• Costs - It's clearly less expensive to do research on-line than using call out, phone banks and auditorium tests.

The biggest obstacle is gathering a sample. With the phone we have a directory that probably includes at least 60% of the phone numbers in the market so we can just grab a bunch of numbers and start dialing. On the Internet it's a little harder. There are millions of email addresses and since the user defines the name there's no system to it. The addresses are also all over the world which won't help you in the middle of Kentucky. We have found more and more databases of email addresses to use. In Canada, with a smaller population base, it's a little easier to find email databases that are sorted by geographic area (postal codes) and ages. For the monthly music testing on-line we build a database from on-air/web promos and for the library (bigger) music tests we use a combo of station and outside databases.

Actually the results have been very useful. The music scores are a little more passionate than we see in the auditorium tests, with less unfamiliarity and higher burn scores. In the perceptual studies we see a lot more longer and more detailed responses to open ended/comment style questions. You can see that when the audience has a little more time to respond and can do it at their pace we get more data and they are clearly a little more 'into it.'

This is the future of research. Some may argue that this system is not as random, doesn't match the systems the ratings firms use, and that we don't have as much control over who we are talking to. All valid points, but having a lot bigger samples, moving away from the outdated phone line, lower costs and getting a lot more data are also big advantages that need to be considered. We all know we need to re-think and re-invent radio to survive in the new media world. We're going to need research and data to do it and a lot of it. On-line systems will have to become a part of the plan sooner than later.