On Wednesday I had a debate with roving Sistema ambassador Marshall Marcus on the BBC World Service. It was an interesting experience, of a kind I’ve never had before, and I learnt a lot about the media in those rather intense 15 minutes. Marshall is a smart guy and, perhaps from having worked at the top level of music institutions for years, he’s also media savvy. He either had a clear strategy worked out in advance, or simply knew what to do in a radio debate, whereas I didn’t. He had messages to get across, whereas I attempted to answer questions. (I now understand why politicians don’t answer questions but just trot out prepared lines. “You know, John, the bigger issue here is [repeats Key Line 1]”).
Anyway, the key issue, as far as I remember, was that of data and anecdotes. Marshall was fairly successful in bringing it down to a question of “my anecdote versus your anecdote,” given the general lack of reliable statistics. However, I think the issue is more complex and more important than that.
To start with, my anecdote and your anecdote may have quite different characters. A well-known figure like Marshall, known to be close to El Sistema’s top brass, may elicit different responses from musicians in Venezuela than someone unknown like me. I struggle to imagine many of the people I interviewed saying to Marshall the things they did to me; some of them were really nervous about word getting back to Abreu, so my lack of connection to the program’s big names was a help.
There’s also the question of interviews. Much of my information comes from long, anonymous interviews, in some cases with senior musicians with more than 30 years’ experience in and around El Sistema, using a flexible but planned set of questions. How many of those who say “in my experience El Sistema isn’t like that” have done this kind of research? On what grounds do they devalue these musicians’ testimony? My mistake, going in there unprepared, was to get drawn into a conversation about anecdotes in the first place. Since Marshall, by his own admission, hasn’t done research on El Sistema, whereas I have, it’s really a question of “my research data versus your anecdote.”
A lot of the debate revolved around the relationship between anecdotes, statistics, and data. Marshall argued that I shouldn’t be questioning the dropout rate without statistics to back up what I had been told by musicians. It’s not my responsibility to collect statistics on El Sistema – it’s El Sistema’s responsibility. If they don’t have, or don’t share, data on their dropout rate, then any claim that the program is successful is seriously undermined. The fact that musicians alleged a high dropout rate is only really the icing on the cake.
As Marshall knows full well, when it comes to publicly-funded social programs, the opposite of “innocent until proven guilty” applies. It is the responsibility of the programs to demonstrate that they work, not of researchers to prove that they don’t. Considering that there is still no reliable data underpinning the loud claims about El Sistema being a “miracle,” Marshall is on very shaky ground here.
Another clever strategy that Marshall adopted was to try to establish a difference between data (reliable) and anecdotes (unreliable), with data implicitly equating to numbers or statistics. He knows that I am primarily a qualitative researcher, so his line of attack was to suggest that only quantitative research really counts. However, qualitative data (collected through fieldwork, interviews, observations) is just as valid a form of data as quantitative data is (and the idea that the two are separate is a fiction: behind every quantitative study are human assumptions, decisions, interpretations, and so on). Of course statistics are important, if handled carefully. But it’s naïve to think that stats are the alpha and omega of research. The attempts to demonstrate El Sistema’s success to date are full of stats, but the studies have not convinced the Inter-American Development Bank. (He asked why the IDB then funds El Sistema. It’s an excellent question, and one that someone with expertise in development economics should really look into.)
Equally, there are many important issues – political, cultural, ethical, and pedagogical – for which stats have little or no relevance, issues that require a qualitative approach. Take the example of sexual abuse in UK’s specialist music schools. How many people believe that since there are no statistics for the prevalence of abuse, only “anecdotes,” then the problem can be brushed under the carpet? People’s personal histories should be taken seriously: they can be revealing and important, and to dismiss them as mere “anecdotes” is disrespectful and – in some cases – highly irresponsible. We now know that in the UK, lives have been lost and damaged through a collective failure to listen to the stories of young musicians about their personal experiences.
Unfortunately, brushing aside the experiences of Venezuelan musicians seems to be widespread within the non-Venezuelan Sistema movement, as I discuss here.
Geoff Baker 