Sistema’s rose-tinted glasses

In a recent article in the LA Times, Mark Swed painted a thumbnail portrait of Venezuela today:

the country on the verge of economic collapse, an increasingly authoritarian government generating a possible constitutional crisis and perpetual demonstrations that could lead to a full-scale revolution

Yet despite being armed with this information, he allowed himself to “put on [his] rose-coloured glasses,” and when Gustavo Dudamel appeared on stage and proclaimed: “With these instruments and this music, we can change the world, and we are doing it,” Swed’s response was: “Despite everything, it seemed at that moment impossible not to believe him.”

Swed’s article summed up a refusal to face the facts and a determination to believe the unbelievable that have become a hallmark of the international Sistema-inspired field (henceforth, Sistema). It is hard to grasp how anyone could be fully aware of the unfolding social, political, and economic disaster in Venezuela today and the simultaneous growth of El Sistema, and yet continue to believe the fantasy that orchestral training is changing the world. But this is not a sphere in which evidence is taken very seriously, at least by those with the loudest voices.

Dudamel’s announcement was made during the Take A Stand Symposium in LA, an occasion for mutual backslapping by the various organizations behind Sistema in the US. Here, too, the gap between rhetoric and reality can be perceived, though it’s subtler.

During the symposium, the headline findings from the U.S. National El Sistema Study (a joint project of Longy School of Music of Bard College and WolfBrown) were presented. As is commonly the case in studies in which Sistema or partner organizations have a hand (Longy is a key Sistema base in the US), the language is effusive. The report indicates significant musical growth, rising academic achievement, and developing socio-emotional skills.

However, it also notes that the initial findings for academic achievement “are not yet statistically significant.” Further down, we read: “by spring 2016 students enrolled in El Sistema-inspired programs exhibited higher levels of academic behaviors at a rate approaching significance.”

As a statistician colleague of mine noted drily, “there’s no such thing as ‘not yet statistically significant’ or ‘a rate approaching significance.” An objective summary of this section, without the advocacy tone, would be: “there is no evidence of difference between the academic achievement of Sistema and non-Sistema children.”

Students responded to a series of self-report measures regarding their socio-emotional beliefs and behaviours. “These measures included domains such as growth mindset, perseverance, empathy, cooperation, peer relations, self-concept, school engagement, and self-efficacy. Our early findings show that young people enrolled in El Sistema-inspired programs outdistance their peers in the area growth mindset.” If these words are an accurate representation of the study, then there were no significant findings in 7 out of 8 domains measured.

This report begs many questions, though they may be answered in the full study rather than its headline findings. There is no detail on what is actually being measured, what the numbers mean, whether the differences are small or large, etc. There appears to be no control group for the musical part of the study, and it’s not clear how the musical growth demonstrated is in any way remarkable for a music education program. The drop in growth mindset over the first year ought to be concerning, given the high dropout rate found in voluntary Sistema programs (including El Sistema itself, in which nearly half of students who were offered a place failed to complete two semesters, according to the IDB’s recent study). Taken at face value, this finding suggests that Sistema may be doing socio-emotional harm as well as good.

But such details aside, the report reveals very modest results for US Sistema programs, if ones that are dressed up in enthusiastic language. It does not provide support for the notion that Sistema is “changing the world.”

***

This sense of pervasive hyperbole has long been a feature of El Sistema in Venezuela, and it now seems to have established itself firmly in the international Sistema sphere. I recently came across a Swedish initiative called El Sistema Play, which is being driven by El Sistema Sweden and its charismatic Venezuelan artistic leader Ron Davis Álvarez. As always, the language is dynamic and the idea sounds funky and cutting-edge – a “digital impact hub” – and the principal photo on the homepage shows RDA in a kind of rock-star pose. But this photo is in fact very revealing: it’s all about the conductor showing off. Most of the kids aren’t even playing – they’re just staring at Ron.

The content of the videos on the site indicates old-fashioned, teacher-centred, classical instruction (something that will be little surprise to anyone who has observed instruction in Venezuela without rose-tinted glasses on). There are five videos in all, and in four of them there are no children and no music-making. In the fifth, children do appear, but only to obey the conductor’s instructions about holding their instruments. In the whole series, we do not hear a single child speak or make their own music. RDA is El Sistema’s star European teacher, and if El Sistema Play is designed to show off the “El Sistema philosophy,” then it does so very well – demonstrating just how far the program is from the most progressive and innovative currents in music education. Behind all the talk of energy, discipline, and passion, there’s very little to get excited about here.

***

A lot of the hyperbole comes from the field itself, and is unsurprising considering that Sistema has become an industry and a brand. What is a little more surprising is the way that this hyperbole has infiltrated into so many other spheres, such as government, policymaking, and the media. This process raises questions about how much attention is paid to detail, even at the highest levels where big decisions are being taken.

(Perhaps I should have written: “especially at the highest levels where big decisions are being taken.” David Laws, a former UK schools minister, admitted yesterday that “the quality of education policymaking is poor. […] A lot of decision-making is not based on evidence but on hunch. I had little coming to me from civil servants that presented the latest academic evidence. Too often, they just serve up practical advice about how the minister can do what he or she wants. But politicians are prone to make decisions based on ideology and personal experience.” Is music education all that different?)

The recently published All-Party Parliamentary Group on Arts, Health and Wellbeing Inquiry Report “Creative Health: The Arts for Health and Wellbeing” took Sistema Scotland as one of its success stories. It simply summarised the headlines of the 2015 Glasgow Centre for Population Health report. Neither the caveats within the report nor critiques of the study were taken into account. The Powers That Be have long since decided that Sistema Scotland is a Good Thing and nothing that goes beyond the rosy executive summary is allowed to intrude on that picture. It is fascinating to see a myth being constructed before one’s eyes.

There is also a thumbs-up to the Liverpool branch of In Harmony Sistema England, which is based at Faith Primary School. This is surprising, considering the question mark that currently hangs over the program, if somewhat silently. It’s true that last year, an evaluation of In Harmony Liverpool concluded that the program “has played a prominent role in children’s educational achievement, both through musical skills development and performance, and in other academic subjects such as Maths and English”; and “has led to improvements in children’s wellbeing, particularly aspiration and resilience, as well as their sense of belonging and pride in their community.”

However, a general inspection by Ofsted (the government’s Office for Standards in Education, Children’s Services and Skills) just a few months ago rated the school as inadequate (the lowest rating). Achievement in Maths and English was disappointing. Music appears only once in the 16-page report:

While there is a strong emphasis on developing pupils’ skills and knowledge in music and the performing arts, and showcasing these talents in concerts, the time pupils spend covering other subjects is not helping them to become successful learners. This means that pupils do not gain a good enough grasp of the basic skills in reading, writing and mathematics.

The report also states baldly: “Pupils lack emotional well-being and resilience.”

The two reports are not necessarily contradictory – advocates of the music program could argue that the Ofsted report would have been even worse without In Harmony Liverpool’s involvement in the school – but there are certainly some serious questions raised here. The previous Ofsted inspection (in 2013) had rated the school as good (the second highest rating). So during the following four years, the school’s performance had declined substantially, despite the operation of In Harmony Liverpool throughout this period; and it had weakened notably in key areas – Maths, English, well-being, and resilience – that In Harmony Liverpool was supposedly strengthening. If I were involved with In Harmony, in Liverpool or anywhere else, I would be concerned by this scenario. However, if there has been any public discussion of this topic, I have missed it.

The Ofsted report raises questions not only about the effectiveness of Sistema as an intervention to improve the academic performance and overall culture of a school, but also about the robustness of the evaluation process. There is no clear evidence that the evaluation was flawed, but the gulf between its findings and the Ofsted report demands critical scrutiny. I have been highlighting the contrast between evaluations and independent research on Sistema programs for some time, and the news from Liverpool underlines the need for more attention to this topic.

***

Although studying the Sistema-inspired field has never been my aim, I have picked up quite a bit of information over the last few years via conversations, emails, social media, articles, videos, and so on. (Simply reading the full GCPH report on Sistema Scotland was a learning experience – I wonder how many others in the Sistema field have read the whole thing.) I was intrigued to hear recently from a musician who had been involved in a Sistema-inspired program for a while. She was impressed by the dedication and skill of her colleagues, but she had a number of broader concerns that chimed with points raised to me by musicians and researchers in other parts of the world.

The first is that, as I argued in relation to Sistema Scotland, Sistema programs are providing a media-friendly façade behind which public music education is declining. In this teacher’s words, Sistema is “a costly band-aid on a much bigger sore (inequity of access to instrumental learning) which is too often used (in parliamentary enquiries, media, etc) as a means to demonstrate the ‘wonderful stuff going on to address the problem’ instead of being seen in the true light – a part of the problem.” In the UK, at least, Sistema looks more like a fig leaf than a strategy in the face of declining music provision in state schools. (It may be that in other countries, Sistema is having more positive effects on public music education, though I would like to see some evidence for this.)

Secondly, in a number of cases around the world, Sistema(-like) programs are simply the 21st-century spin on the older idea of orchestral outreach. As Robert Fink has argued, the LA Philharmonic has benefited handsomely from all the publicity around its associated Sistema project (YOLA), and the same might be said about the Liverpool Philharmonic and a number of other professional orchestras in other countries that have spawned similar programs. This teacher saw the program as in part a way to attract donors, with the goal of transitioning this support to company loyalty. Sistema projects cannot simply be reduced to marketing or fundraising ploys – that would be to take too much away from their employees – but marketing and the essential task of increasing philanthropic support are still an important part of the picture, and their blending with education is, at least for some, a cause for concern.

Thirdly, this particular program was voluntary, and the teacher noted that while press reports reproduced the kind of salvation narrative that underpins El Sistema, in fact most children were drawn from immigrant communities that, if experiencing disadvantage in some ways, were also replete with well-educated, supportive, highly aspirational parents, willing to make considerable efforts for their children to be able to attend the program. In other words, most of these children were not being “saved” from anything, and in the teacher’s words, their parents “would find ways to give their children musical opportunities with or without the program.”

This teacher’s testimony points to questions that are hardly ever asked in public:

Beyond generalizations (and exaggerations, in the case of Venezuela) about disadvantaged/underserved/at-risk populations, who precisely is entering and thriving in voluntary Sistema programs?

Are such programs “changing lives” in significant numbers, or are they providing an avenue for children from supportive, aspirational families – children who would probably find another musical channel if they did not have Sistema, or another constructive channel if they did not have music?

Are Sistema kids really exchanging guns for instruments – the motto of José Antonio Abreu, lapped up by acolytes like Swed – or is this more marketing slogan than reality?

(As a former El Sistema musician recently noted semi-publicly, “every time we went on tour we would be presented as a group of children rescued from extreme poverty, prostitution, and drugs thanks to the benevolence of El Sistema, when the truth is that less than 5% of the orchestra came from such conditions.”)

***

As is my wont, I am focusing here on critique of Sistema, since critique is still much needed and too little heard, but I do not think for a second that Sistema lacks positive aspects. I have met good people in this field, and I have had in-depth conversations with Sistema teachers who, while critical of fundamental elements of the programs, speak warmly of the skill and personality of some of their colleagues. There are great things happening in some places some of the time.

For example, I recently came across Brad Barrett’s work at Conservatory Lab Charter School. This is a Sistema-inspired project with a highly progressive philosophy and practice, focused on creativity and reflection. I watched a video of its work that provides a stark contrast with the Swedish example discussed above. There are no adults in the video; there is no conductor, let alone a rock-star conductor hogging the limelight; the children speak; and all the music is created by the kids. I found the video impressive and moving.

So there are clearly positive examples within this field. But the overall picture I’ve gained from multiple sources is typical of the field of education as a whole: there is a mixture of good, bad, and indifferent.

The problem has much to do with the hyperbolic framing – the claim that Sistema is “changing the world,” and the pervasive belief among the field’s leaders that Sistema is somehow better or more advanced than other forms of music education. In fact, beyond the rhetoric, there is little to suggest that Sistema is substantially different to conventional school ensemble programs, whether in relation to practices or outcomes. Although there are some signs of innovation, there also appears to be a lot of belated adoption of commonplace practices, though frequently without a thorough understanding of their theoretical and historical underpinnings or the debates around them within music education research. The very fact that the culmination of the Take A Stand Festival/Symposium was a performance by a selective youth orchestra under the baton of a famous conductor illustrates just how conservative this field is as a whole, wedded as it still is to the deeply and obviously flawed idea of the symphony orchestra as a model for society.

Whether one looks at Venezuela or the US or Sweden or the UK or anywhere else, robust evidence to support the eulogistic narrative is largely lacking. What’s more, it was only by ignoring Venezuela’s meltdown despite unprecedented investment in orchestral training that the Take A Stand Festival could maintain its habitual cheerleading tone. But what sort of educational program worth its salt depends on putting on rose-tinted glasses and ignoring the elephant in the room?

Advertisements