Department for Culture, Media and Sport
Building on my first Priceless? blog, this week I’d like to issue a challenge: how many measures of ‘cultural value’ can you think of?
Mind your language
OK, so perhaps I shouldn’t say ‘measure’. Why not? Well, one of the major themes running through the blog discussion so far is the need to take care in the language we use when discussing the idea of measuring cultural value. Definitions of the words ‘value’, ‘culture’ and ‘measure’ are all contested and convey different meanings.
When thinking about demonstrating the benefits the cultural sector brings to the public, the word ‘measure’ tends to direct thought towards counting things, and, of course, economic valuation. Yet the broader notion of ‘capturing cultural value’ also allows room for alternative ways of thinking about representing cultural value using quantitative and qualitative methods.
Organisations in the cultural sector have largely been individually grappling with how to provide the most powerful evidence of the value of their efforts. The dominant discourse traditionally is one of metrics, inputs and outputs. Yet there is a parallel concern that this renders broader public benefits invisible or intangible.
A joined-up approach
The Priceless? blog wants to hear about and learn from your experience of measuring or capturing cultural value. What ‘measures’ does your organisation use? What ‘measures’ do you think DCMS should use (or not use)? Is anything important being missed?
It seems that the time has come to share best practice in the sector. Do non-economic approaches to valuation have anything meaningful to offer? Who is at the forefront of developing novel economic or narrative approaches? What can we learn from other policy areas?
‘Measure’ for ‘measure’
I would like to issue a challenge. I bet that there are so many diverse approaches to demonstrating the value of the cultural sector that we can collectively generate a list of one hundred ‘measures’ of cultural value this week. And as an incentive – as if you would need one – there are some DCMS prizes for the most creative suggestions!
The point is that my research wants to draw together a comprehensive list of standard and novel approaches to ‘measuring’ or capturing cultural value, and then later go on to investigate what works best and in which contexts.
I will start the ball rolling by listing some methods Phase One of ‘Measuring Cultural Value’ recommended that DCMS could use.
1. Contingent valuation/stated preference techniques
2. Travel cost/willingness to pay
3. Subjective wellbeing
Now, over to you …
4. Improved equity
5. Enhanced quality of public debate
6. Changes in individual or collective perception/attitudes
7. Contributing to cultural preservation and enrichment
8. Income generated
9. Visitor/attendance figures
10. Visitor/public satisfaction ratings
11. Public participation in events
12. Degree of ‘transformativity’
13. Number of lives changed
14. Social Return on Investment (SRoI)
15. Multi-criteria Analysis (MCA) of various stakeholder views on ‘value’ generated
16. Value of tangible assets
17. Product sales
18. Media mentions
19. Website visits
20. Number of downloads
21. Awards/prizes received
22. Critical acclaim
23. ‘Brand’ visibility
24. Market share
25. Creative industries employment figures
26. Number of arts/cultural sector volunteers
27. Life-changing experience
28. Number of books borrowed
29. Happiness
30. Unhappiness
31. Cost Benefit Analysis
32. Return on Investment
33. Philanthropic bequests
34. Revenue from commercial transsfers (e.g. film and TV productions)
35. Artistic/aesthetic quality
36. Social cohesion
Use the comment section below to share your ‘measures’ of cultural value, your views on these questions or other aspects of measuring cultural value.
Dr Claire Donovan is a Reader in the Health Economics Research Group at Brunel University, London. ‘Measuring Cultural Value (Phase 2)’ is jointly funded by the Arts and Humanities Research Council, the Economic and Social Research Council, and the Department for Culture, Media and Sport.
This interactive blog seeks to stimulate discussion across the cultural sector on the very idea of measuring cultural value. Dr Claire Donovan is an academic working at DCMS to write a report on this issue, and wants to know what you think. Can the value of culture be measured by government in monetary (or other) terms, or is it ‘priceless’?
Here’s a list of 56 possible metrics, broadly construed, that we came up with at the Center for the Study of Interdisciplinarity (http://www.csid.unt.edu) to measure our own impact.
Maybe some could be used to measure cultural value, as well — or at least spur some thinking (we should add that to the list).
1.H-index; 2.G-index; 3.HL-index (universal); 4.HM-index (standard for co-author); 5.Peer review; 6.Place of publication; 7.Number of pubs; 8.Number of citations; 9.Book sales; 10.Article downloads; 11.Website hits; 12.Media mentions; 13.Quotes in media; 14.Quotes in policy docs ; 15.Developing a metric that people use; 16.Rabble rousing; 17.Muckraking; 18.Lawsuits provoked; 19.Meetings with important people; 20.Mention by policy makers; 21.Invitations to present; 22.Invitations to consult; 23.Invitations to evaluate; 24.Protests/demonstrations provoked; 25.Coining phrase or buzzword; 26.Trending in social media; 27.Esteem surveys; 28.Trust/reputation; 29.Rankings; 30.Blog mentions; 31.Student surveys; 32.Student testimonials; 33.Faculty recommendations; 34.Faculty award/prize; 35.Citizen testimonials; 36.Town hall meetings; 37.Social networking contacts; 38.Increased diversity; 39.Degree of ID/TD; 40.Degree of transformativity; 41.Academic rigor; 42.CSID advisors; 43.Special problem requests with CSID faculty; 44.Internationalization; 45.Grant money; 46.Audience size @ CSID events; 47.Success of faculty fellows; 48.Esteem of senior fellows; 49.Success of graduate / UG presentations & grants; 50.Guest posters on blog; 51.Impact factor of journals in which CSID publishes; 52.Number of angry letters received; 53.Textbooks produced; 54.Influencing curricula creation; 55.Participating in public education programs; 56.Branding.
@J Britt Holbrook – Thanks for getting the ball rolling!
I guess this is a very academic take on ‘measures’ applied to assessing academic performance. So you have kindly provided a list of input and output measures of academic quality, markers of academic esteem, and various indicators of academic engagement with the public and policymakers.
I guess that there are three main observations. First, this very much fits with the realm of academia, and the humanities where the written word is the major focus of activity (the arts would include performance, exhibitions, curating, etc.)
Second, the ‘Priceless?’ blog is looking for ‘measures’ of cultural value specific to the public funding of arts and heritage. So I have taken three likely areas from your list and moulded them to this end: ‘Increased diversity’, ‘Degree of transformativity’, and ‘Participating in public education programmes’.
Third: ‘Rabble rousing’, ‘Muckraking’, ‘Lawsuits provoked’, ‘Number of angry letters received’! Really? What do you guys get up to at CSID?!!! Can these ‘measures’ be reasonably used to make the case for public funding of the cultural sector…?
@Claire Donovan “Third: ‘Rabble rousing’, ‘Muckraking’, ‘Lawsuits provoked’, ‘Number of angry letters received’ …. Can these ‘measures’ be reasonably used to make the case for public funding of the cultural sector…? “
To me this is the nub of your problem. If we are looking only for the ways that heritage makes better citizens, the fact remains that value is often expressed in outrage (you don’t know what you’ve got till its gone).
A vibrant Heritage sector supports increased diversity alright, and possibly ‘degree of transformativity’, though I’m not sure how that is measurable never mind a measure. I suspect some of those ‘dangerous’ measures may be a better way at getting to the visceral importance that people actually attach to Heritage for better or worse.
@Claire Donovan @Sarah May — Sarah is getting at what we intended, at least in part. By including these sorts of ‘negative’ indicators, we are attempting to capture something that gets left out of typical measures.
And, no, I wouldn’t want to use only such indicators. The point of including these — and in having 56 (to which we are open to adding more, so I hope to see some good stuff here) — is precisely to suggest that any single measure (or even a small set) will fail to capture the full impact of our activities.
Recently at an event Tyne & Wear Archives & Museums held in Newcastle two people came up to me and told me how our Culture Shock project had changed their lives. I think that’s not a bad measure of the value of culture, particularly as it’s both an instrumental and an intrinsic value!
http://www.cultureshock.org.uk
In terms of your point about ‘is culture just priceless’, consider things which are truly priceless in the economic sense. Then ask an insurer how much the premiums would be to insure said priceless ‘thing’ (sorry to use such a useless word!).
I would imagine, like when you do your car insurance, and gradually lower the car’s value to reduce the premium, the value of anything is only what someone is willing to pay to ensure it never goes away (and for the insurer, the risk they associate with the liklihood of it going away.
Value must therefore be intrinsically linked to ‘risk’ or more importantly, the liklihood of something not being available tomorrow…
Having carried out a contingent valuation study into the National Galleries of Scotland, one of the key issues I found was that the measure of ‘value’ which I reached was not comparable to anything else.
Whilst it was clear that the public valued the galleries at an amount greater than that which it recieves in funding, had I conducted the same type of study into another service – such as a public park, library, or even car park – I may well have found out that the public valued that investment equally or even more greatly.
Much research is produced, both quantitative and qualitative, which shows that people feel culture has the capacity to improve their lives; but what is not explored is the extent to which any of these benefits are exclusive to ‘culture’ (as defined by bodies such as DCMS) or which culture is the most effective tool for their delivery. Does a community arts project more effectively improve participants’ confidence than a literacy course, or gardening project? Is the National Theatre more effective in unleashing creativity than the establishment of a National Computer Games organisation.
If we are measuring, we must be clear about why we are doing it, and be prepared that some of our much loved cultural institutions and projects may no longer be seen to be the best choice for funding in order to reach the stated objectives. Without an ability to compare, any valuation is all relative.
Interesting post.
Being public libraries are (at least in terms of the government – they are of course heavily linked to education and social services as well) in the sphere of Culture and, as at least one commenter has suggested, the measures you list appear biased towards the academic, I would add one or two more library specific ones:
Books loaned.
E-books loaned.
Usage of public access machines.
Comments books: this doesn’t seem to be mentioned anywhere yet one gets a gigantic amount of data from one of these.
Comments told to staff: A lot of people won’t write it down but they’ll tell you. I had a gentleman tell me he’d commit suicide if the library closed down. That is a pretty major measure of impact. We constantly get people saying that they don’t know what they’d do without the library.
Financial benefit: Several websites have simple calculators showing what a public library could save in terms of cost of books lent not bought etc.
There is a ton more of this stuff. Let me know if you want more. There is a complete bibliography of measuring the value of libraries that you may be interested in as well.
Of course, the current biggest indicator is ironically one someone has criticised you for including. Tens of thousands of people have protested cuts to libraries, thousands have joined/formed groups to protest, hundreds perhaps thousands are being forced into volunteering to keep their libraries open. All of these groups are putting “their money [sometimes literally, sometimes in terms of time] where their mouth is and really demonstrating the value of the service they’re trying to save.
Do let me know if you wish further information.
Ian Anstice
Creator of Public Libraries News,
IWR Information Professional of the Year 2011
Since creativity is the root of culture, since the objective of this discussion is to encourage adequate public funding of culture, we should measure the proportion of public funding for culture which goes to arts education in schools.
The public’s top three priorites in public funding for the arts:
– Giving every child the opportunity to access art and artistic experiences
– Enabling access to the arts for people who would not normally have that opportunity
– Encouraging more ordinary people to engage with the arts
(Arts Council England’s “stakeholder focus” report, 2011)
We should measure the extent to which we deliver on these objectives. In particular, the extent to which public funding unleashes creativity. Creativity has intrinsic value, but also enormous instrumental value to the economy.
@Iain Watson – I agree: measuring the social impact of arts, whether personal or community development, is important.
Coming from a community arts background (http://www.soundsense.org) I would say that wouldn’t I, but my personal experience is that “audiencing” as well as participating can be transformative.
For a project on the benefits of participating in community choirs in areas of disadvantage our metrics are likely to include how the project has helped: singing skills; problem-dealing; happiness; decision-making; energy; interest in other people; interest in new things; wellbeing; feeling relaxed; confidence.
One of the major problems remaining with the panoply of measures are the ones which may be acceptable as having appropriate rigour and validity – this was an issue brought out by the performance of ‘heritage’ in comparison to other sectors within the DCMS stable in the CASE (Culture and Sport Evidence Programme) research.
The well-established ‘Heritage Counts’ programme of work undertaken by English Heritage and others over the past decade provides a great wealth of evidence (and this is replicated by organisations in the other home countries as well as overseas, such as within the US National Parks Service).
But the rub comes in the articulation of the evidence in the power struggle for policy-making attention.
It remains surprising to me that so little mention is made of the Treasury’s other evaluation manual – the Magenta Book. Though primarily aimed at the economic language of the Treasury, it contains much of use. The language of articulating value as already noted remains key, as seen in growth of evidence allied to debates on well-being, covered by other posts here.
http://www.hm-treasury.gov.uk/magentabook
Interesting post, and I believe, a noble effort in trying to raise debate around cultural value, particularly in these times of austerity when arts funding is becoming scarcer.
Clearly any attempt to define a measure of cultural value is going to be a tricky one, and there are some good points raised in the debate here.
While the list that is taking shape here has some valuable insights, many of these indicators (attendance figures, product sales, number of citations, etc) are only relevant to established institutions and have limited value to smaller organisations or newer concepts.
From the viewpoint of someone making funding decisions, it may be easier to rely on tried and tested metrics, but this will inevitably lead to traditional institutions gobbling up funding that may have been better spent in a more diverse way. It is for this reason that we need a shift towards using the less definable metrics on your list such as “degree of transformativity”, “number of lives changed” or “happiness”. For while they are hard to quantify they are also much more attuned to what culture really is.
A previous post here mentions the importance of including, for want of a better word, ‘negative indicators’ of cultural value such as popular protest and “Lawsuits provoked”. These are, I feel, exactly the kind of indicators we need to focus our attention on.
So, to add my own metric, for what it’s worth:
‘Number of people inspired’
Because surely that is the ultimate goal of culture.