4.1 Evaluation

 

Evaluation is often viewed as a chore or, even worse, as an unnecessary add-on. The reality is that evaluating a project can be a positive and satisfying experience. Each project is different and there will be a range of motivations for wanting to develop an evaluation process. The first step is to ask 'why am I evaluating the project and what will I gain from this?'

Outlined below is a list of common reasons for evaluating a project. This list is not definitive and you may find other reasons, however it does provide you with some considerations. It will:

Allow ongoing evaluation to assess if the project is developing in line with the stated objectives.

Develop feedback and connections for those involved in the delivery of the project - community and other professionals.

Allow you to assess and present the outcomes and benefits of the project.

Provide the evidence which is required by funders and other stakeholders.

Allow you to weigh up what went well and what could be improved, thus

Improving the delivery of future projects.

 

When assessing the project for funders and business led stakeholders there are six types of economic benefit that should be evaluated and weighted according to the agreed project outcomes.

Attracting investment

Increasing land and property values

Attracting visitors

Increasing tourism

Improving business (productivity)

Improving cultural / aesthetic image

 

Barriers to evaluation

While the benefits of evaluation may be clear, there may also be some reluctance to carrying out the evaluation. This issue may be more prevalent when working with a wide range of stakeholders. Some common reasons you could face are:

Time constraints The artist or any stakeholder may say they are too busy

Lack of benefit  There may be a sense that evaluation is not important or that no clear benefit can be seen.

Responsibility  Often a project lead or artist can feel it is not their responsibility to drive the evaluation.

Undermining  An artist may view an evaluation as an attempt to undermine their work or that an evaluation will fail to grasp the true outcomes of the work.

It is important that you embed the evaluation as an integral part of the project. This must be done from the very beginning, with everyone involved fully informed on the requirement to evaluate. The evaluation of the project lies with the project lead or artist and the evaluation process must be planned before the project begins.

Should a situation arise during or after the project then the project lead must take the lead role. If there is a reluctance to carry out an evaluation, then it should be discussed between both parties. The issue may be legitimate and it is up to the project lead to examine different appropriate forms of evaluation and find a solution which is agreeable to both parties.

 

Time lines for evaluation

As previously stated the evaluation process should be embedded into the whole working process of the project - the artist should be keeping a diary throughout and the other partners should keep similar documentations that can be part of the formal evaluation analysis at the end.This process can be communicated through an evaluation schedule devised as part of the management plan.

Current timescales for the evaluation of permanent public art projects are often too short.There is need for long-term investment in evaluation to identify the sustained value of a project that is revealed over an extended period of time after the art work has been sited.This may be a process that is adopted by other city organisations or the local authority in liaison with the original commissioner/artist.Key times for evaluation can be placed into four areas: on agreement of final funding at the start of project; during the development of the art work; on the completion of the project; and regular monitoring / staged reviews post completion.

 

Types of evaluation

There are three main types of evaluation to be considered:

Outputs

Outputs refer to the practical aspects that are delivered during the project and this usually forms the basis for a statistical evaluation. For example, it could be the number of volunteers working on the project, the money spent, number of hours worked on project or the number attending the opening. This part of an evaluation can be important as it provides a certain base level of evidence which can be provided to funders or other relevant stakeholders.

Outcomes

While there is a tendency to value outputs as the key part of an evaluation, it is outcomes which should always be considered as the most important aspect of any public art project. Often difficult to assess and present, outcomes refer to the expected consequences of the outputs - they are the answer to the question of 'why am I doing this project?'. For example, there may be clear outcomes such as an improved use of space or aesthetic improvements. However there are also outcomes which are not obvious or hidden. For example, consider: how those involved will benefit; how an outcome could be an increase in community participation or a greater sense of community or civic pride; or how it could inspire an individual to get involved further in the arts.

Process

The evaluation of the process is, ultimately, to assess how well the project was managed and delivered - both by artist, stakeholder management and other professionals. This may include an analysis of the different stages and can also include self reflection from all those involved in the project - this is where diary information becomes important. Process evaluation will indicate project strengths, where there are areas for improvement, and recognised failures, to inform how to develop future projects.

Using evaluation as a learning tool

Evaluation, when carried our correctly, can be an important tool for learning. The ability to effectively monitor a project and to use the information gathered to learn and develop, is a key skill which is at the heart of good management.

Before beginning the project, consider the areas you feel you would learn from and build your evaluation around this. For example, this may be set areas of the process such as the selection of the artist or the community consultation, or it can be a reflective process that examines your role in the project. In order to do this an evaluation framework needs to be created, which allows for both strengths and weaknesses to be recorded. For the majority of projects this will require both self-evaluation and the evaluation of others.

The final step is to take the results of the evaluation and process them into useful data. Creating set areas within your evaluation will allow you to look at particular strengths and weaknesses and while all projects are different, it should provide you with some practical steps which can be used in the future. The personal feedback on your role in the project may be more difficult to interpret and use practically. The nature of this aspect of the evaluation, particularly when sought from others, can often be viewed as subjective or time-bound but must be collected and incorporated in a positive and constructive manner.

Evaluation Methods

There are no clear rules on what can and can not be used as part of an evaluation process, as all projects will have their own individual identity.Whatever the process, it is important to understand that monitoring and evaluation needs to be resourced, so this should be built into the project costs. Presented below is a list of those who may be involved in a project and evaluation methods to be considered for each - including an example template to assist the development of a specific project can be found in the 'Evaluation and Monitoring templates' section.

 

The Artist(s)

All artists in the selected projects can be asked to keep an introspective journal throughout the entire project. This could cover the process undertaken, documenting the artistic objective, research and consultation undertaken, reasoning and decisions, relationship with partners, impact the project has had on individual practice and any challenges faced in the project. This journal should be comprehensive and can contain a combination of written and visual documentation. The use of a blog or social networking site can provide a live and interactive approach. Above all, evaluation should be integrated into the artist's development of the project, so that the process becomes part of the artist's genuine working practice.

 

The Participant

In the context of an evaluation process a 'participant' is anyone who is not the professional artist or project manager. As detailed previously, they should be engaged in the evaluation from the beginning and at all key points. Consultation with participants will vary and consideration should be given to: completing questionnaires; asking them to keep a personal journal; participating on social networking site and participating in focus groups and interviews.

The Audience

The audience is classed as the general public who may view the final result of the project. This aspect of the evaluation will differ greatly depending on the type and delivery of the project. For temporary and permanent pieces, a plan should be drawn up on how and when to evaluate. This may include questionnaires, feedback forms or use of social networking.

Feedback provided through questionnaires and interviews will give an immediate perception, however observed behaviour aligned to the aims of the project can provide longer term evidence of the project's impact. For example, if a public art project has an aim of improving and increasing the use of a public space, then monitoring the number and type of use over a structured period will provide further evidence. This can be particularly effective when the exercise is carried out before and after the project, in order to provide a comparator. Examples of methods to do this can range from basic on-street evaluation using worksheets, through to more complex tools such as time-lapse photography.

 

Partners

The vast majority of public art projects will require a series of partners in order to be delivered effectively. This may include the local authority, community groups, the local business community and residents. There is again a range of formal methods of consulting with each group through interviews, questionnaires or meetings - all relatively easy to document and analyse. However informal discussions, either face to face, by telephone or email, can be more difficult to consider as part of the process. Using a diary format that indicates who has been spoken to and on what topic, can allow a clear picture of the informal communication to be presented and included in the overall evaluation.

When consulting with community groups that are not used to filling in questionnaires or who are reluctant to detail information through writing, consider alternative oral methods of collecting information, particularly in small groups.Given thoughtful consideration of group dynamics, this can allow a variety of voices to be heard that would not usually express views.

When developing issues that need to be tested in the evaluation, consider questions that are the same or connected, that can be asked to a variety of different partners (community, artist, developer) so that useful direct comparisons can be made.