23 April 2007
Feuerstein Partnership - Evaluation Report - Executive Summary
Accredited Trainer: Billy O’ Neill
Learning and Teaching Development Officer: Anne-Theresa Lawrie
In the light of decades of worldwide research that implies that the Feuerstein Instrumental Enrichment (FIE) programme has potential to enhance learning and attainment (Romney and Samuels, 2001) a pilot programme was launched in Scottish Borders Council schools in September 2005. Since the programme includes activities to help pupils to control impulsive behaviour, most pupils selected for the programme had a history of underachieving due to social, emotional or behavioural problems. The FIE programme is described in Section 1.
There are two strands to the pilot project: one is equipping teachers to deliver the FIE programme to the most vulnerable pupils, the other is the adoption of the Feuerstein approach to mediating learning across schools. While the first strand is very resource intensive, the second strand can operate with more modest investment. In 2005-2006, 32 primary and secondary teachers, including members of school Senior Management Teams, volunteered for the Feuerstein accredited training and began to deliver the FIE programme with the selected pupils for around 80 minutes per week.
Scottish Borders Newly Qualified Teachers (probationers) also participated for three days in the area of the Feuerstein training that deals specifically with mediated learning. Since there is extensive research evidence that the quality of the teacher’s mediation is a major influence on learning, early career training in mediation was thought likely to yield long-term dividends. This evaluation is one of many of FIE programmes.
The Scottish Borders programme is a pilot project that had been operational for around six months (excluding school holiday weeks) when the evaluation began. Typically, published evaluations are of FIE programmes that have been running for at least two years and often these programmes provided more lessons than in the Borders pilot project.
Project and evaluation aimsThe broad aims of the pilot project relate to the following areas:
- training and support of teachers, including all Newly Qualified Teachers (probationers)
- establishing baseline data to enable longer term impact on pupils to be assessed
- impacting on pupils in ways likely to lead to attainment gains
- involving parents in using the Feuerstein approach at home
- extending the approach through partner agencies such as social work and health visiting
- ensuring the approach would be active in the project schools.
The above project aims were translated into evaluation aims in the tender submitted to SEED. These evaluation aims, with minor amendments agreed at a Steering Group meeting, are as follows:
Aim 1: Assess the overall impact of the training on the teachers involved, particularly in relation to changes in:
- the confidence of staff in their ability to deal with pupil behaviour
- attitudes (defined as beliefs and values) towards learning and learners
- ability to design and implement effective learning environments.
Aim 2: Identify any emerging impact on pupils whose teachers have been involved in the training, particularly in relation to any improvements in:
- confidence and self-esteem of pupils participating in the project
- progress with cognitive functioning
- changes in pupil perceptions of themselves as learners.
Aim 3: Assess the impact of the project on probationer teachers.
Aim 4: Make recommendations for the further improvement and development of the project (including suggestions about how gains might be sustained).
The evaluation comprised three phases, beginning in May and ending in December 2006:
- Phase 1 looked at what has been happening
- Phase 2 involved data gathering and analyses
- Phase 3 focused on integration and comparison of findings with those in other evaluations.
The research method is described in Section 1 and details of the evaluation instruments and data analyses appear in Appendices 1-4.
During this phase the evaluation team built up a picture of what was happening in the project through discussion with the Development Officer and other key informants (such as members of Senior Management Teams in schools and of the Scottish Borders Feuerstein Steering Group). Important documentary sources included the full set of Feuerstein ‘instruments’ (pupil activities) used in the project, the Continuing Professional Development materials prepared by the Scottish Borders Feuerstein Steering Group and the Project Monitor’s reports. The picture of the project was enriched through the observation of an FIE lesson. Revisions were agreed to the data gathering plans in the tender: in particular, the evaluation team agreed to analyse additional data described below that had been gathered by the Development Officer.
The qualitative and quantitative data specified in the tender were analysed by the evaluation team to judge the impact of the pilot project on pupils and teachers. Thus, interview data were analysed from P5-S3 FIE pupils, from FIE trained teachers, probationers and members of Senior Management Teams in schools. Questionnaire and FIE related task data from FIE and control group pupils were analysed. FIE related task data from a small sample of nursery-P4 pupils were also analysed. In addition to the data specified in the tender, the following evidence was analysed:
- Project Monitor interview and reports
- Responses to FIE trained teacher questionnaires administered by the Development Officer
- Three sets of data gathered by the Development Officer which included pupils’ perceptions of themselves as learners, perceptions of their own task completion competences and teachers’ perceptions of pupils’ progress with Feuerstein related cognitive functions.
The findings from all sets of data were integrated, compared with each other and with findings from other evaluations.
|Aim 1: Impact of the training on the teachers involved
The FIE trained teachers evidenced a significant shift in attitudes towards learning and learners and reported that they were acting in accordance with their changed understandings. These shifts relate to a belief in and an ability to engage in a more open and powerful type of dialogue that is likely to develop young people’s ability to learn for themselves. These teachers support the continuance of the programme and the implementation of a mediated learning approach throughout the schools. They believe that the Feuerstein version of mediated learning is likely to enhance the aims of other initiatives (e.g. A Curriculum for Excellence; Assessment is for Learning) and contribute to National Priorities. They also believe that the Feuerstein approach offers a coherent framework for accommodating ideas presented in other CPD programmes and, in particular, provides tools for enhancing the abilities of all pupils. The FIE trained teachers were more enthusiastic about the training than the Newly Qualified Teachers, possibly because they were able to locate the ideas in their richer classroom experience.
|Aim 2: Impact on pupils
One of the most significant findings is that teachers reported progress over six months for over three-quarters of the sample of 67 pupils that they rated on 6 aspects of cognitive functioning targeted by the FIE programme, all of which are connected with school attainment. It is impressive that teachers reported that 98 per cent improved on one particular and important aspect: ‘correction of deficient cognitive functions’. The younger FIE pupils (nursery-P4) had higher scores than their control group on a FIE related task. The older FIE group pupils (P5-S3) achieved higher scores on FIE related tasks than the control group pupils and the differences were statistically significant. It appears that the FIE pupils, with a history of social, emotional or behavioural needs, not only retained what they learned, but also generalised their learning to a new task. Work is ongoing to ensure optimal ‘bridging’ from the Feuerstein activities to mainstream classroom work. Many pupils talked about improving their behaviour “a wee bit”.
Aim 3: Impact of the project on probationers
Evaluation of Aims 1, 2 and 3 – wider implications of findings
Aim 4: Make recommendations for the further improvement and development of the project (including suggestions about how gains might be sustained)
There is enough evidence to support the teachers’ and senior managers’ view that the majority of young people who began the FIE programme in 2005 should remain on it. Continuance is likely to consolidate and increase the gains they have made. Current attainment, attendance and exclusion data for these pupils should be compared with smilar data available when these pupils reach the end of their schooling.
The primary school pupils who participated in the FIE programme as a whole class have made significant gains which might act as a booster to future attainment, although continuation of the programme with this class is a lower priority if funding is tight. However, it is important that this class is followed up to find out if their apparent cognitive gains feed through to higher than expected attainment. Their secondary school attainment should be compared with that of a similar non-FIE pupil group in order to assess programme benefits for young people who do not have social, emotional or behavioural needs.
FIE trained teachers and Senior Management Teams
The teachers and senior management should build on their significant progress towards realising the Feuerstein approach to mediated learning across their schools as well as continuing to ensure that the FIE pupils reap the intended benefits from participation in the programme. In particular:
- Deliberations should continue to determine criteria for the selection of pupils for the FIE programme and to agree on tools to be used to ensure accurate selection
- Senior Management Teams in schools, supported by the Development Officer, should continue the Continuing Professional Development activities that are designed to cascade the Feuerstein approach to mediated learning across whole schools. They should also continue to try to identify which areas would benefit most from the infusion of FIE activities.
The schools should continue their efforts to persuade more parents to attend workshops on mediated learning and to use this approach at home. Schools might consider running a series of workshops that encourage parents to report on their experience of trying to mediate their children’s functioning in their homes.
Support at the local authority level
Senior Management Teams would benefit from support in expanding the training of school Homelink workers in mediated learning so that they can help parents and carers to mediate their children’s learning effectively. Support in extending the three-day training in mediated learning to more teachers and to develop support materials would also be useful.
Feuerstein’s Instrumental Enrichment programme seems to make sense to Scottish Borders teachers, and that is a particularly sound research base. The evaluation evidence suggests that the pilot project had a positive impact on pupils and staff, particularly in equipping teachers to help some of the most vulnerable young people in society. The evidence to date suggests that further financial support would enable staff and pupils to build on what has been achieved and to extend the approach to other Scottish Borders schools.