"Fischer Data under Scrutiny" reports TES

The TES of 12.12.08 reports that teachers say that the use of the Fischer Family Trust's data for estimating pupils' examination results can be "inaccurate and - in non-core subjects - misleading".

"At secondary level, Fischer Family Trust bases its estimates of what pupils might be expected to achieve in GCSEs in each subject, given their test and teacher assessment performance in English, maths and science at key stage 2 and 3.  But teachers say that these are no gauge of ability in drama, music, art and PE. "

Fischer Data

The article quotes Clive Hulme, a drama teacher, who has said that FFT scores were simply being transferred into performance management systems without any checks.
"Most of the teachers we spoke to say that the trust's predicted grades are being used to set performance management objectives as part of the upper pay spine progression."

"Mr Hulme said that that often the problem was that those using the data had not been trained properly in how it should be interpreted.
It's not the trust's fault. They are trying to give schools information to improve results. Unfortunately, what should be used as a carrot to make children strive harder is often used as a stick to beat teachers with."

Mike Treadaway of the FFT denied that the trust's estimates were useless in non-core subjects. He said that it was made clear that "estimates of future performance has a 70% accuracy rate and were not accurate for every child".

Our comment
The Fischer Family Trust's data service provides an important additional source of performance information for schools to complement the official data source - RAISEonline.  Fischer Family Trust data compares the performance of pupils arising from key stage test information with national trends to offer estimates of future performance at the end of future key stages.

These estimates are based on a pupil's performance in English, mathematics and science only, as only these subjects are tested in a standardised way at Key Stage 2.

It is sensible to think that previous test results in English, mathematics and science would be a good predictor of future performance in English, mathematics and science. But one would expect these results to be less approximate predictors of how well pupils will perform in their other subjects.  There is an assumption here of straight line progression in how pupils progress in their core subjects. But there is evidence to suggest that in other subjects, development is much more like a curve.  The TES gives examples of how pupils have 'blossomed' to go way ahead of expectations in subjects like drama. Variation from the linear line of progression should be both celebrated and planned for.

With the disappearance of SATs at Key Stage 3 a second important waypoint and predictor is now lost from the process of estimating how pupils will perform at Key Stage 4. The difficulty for anyone wishing to make use of results in three subjects at age 10 as a predictor in a full range of subjects some 6 years later, is obvious.

This is an example of where schools should ensure they understand how performance data can be used in a predictive manner - and where to exercise caution. There is tension here with school managers' need to set targets for departments and individual teachers.

FFT data provides a useful, but approximate, estimate of future performance.
It provides a dotted background line upon which to project and plan provision for individual learners - whilst having high expectations for all learners. Other essential data tools needed will include tracking devices to support Assessment for Learning. Monitoring progress as it occurs provides scope for intervention and devising short-term targets.

Terminal results will provide the best measure of a school's overall performance and provides the baseline from which subsequent improvement can be compared.
RAISEonline will be the official source of comparison about how well schools and individual departments are performing.  But in our project involving over 150 secondary schools, access to tools that measure Within School Variation has provided another important perspective on the comparative achievement of pupils across their subjects.

The 4Matrix toolkit supports teachers' research into factors that optimise the relationship between teaching and learning. The ability of 4Matrix to forecast the next year's comparative achievement picture provides a future profile upon which action plans can be based. The aim of such plans should be to raise the achievement of all groups of learners where there is significant predicted variation in their performance.

Teachers who make good use of data tools, can demonstrate effective action planning, and can show the impact of their action on standards will live up to performance management expectations based - not on target grades plus a dash of wishful thinking, but on solid professional action informed by effective and confident use of performance data.

One of the most important features of effective teaching is consistency. Out tools uniquely provide numerical measures of the consistency of teaching for every teacher. Although 4Matrix was not developed as an accountability tool, in schools where its use is established, teacher appraisal based on measures of Within School Variation has been a natural development.

If you would like to learn more about how successful schools in our project are using the 4Matrix system to provide alternative measures of their performance, then please contact us using the contact form at www.4matrix.org

Note: 4Matrix is an independent research and development project to provide easy-to-use data tools to support an 'action research approach to school improvement'.

.                            .