4Matrix Frequently Asked Questions

The following are Frequently Asked Questions about 4Matrix.
If your question is not answered here then please contact us.

Q. What is 4Matrix ?
A. 4Matrix is a multi-user application and research tool for use by subject leaders and teachers to research the factors influencing pupil performance in their subject. 4Matrix extracts the most recent GCSE results, analyses them, generates a series of reports and creates an interactive data analysis and research toolkit. Schools can also cut and paste in current Y11 grades from a spreadsheet and forecast comparative achievement in the 2010 exams. 4Matrix contains an SQL database containing details of every examination and all national subject comparative data. This database automatically updates itself so it is always using the most recent information.

Q. We do a lot of data analysis - why would we need to use 4Matrix?
A. Most schools are now data rich but information poor. A school can have a lot of low-level data which it may use in mostly low-level ways. A list of grades for example will tell a school very little that it doesn't already know. What is needed are tools to convert grades and other low-level data into useful management information that can be acted on. 4Matrix interprets low-level data into high-level commentaries.
It is a tool designed for educationalists rather than statisticians.
Barbi Goulding, Principal of Paddington Academy said of this:
     "Having lots of data is not what self-evaluation is about.
      What counts is having the right tools to make top-level judgments on that data.”

Q. What does your system offer that we can't do already?
A. There are many good reasons for using 4Matrix. Our new version, 4Matrix 2010, will:
1. Forecast comparative achievement in the 2010 examinations
2. List pupils in relation to three levels of progress from key stage 2 (feature due Autumn 2010)
3. Display pupil lists for those likely to fall short of 5A*-C grades in 2010
4. Calculate residuals for every specialist teacher's classes at KS4
5. Go much further than grade lists by providing automated performance commentaries
6. Analyse the impact of teaching on every group of learners
7. Offer a real independent alternative to RAISEonline data
8. Provide comprehensive measures of Within School Variation
9. Allow teachers themselves to research the reasons for performance variations
10. Save an assistant head many hours of crunching numbers in Excel trying to do what 4Matrix does automatically......
...That's a start - there are many other good reasons - see this web site for more information.

Q. I reckon we can do everything that 4Matrix does using Excel . . .
A. We have seen some impressive use of Excel by deputy headteachers. However, it is difficult to get much further with Excel than displaying grades and simple charts. We feel that a deputy headteacher's time is too precious to be spent creating the tools they need. We have aimed to create an indispensible professional data-handling toolkit for data managers and those deputy headteachers with a responsibility for standards. It is requests from deputy heads and headteachers that have driven the design of 4Matrix. We are adding new features requested by schools all the time - and these update automatically. Rather than programming Excel, we believe that a deputy head's time can be much better spent working with subject leaders on action-research developments to improve learning. 4Matrix is a highly affordable professional toolkit. There is no good reason not to add this tool to your professional armoury.

Q. We are likely to be inspected soon. How can we be sure if we are doing enough with data?
A. A test question is 'How sure are you of what an inspection report would say about your school?' If you have any doubts at all then you may need to develop your data systems further. Ofsted inspection findings would never be a surprise to a school that knows itself well. 4Matrix provides the independent performance analysis needed when a school is due for inspection because it provides essential data on the performance of the current year 11 - the year that Ofsted will not have information on.

Q. Our results are likely to be a lot better than last year, but we are concerned that Ofsted will judge us on last year's results. Can 4Matrix help?
A. Without effective school-self evaluation that provides new evidence, the inspection report will equate closely to what it says in RAISEonline. (The TES found a high correlation.) Inspectors won't have evidence of the current performance of the terminal year (Y11) pupils and it is hard to collect through observation. 4Matrix has a powerful forecasting function. Schools can use 4Matrix to provide information about current and projected Y11 performance to set against the historical data provided by RAISEonline.

Q. What I really need is something to help me with next year's results. . .
A. One of the most distinctive and useful functions of 4Matrix is to forecast comparative achievement in next Summer's examinations. It will do this for every identifiable grouping of learners and every teaching group.

Q. What is 'Within School Variation' and why is it important?
A. 4Matrix calculates a set of Within School Variation (WSV) measures for every subject, ethnic category and teaching group - or for selected groups of pupils. WSV is important because it reveals where pupils who perform well in some subjects underperform in others. This is the most obvious target for school improvement because it is achievable - we know that the target groups of pupils are capable of doing better - because they demonstrate it across their other subjects.

Q. We publish grade lists and use target grades to focus on improvement.
    What more could we do?
A. 4Matrix goes much further in its analysis of variation than just headline grades. It is a subject leader's research tool for investigating hypotheses about underachievement or high achievement in regard to a large number of factors (including gender, academic range, ethnic group etc.) as well as selected groups (like, pupils from a particular primary school, frequent absentees, differences between teaching groups, main lessons on a Friday, pupils with poor language skills, pupils in the revision club, those students who went on the Geography field trip etc.) It can help answer questions like "Do pupils with good maths skills do better in ICT?" or "Do pupils with high non-verbal reasoning CAT scores do better if they choose Diplomas rather than academic subjects"?

Q. Our Contextual Value Added (CVA) score for KS2-4 is 1028.
    This means that we are good doesn't it?
A. CVA compensates for factors like deprivation. An inner city school will have its value-added totals 'adjusted' so as to try to remove the deprivation factor from a consideration of how well the school is doing. We have undertaken work for a number of local authorities that reveals the relationship between CVA, 'standards' (i.e. %5A-C grades) and 'Value Added Variation'. The evidence of this work shows a non-linear relationship between these three measures.
We are aware of evidence to suggest that CVA often gives an 'inflated' view of the performance of some categories of school. The answer to the question posed here is that a judgment of school effectiveness based only on CVA, or only on 'standards', or both, will not necessarily paint the most accurate picture of how good a school is at the time of an inspection.

Q. The emphasis in the new 2009 inspections will be on attainment.
     How can 4Matrix help our school put pupils' attainment into context?
A. Christine Gilbert, Head of Ofsted, has said that new inspections will use unadjusted data as the starting point for judging the work of schools. The approach described in the 2009 evaluation schedule is to 'take into account' the progress made by pupils and their quality of learning. When judging current standards, inspectors will need to look closely at the current year 11, particularly their current grades, their progress and their quality of learning. A school using 4Matrix can provide a range of additional evidence that Ofsted will take into consideration when making judgments.

Q. Can 4Matrix help a school that is expecting a new style inspection?"
A. Click here for a full answer to this question.

Q. RAISEonline data that the Ofsted team will use won't reflect our current intake.  What can we do?
A. RAISEonline data is the prime source of evidence used to support judgments about the quality of a school's provision. Unfortunately, this means that schools will often be judged on the basis of pupils that are no longer in the school.
The best way to help inspectors to evaluate the current work of the school will be to have secure, independent, quantitative, up-to-date measures of the performance of pupils in the current key stage 4. 4Matrix will help to provide this evidence.

Q. We are a National Challenge school.   How can 4Matrix help us?
A. 'Within School Variation' is a theme of the National Challenge programme to help schools with less than 30% A-Cs to improve. 4Matrix is a well-tested tool designed specifically to promote action-research approaches to school improvement.

Q. We have 65% A*-C GCSE grades. We are a good school aren't we?
A. Probably. But, it really depends on how you wish to define 'good'. A good school may be one with a high proportion of pupils reaching 5 or more higher GCSE grades, but a good school might equally be one that does the best for all pupils. 'Standards' are not necessarily the route to answering this question.
It will always be difficult for a school with a large number of disadvantaged pupils to show that it offers good or better provision than the local selective school down the road.
CVA measures were devised to try to compare schools working in different circumstances. But we believe that additional measures of 'Within School Variation' get closest to measuring how good provision is for every identifiable group of learners. We know that the work of schools is often skewed by attempts to raise their league table positions. This shows up as wider variation, i.e. inconsistent provision. 'Every Child Matters' is the key theme of education. So we think it is important to look at provision where schools are claiming improvements in 'standards'. Schools using 4Matrix can demonstrate that they have high achievement, coupled with a high consistency of provision. This is a 'good' school in our judgment.

Q. How are schools using 4Matrix?
A. 4Matrix isn't just another data tool. It is an action-research approach to school improvement that involves all teachers. 4Matrix has been designed for school leaders and teachers to use for themselves as a process of research and development to find out where particular groups of pupils have the capacity for doing better. In the best practice seen in trials the 4Matrix application was used by subject leaders and teachers to explore and report on variation factors - with subject leaders then reporting to school leaders their plans to reduce negative variation - a process of non-judgmental internal review and development advocated by the National College for School Leadership.