Using Online Testing to Assess Learner Progress and Help Teachers Improve Class and School Performance

This year saw €5m allocated to Teaching, Learning and Assessment which should facilitate online learning and testing. Time-critical feedback is essential to allow teachers to intervene at class or student level, targeting individuals and strands. Continuous knowledge checks will achieve this, but there are teething problems.

This article describes the information recently gleaned about areas of course difficulty in Mathematics encountered by third- to sixth-class learners, and the lessons to be learnt about applying online tests to primary school learners.

The online tests were taken by the learners as baseline tests at the start of the school year. The results, learner by learner and question by question, were presented to their teachers. The purpose was to assess their ability in various areas of mathematics (referred to here as strands) at the start of the year. Against this base level their progress was measured during the ensuing school year. It was measured weekly through online knowledge checks, testing them on the specific areas (strands) dealt with in class that week.

A total of 483 third- to sixth-class learners from fifteen Gaelscoileanna across Ireland completed the questionnaires. Learners in third class had the most difficulty with the strands Fractions, Place Value, Money, and Multiplication. Fifth- and sixth-class learners also had difficulty with Volume, which was not tested in third or fourth class. Money, Time, and Fractions caused problems across all classes from third to sixth. Time as a strand was measured in fourth, fifth, and sixth classes. It can be seen from the table below that it is causing significant difficulties in each.

Below are some examples of questions asked and percentage performances on average by those who attempted these questions.

Fractions

Anna ate 1/2 of her chocolate. She gave 3/8 to John. What fraction had she left?

Select one:

• 3/8
• 1/8 Correct
• 1/4

39% of learners who attempted this question answered it correctly.

Place Value

The children were playing a game of Ring Toss. John scored 2 units and 3 tens. What was his score?

Select one:

• 20
• 32 Correct
• 23
• 30

52% of learners who attempted this question answered it correctly.

Money

Maria is going shopping. She has 3 × 50c coins, 3 × €1 coins and 2 × 20c coins in her purse. How much money has she?

Select one:

• €3.90
• €4.70
• €4.90 Correct

52% of learners who attempted this question answered it correctly.

Time

2 hrs 47 mins + 3 hrs 41 mins + 2 hrs 32 mins = ?

26% of learners from fifth and sixth class who attempted this question answered it correctly.

Multiplication

If Katie’s kitten sleeps for eight hours every day, how many hours does she sleep in a week?

Answer: 56 – Correct

46% of learners who attempted this question answered it correctly.

The table below shows the average level of correct attainment in all questions in each of the five strands Fractions, Place Value, Money, Time, and Multiplication. The percentages cited above relate to the specific questions presented above for illustration. These five strands show the lowest performance of all strands. They are much lower than the average for all strands, as can be seen by comparing individual strand performance levels with the average of all strands shown in the rightmost column.

We have results for a smaller cohort of learners in fourth, fifth, and sixth class in the same Gaelscoileanna, and early indications are that Money, Time, and Fractions continue to be weak through to sixth class.

Next year we intend to complete a more comprehensive study of Gaelscoileanna and English-speaking schools across third to sixth class to see if any differences exist between them. Longitudinal studies in the coming years, following the same learners over third, fourth, fifth, and sixth classes using weekly online questionnaires compared with learners not using them (the control group), will prove conclusively whether frequent online questioning improves learner performance.

We are seeing that frequent online testing improves learner performance. Weekly performance questionnaires are showing an average improvement of 10%. The next step in the research will be to establish how it does so. A possible reason may be the information value of the data focusing teachers’ efforts on areas of difficulty as disclosed by the learner responses. Other factors may be the benefit of reinforcement of learning through repetitive questioning, or better adapting of teaching to different learning styles as teachers focus more on the weakest learners and strand weaknesses disclosed by the testing.

Resource teachers are also a key part of the process, as they can identify the weaker students and work one to one on specific areas or strands.

Problems with Online Surveys as a Measure of School Performance

We are also seeing that 21% of learners are failing to finish tests of 50 questions. Feedback from schools is that they are seeing similar issues with the new online Drumcondra tests, which have now been running for two school years, 2017/18 and 2018/19 (new format 2018). Principals are alarmed at this data and are keen to use online practice testing to highlight these learners who are struggling with online testing – a new concept for primary schools.

The practice tests used in our survey allow teachers and principals to review individual learner weakness, class strand weakness, and learners who may be slow to complete the tests or are unable to do so within the designated times.

Problems with Online Surveys of Learners at Primary Level

The questions then arise: Why do learners not complete the entire questionnaire, and what are the implications for assessing learners and their schools? Possible reasons for non-completion could include:

1. bad timing when completing the questionnaire due to lack of practice
2. tiredness of the learner mid-completion
3. avoidance of difficult questions by the learner.

Failure to complete the questionnaire could lead to a misguided conclusion. For example, if the learner fails to answer the last 30% of the questions, one could conclude that these indicate the learners’ weak areas. But reasons i, ii, or iii may apply instead. Teacher interpretation is important here.

As regards school comparisons, bear in mind that schools whose learners complete the test will show higher performance on a comparative scale. There could also be a false impression of general improvement in mathematical skills of primary school learners in situations where the learners get more used to online testing from year to year. There is much to be learnt in the coming years.

This online test system is currently supplied to schools by Achieve Online Learning Ltd, and the material is available in both English and Irish. In essence, the learners complete the weekly mini-test online when the strand unit is complete, thus informing the teacher on the class’s and students’ understanding before moving to more complex related strands.

Because the results are in digital form, with results by learner and by class average, this informs the teacher on the improvements attained. The purpose is to enable teachers to focus their energies and teaching on the weaker learners and on the specific aspects taught in that week which the class as a whole is finding difficult.

As more schools adopt this system, much more data will be available, with a resulting increase in confidence in the conclusions reached. The system also presents a means to assess the effectiveness of different teaching styles and techniques on a real-time basis over a large sample of learners. Hopefully what follows will be a more informed development of teaching materials and a significant national improvement in mathematical skills among primary school learners.

Table: Average levels of attainment on all questions in each of the five strands