Current Filter: Document>>>>>Case Study>

   

Marking music theory examinations is demanding but ABRSM (the Associated Board of the Royal Schools of Music) has more experience than most. By using DRS e-Marker for electronic marking, ABRSM has improved quality assurance, automated results handling and reduced the risk of losing examination scripts.

Founded in 1889, ABRSM provides teachers and candidates with instrumental and theory examinations. Linked to the Royal Academy of Music, Royal College of Music, Royal Northern College of Music and the Royal Conservatoire of Scotland, ABRSM delivers over 650,000 examinations and assessments each year in 93 countries.

Held three times a year in 200 centres around the United Kingdom as well as locations abroad, over 120,000 theory examinations are completed annually. Before introducing DRS e-Marker technology for the UK, Malaysia and Singapore, examination scripts were manually sorted and distributed for marking. ""The whole operation was quite cumbersome and lots of risk was introduced by moving paper around,"" says IT Director Eugene O'Donnell.

CHANGE REQUIRED
Although everything had worked well from a marking quality point of view, a realisation that other examination boards were automating provided the impetus to change, as O'Donnell notes: ""It was to do with streamlining the process and getting the results to candidates more quickly.""

An evaluation of electronic marking technologies helped ABRSM appreciate the true potential. Although the marking process might be speeded up, the benefits were more likely to come from managing the quality of marking and its progress in real time. Any solution also had to cope with occasional rubbing-out residues on music manuscript-sized paper. Neither was an obstacle for DRS thanks to its specialised scanning technology.

""DRS showed us around their facilities and that engendered confidence. We were paid more attention and they cared more about the requirement,"" says O'Donnell. ""Upon reflection, our processes weren't particularly robust. Partnering with DRS made our processes much more professional.""

EXPERT ADVICE
ABRSM selected the DRS e-Marker service for the first five grades of its eight grade music theory examinations. DRS gave expert advice about changing designs and adding barcodes for traceability and security. An XML interface was built to link DRS e-Marker to the ABRSM candidate database. ABRSM received good information about assessment, marking and setting tolerances from DRS consultants before running a successful pilot test.

Ongoing support is provided by DRS while in-house seminars, computer-based training and a training system now prepare new examiners. Completed examination scripts from the UK, Malaysia and Singapore are sent to DRS three times a year for scanning and processing. Around 70,000 papers were marked online last year by 54 trained examiners.

Thanks to DRS e-Marker's segmentation capability, examiners see a series of individual questions rather than whole examination scripts. The freedom from physical handling enables them to concentrate on accurate marking while their progress is regularly monitored to ensure quality standards. This has made the overall system more robust with real-time information available and consistent marking standards.

CONSISTENT APPROACH
A powerful combination of seeded questions and double-marking ensures consistency. Pre-set tolerances allow the marking to stand, refer the marked question to a moderator for adjudication, or lock out particular examiners for scrutiny. Examiners benefit from the feedback while data gathering assists post-examination analysis.

""We can configure the system to take action when quality is not met. The system can lock examiners out and they have to talk to a senior examiner about exactly what went wrong,"" says O'Donnell. ""We can also more or less guarantee that everything is done by the deadline because we can see what's left days in advance. DRS is an entirely professional organisation which has excellent people who know what they're doing and give good advice.""
More info: www.drs.co.uk