In our four previous articles, Robyn’s team entrusted EPSI with the integration of two tests on the online testing platform, for the assessment of more than 2000 bilingual candidates primarily located in the provinces of Canada, but also from around the world. The tests were adapted and integrated on the online testing platform and the applicants were sent an email inviting them to take the tests.
The fifth article in this series looks at the phase most eagerly awaited by Robyn: receipt of the candidate’s answers and scores.
The beginning of the end of the evaluation process
After just under four months, Robyn can finally see the light at the end of the tunnel, now that the assessment of her 2000 candidates is complete. Yet it is important to remember that the applicants had two tests to complete, one with 150 multiple-choice questions and an analytical test with 10 essay questions. In other words, more than 4000 assessments were given during the seven-day evaluation period. The time has now come to deal with the output from these evaluations, i.e. the results of the 150 multiple-choice questions and the answers to the 10 essay questions.
The online testing platform: a whole world of possibilities!
The online testing platform offers a wide array of functionalities to Robyn and her team. First, it instantaneously corrects all multiple-choice and short-answer questions. Second, it provides access to the candidates’ results via the platform’s “client access” functionality. The platform’s other functionalities include test session management, candidate registration, report generation, assessment statistics compilation, a powerful search engine and many more. But for all the benefits of this process and the assignment given to the firm, and although all of these functionalities were available to Robyn and her team, in view of the limited availability of her team members, who all have heavy workloads, she has decided she only wants to receive the applicants’ scores and answers.
Output management
Because the correct answers to the multiple-choice questions were integrated on the testing platform when the test was set up, the results for these questions are available immediately upon completion of the test.
As you may recall, during the initial discussions between Robyn and her team, as well as with the EPSI advisor, it was agreed that two types of reports would be delivered as part of the tests’ output management. Robyn had communicated to EPSI the correct answer to each question asked in the assessment, which the firm entered directly on the platform in order to simplify the scoring process.
Initially, individual reports are generated, containing two separate sections: the results of the multiple-choice questions and those of the essay questions. The multiple choice questions assessed 3 criterions via 150 questions. The individual report provides Robyn and her team with an overview of the candidate’s performance on all three criteria being assessed. A graphic representation of the candidate’s results provides the team with a benchmark comparison with the previously established pass mark. The latter is accompanied by a small paragraph further detailing the candidate’s performance (i.e., strengths, weaknesses are explained, etc.). Because Robyn’s team members are the ones correcting the essay questions, they will be completing this second section. The section consists of the candidates’ responses to any long answer or essay-type questions; a table indicating expected responses as well as a space where correctors can include comments and the candidate’s final mark. The individual report will also be used by Robyn and her team when time comes for them to provide candidates with feedback upon request.
Next, a summary report is also delivered to Robyn. This summary report includes each candidate’s score for each criterion assessed by the multiple choice questions. These are displayed in a single table, thus facilitating Robyn’s management of the next phase in the process: invitation of the best candidates for interviews.
After these reports have been generated, the EPSI advisor transmits the individual reports in PDF format and the summary report in Excel. The advisor takes care to explain the layout of the reports to Robyn and ensures that she has fully understood the data that they present.
The post mortem: a crucial stage
Naturally, to bring the process full circle, a post-mortem review is conducted upon its conclusion. During this review, the different stages and interventions that took place throughout the process are discussed with Robyn. She shares her remarks while the advisor takes note of them, attempting to understand the experiences of Robyn and her team over the course of the process… which can be read in the next and final article in this series.
1. New strategy for online assessments
2. Integration of tests on the platform
3. Inviting candidates to take a test
4. Completing an online test
5. Managing responses and results
6. Lessons learned
