In the first article of this series, Robyn’s team gave EPSI the task of integrating two tests into an online testing platform to evaluate over 2,000 bilingual candidates from the Canadian provinces, as well as from other parts of the world.
This second article is about online test creation, step by step.
Document analysis
The first step for EPSI’s IT team is to get familiar with the original test content in order to identify the particularities. In the case of these two tests, here are the initial analysis results:
| Multiple-choice test | Analytical test to be developed |
| General instructions (untimed) | General instructions (untimed) |
| 150 multiple-choice questions in total | Basic information to read before answering the questions (timed) |
| Test divided into two 60-minute timed sections with 75 questions each | 10 analytical questions related to the basic information |
| Some questions refer to specific scenarios that must be read first | |
| Total duration: 60 minutes |
The Compmetrica online testing platform can accommodate these test formats. The multiple-choice test in the case study has over 150 questions. As for the platform, it has no restrictions in terms of number of questions or test duration. The multiple-choice test has two separate parts, each with 75 questions and lasting 60 minutes. Online tests can be subdivided into several sequentially-administered sections. These sections can also be independent, in other words, each section will have to be completed before the next section can be started, at which point applicants will be unable to go back to questions in a completed section.
All tests can include one or more sets of instructions, background information related to the entire test or specific scenarios related to one or more tests within the overall exam. The great flexibility of Compmetrica’s online testing platform is therefore suitable for both test formats presented by Robyn’s team.
EPSI’s analysis can also confirm the time period needed for the integration of an online test and can offer a realistic timeline to the client.
Adaptation and programming of online tests
Since Robyn’s tests were originally designed to be administered in paper-and-pencil format, it’s not uncommon to have to adapt the test content for the online platform. EPSI’s experts read the tests carefully in their original format and identify any passages requiring minor changes. For example, the instructions in Robyn’s tests ask the applicant to write legibly and to turn to page 8 in order to read an important document for establishing the context. Texts are once again proposed to Robyn’s team so that the test content is coherent with the test format.
Once this step is completed, EPSI’s IT team starts programming the test and coding the HTML on the online testing platform. The team’s primary objective is to make sure the online tests remain as faithful as possible to the original pencil-and-paper format. The candidate must not have any advantage or disadvantage because of the way the evaluation process is administered, especially when there are as many candidates as in Robyn’s case.
The IT team enters all the information, i.e. test name, duration, question types and content, answer key and test structure. Images or tables can also be included, and Robyn’s two tests are integrated in both English and French.
Programming of evaluation reports / comprehensive results table
Once the tests are integrated online, the IT team programs individual evaluation reports. Each time an online test is administered, an evaluation report is generated. In Robyn’s case, it was also important to create a table representing the compiled results. Since it is unlikely that Robyn and her team would want to download the 2,000 evaluation reports at the end of the process, EPSI offers a response or result extraction service. In just a few minutes, it is possible to obtain all the scores on a multiple-choice test or to create a document compiling the written responses of all applicants.
Final quality assurance
Before wrapping up the online test integration, access codes are sent to Robyn and her team. This allows them to take the tests themselves to validate the format of the test as it will be presented to the candidates. It’s also an opportunity for the team leaders to familiarize themselves with the online testing platform and to confirm that the tests are in good working order in both languages offered.
Once the parties are satisfied with the test integration, EPSI’s team of experts proceeds to the next step, which involves process-management and inviting the selected candidates to take the test.
|
