Scoring Activities

EQAO hires scorers to score the open-response questions completed by students as part of the language components of the primary- and junior-division assessments and the OSSLT. All questions on the mathematics component of the assessments are computer scored. If you are interested in scoring the language components of the assessments, please create or update your EQAO profile. Please note that EQAO is unable to accept job applications or employment inquiries by e-mail or by telephone.

The following terms are most commonly used:

• Scoring Window: a specific period (with predefined start and end dates) during which the e-assessment scoring system is available for training and scoring
• Item: a question or task that appears on an assessment or test
• Response: a student’s response to an item on an assessment or test
• Training Module: material for the training session, which includes an introductory video, a scoring guide, training responses, a practice test and a qualifying test
  • Practice Test: a test that scorers complete as part of the training module, used to prepare scorers for the qualifying test
  • Qualifying Test: a test used to assess a scorer’s ability to score student responses accurately
• Batch: a predefined number of student responses to a test question (Each batch is expected to take an average of one hour to score.)
  • Partial Batch: a batch that is partially scored, or a batch with a partial number of responses to be scored, which was automatically assigned by the system
• Expert Scorer: an experienced scorer with high validity
• Validity: the measure of a scorer’s accuracy assessed by examining the agreement between the scores assigned by the scorer and those assigned by expert scorers. The following indices are computed: percentage of exact agreement, percentage of exact-plus-adjacent agreement, percentage of adjacent-low agreement and percentage of adjacent-high agreement.
• Adjacent: a difference of one score code between the score assigned to a certain response by the scorer and that assigned by the expert scorers
  • Adjacent-Low: the score assigned to a certain response by a scorer is one score code below the score assigned by the expert scorers.
  • Adjacent-High: the score assigned to a certain response by a scorer is one score code above the score given by the expert scorers.
• Non-Adjacent: the difference between the score assigned to a certain response by a scorer and that assigned by the expert scorers is greater than one score code.
• Exact Agreement: the score assigned to a certain response by the scorer and the expert scorers is the same.
  • Cumulative Exact Validity Percentage: the total percentage of validity responses scored that match the score preassigned
  • Cumulative Exact + Adjacent Validity Percentage: the percentage of validity responses scored that are either a match with the score given by the expert scorers or are one score code different (higher or lower)
• Validity Response: a response that matches the anchors in the corresponding scoring guide
• Minimum Validity Threshold: the minimum average rolling validity required to continue to score without retraining
• Cumulative Validity Incentive Threshold: the minimum cumulative exact and cumulative exact + adjacent validity percentage required to receive incentive pay

Training and scoring are self-directed through the e-assessment scoring system, which is available 24 hours a day, seven days a week (except on statutory holidays), during the scoring window. Scorers who meet the technical requirements and qualifications can score on their own time and from any location. All the training and scoring resources are available in the e-assessment scoring system.

Once scorers have completed their training and have passed the qualifying test (available in the e-assessment scoring system), they will be able to select batches, which they will have 12 hours to complete. A batch takes approximately one hour to score.

Scorers are not to select any batches unless they are able to score them in the 12-hour period. If a scorer completes the selected batches before the 12-hour period expires, the scorer can select one or more additional batches, for which a new 12 hours is allotted for completion. On the last day of scoring, EQAO may decrease the 12 hours allotted to complete batches to 2 hours or less, and the number of batches that can be claimed at a time to one (to enable the distribution and availability of batches for everyone). EQAO can also revoke claimed batches at any time to facilitate the completion of scoring.

Online scoring is self-directed. Although scorers do not need to score daily, they must score a minimum of 14 batches per week, if available, during the entire scoring window and meet the minimum validity threshold. These requirements are outlined in the Confidentiality, Non-Disclosure and Employment Agreement for an EQAO Scorer (Online) or Committee Member (Online).

If a scorer falls below the minimum validity threshold, the scorer will have the opportunity to retrain and pass another qualifying test. If successful, the scorer will be able to continue scoring. If unsuccessful, the scorer may be withdrawn from their current scoring assignment. Scorers can score as many batches as are available but should not score for more than 48 hours in a work week (from Sunday to Saturday).

Scoring generally takes place during the latter stages of the administration windows of the Ontario Secondary School Literacy Test and the Assessments of Reading, Writing and Mathematics, Primary and Junior Divisions. Scoring invitations are generally sent to qualified candidates near the start of the administration window.

You will receive an e-mail from [email protected] on the first day of the scoring window. The e-mail will contain a link allowing you to register for an account in the e-assessment scoring system. The e-mail will also include important dates by which you must register for your account, complete your training and start scoring. Please refer to your EQAO confirmation letter for more information.

The name in your EQAO profile must match the name in your profile on the OCT website. If you change your name, you must ensure that you update your name with the OCT. Once the name change is reflected on the OCT website, the name will automatically be updated in your EQAO profile. As EQAO’s database is linked with the OCT’s database, a different name will create communication errors between the two sites. If you have a middle name, you can try adding your middle name to the first-name field or the last-name field one at a time, which may rectify the error. Otherwise, please ensure that your name is the same in both profiles.

If you cannot remember your password, click Forgot Password, and an e-mail will be sent to you with instructions on how to reset your password. If you no longer have access to the e-mail account on file with EQAO, please contact us at [email protected] or 1-888-327-7377.

Some scorers, upon making changes to their EQAO profile, may find that a scoring opportunity is no longer available to them. This is a normal occurrence, as the automated system verifies that the details, qualifications and current employment status in an applicant’s EQAO profile match the requirements for the scoring position. It is therefore important that each scorer’s profile be maintained with accurate information regarding address, qualifications, current role in education, experience, etc.

Since there are a limited number of scoring positions, the scoring positions are filled quickly on a first-come, first-served basis, as applicants accept their offers to score. Scoring positions are sometimes added or become available if scorers withdraw. Applicants are encouraged to check their EQAO profiles regularly, as scoring positions can become available at any time.

Qualified candidates will receive e-mails with offers to score. As e-mails may be stopped by firewalls or sent automatically to junk or spam folders, it is recommended that applicants check their EQAO profiles regularly for scoring offers. Applicants can add the e-mail addresses below to their “safe senders” list to avoid having the e-mail scoring invitations end up in a spam or junk folder. Applicants are also encouraged to check these folders periodically.

Here are EQAO e-mail addresses to add to your “safe senders” list:

• [email protected]
• EQAO/[email protected]
• EQAO/[email protected]

Please note that EQAO is unable to accept job applications or employment inquiries by e-mail or by telephone.

This message means that a scoring opportunity was made available, but the position was filled, as multiple applicants received an invitation for the same position at the same time. Applicants are encouraged to check their EQAO profiles regularly, as additional scoring positions can become available at any time.

When you log into your EQAO profile, all the scoring opportunities that you have accepted are listed under “Status” on your profile dashboard page. For more information about each scoring opportunity, click Details beside each opportunity.

You are seeing more than one opportunity because you have qualified for more than one scoring position. You may accept as many of the scoring opportunities as you like, provided the scoring dates are not conflicting. If the dates conflict, please select and accept only one offer.

To accept an offer to score, select Accept/Decline Position beside the scoring position on your profile dashboard page; then click Accept and continue to follow the instructions until you click Confirm.

Unfortunately, the task you have been offered cannot be changed. However, you may be offered a different task to score based on your validity measure during the scoring activity.

You must withdraw from your accepted scoring position immediately. Scoring with EQAO is considered active employment. EQAO relies on every scorer’s work in order to provide timely results to schools, parents and guardians and the public. If you have accepted an offer to score for EQAO, you are expected to participate in the scoring activity. If you do not withdraw from a scoring activity that you will not be able to participate in, you will be considered a “no show” and may not be invited to participate in future EQAO events.

Do not accept an invitation to score until you can commit to the scoring activity.

Log in to your EQAO profile and locate the offer you wish to decline on your profile dashboard page. Next, select Details and then click Withdraw. If you are not able to withdraw through your EQAO profile, please contact EQAO at [email protected] or 1-888-327-7377.

You must update your proof of enrolment in each year of your program. Once you receive your Ontario College of Teachers (OCT) certificate and your information is on the OCT website, you will need to update your current educator role and enter the OCT number in your EQAO profile. Ensure that the first name and last name in your EQAO profile match those on the OCT website.

Your EQAO profile may not be up to date or your qualifications and experience may not have met the requirements of the scoring position. Additionally, there may not be any scoring opportunities available at the moment.

Please refer to the Payment Guidelines for details on remuneration.