When a recruiter tells us "the AI got this one wrong", we go and read the justification together. Most of the time the grader did exactly what the criterion said. The criterion just did not say what the recruiter meant. That is good news, because criteria are cheap to fix and a role regrades in minutes. It also means the quality of your shortlist is mostly decided in the ten minutes you spend writing the rubric. These are the rules we have settled on.
1. Write things a CV can answer
A criterion is gradeable when a careful person could read a CV and say yes, no, or partly, and point at the line that made them say it. "Strong communicator" fails that test. "Has presented to executive or board audiences" passes. The difference is not that one is stricter; it is that one is checkable.
| Hard to grade | Gradeable |
|---|---|
| Experienced in payroll | Five or more years processing end-to-end payroll in Australia |
| Knows the systems | Hands-on with at least one of MicrOpay, Chris21, ADP or Employment Hero |
| Good with awards | Has interpreted a modern award or enterprise agreement in a previous role |
| Team player, culture fit | Leave it out. Ask about it in the interview, where it belongs. |
If you catch yourself writing an adjective, ask what evidence on a CV would prove it, and write that instead.
2. Split must-haves from nice-to-haves, and weight them
Three to five must-haves and two or three nice-to-haves is enough for almost every role. More than that and the weights get so small that nothing moves the score. Give each a weight so the total is 100, and be honest about the split: if the platform experience is really the thing you cannot train, it should carry the biggest number, not sit third because that is the order it came to mind.
A worked example for a senior payroll officer:
- Must-have, 40: five or more years processing end-to-end payroll in Australia.
- Must-have, 25: hands-on with at least one major payroll platform.
- Must-have, 15: has interpreted a modern award or EBA in a previous role.
- Nice-to-have, 12: has been through a payroll system migration.
- Nice-to-have, 8: exposure to New Zealand payroll.
3. Keep protected attributes out, including the proxies
Nobody writes "under 35" as a criterion. People do write "recent graduate", "digital native" or "energetic", which are the same thing wearing a hat. "Unbroken work history" penalises carers and anyone who took parental leave. "Local experience only" is a proxy you should be able to justify on the role's actual requirements, or drop.
Alert Hire strips names, photos, dates of birth and addresses before grading, so the grader cannot see the attribute directly. It can still see a proxy if you write one into the criteria, which is why the rule is yours to keep. A useful test: if a candidate asked why they were marked down on this criterion, could you say it out loud to them?
4. Put what a CV cannot tell you in the conversation
A CV does not know the candidate's notice period, whether they still have a forklift ticket, whether they can start at 6 am on a Tuesday, or what they expect to be paid. Trying to grade those from a CV produces noise. They belong in the screening questions, where the candidate answers them directly and the answer lands next to the score as a field rather than a guess.
The usual list: notice period and start date, right to work, licences and tickets with expiry, shift or travel flexibility, salary expectation, location or commute, and any hard requirement that a CV tends to leave out.
5. Set the threshold by asking one question
The threshold is the score above which a candidate gets the screening questions the day they apply. It does not reject anyone below it. The question that sets it well is: of the people who score here, would I want to spend ten minutes with all of them? If yes, the line is about right. If you would skip half of them, raise it. If you keep starting conversations with people below it by hand, lower it. You can move it at any time and the role regrades around it.
6. Test the rubric on five people you already know
Before you turn it on for a live role, run it on five past applicants whose outcome you remember: two you hired or would hire again, two you would not, one borderline. If the ranking matches your memory, you are done. If it does not, read the justifications and you will usually find one criterion doing something you did not intend. Fix the criterion, not the grader.
Alert Hire grades every application against per-role criteria like these and starts the screening conversation with everyone above your threshold the same day. How grading works, or book a demo and bring a role.
Sources
- Australian Human Rights Commission with LexisNexis, AI and recruitment compliance checklist. Checked 10 September 2026.
- Office of the Australian Information Commissioner, Australian Privacy Principles guidelines, Chapter 1: APP 1. Checked 10 September 2026.
Every source here was opened and read on the date shown. If one has moved or changed, tell us at admin@alertai.com.au.