Comparing security awareness training platforms comes down to nine criteria: simulation quality and volume, auto-enrolment behaviour, per-person reporting, framework evidence, deployment effort, MSP and branding flexibility, pricing transparency, content freshness, and reporting format. Most platform reviews score vendors on feature checklists that never touch these, so teams end up comparing app-store screenshots instead of programme outcomes. This guide gives you the nine criteria, what each one changes about your results, and a repeatable scorecard for a side-by-side trial.
TL;DR
- Judge platforms on programme outcomes (click reduction, report rate), not feature counts.
- Auto-enrolment of clickers into follow-up training is the single highest-leverage feature.
- Per-person reporting beats per-campaign reporting: a trend needs per-person data to be actionable.
- Evidence for Essential Eight or SMB1001 matters in Australia — ask what the platform exports.
- Trial two platforms on the same staff segment and score them on the same nine criteria.
The nine criteria
| Criterion | What to look for | Why it matters |
|---|---|---|
| 1. Simulation quality and volume | Templates covering current lures: invoice fraud, MFA prompts, shared documents, executive impersonation; difficulty that escalates | A stale template library trains staff on last year's scams |
| 2. Auto-enrolment | Clickers are enrolled into targeted follow-up training the same week, automatically | The fix lands while the mistake is fresh — the highest-leverage feature in any platform |
| 3. Per-person reporting | Click, report and completion metrics per person per month, not just per campaign | A trend you can act on needs per-person data; campaign-level numbers hide repeat clickers |
| 4. Framework evidence | Exports that map to Essential Eight and SMB1001 controls, ready for an assessor | Australian buyers increasingly ask for evidence, not certificates |
| 5. Deployment effort | Directory sync, SSO, or CSV upload; days to first campaign, not weeks | Long setup kills momentum before the first data point arrives |
| 6. MSP and white-label flexibility | Own branding, per-client portals, per-client reporting for MSPs | MSPs need the platform to be invisible to their clients, not branded as a third party |
| 7. Pricing transparency | Public per-seat pricing or a fast quote — not 'contact sales' for a ballpark | Hidden pricing hides value; opaque pricing wastes weeks of evaluation |
| 8. Content freshness | New modules and templates shipped continuously, not annually | Attack tactics shift quarterly; annual content updates train on yesterday's threats |
| 9. Reporting format | One-page executive report plus per-person drill-down, exportable | If leadership won't read the report, the programme stalls |
How to run a two-platform trial without wasting staff goodwill
The temptation is to trial all shortlisted platforms at once, but that burns goodwill and produces noise. The disciplined version:
- Pick two platforms and one staff segment. A department of 20-40 people gives enough data to compare without exhausting patience.
- Run identical simulations. Same cadence, same difficulty, same two-month window. Divergent simulation design makes the comparison meaningless.
- Score the nine criteria after each round. Same scorecard, same reviewers, same weighting.
- Decide on the trend, not the first result. One hard campaign dips the numbers on any platform; the month-over-month line is the verdict.
This is the same discipline the platform comparison page encodes — criteria-based scoring rather than vendor marketing claims.
The three claims that hide the most
Three platform claims sound strong and hide the most:
- 'AI-powered' anything. An AI-scored phishing detection model, for instance, is largely a vendor claim — and irrelevant to staff behaviour.
- 'X templates' as a raw count. Volume without freshness is a museum. Ask how many templates shipped in the last 90 days, not in the library total.
- 'Compliance-only' content packs. Compliance training teaches a policy, not a behaviour — the criteria table above judges platforms on what they train, not what they certify.
Ask yourself for each claim: does this change what staff do when a suspicious email lands? If not, it is not a criterion that belongs in the comparison.
What to verify before signing
- Trial scope. Ask exactly who is in the trial segment, for how long, and what happens to the data after.
- Data ownership. Per-person risk data is sensitive; confirm who owns it and who can export it.
- Reporting cadence. Monthly is the norm; confirm the report lands automatically rather than on request.
- Framework mapping. If an assessor will review the programme, confirm the export format up front.
FAQ
What is the most important feature when comparing awareness platforms? Auto-enrolment of clickers into follow-up training. It closes the loop between measure and fix within days rather than months, which is the gap most programmes die in.
How long should a platform trial be? Two months on one staff segment, identical simulations, same scorecard. Shorter gives one data point per platform; longer burns staff goodwill.
Is per-person reporting a privacy concern? Handled correctly, no. Publish organisation-level numbers broadly, treat per-person data as coaching input, and never attach consequences to a click — the same discipline a phishing reporting culture requires. Publish a shame list once and reporting dries up.
Do MSPs need white-labelling? If clients see a third-party logo on their security reports, the MSP's own brand value erodes. Per-client portals and branded reports are a practical requirement, not a nice-to-have.
How much should a platform cost? It varies by headcount and features; published self-serve rates run from a few dollars per employee per month upward. Budget from current vendor pricing, not a blog figure.
One last thing
The best way to compare platforms is to watch what happens to the click and report numbers over two months on the same staff segment. Features are inputs; behaviour change is the output. Score the output.