A phishing simulation pilot tells you whether your training platform, templates, and reporting workflow actually change behaviour before you commit budget to a company-wide rollout in 2026.
Key takeaways
- Run a phishing simulation pilot on 10-15% of staff for 4-6 weeks before full rollout.
- Baseline click rate first - skipping it makes post-pilot comparisons meaningless.
- A pilot that cannot show a 5-10 point drop in click rate by week 4 needs a template or cadence fix, not a bigger rollout.
- Exclude IT and security teams from the pilot group; their click rates skew results low.
- Tie go/no-go decisions to a fixed threshold set before the pilot starts.
Why this matters
Most security teams skip the pilot and roll phishing simulations out to the entire company on day one. That is how you end up with 400 confused employees emailing IT, a help desk that blocks your own simulation domain as spam, and a click-rate report nobody trusts because there is no baseline to compare it against.
A pilot fixes all three problems in a contained group before the mistake is company-wide. It also gives you a real number to bring to leadership instead of a vendor marketing claim. If you are evaluating a phishing simulation platform for the first time, the pilot is where you find out if the reporting actually holds up under a live audience, not a demo account.
What you will need
- A pilot group of 50-150 employees (roughly 10-15% of headcount for mid-size organisations)
- Platform access with admin rights to build and schedule campaigns
- A pre-pilot baseline click rate, even a rough one from a single test send
- Three to four escalating phishing templates (generic, brand-spoofed, and one targeted/spear variant)
- Sign-off from one executive sponsor who will see the results regardless of outcome
- A 4-6 week pilot window on the calendar, not whenever you get to it
- A fixed go/no-go threshold agreed before the first email sends, not after
The steps
1. Define success criteria before you send anything
Write down the number that decides go/no-go before the pilot starts, not after you see the results. A common threshold is a 5-10 percentage point drop in click rate between the first and third simulation, plus a reporting rate (employees who flag the email instead of clicking) above 20% by week 4.
If you wait until after the pilot to decide what counts as success, you will rationalise whatever number you get. Put the threshold in writing and share it with your executive sponsor on day one.
Common mistake: setting the bar so low that any result looks like a win, which defeats the point of piloting at all.
2. Pick a representative sample, not the easy group
Select 10-15% of the workforce across at least two departments and two seniority levels. Do not pilot on the IT team alone - their click rates run artificially low and will not predict how finance or sales will behave.
Mix in at least one department with high email volume (sales, accounts payable) since those are the roles attackers actually target.
Common mistake: cherry-picking a tech-savvy department to make the first report look good to leadership.
3. Set a baseline before the first real campaign
Send one soft, low-pressure template in week one purely to measure where the pilot group starts. Do not count this as a failure if click rates hit 25-35%, that range is normal for a first send across most industries.
Without a baseline, your week-4 numbers are just numbers. With one, they are evidence you can put in front of a CFO.
4. Run three escalating campaigns over four weeks
Sequence the templates from generic (a fake shipping notification) to brand-spoofed (a fake internal IT request) to targeted (referencing a real project or manager name, built from public info only). Space campaigns 7-10 days apart. Faster than that and you are testing memory of the last email, not genuine awareness.
Common mistake: running all three campaigns in one week to get through it faster, which collapses the data and hides whether training between sends had any effect.
5. Track click, report, and remediation completion separately
Do not just measure who clicked. Track who reported the email to security, and who completed the follow-up training module within 48 hours of a fail. A pilot with a falling click rate but a flat reporting rate means people are getting lucky, not getting sharper.
6. Brief managers before results go wider
Tell department managers what is happening and why, at least a week before the first simulation lands. An unbriefed manager who gets blindsided by an angry employee complaint will kill your pilot credibility faster than a bad click rate ever could.
7. Compare against the threshold and make the call
At the end of week 4-6, put the actual numbers next to the threshold you set in step one. If click rate dropped by your target margin and reporting rate cleared 20%, that is a Go for full rollout. If click rate barely moved, that is a Hold - fix the template mix or cadence and re-pilot on a fresh group before scaling.
A pilot that misses its own threshold and gets rolled out anyway usually fails the same way at 10x the headcount.
Troubleshooting
Click rate hits 0% on the first send. The template was too obvious or IT already flagged the sending domain internally. Check spam filter logs before assuming the pilot group is unusually alert.
IT blocks the simulation emails outright. Whitelist the platform sending domain and IPs with your email security team before campaign one, not after a failed send. This is the single most common reason pilot data comes back blank.
Employees complain about gotcha tactics. Reframe the pilot messaging around skill-building, not punishment, and make sure the exec sponsor communicates that framing directly.
Results plateau after week 2. This usually means the templates are not escalating in difficulty. Swap in a spear-phishing variant referencing a real internal process for campaign three instead of reusing the same brand-spoof format.
No baseline data exists. Run one extra soft-touch send before the official pilot start and treat that as week zero. It costs one week but makes every later number comparable.
Reporting rate stays near 0% even as click rate falls. People are getting more cautious but not reporting proactively. Add a one-line reminder in the post-click training module showing exactly where the report phishing button lives in your email client.
Tools and resources
- A phishing simulation platform with escalating template libraries and per-user reporting
- A shared tracking sheet for click, report, and remediation-completion rates by week
- Email security whitelisting docs, shared with IT before campaign one
- A pre-agreed go/no-go threshold document signed by the exec sponsor
What to do next
Once the pilot clears its threshold, do not jump straight to a full-company blast. Build the rollout as a proper campaign with its own cadence, template rotation, and reporting cycle rather than just widening the pilot audience list.
FAQ
What is the best way to run a phishing simulation pilot in 2026? Pilot on 10-15% of staff across two or more departments for 4-6 weeks, with a baseline send in week one and escalating templates after. Set your go/no-go threshold before the first email sends.
How long should a phishing simulation pilot run? Four to six weeks is standard, enough time for three escalating campaigns spaced 7-10 days apart.
How many employees should be in a phishing simulation pilot group? H6ifty to 150 employees, or roughly 10-15% of headcount for a mid-size organisation.
Is a phishing simulation pilot necessary before full rollout? Yes, if you want data you can defend to leadership. A pilot catches template, delivery, and reporting-workflow problems in a small group instead of a full-company rollout.
What click rate is normal for a first phishing simulation? First-simulation click rates typically fall between 20% and 35% across most industries. That range is a baseline to improve from, not a failing grade.
Should IT staff be included in the pilot group? Include a small number for visibility, but don't rely on IT alone to represent the organisation. Their click rates run artificially low.
What happens if the pilot does not hit its threshold? That is a Hold, not a failure. Adjust the template mix, tighten the send cadence, or extend the baseline period, then re-pilot on a fresh group.
Can a phishing simulation pilot run across multiple offices or remote teams? Yes, but delivery timing across time zones affects open and click data, so schedule sends around each location's business hours.
One last thing
The pilot detail that gets skipped most often is not the template or the sample size, it is whitelisting the sending domain with IT before campaign one. A blocked simulation email does not just skew the click rate to zero, it burns a week of your 4-6 week window before anyone notices the report is empty.