Motivation

Phishing via email remains a major entry point for cybersecurity and privacy incidents in hospitals. In the European Union, the EU NIS2 Directive puts hospitals under regulatory pressure to act. Limited ressources may compell hospitals to default to minimum-effort, off-the-shelf anti-phishing interventions in enterprise email solutions.

If hospitals are vulnerable to phishing and required to act, how effective are these common and readily available anti-phishing interventions?

Methodology

We conducted two phishing simulation studies in a large German university hospital, targeting 7,044 email accounts.

• In Study I, we compared 12 phishing emails using a Plackett–Burman design to assess which characteristics influence phishing susceptibility.
• In Study II, we used a between-subjects design to test 11 widely available in-situ anti-phishing interventions.

Hospital staff were divided into four groups to account for heterogeneity in hospital staff:

• Medical Service
• Nursing & Functional Services
• Administration & IT
• Other Personnel

Every staff group was further divided into 12 sub-groups. Every sub-group received a slightly different email or anti-phishing intervention (details in the paper).

We measured hospital staff risky behavior after receiving a phishing email. We define risky behavior as inputing credentials into a phishing login. The study procedure is depicted in Figure 1.

Main Findings

Already a small number of phishing emails poses a high risk; risk declines after 3 days

In a phishing campaign with 12 emails, we found a 6.5% probability that staff would engage in risky behavior (e.g., interacting with a phishing login) within the first 12 hours. Just 69 delivered emails suffice for a 99% chance of at least one risky interaction within the first 12 hours. Risk declines sharply after 3 days. Nursing and functional services were slightly more vulnerable than medical, administrative, IT, and other staff.

Figure 2 illustrates the hazard rate h(t), which describes the instantaneous risk that an employee will interact with the login prompt on the phishing website at a specific point in time t, given that they have not done so before.

Good to know: Click rates were 9–23 percentage points higher than actual login interactions. Click rates overestimate risky behavior when used in phishing simulations.

Hospital staff reacts differently to the same phishing emails

Features like timing, context, tone, and framing had heterogeneous effects. Emails exploiting email timing, loss aversion, urgency, authority, or liking were more effective. Treating staff as homogeneous leads to misleading results. Even within the same email, login interaction rates ranged from 18–40%.

The following table summarizes which feature of a phishing email increased hospital staff risky behavior (i.e., increased login interaction rates). We compared 10 email features. "-" denotes no effect.

Common in-situ anti-phishing interventions can be useful but are imperfect

Technical interventions were most effective: spam filtering and warning banners reduced risky behavior by 81–94%. Friction-inducing approaches like disabling links reduced it by 61%, and warning pages with nudges by 44%. External tagging ([EXTERNAL]) was inconsistent—effective only when combined with banners or multi-field tagging (53–58% reduction). Subject-line-only or From-field-only tags were negligible. Display name suppression was similarly weak.

Interventions that shift the burden onto users cannot be relied on. Our study likely overestimates the effectiveness of in-situ anti-phishing measures, since people tend to habituate after repeated false alarms. Generic [EXTERNAL] tags in particular are likely less effective in practice than in our study.

Click on the rows in the table below to see an example implementation.

Category / Condition

All Staff

Medical

Nursing & Functional

Admin & IT

Other Staff

Suppress Display Name

[EXTERNAL] Tag: From Field

[EXTERNAL] Tag: Subject Line

[EXTERNAL] Tag: Banner

[EXTERNAL] Tag: Combined

Phishing Banner: Plain Text

Phishing Banner: HTML

Phishing Banner: HTML + Unverified Sender

Friction: Spam Folder

Friction: Link Disabled

Friction: Active Warning Site

Table 2: Effect of in-situ anti-phishing measures on phishing susceptibility, divided by hospital staff groups.

Some hospital staff reacts with strong negative emotional responses to phishing simulations

Among 530 staff who fell for the phishing attempt, 60% reported to feel scared, 41% ashamed, 36% guilty, and 25% hostile towards the phishing simulation. These emotions were strongly correlated. There were no significant differences across staff groups.

These results must be viewed in light of the broader working conditions in healthcare, where EU health and social care workers report the highest levels of psychosocial stress—driven by chronic time pressure and workload.

Acknowledgements

We would like to express our sincere gratitude to the participating hospital, particularly the CISO team, for making this study possible. We thank the IT department, staff councils, and data protection officers for resolving technical, legal, and ethical issues. Above all, we are deeply grateful to the hospital staff, whose participation enabled us to gain a deeper understanding of phishing susceptibility. We also want to thank the reviewers and our shepherd for their incredibly valuable feedback, which helped us improve this paper. This research was supported by the German Federal Ministry of Health with grant number ZMI1-2521FSB801.