Machine Learning Model
How the Scam Message Scanner learns to detect new scams
A scam-detection tool is only as trustworthy as the explanation of how it makes its decisions. This page is the open methodology — what the ScamSupport detection model is, what it learns from, how the training pipeline works, what the model can and cannot do, and what we will tell you when we cannot tell you the answer. The full validation results live on the About the Scanner page; this page is the “how it learns” companion to that “what it does” reference.
The headline things to know up front: every piece of analysis happens inside your browser, so the model never sees your messages and your messages never reach our servers. Training data is contributed by users who explicitly opt in, anonymised before processing, and used only to improve the model that runs locally on your device. The accuracy figures below come from held-out test sets containing examples the model never saw during training. And no machine-learning model is perfect — the limitations section lower down lists the categories where ours is weaker and what to do about it.
Training Methodology
ScamSupport's detection engine is trained on thousands of verified scam examples and legitimate emails. Our model continuously learns from new reports and feedback to adapt to emerging threats.
Feature Extraction
The model combines outputs from the 23 detection checks documented in detail on the About the Scanner page. Those checks are organised across the following feature classes:
- Linguistic patterns and word frequency
- URL characteristics and domain reputation
- Sender header analysis
- Emotional language patterns
- Brand impersonation indicators
- Grammar and spelling quality
- Request patterns (personal info, money, actions)
- Temporal patterns and urgency signals
Model architecture
The 23 detection checks produce a feature vector that feeds a small neural-network aggregator. The aggregator combines the per-check signals into the 0–100 risk score and the verdict tier the scanner returns. The whole model — the check library plus the aggregator weights — is approximately 18 KB and runs entirely inside your browser; inference takes 2–3 milliseconds per message on a modern phone or laptop.
The architecture is deliberately small. The goal is a tool that runs reliably on a five-year-old phone over a flaky connection, not the largest model we could fit. A larger architecture (gradient-boosted trees, deep transformers) would buy at most a marginal accuracy improvement on this kind of structured detection task while making the tool slow, memory-heavy and impossible to run offline. The structural patterns that distinguish scam messages from legitimate ones (sender-domain mismatch, urgency markers, credential requests, callback asks) do not benefit from large-model capacity.
Continuous Learning
Our model improves constantly through:
- Community Feedback: Reports from users help identify false positives and false negatives
- New Threat Detection: Emerging scam patterns are automatically identified and added to the model
- A/B Testing: New algorithms are tested against existing models before deployment
- Periodic Retraining: the detection-check library is updated when new scam patterns are reported and verified; the aggregator weights are retrained against the updated feature set on a rolling cadence rather than a fixed calendar
- Adversarial Testing: Security team regularly attempts to craft messages that evade detection
Performance by Scam Type
The validation set is composed of 351 cases labelled across 7 UK scam categories. The macro-averaged F1 score across the categories is 98.5%, with macro precision and macro recall both at 98.5%. The full per-category breakdown — which checks fired most heavily for each category, what borderline cases look like, what the false-positive failure modes are — is documented in detail on the About the Scanner page, which is the canonical reference for the scanner’s validation results. The per-category numbers there are kept aligned with the next training run.
Data Privacy in Training
All training data is anonymised and processed in compliance with GDPR and UK data protection laws:
- Personal information is removed before analysis
- Training only occurs on user-submitted content with explicit consent
- Model weights are proprietary but detection methodology is transparent
- No individual messages are stored or indexed long-term
Why we publish this
Most consumer scam-detection tools treat their model as a black box. The output is a verdict; the inputs and the training methodology are not disclosed. We deliberately publish the architecture, the feature set, the training pipeline, the held-out performance numbers and the limitations because (a) a tool people are asked to trust with a financial-loss decision should be inspectable, and (b) a published methodology is the only honest way to invite challenge to the cases where the model gets things wrong. If you spot a class of message the model handles badly, the report form feeds straight into the next training run; the issue tracker on our public repository takes structural feedback on the methodology itself.
The other reason is regulatory. UK consumer-protection law and the FCA’s Consumer Duty both place a higher bar of explainability on tools used for harm-prevention decisions than on entertainment products. We are not regulated as a financial firm, but the discipline of explaining how a decision was reached is a reasonable one to adopt voluntarily for any tool that someone might use to decide whether to send money, share personal information, or trust a contact.
Limitations
No ML model is perfect. The Scam Message Scanner has known limitations:
- Sophisticated targeted scams may evade detection
- Legitimate emails with similar characteristics might trigger false positives
- Model lag: new scam tactics take time to integrate
- Language-specific: primarily trained on English messages
- Context-blind: doesn't have relationship/history data about the sender
Bulk Email Tester
Test up to 100 emails at once against the 24-check detection engine
Include subject line, sender, and body for best results. Supports plain text and .eml paste.
Report a False Negative
Did the Scam Message Scanner score a scam too low? Help us improve detection by logging the miss.