I’ve tried giving screenshots of phishing emails to a local Qwen instance and so far it always correctly detected it as scam, even points out the exact elements that it based its judgement on. Sending screenshots to it ad-hoc isn’t too scalable for family and friends. I’d like to be able to either forward emails for screening, or perhaps have it screen everything from a mailbox.
Has anyone done anything like this? Is there anything self-hostable that does this?
You must log in or register to comment.
I’d be pretty concerned about prompt injection risks with feeding a LLM unsanitized data. You definitely need a good harness around it…
Good point. It’ll have to have no access to the internet or anything local outside of its container. Just text in, text out.
You could (and probably should) use a system-one style inference system for spam classification. Much cheaper and the structured output means it’s impossible to go rogue and curl some malware or whatever. It can absolutely misclassify but its output is programmatically structured and just ranks a pre-selected set of output tokens.
In your case that’s
Spam
Not_spam
I only read about that today, so I hadn’t even considered that angle. They’re doing some cool stuff in that space.
jeffis the self-hostable one that does best as far as I know.You actually have a lot of options in this space. You can take the prefill engine of just about any LLM and turn it’s transformer into a classifier by lobotomizing out the decoder. You can also use a diffusion model on single pass to surprisingly competent result.
If Jeff looks easy enough to deploy by all means start there, but don’t discount the idea if Jeff sucks; you have a lot of options.
Yeah if it’s just a basic input with a function call for spam or not spam the risk is low. What’s the worst case outcome, it tricks it into saying no it isn’t a scam and you have to delete it manually? Hahaha
You definitely don’t have to go full LLM for this. There are a wide variety of local spam filtering tools available, depending on what email application you use. If you really want to use modern ML, I’d go with something like Laya as it is tunable and runs locally.
If your email service offers an MCP connection (Google does), you can have your AI connect and examine emails and manipulate them. You can hook that up to qwen using maybe lmstudio. I haven’t tried it self-hosted, but it sounds pretty straightforward.
Just be aware that those phishers are now also scanning their phishing emails to see how they can get better too! The wheels on the bus go round and round! Round and Round! Round and Round!
Kinda what prompted me to think about this. I started seeing some too good phishing emails recently.
Probably better to ask this in an ai community. However fwiw there are a few skills for generic imap checking. I’ve set up a local openclaw gateway and model with an automated task to check my email in the morning and basically prod me into replying. The same concept should be applicable to spam detection too.
The ability to do it aside, if feel it should be brought up that handing off things like this can lead to giving anything that gets through an “unofficial seal of approval” so to speak.
“This email looks phishy, but AI says it’s safe so I’ll ignore my initial concerns” is going to inevitably happen.
There’s also the fact that using a model to detect scams, means you can use that same model to train a different model on how to build scams that bypass the detection.
This also feels a little like a hammer looking for a nail. There have been tools to handle this without the insane computational requirements of an LLM for many years.
deleted by creator
