Combatting Self-XSS (Part 2)

The immediate context to this post is Self-XSS but it may have a wider context in avoiding other Internet scams.

I’d like to frame Self-XSS in terms of a human script execution engine. It’s an interpreter - and probably not JITed, but it works with a broad and poorly specified grammar. Not all humans have the same interpreter, but there’s enough of a dominant language to provide a mono-culture which makes attack easy.

However not all script execution engines are created equal. Some come with advanced scam detection routines. Some have random timeouts and many have surprising execution errors. It’s not my intention to be derogatory, but it’s a fact that not everyone has English skills the same.

So Self-XSS is a script that uses this script execution engine to coerce someone to do the attacker’s bidding, probably to their own detriment. If you're writing a scam script, you need to keep the scripts simple enough to avoid the victims’ scam-detection, timeouts and errors.

These problems mean that we might be able to draw a graph of script complexity vs completion rate (where ‘completion’ is a scam that succeeded). It might look like this:

I’d like to suggest that humans are not that simple and that we shouldn’t expect something as predictable as that.

Many (most?) of the tasks that we complete on computers can be done purely using unconscious / heuristic / system 1 processing (in dual process) terms. That is to say using parts of the brain that are fast, pattern-based and non-analytical. Its less common that we need to use conscious / systematic / system 2 processing (slow, lazy, analytical). (“Thinking, Fast and Slow” is probably the canonical work to understand these terms in more depth)

I’d suggest that, for most people, scam detection is a system 2 operation. So forcing use of system 2 to prevent scams might help. (Obviously this is at odds with the “don’t make me think” approach to good UX design - I’m obviously not arguing that bad design is a good thing).

We don’t have a lot of data for Self-XSS. The data that I know of is:

It seems reasonable that an infinitely simple script would always succeed, and that at some point of complexity the completion rate will drop to 0. So that something like 5 data points.

It would be fair to criticise this as ‘not much data’, which is part of the point:

• If you have more data in places where Self-XSS attacks have worked, please publish them and tell me.
• If you know of research into the use of system 1 vs. system 2 in detecting scams, I’d love to hear about it.

However, given the lack of data, it seems to me that there is a surprising drop-off in completion rate with increasing script complexity. There isn’t much any evidence for attacks that were somewhat complex, and got some people - it looks as though all attacks that were somewhat complex were surprisingly unsuccessful.

So the drop-off rate might look something a bit more like this:

Which might in some small way indicate a point in script complexity where system 2 needed to get involved, and where the analytical part of system 2 said “Wait, am I being scammed?”

Summary: We shouldn't think of scams purely in terms of smart people not getting scammed, and some sort of level of intelligence providing scam protection. Perhaps for many people, scams are detected when system 2 gets involved, and there may be ways in which we can design software to protect people.

How does this affect browser developer tools? I believe that we have solutions [3] to the problem that tailor the addition of complexity only to places where people are most at risk, so 99.99% of people won't be affected. Also the level of complexity that is being added is a low as it can be whilst still being effective.

Notes:

[1]: Pressing Win+R on Windows systems gets you a system level command prompt. I have no evidence that this has been used in Self-XSS attacks, probably due to limitations in cmd.exe forced an attack script that is 5 or more steps.

[2]: I've seen several demonstrations of Self-XSS that involve 4 or more steps, but none that have a high completion rate.

[3]: Relevant bugs: Firefox 994134, 664589, 953166, also Chromium 345205,