Incompleteness

Combating Self-XSS

2011-12-14T12:00:00+00:00

What is Self-XSS?

Dr. Evil has several options for getting his script to execute in the page of another site. Generically we call this XSS. Self-XSS involves using social engineering to coerce a user into manually executing JavaScript using the location bar or developer tools. For more, see socially-engineered XSS attacks.

The recent Facebook attack signals that something needs to be done, but knowing the right response is tricky.

What is Mozilla doing about it?

We're proposing adding a directive to CSP that says 'Please disallow user supplied JavaScript in the context of this resource'.

It will probably look something like this:

X-Content-Security-Policy: no-user-js

We're also going to add a way for developers to opt out of this protection. Effectively saying 'I can take care of the JavaScript that I ask my browser to execute'.

How does this affect other Firefox developer tools?

It doesn't. The Highligher, Style Inspector, Style Editor and Tilt are all unaffected. This is only about JavaScript executed via the Web Console and Scratchpad.

How does this affect users?

User		Is a developer?
User		No	Yes
Can recognize Self-XSS attack?	No	Added Self-XSS attacks protection.	Warned about Self-XSS, may benefit from protection.
Can recognize Self-XSS attack?	Yes	Unchanged	Minor inconvenience of having to set a preference (one time only) to enable user JavaScript on sites using this Self-XSS protection.

I think this is a fairly clear net win: Minor, one time only, inconvenience to a sub-set of web developers, vs. full-time protection for the many that wouldn't recognize a self-xss attack.

Objections: This is a user problem

The goal of educating 7 billion people about what JavaScript can do is lofty, grandiose, ambitious and utterly unattainable. There has to be another solution.

Objections: This is a Facebook problem

While it's true that allowing untrusted, unvetted, third party, dynamic content onto your site is something to avoid, I don't think that fixing this either is going to happen, or would fix the problem. The attack could easily forward you to another site to see the clipboard injecting flash, before returning you to the original site for the 'paste' step.

Objections: This is a Flash problem

While it's true that Adobe's clipboard policies are more relaxed than those of major web browsers, we've seen people caught by instructions that ask people to select/copy their own attack script. We think that the level of pain caused to developers by the CSP solution is low enough that we can justify the additional protection.

How to Eat an Elephant

2011-09-16T13:50:00+00:00

This is about levels of change. It's partly about hg.mozilla.org vs GitHub (or BitBucket). But it's more about our devt/review/commit process.

Moz-central demands patches that solve significant issues, and it's probably right in wanting this, but this bite size comes with problems.

There are 3 levels of change worth discussing:

The detail level we're used to working at: 'Added foo', 'Renamed foo to bar', etc. If working in a small team, this is probably the level we would commit at. At this level it's easy to understand the intent behind each change.
The review level which is an ideal chunk for someone to review. You probably don't want to review every new commit as it's done, but on the other hand, you don't want to be told that you made a mistake weeks ago and need to redo lots of work. So ideally this happens on something like a daily basis. It's the scale that pull requests were designed for.
The patch level which mozilla-central is used to, and which is ideal for commit. The work is 'done' and useful for a much wider audience. When a bug is fixed we can extract a patch to be committed, like this one for bug 684958.

The problem with the Mozilla process is that steps 1 and 2 don't exist at all. The benefits to the existence of more detailed change levels are:

Review is faster and deeper. You can review this pull request far better than a conventional patch because you can see what I'm intending by each change - you don't need to reverse engineer the intent and mentally hold together all the bits of mashed together bits of intent.
Smaller changes are easier. Our current process prevents us from clearing up small things that are not directly related to the bug in hand. We can (for example) entertain the thought of variable renaming because it won't confuse the patch. There may be other reasons not to go crazy with this, but at least the process won't be one.
Progress is visible. Because we can share to a smaller team before we're done, everyone gets to see what's happening. There is a feeling of momentum - progress is the norm. Just like a 6-week cycle is better than an 18-month cycle, a daily commit/pull cycle is better than a weekly patch/review cycle.

In effect we're using GitHub as a tool to develop patches for Firefox, rather than to develop Firefox itself. In some ways I mean this literally.

There are probably reasons why the detail of what works for GCLI might not exactly work in other places, but I'm willing to bet that these ideas could be adapted to be of more general use.

Fundamentally we solve big problems by breaking them down into smaller steps.

I can eat an elephant, but not by opening really wide.

Marketing (according to the browser makers)

2010-09-13T16:47:32+00:00

Apple and Adobe, Obituaries and Idealism

2010-02-1T07:22:00+00:00

Practical reasons for thinking that Flash is dying:

You can't get flash on an iPhone, iPod or iPad
IE9 looks like it could get HTML5 video

Idealistic reasons why Flash should be dying:

The spec is controlled by a single entity.
The Flash wire-format is binary. View-source is important.
The Flash runtime can't be fully open-sourced due to patent encumbered codecs.

It's obvious, but you can't beat a good venn diagram:

The Idealism isn't having much effect.

Apple

There's a parallel post to this one, with the subject being the iPad instead of Flash, and that it's a Bad Thing when you're not allowed to tinker with devices that you own.

Idealistically the world would wait until something Chromey, Androidy, WebOSy or Maemoy came along out of principle. But we all know that's not going to happen however much we complain.

Shame really. All the talk is for nothing.

Collaboration in Bespin

2009-08-13T16:29:02+00:00

Recently I've been working on adding collaboration features to Bespin, Mozilla's web code editor. Today they're getting pushed out into bespin.mozilla.com.

Enhanced collaboration is one of the best reasons to do an IDE on the web. It’s possible in a traditional IDE or code editor, but there are lots of features of a web based editor that make it an obvious thing to do on the web.

Our new collaboration feature is still fairly new, and like the rest of Bespin, still fairly beta, but we’ve got it working in Firefox, Safari and Chrome right now.

We’ve released a webcast showing it off here:

Taking it for a spin

Want a go real quick? Here are the steps:

Sign up or login to Bespin.
Press CTRL+J/CMD+J to open the command line.
Type 'follow joewalker' to get someone to share files with. I've shared a project called pubproj globally.
Type 'project list'. You should see joewalker+pubproj in your list of projects.
Type 'set collaborate on' to turn on shared editing.
Open a shared file by typing 'open /joewalker+pubproj/example.txt'.

Or, in one graphic:

You should then be in a collaborative editing session with the rest of the Internet.

Click on the collaboration icon (2 people) in the top right hand corner show you who's editing with you.

How it Works

There are 2 parts to what's new. Firstly there's a way to decide who you want to share stuff with and secondly there's the shared editor. We’re adding 3 or 4 new commands:

follow/unfollow: allow you to decide whose shared projects you want to see in your project navigator. We've gone for a lightweight Twitter style model so it's easy to get started. As we evolve Bespin this will become a way to get all sorts of information about the people you are working with
group: allows you to put the people you are interested in into groups to make it easy to manage sharing
share: allows you to export your projects read-only or editable to individuals, groups, or to everyone

Expressing an interest in someone is as easy as “follow username”. You can then group these users together to make sharing easier. So for me to add Kevin, Ben and Dion to a group, I would do “group add mozlabs Kevin ben dion”. Grouping people like this is totally optional, but it will make maintaining sharing rules a lot easier going forward, when you’re working on a number of projects.

Then if you want to share a project in read-only mode, to someone that is following you, you just type: “share add projectname username”. Replace username with groupname to share with one of your defined groups, or with “everyone” to just throw everything wide open. To allow editing you add “edit” to the end. So for ultimate wiki-like promiscuity: “share add projectname everyone edit”.

Projects shared to you by people you follow will now show up in your file-explorer, or you can edit them from the command line. Type “load /owner+projectname/path/to/file”.

Future Extensions

We’re currently using Mobwrite to provide the diffing and synchronization with some additions to allow us to share collaborator information. We’re hopeful that we will be able to share collaboration systems with a number of other editors in the future. Various developers are working on plug-ins for GEdit, Eclipse, emacs, etc.

Other things that we’re working on include plans for a quick share mode where you can grab a URL post it to an IRC session and instantly be in a shared edit session with the other IRC users.

We've also been discussing in-page chat. We've left it for now because many Bespin users will already have IM or IRC or even Skype or similar, and we want to do something smart with the chat transcripts other than just throwing them away.

Next we’ll be working on getting smooth undo so you have a time machine that can smoothly walk through changes from the current revision back in time through recent saves, and into VCS history, and then into a more lightweight collaboration where when 2 people have 2 check-outs of 1 file, they can both see each others changes to reduce merge pain, without directly affecting each others work.

Command Lines

2009-05-27T20:08:19+00:00

When I was fresh out of college, my first job was working for the OpenGroup, and my command line looked something like this:

In 2009, fired by the white heat of years of cutting edge research and development, my command line looks a lot more like this:

To be fair, we've now got transparency, anti-aliasing, our shells now do better completion. But that's about it for changes. Considering how much the rest of the industry has changed, that's shockingly glacial.

For the past week or so, I've had the privilege of working on the Bespin command line, and I think there's a lot that we can do to make things better.

This is what my Bespin command line looks like now:

This post is all about how we can make command lines better.

The core of the unix command line philosophy seems to be: small programs and pipe. There’s a lot to be said for loose coupling but there are some drawbacks as well...

Zsh

I love Zsh, and love the fancy completion that it does. Basic file completion has been around since ksh, IIRC:

$ ls [TAB]
Applications/  Mesh/       Projects/
Desktop/       Movies/     Public/
Documents/     Music/      Sites/
Downloads/     Parallels/  backup/
Library/       Pictures/   bin/

But zsh can complete against more than just files:

$ telnet [TAB]
192.168.0.1
192.168.0.12
download.directwebremoting.org
download.dojotoolkit.org
getahead.org
hg.mozilla.com
localhost
people.mozilla.com
sandbox.dojotoolkit.org
svn.directwebremoting.org

Zsh knows that you don’t telnet to a file, so it completes against hosts (Does anyone know where that host list came from BTW?)

Zsh also knows about other commands:

$ git [TAB]
add                 -- add paths to the index
apply               -- apply patch on a git index file and a work tree
applymbox           -- apply patches from a mailbox
applypatch          -- apply one patch extracted from an e-mail
archimport          -- import an Arch repository into git
bisect              -- find the change that introduced a bug
branch              -- create and show branches
checkout            -- checkout and switch to a branch
cherry              -- find commits not merged upstream
cherry-pick         -- cherry-pick the effect of an existing commit
clone               -- clones a repository into a new directory
commit              -- record changes to the repository
...

That list of commands is the result of a 'git help' command, but that's not what I typed. Zsh has been doing something funky behind my back when I pressed [TAB].

Zsh can do different things depending on where in a command you are:

$ git add [TAB]
Applications/  Mesh/       Projects/
Desktop/       Movies/     Public/
Documents/     Music/      Sites/
Downloads/     Parallels/  backup/
Library/       Pictures/   bin/

My point, is that, whilst all of this is very clever, the implementation is also insane.

In order to make this work, the zsh guys have had to reverse engineer every single unix command (or at least all most of the ones that I use). And any time those commands change, then zsh is going to have to change too.

It's very useful, and I'm glad that zsh has done it, and they probably didn't have much of a choice, but this way of going about things is not the Best Idea ever.

I'm sure somewhere the bash guys are working on the same feature themselves, or maybe they've already done it. I don’t care - it just strengthens the point about the lunacy of the situation.

The Unix philosophy of small programs and pipe has a lot to be said for it, but there are serious drawbacks too.

Here's some of the ways that we're innovating when it comes to the Bespin command line, and benefiting from being about to integrate things better.

GUI integration

Traditionally you've had a set of shells (bash, zsh, etc) that work in a set of terminal emulators (xterm, terminal, gnome-terminal, konsole, etc) But there's so much more you could do if you had one program do both.

In Bespin:

If you click on an old command, it gets copied to the prompt (the blue highlight is a mouseover).
If you double click it's re-executed.
Output is not restricted to ascii in a grid. We (predictably) use HTML.
Error output is visually different to normal output
You can minimize or maximize output (the help output has been minimized above) - I hate it when a vcs diff operation takes up 4k lines of scrollback obliterating what I wanted to see.

There's no reason why you couldn't do this in Unix (or even Windows). I don't think you’d want such a gui-shell as your login shell, but since its use is very much from within a GUI, that's OK - you’d probably launch it from the GUI anyway.

In Bespin we don't have pipes yet, but would it be neat to be able to inspect a pipeline visually as it's working, I'm sure it's possible with a gui-shell.

Linking Output to Commands

We link the output from a command to the command that created it. The obvious effect is that output from slow running commands no longer gets confused with what's happening now.

The 'slow' command is just something I hacked up to demonstrate asynchronous behavior.

A nice upshot of this is that we can implicitly add & to the end of every command, and you no longer have to wait for anything.

In essence, each command has it’s own separate stdout, which goes to it’s own div. Maybe there is a reason why unix shells don’t do something similar. Would it mean shells being curses based or something?

Command Meta-Data

With the exception of the reverse-engineering of the command line structure done by zsh, the integration between commands and the shell is limited. In Bespin, a command is a structure like this

{
    name: 'createproject',
    takes: ['projectname'],
    preview: 'create a new project',
    usage: '[newprojectname]',
    execute: function(commandLine, projectname) {
        /* code to do the work */
    }
}

This means that we can provide help as the user is typing. The idea is that TAB will guess what you want to do at all times.

This ties in a lot with GUI integration too. If the command line knows that the next parameter is a file, when the user presses TAB without typing anything to complete on, we can offer a dialog to allow selection of a file, and so on. This isn’t about forcing the user to switch to the mouse, but it is about allowing better feedback from the system about what options are available.

We've only really got started with this part - there’s a lot more work to do. I hope we'll be able to do some of the funky GUI integration that Ubiquity has been pioneering.

Attention Guides

There's a lot to look at on any command line, I want to find ways to keep the most important bits in front of your eyes at all times. I'd like to expire older commands more intelligently than a standard terminal (oldest dies first), Commands are more important if:

They are still in progress
They keep being re-executed
They are waiting for input
etc

We've only basically started on this right now, with the ability to hide/show output, but there's potential for it to make the command line lots more usable.

Plug-ins

Linking everything together is easy if you make one big monolithic lump. That's not the plan - we're evolving a plug-in strategy as we go, and planning on making as much of Bespin itself use the plug-in system. The key to coupling is to get the right level of coupling.

Status Messages

My other integration obsession is about status messages. There's a huge amount we could do to integrate the familiar concept of "what are you doing now" with commit messages, bug trackers, chat, and team tracking. More soon.

What other things could we do to trick out the command line?

Introducing David Marginian

2009-04-15T19:47:48+00:00

The DWR project has a new lead developer - We've just closed a unanimous vote that declares David Marginian to be the new lead.

David had been around DWR since the beginning, and has been consistently one of the most helpful people on the mailing list, and has contributed to many areas, most particularly the documentation.

Q: Why? Because David is already doing a better job of leading DWR than I am, and because for a while I've had too many projects on and been trying to do a better job of less things.

Q: Am I gone for good? No, I certainly don't intend to disappear, and I've still got a lot of love for DWR - this is mostly just me admitting that I'm not the best man for the job, and that DWR deserves better.

Q: What are you doing instead? Quite a bit of Bespin. Oh, and like everyone else, I'm excusing my lack of blog activity to the whole 140 char thing. It's a cheap excuse, but if everyone else can use it...

I'd like to thank the David, and the whole DWR community for helping make DWR a success.

The Power of Bespin

2009-02-16T12:38:54+00:00

Yeah, so Bespin a cool editor, and it makes use of lots of bleeding edge tech, (and it's got several annoying bugs - sorry about those). But I think Bespin has potential way beyond just looking nice ...

Open source development: You hear about project X, use it, and it’s nice, but to use a worn out phrase - there’s an itch to scratch ...

So what do you do?

Open Source Development Before Bespin

You search the project website for the directions to the source control system, you check that you’re up to date with cvs/svn/git/hg/whatever and checkout the source. Next you need to figure out how to build the project with ant/make/maven/paver/rake/whatever, and maybe download new libraries to make the compile clean and you then configure your IDE or editor to the new project. You can then go about editing the code. When you’re done with that you might need to figure out xunit/yunix/zunit to run the unit tests. Then you have to join a mailing list to work out how to send your changes in, which might be a patch or a bundle or a zip or direct commit access. You might discover that you need to sign a CLA so the project knows who you are, so you mess about with printing/signing/scanning/emailing/whatever.

And then you’re done. With complex projects there could be lost more to it.

While you’re doing this you have no idea what the other developers are doing. Maybe they’re working on a totally different branch. Maybe they’ve done something similar already. Maybe they’re making changes that mean you should try a different approach. It would be good to know because you might be wasting your time.

Open Source Development After Bespin

You visit Bespin, and elect to edit project X. When you’re done, you click submit, and the project owners get your change.

Plus you can see what the project owners are doing with the source, and how your changes fit in.

The Power of Bespin

For me, what I find interesting about Bespin isn't the tech, it's the lowering of the barriers to entry and the socializing of open development.

Virtually no software is perfect, and there are billions of people that are affected by the imperfections. There are hundreds of ways to contribute to any development project, and millions of people who could use their skills to help if only we could find ways to get the sources of help to the problems needing work

This application of huge resource to complex problems is what the Internet has always been good at, but so far we've not really done much to help software development. I hope that Bespin can become part of the solution.

This isn’t going to become a Bespin blog (that’s here or we're on twitter). Irregular DWR/Ajax/Security programming will return to my blog and twitter feed shortly.

Undoable Silent Autoupdate

2009-02-4T09:49:23+00:00

Recently:

Aaron Boodman - "I'm a big believer in silent autoupdates"
PPK/Quirksmode - "I hate automatic updating"

Something Aaron didn’t mention - What happens when a silent auto-update breaks something? Anyone that's had to do battle with the demo-pixies silently changing something just before an important demo hates the idea that something might be silently altering their rendering engine as they are opening their laptop on-stage.

PPK's issue basically comes down to testing older versions - how can I test version X-1, when it's just been replaced by version X. On the one hand with silent auto-update virtually everyone will be on version X, so X-1 isn’t important, however I don't think that removes all need for testing on X-1.

Solution?

We should have silent auto-update. It makes the Internet safer, and we badly need that. But we should also have a system that keeps old versions around for some period of time with a way for users to revert. Maybe there is then an entry in the help menu that allows users to say 'it broke take me back', and maybe a preference that configures the number of old versions left lying around.

I have a suspicion that many corporates turn auto-update off in order to test the updates batches before internal release (or to put it another way - it's a cost saving) Maybe with undoable silent autoupdate they would allow the system to work without getting in the way?

I'm a big believer in undoable silent autoupdates.

Bespin and Mozilla

2009-02-4T11:27:33+00:00

I’m very excited to be going to work for Mozilla Labs.

I have very much enjoyed my time working at SitePen - they are doing awesome work with Dojo and DWR, but with the main development on DWR 3 being done, it's time to move on. SitePen has some amazing talent, and it's been a huge privilege for me to have been able to share the ride, so I'm very grateful to Dylan and everyone at SitePen for allowing me to be part of it. I'm not going to be ignoring DWR either, so between me and the excellent team of Mike, Jose, David, Lance and many others I'm sure it has a long future.

My new role will be working with Ben, Dion and Kevin on Bespin.

You might be able to get hints about Bespin from postings that are around already. Oh, and the name - that might be a clue too. Needless to say I’m excited about the project. More soon.