First Alan Williamson asked if Amazon EC2 has become oversubscribed. Then Cloudkick jumped in with graphs illustrating the increased latency seen by spot instances. Amazon has denied there's any fundamental issue. But let's look at that graph:

Something struck me about the timing: the trouble all seems to kick off round the 12th of December: that's the day Amazon announced EC2 spot instances. The way spot instances work is simple: Amazon puts its spare capacity up for auction. Instead of paying a set price, you bid for an instance, and the highest bids that fill up available instances win. If more people turn up demanding instances, the price should rise.

But there's a side effect: assuming spot instances are popular, then we can assume that no matter what the price is, all of EC2 capacity is now being used. What would you expect to happen if that were the case? Well, you'd expect them to start hitting capacity limits -- which is what the ping times seem to suggest is happening.

At the moment this is just my theory. Anybody else got any evidence that might back it up? It would be really nice to see what's been happening to EC2 spot prices over the last month, for instance.