The challenge? Site traffic keeps growing at higher and higher levels. How much higher? An extra 1.5 million visits over the last 30 days, as compared to the 30 days prior to that. That works out to an extra 16 million page views. We’re serving over 2x what we did last year. Although we’ve made lots of progress, with such rapid growth there is more to be done.

We did foresee these increases, and took steps to accommodate them. In July and August we quadrupled our server capacity. However, some parts of the site cannot be scaled by adding more servers - they need to be centralized on a single server. This is the master server that has been talked about so much in the forums of late. We underestimated the effects of the increased traffic would have on the master - and did not anticipate the locking issues. The databases are still getting overloaded from time to time. Our internal experts and consultants are testing some operating system changes that we hope will help in the short term. They will continue working on it throughout the weekend.

We are also working on a long term plan to re-architect the system so we won’t rely on a single master server like we do now. We are looking at new technologies and new techniques. It will involve a lot of work, both in our code and in the way we store the data. But in the end, we will end up with a system that will be much larger in scale and scope, taking us to a whole new level.

We also have some new hardware en route to do some testing, but this will take time.

What this means
We may need to restart our main servers on occasion. This means no site for about 14 (agonizing) minutes if all goes well. Please understand that we realize this is unacceptable, and we’re doing everything in our power to keep it from happening. When it does, we ask you to be patient. If you phone into customer service, please be gentle. They are fielding hundreds of calls, and can’t tell you much more than our ETA to be back up.

To our customers
Again, we thank you for your patience. We will try to keep you updated, but please understand our time is best spent working on the issues. When things are ironed out, and we’re back to running normally, we can explain and expand further.

We appreciate your support. Though we know you are still experiencing painful moments, all of this is moving us toward an amazingly strong new iStock infrastructure. Please stick with us. You won’t be disappointed.

To our artists
We know you are worried about potential loss of royalties. Please do not be: we are selling far more images than we ever have before, even with the site issues.

For those worried about missed transactions. It is possible that a fraction of a percentage of orders are not properly processed when the site starts acting up. We’re investigating, and if we find it to be an issue, we will do a full audit to make it right. Rest assured you’ll be properly paid for images sold.

Again, we appreciate everyone’s understanding.

Comments/Questions:

http://www.istockphoto.com/forum_messages.php?threadid=58279&page=1

The problems we fixed last week had to do with throughput. When those bottlenecks were fixed, it increased the load on the servers by a truly significant amount (average load was higher than our highest peaks–ever). Which is fine. That much more traffic means lots more sales.

Unfortunately we hit a snag along the way. We ran up against a known bug in mySQL 5 (which is what the site runs on). We simply can’t process any more transactions per second. Yeah, it’s a good problem to have. There is a fix in the next version of mySQL, but it’s in beta and a long way from being production-ready (and no, adding more machines doesn’t help in this case).

So our developers have been coding around the problem. It’s slow and meticulous work. Kinda like changing the fan belt on a car doing 60 on the freeway.

We turned off stats because they were hitting the servers especially hard, and aggravating the bug. We’re running the script to update your graphs every afternoon. It takes about 3 hours to run, but in the mean time, your number of downloads and cash balance is always up to date. There is code in place that should allow us to turn it back on–we’ll know if it’s ready to go by late Monday.

Hope that clears things up a bit. Thanks for hanging in there.

Click on the ‘Show Beta’ link on the far right in the search bar and take it out for a spin.  If at any time you would like to go back to the original search, just click on the ‘Hide Beta’ link.

Take it for a spin and let us know what you think.

Join the discussion.

WHAT THIS ISN’T:
This isn’t a punishment for contributors with big portfolios who haven’t disambiguated their legacy uploads. As noted above, all uploads from before Sept. 18, 2006, will retain their Best Match status regardless of disambiguation status. All contributors who have been disambiguating their new uploads as they come have nothing to worry about. Disambiguating and monitoring your keywords is just as much a part of uploading as getting the image on to our site.
This isn’t a total solution. We’ll still be using the Bulk Tools to get the intentional spam and bad disambiguation out of searches. We’ll still be dealing with spammers. We’ll still be working to further refine and translate the Controlled Vocabulary. Business as usual will continue in addition to this new measure being implemented. 

Forum Discussion

While there are still some old credits floating around that will result in the occasional 10 cent per credit royalty, it is the new royalty amounts that vary based on the price of the credit package that I expect will cause the most head scratching.

Maybe it is because he truly is a superhero (or maybe he just has too much free time), but emyerson has whipped up a handy little 2007 Royalty Figure-Outer that does a much better job of explaining things than I could ever do (thanks Ethan).

One important fact to keep in mind is that any combination of credits could potentially be used to download an image, and each credit could have a different net value and result in a royalty not found in the Royalty Figure-Outer. If I may borrow jester’s example (since I’ve copied everything else he said):

For example, a small file downloaded from a non-exclusive using…

one 2004 credit with a 10 cent royalty + one 2007 credit purchased for $1.09 with a royalty of 22 cents = 32 cents royalty

Unrelated to the above, but on the theme of odd royalty amounts, if you should stumble upon an instance where you see a royalty of 0.00 it is the result of someone re-downloading the image after having just paid for it. Downloaders get 24 hours to re-download the file (without additional credit cost) in case there are any connectivity problems at that prevent them from a successful download the first time. Since all file downloads are tracked and recorded you will see the repeat download as a 0 royalty.

Forum discussion

(http://www.istockphoto.com/article_view.php?ID=217

A smarter man than I said that a long time ago, but in the five years we’ve been at this whole iStockphoto thing, I’ve certainly come to appreciate the truth of it. 2005 was a banner year for iStockphoto, with growth of truly unimaginable proportions. Indeed, the stock photography industry itself was an amazing and exciting thing to observe in the past year. Artists of every visual content discipline now have more opportunities than ever before, and we’re certainly proud of the hand we’ve had in the continuing evolution of the digital media landscape. ”

(http://www.istockphoto.com/article_view.php?ID=159