Yesterday at about 1pm EST 4/16/14 our main website http://www.prophoto.com/ went down and was completely unavailable for close to 18 hours. As a result, many of you were unable to purchase designs, upgrade, read tutorials, watch help videos, and contact us for support during that time. We’re not certain, but it is also possible that any support emails you sent us during that window were lost.
For all of this we are extremely, extremely sorry. We know you rely on our site being up for customer support, and that you rely on being able to contact us and get a speedy reply. To make matters worse, the timing was particularly awful: we had just partially released an auto-update to P5 that negatively affected a small percentage of users, temporarily adversely affecting their site’s appearance. Just when we were hoping to quickly help that small but important group of customers, our site went down and our support went offline. To those customers, again, we are extremely sorry. Please contact us again today if you don’t hear back from us, and we will help you get fixed up as fast as we possibly can.
For the sake of transparency we wanted to give you some details about what happened, and what we’re doing about it. You may know that we recommend Bluehost as our preferred webhost. That recommendation is sincere, we host our own site on Bluehost. For the past 18 months we’ve been on one of their top of the line dedicated servers, and have had almost exclusively excellent uptime and performance. Until yesterday. Apparently, a bug in a firmware update brought down many or all of their Virtual Private Server (VPS) and Dedicated Server clients. Because we have a dedicated server (ostensibly to make sure we don’t ever have problems like this, ironically) we went down, and stayed down for far too long.
The good news is that it appears that all of their standard, shared hosting (which 95% of our ProPhoto customers are on) servers were not affected by this outage.
You might ask if this type of problem would give us cause to reconsider our recommendation of Bluehost to customers. That is a good question and it certainly does. Since Bluehost was purchased by Endurance International Group not too long ago, our perception has been that performance and customer support has suffered some. Because we are one of their largest affiliates, we have some pretty high-level contacts at Bluehost. Well before this outage yesterday we have been in dialogue with those contacts about our frustrations, and we have been assured that they are fixing things.
The truth is, at this point we don’t have a better replacement for Bluehost as our default recommendation. Despite some of the recent problems we still encounter fewer host-related problems with customers on Bluehost than on any other hosting company. Until that changes, we will likely keep recommending them. That said, in the last 6-8 months GoDaddy (who used to be one of our least favorite and most problematic hosts) has been making steady strides towards a more reliable, fast, and user-friendly hosting ecosystem. If they keep improving and address some of our last remaining major concerns (we also have high level contacts at GoDaddy), we may at some point in the future switch to recommending them.
We will also be reconsidering hosting our own site with Bluehost in the months to come. If we do not feel that have adequately responded to the incident yesterday, it’s likely that when things slow down for us a bit in the summer months that we will pursue moving out site to another company, and will explore options for mirroring and greater redundancy. We really would like to prevent what happened yesterday from happening again, if possible.
Once again, we’re very sorry for the downtime and any way it may have adversely affected you, our clients. Thank you for your patience and understanding.