Today our main company website http://www.prophoto.com/ was very temporarily hacked. If you’re not interested in all the gory details, here’s the TL;DR version — we eliminated the hack very quickly and verified that no ProPhoto product or customer websites were ever in danger. Also, because we keep no sensitive data about you, our customers, there was no risk of that sort of data having been compromised. If you’re a ProPhoto user, there is absolutely nothing you need to do. If you’re interested in a much more detailed disclosure and details, grab a coffee and read on.
Today at around 2:30pm EST, our main website, http://www.prophoto.com/ was hacked for about 5 minutes, during which time any visitors to our site saw an innocuous web page supplied by the hacker. After we found out about the hack, we very quickly worked with some high-level admins at Bluehost (where our website is hosted) and were able to determine the source of the attack and block it.
As soon as the hack was removed, we immediately began working to find out if there were any security ramifications for our users. The first thing I want to make absolutely clear is that we do not and have never collected or retained any truly sensitive information about our users. The best security is ignorance, because an attacker can not steal what we do not know. We do not process credit card transactions, so we never ever have access to your credit card numbers, any financial data whatsoever, or any password of yours. Because we’re not a webhost and you have no account with us, our website has no information of any kind about any of your passwords, and we have no access to your website or email. Because we never know that information, we never store that information, and there is no possibility that any of that type of information could have been compromised.
The second thing we immediately checked was if there was ever any breach of the security of our ProPhoto or Proofing plugin product files. As you know, we sell products which our customers download and then install into their WordPress websites on their own web hosting accounts. After we found out about our site being hacked, we immediately removed the ability for any customer to purchase, download, or auto-update their websites, so that we could inspect the integrity of those product files.
After careful inspection, we were able to confirm 100% that the files that users were downloading, installing, and auto-updating from were never touched or modified in any way by the hack that affected our website temporarily. We made that confirmation in three ways. First, we compared the last modification timestamp on those files with the last time we made any change (which we keep an internal record of, for exactly this type of scenario), and the timestamps perfectly matched. The last time those files were modified matched exactly the last time we internally updated the files. Second we examined the files themselves and ran a diff tool on that is capable of alerting us if even a single byte doesn’t match between the files on our server and our local copies. The diff tool found zero discrepancies in any of our product files. Thirdly, we were able to analyze access logs to verify that those files were not accessed or modified in anyway.
During this time, we were also working with several high-level security system administrators from Bluehost, who were verifying and confirming our internal investigations, and providing helpful support. They were able to find evidence that also confirmed our knowledge about what the hack affected, and what it did not.
We currently have a pretty good picture of how the hack was executed, but we are still working on understanding the exact implementation details. Right now 100% of the evidence points to a security vulnerability that was totally unrelated to the ProPhoto product that our customers run on their web servers. Again, the hack occurred on our website, not within ProPhoto itself. Our website is not directly related to any of our customers websites at all. We are not a hosting company, so we do not host any of our customers websites. Furthermore, our hosting account at Bluehost is on a completely dedicated server, so there is no possibility that the vulnerability exploited on our website could leak over to any other accounts hosted by Bluehost since we are not on a shared server.
Once we know more about the exact attack vector, we will disclose anything pertinent on this blog. Our website runs WordPress (although it does not run ProPhoto), so if we determine that the attack came through WordPress and that there is a new or unidentified vulnerability in WordPress, we will pass that information on to the WordPress team. Right now we have no evidence that ProPhoto itself was the vulnerability, but if it turns out that we were wrong about that, we will absolutely disclose that information on this blog as well.
Since we started selling and supporting ProPhoto, we’ve never experienced an exploited security vulnerability in the ProPhoto theme or Proofing plugin. That doesn’t mean that we can relax — writing secure software is incredibly hard and it is absolutely our duty to be constantly vigilant to ensure the security of our customers by the strictest caution in the code we write. But it is good to know that we’ve never encountered a security problem caused from the ProPhoto codebase itself. The closest we’ve ever come to that was about 5 years ago we found a hacked plugin that was modified to try to gain access to the ProPhoto3 theme. That attack was not through a vulnerability in ProPhoto, it was through a hacked plugin that was attempting to access P3. Also, at that time we never able to determine if the hack was widespread or isolated to the single compromised web hosting account we were working with.
Thoughts on disclosure
In the seven years since I wrote the first version of ProPhoto and started this company I’ve seen many small and large companies deal with hacks and security exploits. I’ve often considered how I would handle a similar situation. At one point I even uncovered a small but serious security problem at a web hosting company. During the process of working directly with that company I remember being frustrated by the owner considering hiding the security issue from his users. I promised myself that if put in a similar situation I would absolutely err on the side of transparency and disclosure, even if it meant egg on our face. This blog post is my attempt to make good on that promise.
Thankfully, it looks like this scare has turned out to have been very minor. It could have been much, much worse. In the end, it may end up being a blessing in disguise. We got to have a dress-rehearsal of a real security scare without any true damage to our customers websites or data. I promise you that we will learn and grow from this experience. Even though our customers were not affected in any way, it is not acceptable to us that our site was compromised in any way for any length of time. We have to do better, and we will absolutely strive to do so and to continue earn the trust you put in us.
Out of an abundance of caution, our team has decided to disable all purchasing of ProPhoto for the rest of this weekend. We will resume selling ProPhoto and the Proofing plugin on Monday morning, February 16. This is not at all because we have any reason to believe there is any ongoing problem with our site or product, as I explained at length above. Rather, it is to add one more layer of confidence before we resume normal activities. Waiting until Monday to resume selling as usual will cost us some revenue, but it will also give us one more stage of verification as we will be able to examine our raw access logs and files to see if there has been even a hint of suspicious activity since we first identified and resolved the hack.
Second, very shortly after we identified the hack, we chose to sent a pre-emptive email out to anyone who had purchased any of our products within the last 24 hours notifying them that we had an unresolved situation. It turns out that those emails were unnecessary because we have determined that none of those users (or any others) were ever at risk. Even so, we feel good about the decision to alert those customers. We feel it was better to err on the side of communication and caution, even if it temporarily made us look bad to some new customers. We will be re-contacting those customers to let them know the result of our investigation.
We will also perform a very intensive post-mortum to learn from this incident, which process has already begun. We will be working with the security team at Bluehost and possibly other experts to make sure we’ve done everything we know how to do to prevent any future incident, be it small or large.
If you have any questions, please feel free to leave a comment below and I will try to personally answer. Thank you so much for being our customers, and we will keep striving to and earn and retain your trust.