I want to give you a heads-up on something you'll probably hear about, and we'd rather you hear it from us.
On February 22nd we noticed some unusual activity in our server logs: An ip address from Hanoi was poking around where they shouldn't be. Within a couple hours we had it isolated and stopped, a little less than 24 hours after it started.
Because this was on our ecommerce server we hired a data security firm to do a forensic investigation and sure enough, they found that a pocket of data had been compromised which contained credit card numbers.
It was a relatively small breach, about 1% of the national average, and the first we've experienced in 15 years of online transactions. We reported it to all the card brands as well as the FBI, and sent notification letters to all the customers we could identify as being potentially compromised. (FWIW, most merchants don't do any of that and hope for the best.)
Unfortunately, this kind of activity is so common these days (175,000,000 records compromised in 2011) that there's no way of knowing for certain if any specific card was fraudulently charged as a result of this incident.
The situation begs two questions: "How did this happen?," and "What are you doing about it?"
I can't go into detail about where the vulnerability was, but our former security plan held for 15 years. That was largely due to a security service we hired to regularly "attack" us and report any security holes, which we then closed.
After the breach we made immediate and major changes to avoid similar problems in the future:
1. Online checkout was shut down for three days while we installed a system that uses a method called "tokenization." In short, credit card data is sent directly to a subsidiary of VISA and is stored there. It is no longer stored on our server, nor do we have any way to view it.. So, in the unlikely event we are breached again, there's no sensitive data to find.
2. We installed triple redundant defenses against the attack vector that was used to breach the server.
3. We engaged a nationally-recognized data security firm to analyze our operation, conduct penetration testing, and help us shore up any remaining weaknesses.
It's been a very stressful and emotional few weeks as we worked through all these changes. Bly and I feel nothing short of mortified and are doing everything in our power to make sure nothing like this happens again. We take our responsibility to you very seriously and this is not what we had in mind.
Regardless, we deeply regret any inconvenience you may have been caused and we will work hard to regain your trust.
On February 22nd we noticed some unusual activity in our server logs: An ip address from Hanoi was poking around where they shouldn't be. Within a couple hours we had it isolated and stopped, a little less than 24 hours after it started.
Because this was on our ecommerce server we hired a data security firm to do a forensic investigation and sure enough, they found that a pocket of data had been compromised which contained credit card numbers.
It was a relatively small breach, about 1% of the national average, and the first we've experienced in 15 years of online transactions. We reported it to all the card brands as well as the FBI, and sent notification letters to all the customers we could identify as being potentially compromised. (FWIW, most merchants don't do any of that and hope for the best.)
Unfortunately, this kind of activity is so common these days (175,000,000 records compromised in 2011) that there's no way of knowing for certain if any specific card was fraudulently charged as a result of this incident.
The situation begs two questions: "How did this happen?," and "What are you doing about it?"
I can't go into detail about where the vulnerability was, but our former security plan held for 15 years. That was largely due to a security service we hired to regularly "attack" us and report any security holes, which we then closed.
After the breach we made immediate and major changes to avoid similar problems in the future:
1. Online checkout was shut down for three days while we installed a system that uses a method called "tokenization." In short, credit card data is sent directly to a subsidiary of VISA and is stored there. It is no longer stored on our server, nor do we have any way to view it.. So, in the unlikely event we are breached again, there's no sensitive data to find.
2. We installed triple redundant defenses against the attack vector that was used to breach the server.
3. We engaged a nationally-recognized data security firm to analyze our operation, conduct penetration testing, and help us shore up any remaining weaknesses.
It's been a very stressful and emotional few weeks as we worked through all these changes. Bly and I feel nothing short of mortified and are doing everything in our power to make sure nothing like this happens again. We take our responsibility to you very seriously and this is not what we had in mind.
Regardless, we deeply regret any inconvenience you may have been caused and we will work hard to regain your trust.
