Delcampe is one of the online Belgian success stories that nobody talks about but about which we should be very proud.
It was clear that the last months they have faced enormous technical difficulties and that as a solely online business they were bleeding at the heart of their existence.
In an email to its members the president of the company tells all and for managers in ebusiness projects, it is a very interesting read. The only thing that astonishes me is that the previous hoster didn't notice that 2000 GIGA's of data were downloaded from the site in 1 week. Does really no one look at the logs over there ? It could be that the website was stolen or even that a professional attack was being prepared - if you see that the database structure is being analysed and downloaded.
In their own words
"In these difficult months, we have not been able to keep you informed step by step. The security of the site depended on our discretion. Now that the storm is over, it is time to tell you what happened to us, what happened to you.
For over 6 years, our site was hosted with a partner that we trusted and until recently gave us satisfaction. The site has grown, more and more, faster and faster! At the beginning of this year, it was very clear : it was time to install new servers. After analysing the needs in hardware and gathering the necessary funds, we made them buy a set of new servers during the month of April. Everything should have been up and working very quickly and we were hoping for a significant improvement within a few weeks. Unfortunately, the installation of the set was very slow.
In this context, we have had to face a major crisis for our site on June 9. That day, the whole of our database was lost. We trusted our web hosts and thought regular backups were done. But, we discovered it was not the case. The last back up was 10 days old. 10 days! That is such a long time for a site like ours; 2 500 lost registrations, 350 000 items listed for sale lost, 100 000 sold items lost, …
It then became very clear to us that we had to bring back the site in Brussels very urgently. The situation was serious and very dangerous, all our database and all the images of the sales were on servers that appeared at that time so far from our reach and control... Much too far!
We have therefore announced to our web hosts that the servers and the data had to come back to us as soon as possible and that we counted on their taking part in these operations. Communication between us and them then degraded more and more since the end of June. As our hosts were sometimes very quiet, sometimes opposed to our project, we understood that talking would not solve the situation and might even make things worse. Each day gone by brought further danger. We risked loosing our data.
The situation was critical : the site was not working properly, the members were worried for the site’s future, our staff worried for their jobs… And above all that, we couldn’t keep you informed what was going on for our own safety and yours.
This is why we decided to keep things steady with our host and, discreetly, take contact with new partners. Among the companies we sought, we have chosen Evonet and Tigron who came up with a professional solution, in order to firstly save the site, and secondly make it pleasant to use for all our members. As there was great risk to draw the attention of our previous hosts, we have had to download slowly and secretly the whole of our data : More than 2 000 gigabytes, a huge database and 60 million images !
This download took a little more than a week. During this time, a brand new server structure had to be set up in Brussels in just a few days. Unfortunately, our previous hosts couldn’t do it in months.
We have invested a great deal in our new Belgian hardware. This has been made possible thanks to a leasing plan with our new partners who strongly believe in the Delcampe project.
As we have been through so much, surpassed so many difficulties, we are finally going to be able to realize all the projects that were impossible for us to set up. Our servers and data are now physically within range and easily reachable and this security strengthens our collaboration with our new partners "
I'll for sure wish you all the best and maybe it may convince some Belgian hosters that it is always better to have your data and business close to home than somewhere far away (and for the Belgian hosting business to professionalise and standardize their business just as other professions without any internal reglementation (for example real estate) have done)
It has been all over the news in the more serious technical magazines and papers here, there is a report that says that next year we will have more power outages and difficulties and that the situation may worsen year after year. The reason : we don't produce enough electricity ourselves and so we will have outages.
This means that if you have a datacenter or critical infrastructure the backup batteries (UPS) and failover strategies (providers) and own electricity production becomes an important aspect of your business continuity plan.
Some numbers from 2005 show that Belgium is quite dependent on import of electricity (and other energy resources) http://www.eia.doe.gov/emeu/world/country/cntry_BE.html
Since a few weeks we get more and more spam that passes the antispam filters from Yahoo, but recently even mails (phishing) that claims to come from Yahoo has passed their filters and even their domainkeys should normally have stopped this mail because normally Yahoo would have seen that this mail couldn't come from Yahoo and should have blocked it.
There is nothing worse than telling people that you have built a system that will identify if a mail comes from a certain mailserver so that people will trust mail that says it comes from your services (but not from your mailserver) and letting it degrade.
I can understand that Yahoo is going through rough times and maybe people got fired, replaced or aren't doing their job properly any more, but the biggest advantage of Yahoo the last years was that it was one of the best antispam and antiphishing services around.
This is changing and fast. Maybe it is because Trend Micro is also degrading fast - which is the reason people are throwing Trend Micro solutions out of the window and choosing other solutions. But there is definitively a problem with domain keys here
Or before there was a lonesome soldier having a human watch on the traffic and adapting strategies to the tactics of the spammers.
These are the headers of the mail - and this is the reason you should always look at the headers if you aren't sure (or at the source of the page - rightclick)
There was a time that the belgian banks were very proud of their security online and said that what you were reading about online banking elsewhere couldn't happen here or nearly.
The reason was that to transfer money you had to be in the possession of three things. You needed a smartcard or IDcard and a card reader, a password and a pincode you typed on the card reader. So it would be much more difficult to intercept and transfer the money for online hackers.
But because they wanted to be more friendly for their clients and they said it would all be as safe (or tried to make us believe this) they reduced the security online to what you know (password).
It is also clear from the incident that as long as the Belgian online consumers law is protecting the Belgian surfers the consumers themselves won't pay for the stupidity of the banks in the field of security. But changing the law can change all that.
It is also clear that there is no control of the machines before coming in the online banking forms if your machine is infected or not or if it has the most essential security software like an antivirus software that is uptodate.
Some people were the victims of their own stupidity by downloading russian cracks for programs with non-detectable trojans in them that got passed the laughable downgraded defenses from the online banking in Belgium.
The problem of online russian attacks with a very high sophistication level and a precision of targeting can be bigger than the media led us to believe.
When I am upset I say no and when not i say 'yes, but you have to....' and than people look at me with great eyes and say 'I can't do that' and than I say 'sorry'. Most of the times, it takes some more time and things get done, but as they should be, not as they could be.
I don't know if you know the story about the naked king ?
There was once a king and he wanted always to have the best clothes. He wanted to look at his finest the whole day every day. Every day he paraded with his new clothes in the streets of the capital. One day two smart crooks came by and said they would make the most beautiful clothes. He believed them and made so-called invisible clothes for him. His friends, family and court didn't have the courage that the king was naked when he went outside in the capital to parade in his invisible clothes. It was a good laugh for the citizens of the capital and a very embarassing moment for the naked king.
The ITsecurity advisor should say to the king that if he places these services or servers on the network his whole system will go down and that he will be the laughing stock of the internet. Even if he gets fired for that.