Aegir stung me and Linode couldn’t heal me

I came across a nasty bug in Aegir today. Apparently it is still possible to loose data even if you try to do all the right things. I wanted to test some new stuff on my site. I cloned my production site to test the new stuff out on the clone. For some reason the clone never finished, so I deleted it. And did not think more about it. Then I went out of the office for a couple of hours. Coming back, I see that my production site is down with no database access.

I quickly went the the mysql database to see whether I could login in another way. But mysql SHOW TABLES did not help me at all. My database was gone. I went to #aegir on IRC, and I was not the only one who experienced this. It is an edge case, but a fairly critical edge case.

Other than that, I really like Aegir. I am just sad, that I did not install it on Barracuda and Octopus instead.

You can loose up to six days of work with Linode’s backup plan

I thought that I was secure when buying the backup plan at my hosting provider, Linode.

In my opinion the backup plan at Linode leaves a lot of room for improvement. They have one daily backup. Two weekly backups. And one manual backup. In my case the backup was run just before, I discovered that Aegir swallowed my database, but after it was gone.

Today is Tuesday. The weekly backup is on Sundays. That means that the most recent backup before the database disappeared is from Sunday. That means that two days of work is gone. Say it had been Saturday, six days of work is gone.

Support even told me today, that they are not planning on changing that behavior:

“We do not plan on making any changes to our backups system at this time - there will always be one daily backup slot, two weekly backups slots, and one manual backup slot.

… Please note that you can take a manual snapshot at any time - this will not be overwritten until you take another manual snapshot.”

And for that, they are charging 5$ a month. I think they should make a backup a day for a week available, and I think they should provide r1soft backups, so you can retrieve any file and not only restore to full system images.

Other than that, I have been very satisfied with Linode. I am just sad, that one first discovers failures in the chosen backup plan when it is too late.”

Aegir and backup contrib modules

Based on my experience, I also quickly installed hosting_backup_queue and hosting_backup_gc. Still haven’t decided what to do about the Linode backups. But really I should use Barracuda and Octopus. Barracuda comes with an integrated backup solution. But for that I need a clean server, and where should all my sites go in the mean time?