Pitfalls of Bitcoin Ownership: Disk Failure

I was in the middle of browsing the web when suddenly my system stopped responding to me.  The mouse was still moving, but none of the windows were registering actions.  My system froze and locked up on me.   After cycling the power, I noticed the kernel message log spewing out all kinds of errors related to the disk drive that housed my “/home” partition.  My disk was failing.  Normally, I’d just throw a new disk in, recover from backups, clone the appropriate git repos, and proceed.  However, this time was slightly different because  my bitcoin wallet was on the failing drive and wasn’t backed up yet.

Thankfully, the value of my bitcoins doesn’t amount to much more than a few dollars.  Nevertheless, had I been reliant on bitcoins for substantial financial purchases, this could have been a disaster scenario.  There is much talk on internet forums about people losing thousands of dollars worth of bitcoins due to hardware failure.  Even though there was more than adequate warning to pursue backups,  I didn’t consider that I’d be confronted with such a scenario so early on in my experience with crypto-currency.  (I sort of figured the dollar value of my holdings wasn’t substantial enough to care, and I had only had the coins for a week or two before the disk failed.)  However, after the disk failed, I was determined to get my coins back and I managed to succeed.

This post details roughly what I did to recover my wallet file on a linux-kernel based operating system with an ext4 filesystem.  Unfortunately I don’t have screenshots and the process I describe below may be somewhat imprecise.  However, my post might serve as a rough outline of how to retrieve your wallet in the event you start seeing evidence of disk failure.  Please note that I can’t make safety/success guarantees about these steps and they may not work for you (in fact, they may even cause damage.)  If you have substantial bitcoin value at risk, you may want to consider reading a few different sites before trying these steps or, better yet, find a data recovery specialist or expert.

In my situation, I use an ext4 file system.  Because my system was for general purpose use, I did not have the /home partition encrypted.  When the disk stopped operating, I took the following procedures to recover my bitcoins:

1. I downloaded an Ubuntu 12.04 LiveCD
2. After booting from the CD, I clicked “Try Ubuntu”
3. I loaded FireFox and downloaded TestDisk  and unpacked the files.
4. After launching XTerm, I ran TestDisk (as root/sudo’d).  I then navigated to the partition I knew was my ext4 partition on the disk device I knew was failing (in my case, a partition on /dev/sdb).  I then used the file system utilities  menu (“Filesystem Utils”) and tagged the partition (via the [T]ype operation in TestDisk) as an LVM->Ext4 partition.  Before using the [T]ype operation, TestDisk showed the ext4 partition as just “MS Data”.  Specifying the type made TestDisk “see” the partition correctly.
5. I tried to list the files in TestDisk (I think it was the “L” command); this failed.  Don’t panic (yet.)
6. Since listing the files failed, I used the TestDisk superblock option to get the appropriate disk information from the backup superblocks on the filesystem.  TestDisk gives some fsck.ext4 command to attempt to repair the filesystem, but I ignored it and just went back to the menu and retried the “list” operation
7. Once the files started listing, I navigated to the /home/<username>/.bitcoin directory, and then I copied the wallet.dat to a USB memory stick that I had inserted.

Once the wallet.dat was recovered and the disk drive was replaced, I recovered /home from my usual backups and I placed the file into a new .bitcoin directory on my system.  ‘bitcoin-qt’ (my wallet app) accepted the file and successfully recovered all of my bitcoins.  It began synchronizing with the p2p network.

To prevent future scenarios like this, I configured RAID and put a script in place to routinely backup the wallet file.  For the record, TestDisk is amazing software.  I just hope I never have to use it again.  🙂

Good luck to those who are ever in this situation.  I hope the steps above help you, or at least point you in the direction of the right tools to recover your coins.