Something has corrupted my ext4 partition on my debian server. The server had an uptime of about 1700 days, and unfortunately the UPS did not have enough battery to last a whole day power outage.
I was running Debian sid on it (so stupid), and about the time systemd came into the picture i stopped doing apt-get dist-upgrade's. (more stupid)
After the power outage, it booted into systemd emergency mode, and it was not possible to mount the drive at all. (/dev/sdb1 not found). I tried apt-get dist-upgrade to see if it would fix the problem, but it failed after a while, and after rebooting I could not log in anymore because systemd failed. After a while I found sysvinit in the bootmenu, and could then boot into linux. (Should've done that first! doh)
The partition that's become corrupted is on a 750 GB harddrive and is used for /home. There should be about 100 GB of used space, but it's reported as 968 MB.
And somehow, the root directory structure for / (/dev/sda1) has been copied to /home (/dev/sdc1) ???
root@server:/home# ls -lah ls: cannot access 'run': Input/output error ls: cannot access 'sys': Input/output error ls: cannot access 'etc': Input/output error total 92K drwxr-xr-x 23 root root 4.0K Jan 14 2013 . drwxr-xr-x 23 root root 4.0K Feb 11 14:23 .. drwxr-xr-x 2 user2 user2 4.0K Nov 2 2010 bin drwxr-xr-x 2 user1 user1 4.0K Feb 3 2013 boot drwxrwxr-x 3 user2 user2 4.0K Aug 19 2005 dev d????????? ? ? ? ? ? etc drwxr-xr-x 2 user1 user1 4.0K Jan 15 2013 home lrwxrwxrwx 1 root root 30 Jan 3 2013 initrd.img -> /boot/initrd.img-3.2.0-4-amd64 drwxr-xr-x 14 root root 4.0K Jan 9 2013 lib drwxr-xr-x 2 root root 4.0K Jan 3 2013 lib64 drwx------ 2 root root 16K Jan 3 2013 lost+found drwxr-xr-x 2 user1 user1 4.0K May 31 2010 media drwxr-xr-x 6 user1 user1 4.0K Jan 15 2013 mnt drwxr-xr-x 2 user1 user1 4.0K Feb 5 2006 opt drwxr-xr-x 2 user1 user1 4.0K Feb 3 2013 proc drwxr-xr-x 2 user1 user1 4.0K Feb 3 2013 root d????????? ? ? ? ? ? run drwxr-xr-x 4 user3 user3 4.0K Mar 26 2013 sbin drwxr-xr-x 2 user1 user1 4.0K Jan 15 2013 selinux drwxr-xr-x 2 user1 user1 4.0K Feb 3 2013 srv d????????? ? ? ? ? ? sys drwxr-xr-x 3 user4 user4 4.0K Nov 27 23:05 tmp drwx------ 2 user5 user5 4.0K Aug 17 2009 usr drwx--x--x 13 user2 user2 4.0K Jan 12 2012 var lrwxrwxrwx 1 root root 26 Jan 3 2013 vmlinuz -> boot/vmlinuz-3.2.0-4-amd64 (usernames changed)
This looks identical to / (/dev/sda1).
But inside these folders are files which shouldn't be where they are, but are readable. For example, inside "opt" there are jpegs from somewhere in my own userdir. Inside "var" are files and folders from one user almost completely intact with correct directory structure and filenames, just files missing on at least one place.
I did a dd copy of the partition to another harddrive, and then tried fsck on it, it apparently "fixed" a lot of things, but then it wasn't possible to mount it any longer. "wrong fs type, bad option, bad superblock on /dev/sdc1, missing codepage or helper program, or other error."
I have no idea how this happened, or how to fix it, and I don't have high expectations that it will be possible. This is the weirdest thing I have seen in linux, and I mostly want to know how it happened. If I can recover all files though, that would make me a very happy man!
I know about PhotoRec, and plan on using it if I can't repair the original directory structure.
