mounting a ddrescue...
 
Notifications
Clear all

mounting a ddrescue image file from a failed Linux software md RAID10 member  

  RSS
RtheK
(@rthek)
New Member

I have an external e-sata that has 4 drives and was configured RAID10 on Fedora 32 via mdadm. 2 failed. I still have access to all of them in Linux, e.g., they are labeled /dev/sd[a-d].

When I try ddrescue it creates an image and when I try to mount I get:


mount sda3.img /mnt/test/
mount: /mnt/test: unknown filesystem type 'linux_raid_member'.

Are there any other tools that can convert the filesystem type or anything that will open/mount this image? I'm also trying TestDisk (taking a long time) and a program called Foremost found lots of files but renames them all as such:

Num Name (bs=512) Size File Offset Comment

0: 402037.jpg 30 KB 205843252
1: 402821.jpg 83 KB 206244545
2: 400108.zip 1 MB 204855558

So far on deep scan TestDisk has found:


Disk /dev/sdc - 2000 GB / 1863 GiB - CHS 243201 255 63
Analyse cylinder 198186/243200: 81%

Linux RAID 0 32 33 243184 219 51 3906762760 [ourserver:2]
Linux 45320 133 62 226682 174 52 2913583104
Linux 45593 248 32 410363 118 27 5860021856
Linux 45594 26 1 410363 150 59 5860021856
Linux 45596 166 11 410366 36 6 5860021856
Linux 45600 251 28 410370 121 23 5860021856
Linux 45601 93 61 410370 218 56 5860021856
Linux 45611 112 6 410380 237 1 5860021856
Linux 45624 47 56 410393 172 51 5860021856
Linux 45625 85 29 410394 210 24 5860021856
Linux 45638 183 50 410408 53 45 5860021856
Linux 45640 128 57 410409 253 52 5860021856
Linux 45641 36 28 410410 161 23 5860021856
Linux 45642 204 3 410412 73 61 5860021856
Linux 0 32 33 181362 73 23 2913583104
Linux Swap 181362 105 56 182401 67 50 16689136

Would appreciate any suggestions!
Quote
Posted : 11/06/2020 4:57 pm
Topic Tags
jaclaz
(@jaclaz)
Community Legend

You may try dmde (it has Linux versions), the Windows version works well with hardware Raids, cannot say the Linux version on mdadm:

https://dmde.com/download.html

 

But before that, you cannot mount directly a RAID image, see here:
https://forums.gentoo.org/viewtopic-t-1053226-start-0.html

and:

https://www.forensicfocus.com/forums/general/linux-raid-reconstruction/

you need to use losetup and mdadm to make a new RAID from the single image.

 

jaclaz

ReplyQuote
Posted : 12/06/2020 1:44 pm
RtheK
(@rthek)
New Member
Posted by: @jaclaz

You may try dmde (it has Linux versions), the Windows version works well with hardware Raids, cannot say the Linux version on mdadm:

https://dmde.com/download.html

I've been running this on the RAID members and it's finding files from BEFORE I even made this a RAID10. The "raw" options returns mostly broken images, some being just half ok, and then the other half of the image being solid gray, and again from before this was turned into a RAID 10. I'm still trying scans and looking at the build a RAID option. I would thing I can just choose the ExtFS option and ignore Raw, since Raw files get renamed and pretty useless to us.

https://www.forensicfocus.com/forums/general/linux-raid-reconstruction/

you need to use losetup and mdadm to make a new RAID from the single image.

I've seen reference to that and tried but I could never mount anything, as I'd get errors like not enough disks. Perhaps mounting 1 drive of a RAID 10 won't work using this method?

I'm just stunned that TestDisk, DMDE, Foremost, and Scalpel all find the pre-RAID data, and not the RAID10 files. Any other suggestions?

 

ReplyQuote
Posted : 13/06/2020 5:04 am
jaclaz
(@jaclaz)
Community Legend
Posted by: @rthek
Posted by: @jaclaz

You may try dmde (it has Linux versions), the Windows version works well with hardware Raids, cannot say the Linux version on mdadm:

https://dmde.com/download.html

I've been running this on the RAID members and it's finding files from BEFORE I even made this a RAID10. The "raw" options returns mostly broken images, some being just half ok, and then the other half of the image being solid gray, and again from before this was turned into a RAID 10. I'm still trying scans and looking at the build a RAID option. I would thing I can just choose the ExtFS option and ignore Raw, since Raw files get renamed and pretty useless to us.

 

Well, you most probably have not managed (not that I know if it is possible at all) to re-build the RAID in dmde (as well as you didn't manage to rebuild a mounted raid in plain Linux) from the single dd image so you are probably looking at a "sequential mish mash" of ex-RAID stripes, so - basically - you should be able to find "good" files only if they are smaller than the stripe size.

As said I have not any experience with software (mdadm) RAIDs, but the general idea should be not much different from hardware RAIDs, with them, and in the Windows version of dmde, as well as in other similar software, you would make 4 (a separate dd image for each physical disk)  images then once determined the RAID parameters re-assemble the RAID virtually.

Since that is a RAID10:
https://en.wikipedia.org/wiki/Nested_RAID_levels#RAID_10

It should be easy to find which 2 disks (images) are the striped set (one of the two RAID1) and which 2 disks (images) are the mirrored striped set (the other of the two RAID1), then you only need to rebuild virtually one of the two RAID1 sets, and I believe that the needed parameters you can still recover from the mdadm settings.

jaclaz

ReplyQuote
Posted : 13/06/2020 9:38 am
passcodeunlock
(@passcodeunlock)
Senior Member

Make dd images of your drives and build a virtual raid with R-Studio, then recover whatever you need.

ReplyQuote
Posted : 13/06/2020 8:48 pm
RtheK
(@rthek)
New Member
Posted by: @jaclaz

Well, you most probably have not managed (not that I know if it is possible at all) to re-build the RAID in dmde (as well as you didn't manage to rebuild a mounted raid in plain Linux) from the single dd image so you are probably looking at a "sequential mish mash" of ex-RAID stripes, so - basically - you should be able to find "good" files only if they are smaller than the stripe size.

OK I picked 2 drives and tried rebuilding a RAID 0, since RAID 10 is like RAID 1 + RAID 0. There seems to be a lot of file systems on there, have a look at this screenshot of DMDE running, 5 hours 25% done. Based on this screenshot, does that mean there are that many EXT file systems that the program found? It's odd that this external e-sata drive only holds 4 disks and 3 were 1.5 TB and 1 was 2TB. Just wondering how the files systems with 15.8TB were found. The 1st entry shows a Check value of 17 with a "B" for Indicator. I take it that's the most likely option I want? Do I have to wait for the program to complete?

It should be easy to find which 2 disks (images) are the striped set (one of the two RAID1) and which 2 disks (images) are the mirrored striped set (the other of the two RAID1), then you only need to rebuild virtually one of the two RAID1 sets, and I believe that the needed parameters you can still recover from the mdadm settings.

jaclaz

I should've kept track of the UUID's. Power cycling the server and the e-sata causes the /dev/sdX names to change. So I'm making an educated guess on which were a part of the RAID 10.

Also I've had my WiFi connection apparently be unstable for a split second as I open the program on Linux via SSH, i.e., then X-Windows/X11 and I've gotten


Network error: Software caused connection abort.

Does the log get saved so I can restart DMDE where it left off?

Thanks for the tips.

ReplyQuote
Posted : 13/06/2020 11:46 pm
athulin
(@athulin)
Community Legend

Jaclaz wrote:

It should be easy to find which 2 disks (images) are the striped set (one of the two RAID1) and which 2 disks (images) are the mirrored striped set (the other of the two RAID1), then you only need to rebuild virtually one of the two RAID1 sets, and I believe that the needed parameters you can still recover from the mdadm settings.

If forensic soundness is an issue, this approach also requires some kind of indication that the RAID is sound, i.e. there are no pending errors (in the mirroring). This information needs to be collected before the imaging is done.  Only if it is known that some particular tool is able to perform the equivalent function, can it be left until later -- but that is clearly not the case here.

ReplyQuote
Posted : 14/06/2020 6:41 am
jaclaz liked
jaclaz
(@jaclaz)
Community Legend
Posted by: @athulin

If forensic soundness is an issue, this approach also requires some kind of indication that the RAID is sound, i.e. there are no pending errors (in the mirroring). This information needs to be collected before the imaging is done.  Only if it is known that some particular tool is able to perform the equivalent function, can it be left until later -- but that is clearly not the case here.

Yep, but this here is seemingly a case of data recovery, and RAID 10 is (should be as mdadm is used) a spanned set (Raid 0) of two mirrored sets, Raid 1.

https://en.wikipedia.org/wiki/Non-standard_RAID_levels#LINUX-MD-RAID-10

hopefully the mdadm set (of four disks) is like the "standard" RAID 10.

In such a setup:

Disk #1=A1
Disk #2=A1
Disk #3=A2
Disk #4=A2

So you can try recovery on 4 "Raid 0" sets:
Disk #1+Disk #3
Disk #1+Disk #4
Disk #2+Disk #3
Disk #2+Disk #4

@RtheK

I don't know, what these tools can see is often the results of the parameters of the RAID you feed them with, and of course the set (couple) of disks need to be the "right" set and in the right order.

See also:
https://www.forensicfocus.com/forums/general/raid-recovery/
https://www.forensicfocus.com/forums/general/reconstructing-raid-using-linux/

If you are not "limited" to Linux, there are other programs usually capable to rebuild virtually a RAID set for windows (though as said dmde worked for me in windows, but all RAiD's I have ever dealt with were hardware sets), as an example:
https://www.runtime.org/raid.htm

jaclaz

 

 

 

 

ReplyQuote
Posted : 14/06/2020 11:06 am
Share: