Warning on keeping Digital Sound Recordings and Backup (1 Viewer)

[Jon.Bryant]

Probably obvious to everyone (except me!), but I have learnt the hard way not to use a dual slot HDD caddy to house the source and backup for my sound files.

I had a caddy for several years that I used in this manner, then recently the device suddenly corrupted the allocation tables on both drives simultaneously! - all of a sudden, I lost the originals and the backup. I know that the golden rule of three, but for large sound collections it can be hard to keep two backups (particularly with one in the Cloud).

I purchased Disk Drill and fortunately the software managed to recover the files (actually recovery was more successful on the backup set - this was not quite current (another mistake), so I lost a bit of stuff that was not too important). Disk Drill was amazing, but it took an inordinate amount of time (circa 24 hours), during which I feared the worst.

From my experience, I would say you can use an external HDD caddy for source or backup, but never use a dual slot caddy to house both HDDs simultaneously, unless you abide by the rule of three and have another backup you can call upon.

 

[Ted Y.]

How can a caddy corrupt a HDD?
What kind of caddy was used?

 

[Jon.Bryant]

I think caddy is the wrong term? The device blurb says HDD docking station.

To be honest, I do not know how the USB device corrupted the disks. What I do know is that the file allocation tables for both disks became garbaged. I had never seen a disks showing a 'RAW' format instead of EXFat, NTFS or similar. Looking on the internet it suggested that file allocation tables can become corrupted if a disk does not spin down properly. Putting 2 and 2 together, I suppose that the device power supply or the internal power supply to the disks became unstable. Alternatively, I think that poor USB connection can also create the problem, so perhaps there was some issue with the device USB - but not the computer USB which still works fine.

What I do know is that once I had recovered the data and (re)formatted the disks, every time I turned the device off and off, the allocation tables on both disks were lost again. I have since used the reformatted disks in other USB HDD enclosures and they work without issue. Needless to say, I have thrown away the troublesome device!

The device I was using was an 'Inateck Aluminum USB 3.0 To SATA Dual-Bay USB 3.0 Hard Drive Docking Station with Offline Clone Function for 2.5 Inch & 3.5 Inch HDD SSD SATA I/II/III', bought in 2020 from Amazon UK. Not that expensive, and I was probably foolish to trust it to house HDDs storing source and backup files for so long.

 

[Joop99]

Never ever use HDD in a external portable medium like a recorder or camera. Always use (today SSD) cards, they can be copied easily with multiple systems. Your HDD in your computer can be backupped to an external device on the USB port. Always use different devices and be keen how things go, so do't walk away from a backup in proces, look at the light what it will do, its a machine, so the proces should go every time in the same way. If not, this can be a pointer that things are going to break. Always use a card in a recorder, because with copy them to the computer you need to eject the card from the slot and insert it into a cardreader on the computer. Don't be lazy, just do it this way. Why? Because during eject and insert and whole proces return the contacts are cleaned be removing "rust" and other stuff which can block good contact. Mind your outside with the recorder and the environmoent there is not exactly the same as in house.

 

[Jon.Bryant]

Probably should have said, that the collection of sounds included quite a lot of NocMig Recordings that I was planning to review, which is why I was storing the recordings and the backup on external HDDs (in the same docking station - 2 x 2TB) and didn't have the recording on my computer HDD, which simply didn't have enough free space.

I now have the master copy on a NAS with RAID (so one disk can fail and be rebuilt without losing data), then a backup on an external HDD. I will probably also keep copies of processed and finalized recordings (which will take up a lot less memory) in the Cloud, or on M-Disk (just in case). It can take a lot of time to process sound files, and it is very depressing when you think you have lost your recordings coupled with all the time you put into post processing!

I would say that if you use backup software rather than just keeping two copies, you don't need to watch the process - most backup software is pretty good at reporting on whether the backup has failed or succeeded - and incremental backups can take hours, so it would be a tedious process to monitor.

 

[Ted Y.]

Probably should have said, that the collection of sounds included quite a lot of NocMig Recordings that I was planning to review, which is why I was storing the recordings and the backup on external HDDs (in the same docking station - 2 x 2TB) and didn't have the recording on my computer HDD, which simply didn't have enough free space.

I now have the master copy on a NAS with RAID (so one disk can fail and be rebuilt without losing data), then a backup on an external HDD. I will probably also keep copies of processed and finalized recordings (which will take up a lot less memory) in the Cloud, or on M-Disk (just in case). It can take a lot of time to process sound files, and it is very depressing when you think you have lost your recordings coupled with all the time you put into post processing!

I would say that if you use backup software rather than just keeping two copies, you don't need to watch the process - most backup software is pretty good at reporting on whether the backup has failed or succeeded - and incremental backups can take hours, so it would be a tedious process to monitor.

Something to learn from.

 

[Joop99]

I now have the master copy on a NAS with RAID (so one disk can fail and be rebuilt without losing data), then a backup on an external HDD.

Be sure you can use your disks also in an other system. Sometimes the disks can ONLY be used in THAT particular NAS. What's wrong setting a date in your agenda and do a backup on a external disk via USB. To be sure do it also with a second disk. Trouble with most backup software is that they backup all, not just a few files or a directory.

 

[Ted Y.]

do a backup on a external disk via USB

A few files using USB 3.0, OK.
A lot of files, scary.
A huge partition, horror.

I understand software can be used to copy data for a dedicated NAS drive to a Windows one. It is not free, but in this "business" better to do it by the book.

I now have the master copy on a NAS with RAID

At least a RAID does not need software to compare backups.

 

[Jon.Bryant]

At least a RAID does not need software to compare backups.

The recommendation is to still backup data on a NAS even with RAID configuration.

With RAID 1 (Disk Mirroring) one drive can die, but you only get 1/2 the capacity of the smaller disk if they are unequal. With Raid 5 (which requires a minimum of 3 disks), you get greater storage capacity - if all disks were equal, 3 disks would be 2/3, 4 disks would be 3/4, 5 disks would be 4/5 etc. of total capacity. Again, one disk can die and be replaced - the data is spread across all drives and a parity is kept, so that when a disk dies, the parity and the remaining data is used to recalculate 'the' data on the missing disk. It works but it can take hours for the NAS to recalculate and reconstruct the data on a large disk. RAID 6 is a similar to 5, but with requires a minimum of four disks and there is a greater hit on overall capacity. With RAID 6 there is greater redundancy and 2 disks can die simultaneously in the pool. Once the 2 disks are replaced the data on the disks can be reconstructed from the data on the remaining disk, and you are once more good to go.

I suppose the problem is that is possible for something catastrophic to happen and for you two loose more than 1 disk (or 2 disks with RAID 6) simultaneously - a bit like what happened in my 2 drive HDD docking station, hence the reason for backup. Perhaps a large power surge or burn out of a component in the NAS etc, could corrupt all HDDs in the pool.

The rule of 3 is that you have two backups of your data and keep the backups in different locations (perhaps one on disk and one in the Cloud), but this is pretty tough to do, particularly for an amateur enthusiast, having to pay for storage. I (now) try to keep storage and backup apart to be cautious. This is why an external drive can be useful. I have started to use the rule of three for stuff that I consider really important, with a copy on the Cloud or for stuff I will never edit again Blue-Ray M-Disk.

Sometimes the disks can ONLY be used in THAT particular NAS

Yes. For example, Synology have Hybrid RAID, this is kind of like a mix and match of RAID configurations to get the most capacity out of your drives if they are of different capacities. With standard RAID configurations the capacity is governed by the size of the smallest disk - say you had 4 3TB in RAID 5 (9TB of overall capacity) and you upgraded 2 drives to 6TB, you would see no change in capacity, as the data is spread across all the disks. With Hybrid RAID the lost capacity on the two disks would be used as an additional RAID 1 configuration, so you would get an extra 3TB (total 12TB).

The problem is this is Synology configuration (not an industry standard), so if the NAS drive dies, you would need to by a new Synology NAS - you couldn't use the disks in a QNAP NAS or another brand. I understand with standard RAID configurations the disks should work in any NAS (with the same or greater number of bays)

I understand software can be used to copy data for a dedicated NAS drive to a Windows one. It is not free, but in this "business" better to do it by the book.

Yes, I use Arconis True Image (now Arconis Cyber protect home office) - other software is available. Arconis currently costs £34.99 annually (they quite often have sales though). The software does disk cloning, backups (including incremental backups) etc. You can create multiple backup sets, with each set backing up specific files, folders, a group of folders or a drive (including files or folders on a NAS). You can also select different targets for each backup set, so can keep different backups on different hard disks. Each backup set can run automatically at a set interval/time, or you can trigger the backup to run manually. The only issue is that incremental backups do take a long time to complete - not sure why this is?

I think I had got the backup software right - some time ago the main disk on my PC died, so I invested in the software and a new SDD at that time to make recovery of the system and data easy. As I say my main mistake was keeping the original and the backup in the same location, and I think you have teh same risk if you use a NAS for both. Better to separate source and backup physically.

 

[Ted Y.]

This is why an external drive can be useful. I have started to use the rule of three for stuff that I consider really important, with a copy on the Cloud or for stuff I will never edit again Blue-Ray M-Disk.

Yes, I use Arconis True Image (now Arconis Cyber protect home office) - other software is available. Arconis currently costs £34.99 annually (they quite often have sales though). The software does disk cloning, backups (including incremental backups) etc. You can create multiple backup sets, with each set backing up specific files, folders, a group of folders or a drive (including files or folders on a NAS). You can also select different targets for each backup set, so can keep different backups on different hard disks. Each backup set can run automatically at a set interval/time, or you can trigger the backup to run manually. The only issue is that incremental backups do take a long time to complete - not sure why this is?

I did not knew Acronis is so good and at so affordable price.
I do not use incremental backups, just full clone of the disk. I rotate two SSD as backup. True... I do not have lots and lots of data.

 

[Elkhornsun]

Safest is to use a drive enclosure that supports RAID where data is writen to more than one drive. With RAID1 and two drive all the data goes to both drives and if one fails you have all your data on the other one and this data can then be written to the new replacement drive. I use RAID6 with 5 drives and I would lose data only if 2 drives were to fail at the same time.

The NAS boxes I use all have Ethernet ports so they are attached to my wireless router and all our computers and laptops have access to the files from anywhere in our house.

 

[Ted Y.]

In the RAID 6 system, up to two hard disks can fail at the same time without endangering the operation of the system.

 

[Jon.Bryant]

Safest is to use a drive enclosure that supports RAID where data is writen to more than one drive. With RAID1 and two drive all the data goes to both drives and if one fails you have all your data on the other one and this data can then be written to the new replacement drive. I use RAID6 with 5 drives and I would lose data only if 2 drives were to fail at the same time.

I know my failed docking station was not the same as a NAS, however I think my message still persists - there is a risk if you only rely of a bunch of disks in the same device. From what I understand, even if you use a NAS with RAID, it is still recommended that you backup key stuff elsewhere. Not trying to tempt fate, but image if the cooling fan on the NAS stopped working and before you realized what was going on, the drives were cooked, or worse still there was a small electrical fire due to a component burning out in the NAS. Perhaps this is catastrophizing, but I suppose backup is the same as any other type of insurance - it's a waste of time and money until one day it isn't!

I have bought a standard (not NAS grade HDD) which is housed in an external enclosure. I now periodically back up key data from the NAS to it. If I need more backup space, I will split the backup set and buy an additional drive to swap in and out of the same enclosure.