6 disks 4x1TB and 2x2TB. How best to sync them for backup.

RimBlock · 21 Jun 2011 at 08:34

Hi,

I have SBS 2011 and am looking at the best way to backup my files.

I have 4x1TB live drives and 2x2TB backup drives.

Currently the live drives are in a striped array but i am not so sure that is needed anymore so I could probably split them. Having them in a striped array makes it a pain to expand as space runs out.

The 2TB drives are stand alone and the live drives data has filled one and is now new data is being copied to the second one manually.

Buying new drives is unlikely at this point.

What is the best way to make sure I have a copy of the live data on the backup drives. Pref a nightly check and copy of new/changed files if there have been any changes. The data is only TV/Music/Movies but I don't want ot have to spend days re-ripping them and I do not need redundancy on the live data if I have a full backup set.

Important data (family docs/pics etc) will be on separate drives and backed up one for one nightly. It is the 4 (or one big array) into 2x2TB I am unsure of how to automate.

Using the 4 drives like a pool and the backup also as a pool and backing up one to the other seems the best but I am not sure if that is available in SBS 2011 standard and having used a JBOD array before and loosing everything when a single disk got corrupted, I hope pooling would not be affected the same.

Thanks

RB

Rob7865 · 21 Jun 2011 at 11:41

SBS supports backing up to external USB drives - Just run the backup Wizard and it will take care of it...

With SBS ALWAYS USE THE WIZARDS - Don't try setting things up without USING THE WIZARDS - It will come back and bite you on the bum later!!

PS - I would not use Stripe on a Server OS, Mirror Min

Rob

mikeh501 · 21 Jun 2011 at 12:27

Use robocopy. Works great for this kind of thing and its free and very flexible with how it handles your files/folders etc. Once you've written the robocopy script you can then schedule it via windows scheduler

RimBlock · 21 Jun 2011 at 15:02

Rob7865 said:
SBS supports backing up to external USB drives - Just run the backup Wizard and it will take care of it...

With SBS ALWAYS USE THE WIZARDS - Don't try setting things up without USING THE WIZARDS - It will come back and bite you on the bum later!!

PS - I would not use Stripe on a Server OS, Mirror Min

Rob

Having problems with the wizards at the moment. The setup the internet wizard keeps telling me my internet router has DHCP enabled when it doesn't.....

The backup wizard only allows me to backup the OS drive plus some other stuff. These disks do not include the disk that has the OS on it. The OS is on another disk altogether. It seems the backup wizard will not allow me to backup anything apart from the OS drive plus other stuff so I cannot do a data drive only to a backup data drive.

Cheers
RB

RimBlock · 21 Jun 2011 at 15:03

mikeh501 said:
Use robocopy. Works great for this kind of thing and its free and very flexible with how it handles your files/folders etc. Once you've written the robocopy script you can then schedule it via windows scheduler

Will take a look at that, thanks Mike.

I hear there is a powertoys sync bit of software too but I also understand it is a bit slow with large amounts of data and as my new rips are around 40GB it may not be so good.

RB

mikeh501 · 21 Jun 2011 at 15:44

RimBlock said:
Will take a look at that, thanks Mike.

I hear there is a powertoys sync bit of software too but I also understand it is a bit slow with large amounts of data and as my new rips are around 40GB it may not be so good.

RB

The "synctoy" tool is simliar to robocopy (itself another microsoft add-on) but nowhere near as configurable and does not need a interface like synctoy. I've used both thinking the new synctoy was better but soon went back to robocopy.

I use robocopy to backup about 4.5Tb of movies, music and photos on my home server to a couple of 2tb USB drives hanging off another server, so it can handle large jobs without a problem. I also have mine configued to log its actions so I can see whats it done.

I have mine set to echo the source to destination, but you can configure it for two way synchronisation and also echo without delete etc. Very good tool.

JayMax · 21 Jun 2011 at 23:00

An additional negative of synctoy is it's bug with long filepaths, it'll just ignore them and not log any error or warning.

RimBlock · 22 Jun 2011 at 02:46

Cool, robocopy looks like the one then.

Any idea how best to copy the 4x1TB -> 2x2TB ?. The 4x1TB can be made in to 2x2TB I guess as long as it can be shared as a 1x4TB drive via something like Namespaces.

Ideally I want one pool of space for movies, TV and music and the movies portion is currently larger than a single drives capacity. I am not so bothered about how the data is stored as long as it is shared as three shares, "TV", "Music" and "Movies".

Also while I am on the subject, any idea how to hide the other shares from the media player (ie. sysvol, redirectedfolders, usershare) as I would just like the three shares to be visible and not every share on the server for that specific user used only for logging in for media shares (user: Share).

Thanks
RB

mikeh501 · 22 Jun 2011 at 14:05

RimBlock said:
Cool, robocopy looks like the one then.

Any idea how best to copy the 4x1TB -> 2x2TB ?. The 4x1TB can be made in to 2x2TB I guess as long as it can be shared as a 1x4TB drive via something like Namespaces.

Ideally I want one pool of space for movies, TV and music and the movies portion is currently larger than a single drives capacity. I am not so bothered about how the data is stored as long as it is shared as three shares, "TV", "Music" and "Movies".

Also while I am on the subject, any idea how to hide the other shares from the media player (ie. sysvol, redirectedfolders, usershare) as I would just like the three shares to be visible and not every share on the server for that specific user used only for logging in for media shares (user: Share).

Thanks
RB

I might be missing your point here

But why cant you just generate a stripe set in windows for both the live and backup "drives"

4 x 1Ttb Disks = 4tb Volume "Live" drive
2 x 2Tb Disks = 4tb Volume "Backup" drive

Then put the TV, movies and music folders on the Live volume containing everything you have.

Then share out the TV movies and music folders for consumption around the house.

Then setup robocopy to copy the folders within the Live volume to backup volume.

The above is exactly how my setup works, but my backup drives are attached to a different server thats all.

RimBlock · 23 Jun 2011 at 01:49

mikeh501 said:
I might be missing your point here

But why cant you just generate a stripe set in windows for both the live and backup "drives"

4 x 1Ttb Disks = 4tb Volume "Live" drive
2 x 2Tb Disks = 4tb Volume "Backup" drive

Then put the TV, movies and music folders on the Live volume containing everything you have.

Then share out the TV movies and music folders for consumption around the house.

Then setup robocopy to copy the folders within the Live volume to backup volume.

The above is exactly how my setup works, but my backup drives are attached to a different server thats all.

Hi Mike,

Yes, that would seem to be the sensible way to do it but....

The backup drives are WD Green 2TB drives and as I have had 5xWD Green 1.5TB drives all fail on me within a few months when I put them in a raid-5 array, it seems they are not suited to being in an array.

The reason for the different drives is to have the 4x1TB WD Blacks (faster but more expensive) for live to give good speeds for multiple HD media streams and have the WD Greens (cheep and large) for archive storage especially as redundancy is not needed for the home environment.

Now it could be that the WD Green 1.5TB drives just had an issue with being in arrays that has been ironed out with the 2TB drives and if so then I will give it a go but would really like someone to confirm they have no problems with this before I entrust my data to it.

Thanks
RB

mikeh501 · 23 Jun 2011 at 11:05

RimBlock said:
Hi Mike,

Yes, that would seem to be the sensible way to do it but....

The backup drives are WD Green 2TB drives and as I have had 5xWD Green 1.5TB drives all fail on me within a few months when I put them in a raid-5 array, it seems they are not suited to being in an array.

The reason for the different drives is to have the 4x1TB WD Blacks (faster but more expensive) for live to give good speeds for multiple HD media streams and have the WD Greens (cheep and large) for archive storage especially as redundancy is not needed for the home environment.

Now it could be that the WD Green 1.5TB drives just had an issue with being in arrays that has been ironed out with the 2TB drives and if so then I will give it a go but would really like someone to confirm they have no problems with this before I entrust my data to it.

Thanks
RB

Drives will and do fail; and ive seen them do it in groups before. I wouldnt worry about it myself; thats what your backups are for.

My advice with so few disks is to stripe them in RAID0 which will give you excellent read/write performance for streaming around the house. If you go this route you obviously need a backup strategy, which I think robocopy (and checking of logs) will give you to the backup drives. It would be better if those backups drives were attached to a seperate machine, but hey ho.

malman · 23 Jun 2011 at 12:24

RimBlock said:
Hi Mike,

Yes, that would seem to be the sensible way to do it but....

The backup drives are WD Green 2TB drives and as I have had 5xWD Green 1.5TB drives all fail on me within a few months when I put them in a raid-5 array, it seems they are not suited to being in an array.

The reason for the different drives is to have the 4x1TB WD Blacks (faster but more expensive) for live to give good speeds for multiple HD media streams and have the WD Greens (cheep and large) for archive storage especially as redundancy is not needed for the home environment.

Now it could be that the WD Green 1.5TB drives just had an issue with being in arrays that has been ironed out with the 2TB drives and if so then I will give it a go but would really like someone to confirm they have no problems with this before I entrust my data to it.

Thanks
RB

If you are using hardware raid you are probably finding the drives seem to die because of not supporting TLER/ERC/CCTL timeouts. Basically the raid controller wants the drive to give up trying to read and report an error so it can use the parity/mirror stored on the other disk to fix the read issue but as the drive timeout is longer than the raid controller timeout the raid controller marks the whole drive as bad as it thinks the disk has gone awol.

In single disk non-raid you want the drive to retry that read for quite a while before it gives up as thats the only chance you have of getting your data so they have longer timeouts.

You will find that some manufacturers have specific models of disk with the timeout set shorter for use in raid arrays.

google TLER/ERC/CCTL (various manufacturers use different names) for your model of drive. Some of the western digitals can have the firmware adjusted with a util to support TLER - some can't be permanently set and some can't be set at all.

malman · 23 Jun 2011 at 12:35

RimBlock said:
Having problems with the wizards at the moment. The setup the internet wizard keeps telling me my internet router has DHCP enabled when it doesn't.....

The backup wizard only allows me to backup the OS drive plus some other stuff. These disks do not include the disk that has the OS on it. The OS is on another disk altogether. It seems the backup wizard will not allow me to backup anything apart from the OS drive plus other stuff so I cannot do a data drive only to a backup data drive.

Cheers
RB

SBS 2008 had a 2GB partition limit on the in built backup system (nice system when it works) which is probably why you can't see the other drives in the wizard. I can't find anything that says the limit was removed in SBS2011 or that its still there either, so thats a guess at what you are seeing

RimBlock · 24 Jun 2011 at 02:06

mikeh501 said:
Drives will and do fail; and ive seen them do it in groups before. I wouldnt worry about it myself; thats what your backups are for.

Sure they do but 5 drives bought in two batches 8 months between each other (2 drives then 8 months later another 3 drives) with the different batches having different firmware revisions and all 5 drives failed with excessive bad sectors within 3 months of putting them in a raid 5 array (prior to that they were used as single drives) ....

mikeh501 said:
My advice with so few disks is to stripe them in RAID0 which will give you excellent read/write performance for streaming around the house. If you go this route you obviously need a backup strategy, which I think robocopy (and checking of logs) will give you to the backup drives. It would be better if those backups drives were attached to a seperate machine, but hey ho.

Yep the 4xWD Blacks are striped and work very well, it is just the matching backup strategy I am working on using the 2 WD Greens.

The backup will not be on another machine and all things being equal, if I lost the data then it would be a pain but not a massive issue as it is just media on these drives. My personal stuff I will backup to a usb drive as well as having the live and possibly an internet drive (cloud but need to check ramifications on privacy first for that).

Thanks again Mike.
RB

RimBlock · 24 Jun 2011 at 02:33

malman said:
I...google TLER/ERC/CCTL (various manufacturers use different names) for your model of drive. Some of the western digitals can have the firmware adjusted with a util to support TLER - some can't be permanently set and some can't be set at all.

Yeah I am very aware of the TLER issues having done a great deal of research to try and find out why the drives were giving me so many issues

. Unfortunately this was not directly the cause of my drive failures as they didn't drop out of the array but failed with an excessive bad sector count.

The symptoms were that the array would go fine until it read data from a specific drive (same drive every time) and then the drive light would just stay on and the video would freeze for a 5->10 of seconds. The video would then continue to play until data needed to be read from the same drive again. Checking the smart data showed a warning on excessive bad sectors and recommended RMA. I returned the drive and got a replacement which I put in the array and rebuilt it only to have a second drive exhibit the same behaviour (different drive bay as well so not an issue with the drive bay). RMA'd this drive and tried again to have a third drive do the same. All three drives were a mixture of two different batches of the same drive. After the third drive failed and was RMA'd I broke the array and used the two remaining drives as standalone drives and they both failed within another couple of months with bad sectors. THese were WD Green 1.5TB drives and my WD Green 2TB drives have shown on issues at all but have also not been in an array at all.

There is much talk about the WD Greens auto part being set to 8 secs and that is most likely that is playing up with the drives as they try to spin up in time for the array to use them. The array was a Linux software array (mdadm) and worked flawlessly with my WD Black drives.

Have a dig around for WD Green 8sec park" in google for more info on this issue. I am hoping this has been 'fine tuned' for more recent WD Green drives.

RB

RimBlock · 24 Jun 2011 at 02:35

malman said:
SBS 2008 had a 2GB partition limit on the in built backup system (nice system when it works) which is probably why you can't see the other drives in the wizard. I can't find anything that says the limit was removed in SBS2011 or that its still there either, so thats a guess at what you are seeing

SBS2011 will only allow me to set one backup that must include the system partition. I can see all the drives in the backup drives options even though my array is over 4GB. What I would like to do is setup 3 backups, one for the system partition, one for my media area and one from my personal data all going to separate backup drives. It seems I cannot do this with the tool built in to SBS2011.

Cheers
RB

malman · 24 Jun 2011 at 23:25

you can if you don't use the wizard. you will not be able to backup any partition that has more than 2Tb as the backup is still limited by the VHD file format as far as I know.

If you want an easy bare metal restore type backup of the system then you can use wbadmin scripts or you can run the backup console from administrator tools and setup a backup job there, where you can choose what to backup. You could exclude your data from the image backup. I

f you want the system backup and the data backup (robocopy)on the same drive then setup the backup to go to a network share then share the backup disk on the server and point the backup to that share. otherwise if you point it directly at the disk it will partition, format and then remove the drive letter so that it is only available to the backup system.

The image based stuff is great for a system restore when you loose the OS disk and its equally good for just picking out an individual file when needed.

RimBlock · 27 Jun 2011 at 02:09

malman said:
you can if you don't use the wizard. you will not be able to backup any partition that has more than 2Tb as the backup is still limited by the VHD file format as far as I know.

If you want an easy bare metal restore type backup of the system then you can use wbadmin scripts or you can run the backup console from administrator tools and setup a backup job there, where you can choose what to backup. You could exclude your data from the image backup. I

f you want the system backup and the data backup (robocopy)on the same drive then setup the backup to go to a network share then share the backup disk on the server and point the backup to that share. otherwise if you point it directly at the disk it will partition, format and then remove the drive letter so that it is only available to the backup system.

The image based stuff is great for a system restore when you loose the OS disk and its equally good for just picking out an individual file when needed.

Great, this is the funky workaround I expect to get from the people here

.

SBS 2011 is currently giving me more issues than I am happy with for a home setup so may revery to WHS2011 and see if I can find addons for the extra functions and do without those I cannot. For a business env then a I can tolorate a bit more pain for the gain but at home it is becoming a bit too much of a drag.

Thanks I will give the backup a go.

RB

Rob7865 · 27 Jun 2011 at 15:44

I think for what you are trying to achieve WHS 2011 will be ideal for you.

Rob

RimBlock · 28 Jun 2011 at 02:58

Rob7865 said:
I think for what you are trying to achieve WHS 2011 will be ideal for you.

Rob

Yep you are probably right. The extra 8GB arrived from the US yesterday and it made a bit of a difference in the responsiveness of the server but not as much as I would have liked.

Getting three media players to connect at the same time is a pain as I am guessing you can only login from a single account using a single ip address at any one time. Created three separate accounts and now it is a case of remembering the account names and passwords and trying to put them in using a screen keyboard using a tiny remote.

Anyway, 15 days trial remaining and so at the weekend I will try going back and installing WHS2011.

Thanks for the help and suggestions everyone.

RB