Am I expecting too much or is there a problem - RAID 10

Annoyingly the transfer went really slow for a few hours - it was doing a whole load of writing to the System Volume Information - something like 50 different streams giving a queue of over 50!

It's now up to 170+MB/Sec so hopefully shouldn't be too much longer.

170MBSec.jpg
 
You're getting pretty decent speed out of that RAID controller. And you're lucky you have the spare room to copy everything across.

I still don't understand your use of the VHDX (replication to where?), but I get the point that you're doing it for learning.

Windows Server 2016 has some really cool storage replications stuff.
 
I have an ML330 doing nothing but the hooky server licence I had for it had it's key cancelled. When I can afford a real one I'll leave it at my parents with it replicating.

Started the first of a couple of tests on the now blank array.

256KiB strip with 4k allocation for the VHDX. It's creating the disk now at an impressive 714MB/Sec - but that would be ideal conditions!
 
Thick = Fixed rather than dynamic? Yes :)

A microserver WOULD be ideal - but I already own the 330 and have already way overspent so buying another server would be a bit silly - and it's not my power bill :p besides, reciprocally I'll be backing up their data (hence why I have plenty of room on my NAS.
 
I suggest CrashPlan. It is free to use between "friends" (i.e. you and your parents back each other up). That way you can avoid this VHDX malarkey and stick with standard NTFS.
 
Done a few tests on different strip size, different allocation size for the underlying array and then the vhdx itself.

Test A: Host 256KiB Strip 4096 Allocation
Max read: 1868134 @256
Max write: 4072374 @512

Test1: VHDX 4096 Allocation
Max read: 1807420 @512 (1744830 @256)
Max write: 2756549 @512

Test2: VHDX 64k Allocation
Max Read: 1763444@512 (1766211@256)
Max Write: 2857110 @512

Test B: Hose 256KiB Strip 64k Allocation
Max read: 1860443 @512
Max Write: 3654443 @512

Test1: VHDX 4096 Allocation
Max Read: 1785161 @512
Max Write: 2677667 @512

Test2: VHDX 64k Allocation
Max Read: 1767745 @512
Max Write: 2744601 @512

TestC: 64KiB Strip 4096 Allocation
Max Read: 1833217 @256 (1819901 @512)
Max Write: 3231778 @512

Test1: VHDX1 4096 Allocation
Max Read: 1894070 @512
Max Write: 2909152 @512

Test2: VHDX 64k Allocation
Max Read: 1776411 @512
Max Write: 2885681 @512

I've underlined fastest VHDX result - note all ATTO tests were done using default settings. Not that much variation overall.

I did just quickly run the same Test C and Test2 using a queue of 10 to match your test above.

Underlying array:
Max Read: 1897155 @256
Max Write: 3892314 @512

VHDX:

Max Read: 1877475 @256
Max Write: 3882607 @512

So an acceptably small dip in performance from the host disks to the VHDX. Read speed of 1800MB/Sec and Write 3800MB/Sec - I'll take that. Although I'm quite surprised you mangaged to get 1000 out of a single drive!

Need to find the time to robocopy everything back across and change the drive letter over and see how it behaves in the real world.
 
The 1000 is fake -- it is the result of the array controller's 512MB cache and the 10 Queue Depth; essentially ATTO is throwing 10 x IO in parallel, and the small block sizes result in a small overall amount of data that fits in the P410 cache, therefore appearing super fast. Hence why the larger block sizes at the bottom are all fairly stable at around 120MB/s. The higher numbers at smaller block sizes represent what real-world random usage will feel like (optimised and cached by the controller), but copying a bunch of giant MKVs will behave like the big block sizes at the bottom. You have a much gruntier array controller, hence you are seeing much higher speeds at small block sizes.

What are the speeds at the larger block sizes? ATTO isn't the most professional of tools, because to get a true feel for speed, tests need to be much larger so that they overwhelm any cache, giving a true representation of the underlying storage. In other words, don't rely on the high speeds at small block sizes, they aren't even making it to disk (they are, eventually, but masked by the cache).

So... just to finally understand why you are using a VHDX -- this is a VM? All this time I thought it was a physical box that we were talking about.
 
For some reason I assumed direct i/o bypassed any cacheing...

Sorry yes, the vhdx is attached to a server 2012 r2 vm :)

Final%252520FolderRedir.png
 
Beautiful. So as long as you're getting roughly that 1288 figure, you should be golden.

Glad I finally understood your setup! =D
 
Have you got the /MT flag? Makes a MASSIVE difference for lots of small files.
 
Nope - first time I've ever used Robocopy and the guide I've been following made no mention of it - does look like it would have been useful!
 
Just stop it and re-start it with the new /MT flag. It does a differential copy so will skip everything that has been copied already.
 
I had to after it found another file that it couldn't deal with and I couldn't do anything with either - it was definitely flying through the files a lot quicker. All up and running now :)

When my wife gets home from her mother in laws I'll get her to hammer it again to see if it was all worth the effort!
 
You must log in or register to reply here.
Back
Top Bottom