Preventing IO Limits on Hard Drives

Admetos · 13 May 2015 at 04:58

Ok, good stuff.

Go into Administrative Tools -> Performance Monitor

Then 'Add Counters' from right clicking on the graph area.

Then look for 'ReadyBoost Cache'

You want to add 'Bytes cached' and 'Cache reads/sec'.

The Bytes cached will increase over time, and Cache reads shows were reads are done by the USB stick that otherwise would be done by the HDD.

For best effect you need a USB stick with high random access. You can also add multiple USB sticks to use as Readyboost and they automatically work together for more performance.

kayomani · 13 May 2015 at 06:00

Those seagate archive drives could become quite slow if you are performing writes in addition to any reads due to their shingled design.

xchaotic · 13 May 2015 at 06:44

Have you considered putting these drives in some sort of array, like Windows Storage spaces? For example a dual-parity setup means that the reads will be split between two drives and writes will be just as fast as a single drive.
Adding more RAM can help and then making sure processes use advantage of that.
For example, if you're burning to Blu-Ray, a 64GB RAM setup could mean your whole image fits in memory.
The opposite is also true - if you're downloading an image, it can be kept in-memory and only flushed when slower storage tier is ready.
Having said all that, I am still surprised that you're hitting that problem - I've had similar scenarios on a laptop - downloading and burning a BluRay simultaneously and it wasn't a problem.
Your download setup must be doing a lot of random write, so reconfigure the program, use another one, check the cable, check disk SMART or get a small cache SSD so your random reads/writes are not hitting spinny platters.
Make sure that the HDDs are on different SATA/SAS channels.
Go to BIOS and verify that the disks are operating in AHCI mode, use SMART again to check that the drives use NCQ.
Consider attaching one or more of the drives to the motherboard built-in port(s)
Finally consider reinstalling Windows, and defragmenting the HDDs, if using built-in controller, install Intel RST and storage drivers.
Use benchmarking tools such as crystal disk mark to verify that the drives are operating at speeds you are happy with.

lude1962 · 13 May 2015 at 09:57

Actually its sounding more like a windows problem (or just the way windows works)combined with your setup
what cpu etc, windows version and how many drives /burners etc do you have connected and how?

Have you tried setting performance to background services as opposed to programs?

David Bisset · 13 May 2015 at 09:57

holygamer said:
... Don't think fragmentation is an issue as the drives just have large videos on them so they don't really get fragmented...

If you're torrenting then fragmentation is pretty likely so worth considering a defrag - as they come in small parts and normally people torrent several things at once. No idea how your specific torrent software works, it could handle this elegantly but I'd guess it's unlikely.

joeyjojo · 13 May 2015 at 10:15

David Bisset said:
If you're torrenting then fragmentation is pretty likely so worth considering a defrag - as they come in small parts and normally people torrent several things at once. No idea how your specific torrent software works, it could handle this elegantly but I'd guess it's unlikely.

This is a good point, see if you can enable preallocating files in your torrent prog OP.

As for "waiting for hard disk to reach minimum activity level", if you're certain that dedicated different drives for different tasks isn't going to help then the only thing to do is to buy more drives and set them up in an array to increase the bandwidth. A (preferably hardware, might involve buying a card) RAID 5 array (min. 3 drives, the more the better) would give you n times the read performance and n-1 times the write performance, with tolerance for losing 1 disk.

http://en.wikipedia.org/wiki/Standard_RAID_levels

holygamer · 13 May 2015 at 10:26

I've got a cable connected to my power supply. Other end is 2 x molex 4 pin to sata power. Then I have 1 Silverstone Sata Power to 4 Sata power Silverstone (has capacitor on it) connected to that. I have 4 3.5" 8TB hard drives connected to that. If I remove one of the drives and connect the SSD C Drive, the drive works.

On the other molex I have the same thing - another 4 drives

I've also got another cable connected to my power supply. Other end is 2 x molex 4 pin to sata power. I have a a 4 x Sata power cable (no capacitor on the cable) connected to that. Two Bluray writer drives are connected to that cable. Two SSDs are connected to that cable. One SSD is a data drive. All drives work apart from the C Drive. Bizarrely if I disconnect the Bluray drives and data drive and only connect the SSD C drive to that cable, it STILL doesn't work! But it does work if I replace that cable with a Silverstone one with a capacitor in it!

Is that normal? A while ago I bent some of the Sata power pins by forcing a cable onto the C Drive. I bent them down back in place so I can still connect a sata power cable to it but there's still not completely flat. Could that be what's causing the issue?

joeyjojo said:
This is a good point, see if you can enable preallocating files in your torrent prog OP.

As for "waiting for hard disk to reach minimum activity level", if you're certain that dedicated different drives for different tasks isn't going to help then the only thing to do is to buy more drives and set them up in an array to increase the bandwidth. A (preferably hardware, might involve buying a card) RAID 5 array (min. 3 drives, the more the better) would give you n times the read performance and n-1 times the write performance, with tolerance for losing 1 disk.

http://en.wikipedia.org/wiki/Standard_RAID_levels

That sounds interesting. I don't really understand anything about RAID and how doing what you said would improve performance. Is there a guide somewhere?

joeyjojo · 13 May 2015 at 10:35

holygamer said:
That sounds interesting. I don't really understand anything about RAID and how doing what you said would improve performance. Is there a guide somewhere?

The idea is that all files are split up over multiple drives, so the load is balanced across several drives instead of just 1.

You'll need to do your research though. In particular be aware that building/destroying an array involves losing all data on them. Also, if 1 disk happens to die it must be replaced ASAP, since losing another disk before rebuilding the array means goodbye data.

Your motherboard might already be able to do RAID 5, but be aware it'll be a "software" RAID (so will steal some CPU time IIRC), and will probably not survive should the mobo die or need replacing. This is why people buy (expensive) hardware RAID cards.

P.S. your power wiring sounds like a nightmare, get that tidied up! I'm a little concerned by "1 Silverstone Sata Power to 4 Sata power" connectors - might be starving drives for current. You should use a PSU with enough connectors without resorting to extensions/splitters!

lude1962 · 13 May 2015 at 10:50

'I've got a cable connected to my power supply. Other end is 2 x molex 4 pin to sata power. Then I have 1 Silverstone Sata Power to 4 Sata power Silverstone (has capacitor on it) connected to that. I have 4 3.5" 8TB hard drives connected to that. If I remove one of the drives and connect the SSD C Drive, the drive works.

On the other molex I have the same thing - another 4 drives'

8 drives off one cable !!!!!!!!!!

Admetos · 13 May 2015 at 19:31

holygamer said:
OK I've done that. So where do I find the stats to show it's working?

Did you try this?

AtaRo · 13 May 2015 at 21:30

You can try increasing your torrent program cache size to reduce disk access/trashing.

holygamer · 14 May 2015 at 19:34

lude1962 said:
'I've got a cable connected to my power supply. Other end is 2 x molex 4 pin to sata power. Then I have 1 Silverstone Sata Power to 4 Sata power Silverstone (has capacitor on it) connected to that. I have 4 3.5" 8TB hard drives connected to that. If I remove one of the drives and connect the SSD C Drive, the drive works.

On the other molex I have the same thing - another 4 drives'

8 drives off one cable !!!!!!!!!!

Do you have some sort of proof that says you can't run 8 drives off a cable? Because it works perfectly fine. It's just that when I try and connect my C Drive it doesn't work.

lude1962 · 14 May 2015 at 21:52

'Because it works perfectly fine.'
'It's just that when I try and connect my C Drive it doesn't work.'

Do you have some sort of proof that says you can't run 8 drives off a cable?

Post your entire system specs inc psu.

joeyjojo · 15 May 2015 at 07:48

holygamer said:
Do you have some sort of proof that says you can't run 8 drives off a cable? Because it works perfectly fine. It's just that when I try and connect my C Drive it doesn't work.

You're probably hitting some max rated currents, especially at spin up, depending on how they're daisy-chained. A molex connector is rated to 11 A/pin, and HDDs typically will consume 2 A max at spin up, so 8 drives through one connector is well above the rated max.

4 drives off one Molex is probably just about OK, but it's pushing it.

SATA is lower, only 1.5 A/pin over 3 pins = 4.5 A/connector. The drives connected to the 1->4 SATA power connectors are definitely running way over spec.

If at all possible you should use a PSU with enough SATA power connectors to plug every drive in without using splitters or adaptors. If you must use splitters, try to connect as few drives as possible to each PSU cable connector.

Edit: I looked up the specs for your drive (didn't have 4 TB but 3 TB is there). Peak current is 1.78 A, close to the 2 A I guessed. Also read/write they average 6 W, or 1.2 A @ 5 V. So the 4 drives connected to the single SATA connector are probably running at 160% of rating.

http://wdc.custhelp.com/app/answers/detail/search/1/a_id/1679#