Nice!
4Gb optics are cheap but make sure you buy ones that are stated working, they are consumables and do break a lot...
So now I'm back to working out which SAN distro to use while I wait for another JBOD controller to arrive so I can build the SAN server![]()
-----------------------------------------------------------------------
CrystalDiskMark 3.0.1 (C) 2007-2010 hiyohiyo
Crystal Dew World : http://crystalmark.info/
-----------------------------------------------------------------------
* MB/s = 1,000,000 byte/s [SATA/300 = 300,000,000 byte/s]
Sequential Read : 82.080 MB/s
Sequential Write : 105.279 MB/s
Random Read 512KB : 28.259 MB/s
Random Write 512KB : 32.671 MB/s
Random Read 4KB (QD=1) : 0.469 MB/s [ 114.5 IOPS]
Random Write 4KB (QD=1) : 0.824 MB/s [ 201.2 IOPS]
Random Read 4KB (QD=32) : 1.325 MB/s [ 323.5 IOPS]
Random Write 4KB (QD=32) : 0.497 MB/s [ 121.2 IOPS]
Test : 1000 MB [C: 36.3% (18.0/49.7 GB)] (x5)
Date : 2013/04/12 21:35:40
OS : Windows NT 6.2 Server Standard Edition (full installation) [6.2 Build 9200] (x64)
-----------------------------------------------------------------------
CrystalDiskMark 3.0.1 (C) 2007-2010 hiyohiyo
Crystal Dew World : http://crystalmark.info/
-----------------------------------------------------------------------
* MB/s = 1,000,000 byte/s [SATA/300 = 300,000,000 byte/s]
Sequential Read : 344.586 MB/s
Sequential Write : 79.734 MB/s
Random Read 512KB : 319.910 MB/s
Random Write 512KB : 89.099 MB/s
Random Read 4KB (QD=1) : 32.256 MB/s [ 7875.0 IOPS]
Random Write 4KB (QD=1) : 21.507 MB/s [ 5250.7 IOPS]
Random Read 4KB (QD=32) : 275.660 MB/s [ 67299.7 IOPS]
Random Write 4KB (QD=32) : 32.078 MB/s [ 7831.5 IOPS]
Test : 1000 MB [F: 1.5% (0.6/40.0 GB)] (x5)
Date : 2013/04/12 21:27:17
OS : Windows NT 6.2 Server Standard Edition (full installation) [6.2 Build 9200] (x64)
-----------------------------------------------------------------------
CrystalDiskMark 3.0.1 x64 (C) 2007-2010 hiyohiyo
Crystal Dew World : http://crystalmark.info/
-----------------------------------------------------------------------
* MB/s = 1,000,000 byte/s [SATA/300 = 300,000,000 byte/s]
Sequential Read : 220.104 MB/s
Sequential Write : 75.649 MB/s
Random Read 512KB : 178.547 MB/s
Random Write 512KB : 39.126 MB/s
Random Read 4KB (QD=1) : 12.100 MB/s [ 2954.1 IOPS]
Random Write 4KB (QD=1) : 10.216 MB/s [ 2494.2 IOPS]
Random Read 4KB (QD=32) : 21.660 MB/s [ 5288.0 IOPS]
Random Write 4KB (QD=32) : 10.481 MB/s [ 2558.8 IOPS]
Test : 1000 MB [F: 1.5% (0.6/40.0 GB)] (x5)
Date : 2013/04/12 21:57:08
OS : Windows 7 Ultimate Edition SP1 [6.1 Build 7601] (x64)
-----------------------------------------------------------------------
CrystalDiskMark 3.0.1 x64 (C) 2007-2010 hiyohiyo
Crystal Dew World : http://crystalmark.info/
-----------------------------------------------------------------------
* MB/s = 1,000,000 byte/s [SATA/300 = 300,000,000 byte/s]
Sequential Read : 224.799 MB/s
Sequential Write : 186.083 MB/s
Random Read 512KB : 184.480 MB/s
Random Write 512KB : 188.470 MB/s
Random Read 4KB (QD=1) : 8.679 MB/s [ 2118.9 IOPS]
Random Write 4KB (QD=1) : 7.547 MB/s [ 1842.4 IOPS]
Random Read 4KB (QD=32) : 95.861 MB/s [ 23403.7 IOPS]
Random Write 4KB (QD=32) : 66.651 MB/s [ 16272.1 IOPS]
Test : 1000 MB [C: 44.5% (22.1/49.7 GB)] (x5)
Date : 2013/04/14 15:41:00
OS : Windows NT 6.2 Server Standard Edition (full installation) [6.2 Build 9200] (x64)
-----------------------------------------------------------------------
CrystalDiskMark 3.0.1 x64 (C) 2007-2010 hiyohiyo
Crystal Dew World : http://crystalmark.info/
-----------------------------------------------------------------------
* MB/s = 1,000,000 byte/s [SATA/300 = 300,000,000 byte/s]
Sequential Read : 230.229 MB/s
Sequential Write : 206.861 MB/s
Random Read 512KB : 184.449 MB/s
Random Write 512KB : 192.574 MB/s
Random Read 4KB (QD=1) : 8.360 MB/s [ 2041.1 IOPS]
Random Write 4KB (QD=1) : 7.189 MB/s [ 1755.0 IOPS]
Random Read 4KB (QD=32) : 96.432 MB/s [ 23542.9 IOPS]
Random Write 4KB (QD=32) : 92.672 MB/s [ 22624.9 IOPS]
Test : 1000 MB [C: 37.8% (37.6/99.7 GB)] (x5)
Date : 2013/04/14 15:43:20
OS : Windows NT 6.2 Enterprise Edition [6.2 Build 9200] (x64)
Am I likely to see an improvement in this area by using a 20GB SSD for ZIL?
Yes I'd have thought it would run faster than that on the SSDs you have.
I'm running into a bit of an issue now - copying a couple of TB of mixed files (some small, some very big) over 10GbE onto a 3TB LUN mounted as RDM on a VM, I'm finding it's causing the other VMs to bog down. A couple have even been reset by HA after not responding. I've been adjusting the disk resource allocations to lower the priority of the file servers I'm working on and raise the priority on a couple of the other VMs, which I think is helping.
Those are good results. But i would have thought it could go higher.
But i would have thought it could go higher. At most to 500mbyte which is 6gbps sata limitation. But considering it is in raid configuration it theoretically should be able to go more than 6gbit to the 10gbit max from the interface. Otherwise it is not realy utilizing the 10gigabit. I would be interested to see a comparison with 4gigabit iscsi or fc using same disks etc.
root@fcsan1:/volumes# zfs get all FCSAN1_Vol1
NAME PROPERTY VALUE SOURCE
FCSAN1_Vol1 type filesystem -
FCSAN1_Vol1 creation Sat Apr 13 12:43 2013 -
FCSAN1_Vol1 used 7.83T -
FCSAN1_Vol1 available 1.05T -
FCSAN1_Vol1 referenced 162K -
FCSAN1_Vol1 compressratio 1.14x -
FCSAN1_Vol1 mounted yes -
FCSAN1_Vol1 quota none default
FCSAN1_Vol1 reservation none default
FCSAN1_Vol1 recordsize 128K default
FCSAN1_Vol1 mountpoint /volumes/FCSAN1_Vol1 local
FCSAN1_Vol1 sharenfs off default
FCSAN1_Vol1 checksum on default
FCSAN1_Vol1 compression on local
FCSAN1_Vol1 atime on default
FCSAN1_Vol1 devices on default
FCSAN1_Vol1 exec on default
FCSAN1_Vol1 setuid on default
FCSAN1_Vol1 readonly off default
FCSAN1_Vol1 zoned off default
FCSAN1_Vol1 snapdir hidden default
FCSAN1_Vol1 aclmode discard default
FCSAN1_Vol1 aclinherit restricted default
FCSAN1_Vol1 canmount on default
FCSAN1_Vol1 xattr on default
FCSAN1_Vol1 copies 1 default
FCSAN1_Vol1 version 5 -
FCSAN1_Vol1 utf8only off -
FCSAN1_Vol1 normalization none -
FCSAN1_Vol1 casesensitivity sensitive -
FCSAN1_Vol1 vscan off default
FCSAN1_Vol1 nbmand off default
FCSAN1_Vol1 sharesmb off default
FCSAN1_Vol1 refquota none default
FCSAN1_Vol1 refreservation none default
FCSAN1_Vol1 primarycache all default
FCSAN1_Vol1 secondarycache all default
FCSAN1_Vol1 usedbysnapshots 0 -
FCSAN1_Vol1 usedbydataset 162K -
FCSAN1_Vol1 usedbychildren 7.83T -
FCSAN1_Vol1 usedbyrefreservation 0 -
FCSAN1_Vol1 logbias latency default
FCSAN1_Vol1 dedup off default
FCSAN1_Vol1 mlslabel none default
FCSAN1_Vol1 sync standard default
-----------------------------------------------------------------------
CrystalDiskMark 3.0.1 x64 (C) 2007-2010 hiyohiyo
Crystal Dew World : http://crystalmark.info/
-----------------------------------------------------------------------
* MB/s = 1,000,000 byte/s [SATA/300 = 300,000,000 byte/s]
Sequential Read : 292.939 MB/s
Sequential Write : 83.412 MB/s
Random Read 512KB : 257.021 MB/s
Random Write 512KB : 100.759 MB/s
Random Read 4KB (QD=1) : 14.634 MB/s [ 3572.7 IOPS]
Random Write 4KB (QD=1) : 11.121 MB/s [ 2715.0 IOPS]
Random Read 4KB (QD=32) : 22.055 MB/s [ 5384.6 IOPS]
Random Write 4KB (QD=32) : 15.636 MB/s [ 3817.4 IOPS]
Test : 1000 MB [C: 38.2% (19.0/49.7 GB)] (x5)
Date : 2013/04/15 18:06:31
OS : Windows NT 6.2 Server Standard Edition (full installation) [6.2 Build 9200] (x64)
FCSAN1_Vol1
mirror-0
c13t0d0
c13t1d0
mirror-1
c13t2d0
c13t3d0
mirror-2
c13t4d0
c13t5d0
-----------------------------------------------------------------------
CrystalDiskMark 3.0.1 x64 (C) 2007-2010 hiyohiyo
Crystal Dew World : [url]http://crystalmark.info/[/url]
-----------------------------------------------------------------------
* MB/s = 1,000,000 byte/s [SATA/300 = 300,000,000 byte/s]
Sequential Read : 1476.637 MB/s
Sequential Write : 1030.035 MB/s
Random Read 512KB : 1172.646 MB/s
Random Write 512KB : 1146.336 MB/s
Random Read 4KB (QD=1) : 57.557 MB/s [ 14051.9 IOPS]
Random Write 4KB (QD=1) : 83.092 MB/s [ 20286.2 IOPS]
Random Read 4KB (QD=32) : 490.447 MB/s [119738.0 IOPS]
Random Write 4KB (QD=32) : 197.807 MB/s [ 48292.8 IOPS]
Test : 1000 MB [G: 0.0% (0.2/892.1 GB)] (x5)
Date : 2013/04/20 18:55:07
OS : Windows NT 6.2 Server Standard Edition (full installation) [6.2 Build 9200] (x64)
-----------------------------------------------------------------------
CrystalDiskMark 3.0.1 x64 (C) 2007-2010 hiyohiyo
Crystal Dew World : http://crystalmark.info/
-----------------------------------------------------------------------
* MB/s = 1,000,000 byte/s [SATA/300 = 300,000,000 byte/s]
Sequential Read : 900.838 MB/s
Sequential Write : 508.524 MB/s
Random Read 512KB : 162.570 MB/s
Random Write 512KB : 146.932 MB/s
Random Read 4KB (QD=1) : 1.499 MB/s [ 366.0 IOPS]
Random Write 4KB (QD=1) : 2.064 MB/s [ 503.9 IOPS]
Random Read 4KB (QD=32) : 8.557 MB/s [ 2089.1 IOPS]
Random Write 4KB (QD=32) : 2.631 MB/s [ 642.2 IOPS]
Test : 1000 MB [F: 0.0% (0.4/7449.9 GB)] (x5)
Date : 2013/04/20 19:04:04
OS : Windows NT 6.2 Server Standard Edition (full installation) [6.2 Build 9200] (x64)
-----------------------------------------------------------------------
CrystalDiskMark 3.0.1 x64 (C) 2007-2010 hiyohiyo
Crystal Dew World : http://crystalmark.info/
-----------------------------------------------------------------------
* MB/s = 1,000,000 byte/s [SATA/300 = 300,000,000 byte/s]
Sequential Read : 294.316 MB/s
Sequential Write : 346.895 MB/s
Random Read 512KB : 286.544 MB/s
Random Write 512KB : 325.697 MB/s
Random Read 4KB (QD=1) : 37.293 MB/s [ 9104.8 IOPS]
Random Write 4KB (QD=1) : 13.491 MB/s [ 3293.8 IOPS]
Random Read 4KB (QD=32) : 210.328 MB/s [ 51349.6 IOPS]
Random Write 4KB (QD=32) : 73.196 MB/s [ 17870.0 IOPS]
Test : 4000 MB [E: 0.0% (0.2/892.1 GB)] (x5)
Date : 2013/04/21 8:43:08
OS : Windows NT 6.2 Server Standard Edition (full installation) [6.2 Build 9200] (x64)