Impressive de-dupe Server 2012 R2
- Thread starter nutcase
- Start date
Soldato
- Joined
- 11 Oct 2008
- Posts
- 3,833
- Location
- London
DVDs and Photos are some of the least 'dedupable' files going.
I'm guessing you have those files duplicated somewhere else on that volume.
I'm guessing you have those files duplicated somewhere else on that volume.
I keep adding DVDs and photos to my storage server, and the space barely changed.
Quite shocked to be honest:
Could've bought smaller, faster drives!
No, you need the large drive to store the original files until the post processing is complete to dedupe the files. The problem you have is you have looked at the folder size, which indeed will be small but you need to look at the volume usage to see what has been saved.
I wrote a guide for enabling dedupe on windows 8 you can see it here (http://forums.overclockers.co.uk/showthread.php?p=24068473). There's the powershell in there to show the actual usage, as you'll see that the system information folder (hidden system folder) on the volume has the actual files use to make up that deduped folder.
Code:
get-dedupstatusYou also need to make sure that you run garbage collection, to save space from deleted files. You can set the age of the file before it will be considered ready for dedupe, I believe this is to prevent overheads of files changing during initial creation before they end up as achieved.
Soldato
- Joined
- 11 Oct 2008
- Posts
- 3,833
- Location
- London
It's mostly to prevent Windows stressing itself out about deduping transient files. If you're reading in a bunch of temp files only to delete them shortly after, eg, uncompressing archives, it's a waste of time and resources for Windows to even look at them for deduping. Think about what would happen on a file server with hundreds of users and DFSr running.
I knew there was a few duplicated photos but just had a bit of a surprise - things like our high-res wedding photos were on there 3 times!
The vast bulk definitely isn't duplicated though, including all the DVDs.
The de-dupe is set to 7 days which I think is the standard? I'm adding stuff slowly so it gets time to reach the threshold and get processed.
Had a closer look though and yes that was a bit optimistic - it's actually a 43% depude rate (833GB saved) - so still not bad lol
The vast bulk definitely isn't duplicated though, including all the DVDs.
The de-dupe is set to 7 days which I think is the standard? I'm adding stuff slowly so it gets time to reach the threshold and get processed.
Had a closer look though and yes that was a bit optimistic - it's actually a 43% depude rate (833GB saved) - so still not bad lol