Bit-level file synchronisation

mr.bond

mr.bond

mr.bond

Associate
Joined
16 Jan 2006
Posts
672
Location
Surrey
Morning all,

Scenario:

Multiple sites, a datacentre and remote users.

The remote users home directories (redirected via Group Policy) are all held on a file server in the datacentre, this is where the Firewall livess where they VPN in for remote access. All use the standard Microsoft offline files to synchronise their data.

When the remote users visit a site they synchronise their files back to the server in the datacentre, this places a significant load on the respectives sites WAN (especially if there are multiple people on the same site doing the same thing).

We're about to implement COS and QOS which should alleviate some of this, but I'm wondering how the rest of you tackle this issue at a technical level. Are you aware of any software which will allow me to synchronise these home directories across multiple sites file servers and update changes at bit level - incrementally. I can force syncs easily using standard file copies and implement DFS but the issue is the amunt of files, any changes suncs the full file, I'm intrested in bit level syncing software that could enable me to accomplish this task.

Any ideas?
 
I worked for a arm of a french bank and they had the same issues and eventually opted for Citrix farm in the HQ (the datacenter in your example) and then home workers VPN'd then ICA'd in, and the remote sites ICA'd over the WANs. Not a great solution.
I seems like a fairly common setup these days so a good solution should be available.
 
sounds like you are looking for a byte caching product, take a look at what bluecoat have to offer, i think riverbed have something in this space too.
 
You can use rsync which copies only the deltas (changes). E.g if you have a 700mb and change 1 bit of it, rsync will computer the checksums of all the tiny chunks and send only the one with the changed bit. There is a windows version called deltacopy which uses cygwin. The only problem for you may be that the file permissions will be lost.
 
It's been some time since i looked at this kind of thing from a technical level, but the last time we talked about this internally we were looking down the route of Cisco WAFS (Wide Area File Services) which i think is being replaced now by WAAS (Waide Area Application Services) which is all about branch office consolidation and WAN optimisation, as soon as i read your post i immediately thought of that. Not software based as you asked, but possibly an option if you have the funding for it?
 
As pistol has pointed out remote desktops are infinitely more manageable but can be expensive to provision. If you work for a small company, a really cheeky trick is to just enable concurrent remote desktop connections on a decent workstation with the termsrv.dll hack and have your remote users log into it via vpn. Under the hood is is basically the same thing as Windows 2003 terminal server.
 
Does Windows sync really just copy the full updated file to the server?
So, if I inadvertently edit a CD image I'd be looking at a world of pain in this scenario?
If so, rsync might work but it requires you to alter the clients. Hence not really an answer to your question.
 
Last edited:
kefkef said:
It's been some time since i looked at this kind of thing from a technical level, but the last time we talked about this internally we were looking down the route of Cisco WAFS (Wide Area File Services) which i think is being replaced now by WAAS (Waide Area Application Services) which is all about branch office consolidation and WAN optimisation, as soon as i read your post i immediately thought of that. Not software based as you asked, but possibly an option if you have the funding for it?
Click to expand...

It's complicated to deploy but once in place can give huge benefits, specially in the situation outlined above.
 
You must log in or register to reply here.
Back
Top Bottom