Robocopy the changes only and long path name problem

JamesLucas · 20 Feb 2020 at 11:17

Using RoboCopy to copy files from windows server 2008 R2 to windows server 2012. The script copy files fine but is set to scan the source folder 10 times for files. The source is about 103vG of Files and Folders. we have daily about 1 G changes, The scanning task never seems to compete. it is still scanning a folder for changes!
- Does Robocopy scan each directory for new changes or can it be set to only scan timestamp changed folders?
-How can I copy a source with a long path name over 300 Characters? someone Advised me to start renaming the folders and files with RoboCopy by moving the folders to the root, rename them, then move them back, but I think it will not help with more than 60 paths!

LaurenBlake · 22 Feb 2020 at 09:23

We know that robocopy going to check each and every file and splitting the job into several "smaller" ones will take too much time. Just make sure your target is set to be one level down also, then just schedule them as separate batch files and you'll definitely cut the time down.
As for the path names, are they all in subdirectories or spread out all over the place? if they're all over the place, creating 100+ separate jobs would be counterproductive. If they are within the same tree structure, create a batch that starts further down into the subdirectories, then you can exclude that in the batch file that copies the rest of the structure.
The switch to exclude a folder is /XD dir (so, it would look like /XD \\server\share\sub1\sub2\sub3) which will definitely shorten your path names. Also, you have alternatives to robocopy that can copy updated files easily and solve long path names problem if you don't need to consume much time like securecopy quest and gs richcopy360 ... just check my solutions and feed me back and I will provide you with batch script that allows you to copy modified and new files and folders if you want.

JamesLucas · 23 Feb 2020 at 10:51

Amazing, I think that your comment is logical but it will take much time
I will google securecopy quest and gs richcopy360 to check if any of it will help! , can you provide me with the script plz?

Armageus · 23 Feb 2020 at 11:33

Also worth trying the copy from either end
E.g. pushing the files Vs pulling the files
Can make a difference.

JamesLucas · 24 Feb 2020 at 12:57

Armageus , How can I apply your idea?
LaurenBlake, can you provide me with the script, please?
Notice: I did a full search, there is a big difference between the two programs, gs richcopy360 is affordable and securecopy quest is very expensive ! and they almost do the same job! Am I miss something?

Armageus · 24 Feb 2020 at 13:25

Pushing rather than Pulling is just a case of running the batch file on either the Source server (Pushing), or the Destination server (Pulling)

LaurenBlake · 25 Feb 2020 at 13:56

check this script , it copies the new files and changed files

Code:

@echo off
:: Incremental_Backup.bat Created by HS
:: It is a total copy first and then incrementally,
:: ie, it just copies the new files and changed files.
mode con cols=95 lines=5 & color 0E
Title %~nx0 for Incremental Backup with XCopy Command by Hackoo 2017
set "Settings=%~dp0Settings.ini"
Set "FirstFull_CopyLog=%~dp0%~n0_FirstFull_CopyLog.txt"
Set "LogFile=%~dp0%~n0_Incremental_CopyLog.txt"
If not exist "%Settings%" (
    Call :BrowseForFolder "Please choose the source folder for the backup" SourceFolder
    Setlocal EnableDelayedExpansion
    If defined SourceFolder (
        echo(
        echo             You chose "!SourceFolder!" as source folder
    ) else (
        echo(
        Color 0C & echo                    The source folder is not defined ... Exiting ......
        Timeout /T 2 /nobreak>nul & exit
    )
    Call :BrowseForFolder "Please choose the target folder for the backup" TargetFolder
    If defined TargetFolder (
        echo(
        echo             You chose "!TargetFolder!" as Target folder
    ) else (
        echo(
        Color 0C & echo                    The Target folder is not defined ... Exiting ......
        Timeout /T 2 /nobreak>nul & exit
    )
Timeout /T 3 /nobreak>nul
    (
        echo "!SourceFolder!"
        echo "!TargetFolder!\Backups_%ComputerName%\"
    )
cls & echo( & echo(
echo         Please wait a while ... The Backup to "!TargetFolder!" is in progress...
Call :Backup_XCopy "!SourceFolder!" "!TargetFolder!" "!FirstFull_CopyLog!"
Timeout /T 1 /nobreak>nul
Start "" "!FirstFull_CopyLog!" & exit
) else (
Setlocal EnableDelayedExpansion
for /f "delims=" %%a in ('Type "%Settings%"') do (
    set /a idx+=1
    set Param[!idx!]=%%a
)

Set "SourceFolder=!Param[1]!"
Set "TargetFolder=!Param[2]!"
echo(
echo        The source Folder from Settings.ini is : !SourceFolder!
echo        The Target Folder from Settings.ini is : !TargetFolder!
Timeout /T 1 /nobreak>nul & cls & echo( & echo(
echo       Please wait a while ... The Backup to !TargetFolder! is in progress...
Call :Backup_XCopy !SourceFolder! !TargetFolder! !LogFile!
)
Timeout /T 1 /nobreak>nul
Start "" !LogFile! & exit
::****************************************************************************
:BrowseForFolder
set "psCommand="(new-object -COM 'Shell.Application')^
.BrowseForFolder(0,'%1',0,0).self.path""
for /f "usebackq delims=" %%I in (`powershell %psCommand%`) do set "%2=%%I"
exit /b
::****************************************************************************
:Backup_XCopy <Source> <Target> <LogFile>
Xcopy /c  /d  /e  /s  /i  /y %1 %2 > %3 2>&1
Exit /b
::****************************************************************************

and you are right gs richcopy360 is affordable and securecopy quest is very expensive but it is depending on what you need .

Scort · 27 Feb 2020 at 14:36

I use these switches, to copy only changed files:

/e /xo /copyall /FFT /XA:H /R:1 /W:1 /V /TEE

JamesLucas · 2 Mar 2020 at 11:49

LaurenBlake said:

check this script , it copies the new files and changed files

Code:

@echo off
:: Incremental_Backup.bat Created by HS
:: It is a total copy first and then incrementally,
:: ie, it just copies the new files and changed files.
mode con cols=95 lines=5 & color 0E
Title %~nx0 for Incremental Backup with XCopy Command by Hackoo 2017
set "Settings=%~dp0Settings.ini"
Set "FirstFull_CopyLog=%~dp0%~n0_FirstFull_CopyLog.txt"
Set "LogFile=%~dp0%~n0_Incremental_CopyLog.txt"
If not exist "%Settings%" (
    Call :BrowseForFolder "Please choose the source folder for the backup" SourceFolder
    Setlocal EnableDelayedExpansion
    If defined SourceFolder (
        echo(
        echo             You chose "!SourceFolder!" as source folder
    ) else (
        echo(
        Color 0C & echo                    The source folder is not defined ... Exiting ......
        Timeout /T 2 /nobreak>nul & exit
    )
    Call :BrowseForFolder "Please choose the target folder for the backup" TargetFolder
    If defined TargetFolder (
        echo(
        echo             You chose "!TargetFolder!" as Target folder
    ) else (
        echo(
        Color 0C & echo                    The Target folder is not defined ... Exiting ......
        Timeout /T 2 /nobreak>nul & exit
    )
Timeout /T 3 /nobreak>nul
    (
        echo "!SourceFolder!"
        echo "!TargetFolder!\Backups_%ComputerName%\"
    )
cls & echo( & echo(
echo         Please wait a while ... The Backup to "!TargetFolder!" is in progress...
Call :Backup_XCopy "!SourceFolder!" "!TargetFolder!" "!FirstFull_CopyLog!"
Timeout /T 1 /nobreak>nul
Start "" "!FirstFull_CopyLog!" & exit
) else (
Setlocal EnableDelayedExpansion
for /f "delims=" %%a in ('Type "%Settings%"') do (
    set /a idx+=1
    set Param[!idx!]=%%a
)

Set "SourceFolder=!Param[1]!"
Set "TargetFolder=!Param[2]!"
echo(
echo        The source Folder from Settings.ini is : !SourceFolder!
echo        The Target Folder from Settings.ini is : !TargetFolder!
Timeout /T 1 /nobreak>nul & cls & echo( & echo(
echo       Please wait a while ... The Backup to !TargetFolder! is in progress...
Call :Backup_XCopy !SourceFolder! !TargetFolder! !LogFile!
)
Timeout /T 1 /nobreak>nul
Start "" !LogFile! & exit
::****************************************************************************
:BrowseForFolder
set "psCommand="(new-object -COM 'Shell.Application')^
.BrowseForFolder(0,'%1',0,0).self.path""
for /f "usebackq delims=" %%I in (`powershell %psCommand%`) do set "%2=%%I"
exit /b
::****************************************************************************
:Backup_XCopy <Source> <Target> <LogFile>
Xcopy /c  /d  /e  /s  /i  /y %1 %2 > %3 2>&1
Exit /b
::****************************************************************************

and you are right gs richcopy360 is affordable and securecopy quest is very expensive but it is depending on what you need .

I tried this script and it works but now I'm using gs richcopy360 for copying the changes and for long path names, I am happy that my problems solved.

andyinv · 26 Mar 2021 at 13:22

If your pathname is too long, consider using SUBST to map a drive down to a lower level and start your copy from there.

eg

c:\folder1\folder2\folder3\folder4\folder5
Subst L: c:\folder1\folder2\folder3\folder4

Robocopying from L: would take everything starting at that level, and down, but you'd be in essence masking the leading additions to the path name.

JamesLucas · 1 Dec 2021 at 09:50

Hi andyinv , sorry for replying late, I was stuck in some projects. Starting the copy from the lower level would be a good idea but it will not be workable for over 60 paths, No worries now, Gs Richcopy30 did the job and still works fine for me