bash script help

MowyTone · 21 Feb 2014 at 12:35

Hi All,

Been trying to create a script that will delete lines from multiple files according to the contents of a template file but I can't seem to get it right.

Code:

file_location="/tmp/*"
template_file=bleh.txt

for file in $(ls -1 ${file_location})
do
     for remove in $(cat ${template_file}
       do
          sed '/'${remove}'/d' ${file}
    done
done

I'm thinking of something like the above, could anyone help?

I think I also have to output the edits to a new file

Code:

sed '/'${remove}'/d' ${file} >${file}_new
mv ${file}_new ${file}

but not sure how t go about this

MowyTone · 21 Feb 2014 at 13:44

ok so having this:

Code:

file_location="/tmp/*"
template_file=bleh.txt

for file in $(ls -1 ${file_location})
do
     for remove in $(cat ${template_file}
       do
          sed '/'${remove}'/d' ${file} >${file}_new
    done
    mv ${file}_new ${file}
done

only saves the last edit to the output file..I can see why but how can it get all the changes for one file saved to the output file?

philldutton · 21 Feb 2014 at 19:42

My bash is a little rusty, but don't you need double > to append to a file?

Code:

sed '/'${remove}'/d' ${file} >>${file}_new

melvis · 21 Feb 2014 at 20:50

think philldutton is right, might want to tee append it instead so you can verify the output during testing at least

Code:

sed '/'${remove}'/d' ${file}  | tee -a ${file}_new

ChrisB · 21 Feb 2014 at 21:23

Try it in place with -i

RobMunfy · 22 Feb 2014 at 09:43

Yep if you want to keep the filename the same, use sed -i. It actually creates a new file then renames it to the old file, but has the desired effect. Outputting to a new file can be done by either philldutton or melvis' answers..

MowyTone · 24 Feb 2014 at 08:07

sed -i is not an option on this box (bash on AIX) and the only reason I am writting it to a new file is because its the only way it seems to work.

Code:

sed '/'${remove}'/d' ${file} >>${file}_new

using above takes the one line out and appends the remainder of the file to ${file}_new. This means you end up with multiple versions of the file in one file.

ChrisB · 24 Feb 2014 at 09:22

Depending on the size and content of template_file, it might be worth trying a sed 'or' - that way you'll avoid the inner loop, and the the problem with duplicate lines, i.e.

sed 's/$'$remove1'\|'$remove2'$/d'

MowyTone · 24 Feb 2014 at 10:11

I've found the answer. If I put the file move within the 2nd loop it will take 1st instance of remove out and move it to the original file. this will then be the file that is used to look for the 2nd instance to delete.

Code:

file_location="/tmp/*"
template_file=bleh.txt

for file in $(ls -1 ${file_location})
do
     for remove in $(cat ${template_file}
       do
          sed '/'${remove}'/d' ${file} >${file}_new
          mv ${file}_new ${file}
    done
done