Xargs

xargs is a very handy command line utility that allows you to perform operations on a list. For example, you can take a list of files and tell xargs to run the "rm" command on the list to remove each file on the list. Or you can use xargs to unzip a list of files. Think of it as a quick-and-dirty way to do a bash loop like

for i in `cat list_of_files`; do
command $i
done

It has the additional advantage that it performs the command once for each argument in the list. To understand this as an advantage, see wikipedia:xargs - the example they give is for the "rm" command, which cannot take extremely long lists of files. To circumvent this, xargs will run the "rm" command once for each file in the list.

Usage

Basic Usage

xargs is almost always called to receive information from a pipe.

One of the most convenient uses of xargs is to grep through files, or to remove lists of files.

For example, to search through C++ code files for the word "turbulence," you could use xargs as follows:

$ find . -name "*.cc" | xargs grep "turbulence"

./Arches.cc:  //db->require("turbulence_model", turbModel);
./BoundaryCondition.cc:    // --- new turbulence inlet flow generator --- 
./BoundaryCondition.cc:      turb_db->get("turbulence_intensity",intensity);
./CompDynamicProcedure.cc:// Schedule recomputation of the turbulence sub model 
./CompDynamicProcedure.cc:// Schedule recomputation of the turbulence sub model 
./CompLocalDynamicProcedure.cc:// Schedule recomputation of the turbulence sub model 
./CompLocalDynamicProcedure.cc:// Schedule recomputation of the turbulence sub model 
./EnthalpySolver.cc:    // if it is set in turbulence model
./IncDynamicProcedure.cc:// Schedule recomputation of the turbulence sub model 
./IncDynamicProcedure.cc:// Schedule recomputation of the turbulence sub model 
./OdtClosure.cc:// Schedule recomputation of the turbulence sub model 
./ReactiveScalarSolver.cc:    // if it is set in turbulence model
./ScalarSolver.cc:    // if it is set in turbulence model
./ScalarSolver.cc:    // we only need to set mixture fraction Pr number in turbulence model
./ScaleSimilarityModel.cc:// Schedule recomputation of the turbulence sub model 
./SmagorinskyModel.cc:// Schedule recomputation of the turbulence sub model 
./SmagorinskyModel.cc:// Schedule recomputation of the turbulence sub model 
./SmagorinskyModel.cc:// Schedule recomputation of the turbulence sub model

Or, to remove a set of files contained in a list contained in a file, you could use:

$ cat list_file | xargs rm

REMEMBER: before running "xargs rm" with ANYTHING, you should run the plain command by itself first, or run "xargs cat", to print the names of the files that will be removed. This prevents you from shooting yourself in the foot:

for i in $EXTREMITIES; do
  rm $i;
done

Double-Checking Xargs Commands

The -p (lowercase p) and -t flags can be very useful in making sure that you are executing the commands correctly.

Using the -p flag causes xargs to print out each command that it will execute and ask for confirmation from the user.

Using the -t flag causes xargs to print out each command that it will execute to stderr before it executes the command.

Advanced Usage

Dealing with Newline Characters

One of the problems with combining find with xargs is if you've got spaces or newlines or weird characters in your list of files. In this case, you can use some arguments for find and xargs to prevent you from screwing stuff up:

$ find . -name "*.cc" -print0 | xargs -0 grep "something"

The -print0 argument for find is explained in the man page for find (man find):

     -print0
             This primary always evaluates to true.  It prints the pathname of the current file to standard output, 
             followed by an ASCII NUL character (character code 0).

Likewise, the -0 argument for xargs is explained (in man xargs):

     -0      Change xargs to expect NUL (``\0'') characters as separators, instead of spaces and newlines.  This is 
             expected to be used in concert with the -print0 function in find(1).

Manipulating the Argument List

Sometimes you don't just want to feed the argument list, one at a time, to a command. You might want to insert the name of the file somewhere as a part of the command - "mv" and "cp" are examples of this. For example, you don't say "cp file" (which is what would happen if you pipe the names of files to "xargs cp"), you say "cp file /destination/dir/".

You can insert the name of the file by using the -I flag. From the xargs manpage (man xargs):

     -I replstr
             Execute utility for each input line, replacing one or more occurrences of replstr in up to 
             replacements (or 5 if no -R flag is specified) arguments to utility with the entire line of 
             input.  The resulting arguments, after replacement is done, will not be allowed to grow beyond 
             255 bytes; this is implemented by concatenating as much of the argument containing replstr as 
             possible, to the constructed arguments to utility, up to 255 bytes.  The 255 byte limit does 
             not apply to arguments to utility which do not contain replstr, and furthermore, no replacement 
             will be done on utility itself. 
             Implies -x.

Xargs Jujitsu

SVN

I use xargs in pipes to do some handy stuff. For example, if I am in a directory that is part of an SVN repository, I can run the command "svn status" to tell me about files that have been changed, etc.:

$ svn status .
M      Core/Grid/Patch.h
M      Core/Grid/Level.cc
M      Core/Grid/Level.h
M      Core/Grid/sub.mk
?      Core/Grid/AMR_CoarsenRefine.cc
?      Core/Grid/LevelP.h
?      Core/Grid/AMR_CoarsenRefine.h
M      Core/Math/Matrix3.cc
M      Core/Math/Matrix3.h
M      Core/Parallel/Parallel.cc
M      Core/Parallel/Parallel.h
M      Core/Containers/Handle.h
M      Core/Containers/RunLengthEncoder.h

Those "?"s mean the files have not been added to the repository yet. In this case, there's only 3, so it would be straightforward to copy-and-paste the file names into an "svn add" command to add the files to the repository. But the smart solution is to make the computer work for you!

First, use grep to reduce the list only to files that haven't been added to the repository yet:

$ svn status . | grep "?"
?      Core/Grid/AMR_CoarsenRefine.cc
?      Core/Grid/LevelP.h
?      Core/Grid/AMR_CoarsenRefine.h

Then use Awk to strip the leading "?":

$ svn status . | grep "?" | awk -F" " '{print $2}'
Core/Grid/AMR_CoarsenRefine.cc
Core/Grid/LevelP.h
Core/Grid/AMR_CoarsenRefine.h

And finally, use xargs to add each file, one-at-a-time, to the repository:

$ svn status . | grep "?" | awk -F" " '{print $2}' | xargs svn add
A      Core/Grid/AMR_CoarsenRefine.cc
A      Core/Grid/LevelP.h
A      Core/Grid/AMR_CoarsenRefine.h

Or, if we wanted to delete them (this would be the case if we wanted to "clean out" and revert a directory to the repository version), we could replace "svn add" with "rm":

$ svn status . | grep "?" | awk -F" " '{print $2}' | xargs rm

REMEMBER: before you run the "rm" command on anything, leave off the "xargs" portion, and make sure that you're slicing-and-dicing correctly (doing everything up until the awk command will just print the names of the files on which "rm" will operate).