Posts Tagged ‘find’

Find files before/after/in between specific dates

February 14, 2011

Hello folks. \\//,

Well here I am, reading a technical research paper on membrane computing and Petri nets when I decided to download some of the reference materials, conference/journal technical papers, of the paper I’m currently reading.  So I download the reference materials from IEEE Xplore, plus other non-reference materials but still related to the topic I’m reading.

Eventually, I download around 10 PDF files, with filenames made up of purely integers and of course the .pdf extension. Problem is, I’m too lazy (not that much really, I just don’t want to keep doing this manually over and over) to specify each PDF filename, then add them to a compressed archive (this case a RAR file) and then send them to another machine or person or account of mine for archiving.  Essentially, I just want that: to get those PDF files I recently downloaded for this specific topic I’m currently in, not include other PDF files in the same directory, and then compress them into a single RAR file.

How to go over doing this?

Luckily there are for loops and the nifty find command in *nixes like my Ubuntu box. 🙂 So what I simply do is

for i in `find . -type f -cnewer startdate ! -cnewer enddate`; do rar a myrarfile.rar $i; done

which means given 2 reference points, files startdate and enddate, I loop over all files in between the modification times of these two files and then add them to the RAR file myrarfile.rar.

Presto. Problem solved, learned something new. Now, back to reading research papers. 🙂



Doing cool things in Bash and in Linux

February 29, 2008

Finally, I’m able to cut-in one last post for this month. I’ve some little free time and the things I’ve read and learned again inspired me to write another post.

This post is concerned with the cool things you can do with the bourne again shell or bash for short. If you want more info afterwards on other cool Linux and bash commands, you can consult my earlier post here. This post would probably mean it’s an extension of that previous post of mine. Difference is that bash can be installed not only in Linux but other Unix or *nix related operating systems such as Mac OS or solaris, and of course, Unix itself.

Anyhow to start of, I recently re-learned the wonderful use of brace expansion. Brace expansion is performed by issuing a command whose arguments are strings (alphanumeric for example) enclosed in curly braces and separated only by commas such as:

$ echo {s,sd,sdf}

Which outputs the following

s sd sdf

Note that there musn’t be any spaces inside the braces. But then that alone doesn’t seem to be anything marvelous right? So we extend that further to make it a bit more interesting such as these:

$ echo {red,yellow,blue,black,pink}_mask
red_mask yellow_mask blue_mask black_mask pink_mask

$ echo {"red ","yellow ","blue ","black ","pink "}mask
red mask yellow mask blue mask black mask pink mask

$ echo {red,yellow,blue,black,pink}" mask"
red mask yellow mask blue mask black mask pink mask

Or even nest braces like so:

$ echo {{yellow,red}_color,blue,green} yellow_color red_color blue green

At this part you might be saying to yourself the big “So what???” What use would we (Linux/Bash users, administrators, shell coders etc) have for brace expansion? A lot actually. One example is if I want to rename my files (for backup for example). Instead of typing a lot of file names (even with the help of tab completion, that is a lot of work, especially if you’re working with files from another directory). If say I have a configuration file at /home/f/F/RoR/sample_config_file.conf and I want to create a back up copy of the file on the same directory, I just have to do this

$ cp -v /home/f/F/RoR/sample_config_file.conf{,.bak} `/home/f/F/RoR/sample_config_file.conf' -> `/home/f/F/RoR/sample_config_file.conf.bak'

Which is equivalent to doing this

$ cp -v /home/f/F/RoR/sample_config_file.conf /home/f/F/RoR/sample_config_file.conf.bak `/home/f/F/RoR/sample_config_file.conf' -> `/home/f/F/RoR/sample_config_file.conf.bak'

So instead of executing the last command, the command using brace expressions is much shorter (and looks better to).

Next is command substitution which is very helpful at assigning values to variables and especially at shell scripting. Substitution is made possible by enclosing strings inside a $( ) combination. An example is the following

$ uname -a Linux foxhound3 2.6.22-14-generic #1 SMP Tue Feb 12 07:42:25 UTC 2008 i686 GNU/Linux

Using substitution I can do it this way

$ info=$(uname -a) $ echo $info Linux foxhound3 2.6.22-14-generic #1 SMP Tue Feb 12 07:42:25 UTC 2008 i686 GNU/Linux

Which might seem longer but you have to realize that a lot more can be done with this technique such as the next one. Next I issue a two commands: the output of the first (which is actually two commands joined together) becomes the input of the second (outer command) like so:

$ ls -lh $(find . |grep txt)

The inner commands in the previous command lists all files with the string txt in their file names. The outer command ls -lh shows the time stamp (creation/modification date), file owner, file size etc of the files containing the string txt in their file names.

Output redirection of standard error is next. When you execute a command which you know will produce a lot of errors such as using find to look for files on the topmost / (slash root) directory knowing full well that you don’t have read access to many directories, such as this one (which assumes you’re looking for the file php.ini):

$ find / -name php.ini

find: /etc/cups/ssl: Permission denied
find: /etc/ssl/private: Permission denied
find: /tmp/gconfd-root: Permission denied
find: /tmp/orbit-root: Permission denied
find: /var/cache/system-tools-backends/backup: Permission denied
find: /var/tmp/kdecache-guest: Permission denied
find: /var/tmp/kdecache-ma3x: Permission denied
find: /var/tmp/kdecache-root: Permission denied
find: /var/spool/postfix/flush: Permission denied
find: /var/spool/postfix/deferred: Permission denied
find: /var/spool/postfix/defer: Permission denied
find: /var/spool/postfix/active: Permission denied
find: /var/spool/postfix/trace: Permission denied
find: /var/spool/postfix/hold: Permission denied
find: /var/spool/postfix/private: Permission denied
find: /var/spool/postfix/saved: Permission denied
find: /var/spool/postfix/maildrop: Permission denied
find: /var/spool/postfix/corrupt: Permission denied
find: /var/spool/postfix/bounce: Permission denied

<other output truncated>

You get the idea. What you can do is to send them to /dev/null if you’re not interested in the error messages like so

$ find / -name php.ini 2> /dev/null /etc/php5/cli/php.ini /etc/php5/apache2/php.ini /etc/php5/cgi/php.ini

Which produces a cleaner output. Or if you want to view error messages later you can issue a

$ find / -name php.ini 2> error_messagest.txt

And then view the text file later for error messages. If you want to redirect both the error and standard output messages to the same file you can do a

$find / -name php.ini >output.txt 2>&1

The important thing to note here is that the combination of the two messages (error and standard) is done at the end of the command that generates the output (in this case find).

When it comes to searching through the history of bash commands executed, the usual way would be to scroll up or down using the arrow keys or enter the command history, view the history number of that long line of commands you entered, then do a !<history number of command here> to execute the command again without typing it. For example


<output truncated>

531 cat output.txt 532 find / -name php.ini >output.txt 2>&1 533 history

then to execute command number 531


which is equivalent to executing

$cat output.txt

again. But what if your history of commands is already in the hundreds or above a thousand? What you can do is press ctrl and r at the same time which gives you this

$ (reverse-i-search)`':

A command history search. You can start typing the first letter of your previous command and then bash searches for the most recent and closest command you entered.

The last trick is looping. A lot of people think that looping is only good for writing programs, but one can actually put it to good use even in system administration, or plain ‘housekeeping’ of your files etc. For example, I have the files a.txt, b.txt, and c.txt in a directory. In order to make backup copies of them instantly, I do a

$ for something in *; do cp -v $something $something.bak; done

Which gives me the output

`a.txt' -> `a.txt.bak' `b.txt' -> `b.txt.bak' `c.txt' -> `c.txt.bak'

And voila! Instant backup of my files. One story I’ve read is that a system administrator’s machine lost so much memory that even basic commands like ls failed because of insufficient memory. But the administrators know that a certain file was the one wreaking havoc on the machine. So what they apparently did was this

$ for var in *; do echo $var;done

Which actually displays the files in the current directory, and basically solved the problem (replacing ls for the meantime). This is because those commands including echo are part of bash and are already loaded in memory. “Wonderful!” I said to myself (^)__(^)

Thanks to Linux Journal’s issue 132 in April 2005 (my back issue which I actually re-read) before writing this post.

Quick and cool Linux commands!

November 28, 2007
  • Redirection in Linux
  • 0 = stdin
    1 = stdout
    2 = stderr

  • The following command saves stdout and stderr to the files “out.txt” and “err.txt”, respectively.
[root@server /root]# ./cmd 1>out.txt 2>err.txt
  • Using the tee command to save stdout to tee.txt. stdout is still displayed on the screen.
[root@server /root]# cmd | tee tee.txt
  • Using the tee command to append stdout to tee.txt. stdout is still displayed on the screen.
  • [root@server /root]# cmd | tee -a tee.txt
  • This is a cool ‘find’ command – looks in each file for ‘here’. First we create 3 files with the string ‘here’ in 2 of the 3 files
f@foxhound1:~/temp/findtest$ echo "searchstring is here">text1
f@foxhound1:~/temp/findtest$ echo "searchstring is also here">text2
f@foxhound1:~/temp/findtest$ echo "search string">text3
f@foxhound1:~/temp/findtest$ echo "string">text3

Then we issue the command

 find . -type f -exec grep -i here  \{\} --with-filename \;

And the corresponding output is:

f@foxhound1:~/temp/findtest$ find . -type f -exec grep -i here  \{\} --with-filename \;
./text1:searchstring is here
./text2:searchstring is also here

From the output, we can see that the command was able to search all three files we created, but only text1 and text2 contained the string ‘here’. To check that it does search all three files, we issue the command (with the corresponding output)

f@foxhound1:~/temp/findtest$ find . -type f -exec grep -i here  \{\} --with-filename \;
./text1:searchstring is here
./text2:searchstring is also here

Which proves that it does search all files (in this case, all 3 files we just created) since all 3 files have ‘string’ in them, as shown in the ouput.

The difference with using ‘find’ this way as compared to ‘xargs’ (see comment below) is that you can execute any command after the ‘exec’ parameter of ‘find’. You can delete (rm), view (cat) etc files found by ‘find’ this way.

Another neat and quick trick (especially when doing coding/software development) is to find a specific string (see above) within a directory, but to not include certain files containing the string you’re looking for (as with the case of using a version control system such as subversion):

find /var/www/@dmin/ -type f -exec grep   \{\} --with-filename \;|sed -e '/svn/d'|sed -e '/~/d'

This chain of command (coupled with the above ‘find’ command) removes from the list of files ‘find’ found all the files that have ‘svn’ in their filenames, including the files that have a ‘~’ (tilde), which in Linux is equivalent to a backup/temporary file.
More quick and cool Linux commands can be found here.


The following logs the date, the machine’s motherboard and CPU temperature (assumes you have lm-sensors properly setup) and the voltage level of a certain rail in the power supply, and puts all of those in a file called datemp.log:

date +%m/%d/%y%X|tr -d 'n' >>datemp.log&& sensors|grep +5V|cut -d "(" -f1|tr -d 'n'>> datemp.log && sensors |grep Temp |cut -d "(" -f1|tr -d 'n'>>datemp.log

Which after 5 times of iteration (say in a loop for logging, taken every second) gives the following nice output on datemp.log:

02/10/0805:53:48  PHT+5V:         +4.95 V  M/B Temp:    +52.0°C  CPU Temp:    +51.0°C  Temp3:       +17.0°C
02/10/0805:54:08  PHT+5V:         +4.92 V  M/B Temp:    +52.0°C  CPU Temp:    +51.0°C  Temp3:        +3.0°C
02/10/0805:54:29  PHT+5V:         +4.92 V  M/B Temp:    +52.0°C  CPU Temp:    +51.0°C  Temp3:       +13.0°C
02/10/0805:54:50  PHT+5V:         +4.92 V  M/B Temp:    +52.0°C  CPU Temp:    +51.0°C  Temp3:       +17.0°C
02/10/0805:55:11  PHT+5V:         +4.92 V  M/B Temp:    +52.0°C  CPU Temp:    +51.0°C  Temp3:       +30.0°C

This one-line logging technique is beneficial in two ways: the output log (in this case datemp.log) filesize is kept at a minimum, and makes it easier to look for desired values using ‘grep’ since values for every iteration is on a single line.