Learn UNIX
September 18th, 2008 — Basic stuff, Scripts, Unix
Today I'd like to expand a bit more on the basic calculations in Unix scripts.
In Basic Arithmetic Operations post I've shown you how expression evaluation can be used to calculate simple math expressions in your Unix shell:
ubuntu$ START=1 ubuntu$ FINISH=10 ubuntu$ ELAPSED=$(($FINISH - $START)) ubuntu$ echo $ELAPSED 9
September 4th, 2008 — Basic stuff
Today I’d like to show you the basic usage of rsync – a wonderful, old and reliable tool for incremental data transfers and synchronization of local directories or even data between different Unix systems.
rsync is quite a complicated command, so don’t expect this first post to explain everything and cover every possibility. Like I said, this is only the beginning.
August 26th, 2008 — Basic stuff, Scripts, Unix
If you're just getting started with your Unix scripting or new to Unix shell, today's little tip will make your life much easier: I'm going to show you how to update your shell variables.
August 21st, 2008 — Basic stuff
In my system administration job, I come across this particular requirement quite often: run a command, then confirm how long it took to complete the job. You guessed it right – there's a very easy way to do it.
What you need to do is to use the time command, which takes as a parameter the full command line you need to run. Upon completion of the command, time will report the system resources usage during the execution. All the reported numbers are presented in a simple table.
July 14th, 2008 — Basic stuff
Still seeing this request quite frequently in my website logs, I thought I'd expand the topic a bit more, given the fact that I've somehow left out the most obvious way to confirm the group membership for a Unix user.
June 18th, 2008 — Basic stuff, Linux
If you're logged in at some remote Linux system and need to quickly confirm the amount of available memory, there's a few commands you will find quite useful.
free command is the most obvious choice for a first command when it comes to your RAM.
June 13th, 2008 — Basic stuff
If you need to find all the files owned by a certain Unix user or group in a given directory, it's actually very easy to do using find command.
If you know the username, this is the command you might use to locate all the files which belong to it:
ubuntu$ find /var -user www-data /var/cache/apache2/mod_disk_cache /var/lib/awstats ...
In this case, we're looking for all the files and directories owned by a webserver used www-data under /var.
If you're doing a broader search, you may be interested in identifying all the files owned by a Unix group. Here's how you can do it:
ubuntu$ find /usr -group staff /usr/local/lib/site_ruby /usr/local/lib/site_ruby/1.8 /usr/local/lib/site_ruby/1.8/x86_64-linux /usr/local/lib/python2.5 /usr/local/lib/python2.5/site-packages /usr/local/lib/python2.4 /usr/local/lib/python2.4/site-packages /usr/local/share/fonts
In this example, we're using find to locate all the files in /usr owned by the staff group.
If you're more comfortable dealing with Unix user IDs (UIDs) and group IDs (GIDs), you can use them with find command as well. In this example, I'm looking for the temporary files created by myself (my UID on that system is 1000):
ubuntu$ find /var/tmp -uid 1000 /var/tmp/photos.rar /var/tmp/mysql.tar.gz ...
I presume this post answers your questions on the topic, but do let me know if there's anything else you'd like me to explain!
June 9th, 2008 — Basic stuff, Unix
Certain situations require you to quickly confirm which files between two directories are different, and while your particular requirements may suggest writing a script for this task, I want to make sure you're familiar with the basics first – majority of directory comparisons can be done using diff command (yes, that's right – the same one used for comparing files).
First of all, let's agree on why you may need to compare directories. There's a few possible reasons:
For today's post, I've created a set of directories and files to show how you can compare them. Here is the setup:
ubuntu$ find /tmp/dir1 /tmp/dir2 /tmp/dir1 /tmp/dir1/file1 /tmp/dir1/file2 /tmp/dir1/dir11 /tmp/dir1/dir11/file11 /tmp/dir1/dir11/file12 /tmp/dir2 /tmp/dir2/file1 /tmp/dir2/dir11 /tmp/dir2/dir11/file11 /tmp/dir2/dir11/file12 /tmp/dir2/file3
As you can see, I've got two directories: /tmp/dir1 and /tmp/dir2, with a dir11 subdirectory in each of them. There's also a few files here and there, some of them missing from one of the directories specifically to be highlighted by our comparison exercises.
The easiest way to get started is to simply invoke diff command and specify two directories as command line parameters. Here's what you will probably see:
ubuntu$ diff /tmp/dir1 /tmp/dir2 Common subdirectories: /tmp/dir1/dir11 and /tmp/dir2/dir11 diff /tmp/dir1/file1 /tmp/dir2/file1 1d0 < New line Only in /tmp/dir1: file2 Only in /tmp/dir2: file3
This output confirms that /tmp/dir1 and /tmp/dir2 both contain a dir11 directory, and also shows that /tm/dir1/file1 and /tmp/dir2/file1 are actually different files even though they have the same name. By default, diff compares such files and you can see the result of each comparison in the output. Also included are pointers to the files which are present only in one of the compared directories: you can see that file2 can only be found in /tmp/dir1 and file3 was present only in /tmp/dir2.
From the example below, it is easy to deduct that the command line for identifying files missing in one of the directories will be this one:
ubuntu$ diff /tmp/dir1 /tmp/dir2 | grep Only Only in /tmp/dir1: file2 Only in /tmp/dir2: file3
If you're only interested in files which exist in both directory structures, but are different – you can use a special command line option. It will simply point the files out, without getting into any further details. You'll probably notice how this output is very similar to the default one:
ubuntu$ diff --brief /tmp/dir1 /tmp/dir2 Common subdirectories: /tmp/dir1/dir11 and /tmp/dir2/dir11 Files /tmp/dir1/file1 and /tmp/dir2/file1 differ Only in /tmp/dir1: file2 Only in /tmp/dir2: file3
Note how instead of showing the difference between file1 in /tmp/dir1 and /tmp/dir2, this time you only get told that these two files are different.
If you're dealing with a complex directory structure, you'll be glad to know that –recursive parameter for the diff command compares not only the immediate directories pointed to from the command line, but also walks through the full tree of subdirectories:
ubuntu$ diff --recursive --brief /tmp/dir1 /tmp/dir2 Files /tmp/dir1/dir11/file12 and /tmp/dir2/dir11/file12 differ Files /tmp/dir1/file1 and /tmp/dir2/file1 differ Only in /tmp/dir1: file2 Only in /tmp/dir2: file3
Feel better now? Many directory comparison tasks can be accomplished using the diff command, but if you're stuck with a particular problem which can't be solved using my examples – please leave a commend and I'll come up with a solution.
May 7th, 2008 — Basic stuff, Scripts, Unix
Any Unix shell script longer than a line will most likely involve using variables. Variables are used to store temporary values to simply using them in various Unix commands of your script. The beauty of using variables is that they can be evaluated and set ones, but then reused as many times as you like without your shell interpreter having to re-evaluate them again.
To specify a value for a variable, you need to decide on the variable name – can be any word or combination of English alphabet symbols and digits, and specify the value.
April 7th, 2008 — Basic stuff, Linux
Very quick tip for you today, I just see that many of visitors of this block are curious how they can find a directory in Unix – and so here's a command to help you do just that.
There's nothing better than to employ the find command. As you might remember, among many things, this wonderful tool allows you to search files by their type. Since nearly everything in Unix is a file, this means you can find directories.
