When I was writing a post about using date command to confirm date and time in your Unix scripts, I made a note in my future posts list to cover the date calculations – finding out the date of yesterday or tomorrow, and so on. Today I'll show you the simplest way to calculate this.
Entries Tagged 'Unix' ↓
Easy date calculations in Unix scripts with GNU date
September 19th, 2008 — Basic stuff, Scripts
Another way to use math expressions in shell scripts
September 18th, 2008 — Basic stuff, Scripts, Unix
Today I'd like to expand a bit more on the basic calculations in Unix scripts.
Use Parenthesis to simplify math expressions
In Basic Arithmetic Operations post I've shown you how expression evaluation can be used to calculate simple math expressions in your Unix shell:
ubuntu$ START=1 ubuntu$ FINISH=10 ubuntu$ ELAPSED=$(($FINISH - $START)) ubuntu$ echo $ELAPSED 9
Updating Values of Your Shell Variables in Unix
August 26th, 2008 — Basic stuff, Scripts, Unix
If you're just getting started with your Unix scripting or new to Unix shell, today's little tip will make your life much easier: I'm going to show you how to update your shell variables.
Continue reading →
How To Parse Text Files Line by Line in Unix scripts
August 13th, 2008 — Questions & Answers, Scripts, Solaris, Unix
I'm finally back from my holidays and thrilled to be sharing next of my Unix tips with you!
Today I'd like to talk about parsing text files in Unix shell scripts. This is one of the really popular areas of scripting, and there's a few quite typical limitations which everyone comes across.
Reading text files in Unix shell
If we agree that by "reading a text file" we assume a procedure of going through all the lines found in a clear text file with a view to somehow process the data, then cat command would be the simplest demonstration of such procedure:
redhat$ cat /etc/redhat-release Red Hat Enterprise Linux Client release 5 (Tikanga)
Unix Scripting: Time and Date
June 11th, 2008 — Scripts, Unix
If you have followed this blog for a while, you should remember how to use variables in Unix shell scripts.
Going further, I'd like to show you some basics of working with time and date in your scripts – generating all sorts of timestamps and timing some parts of your script for reporting purposes.
How To Compare Directories in Unix
June 9th, 2008 — Basic stuff, Unix
Certain situations require you to quickly confirm which files between two directories are different, and while your particular requirements may suggest writing a script for this task, I want to make sure you're familiar with the basics first – majority of directory comparisons can be done using diff command (yes, that's right – the same one used for comparing files).
Why compare directories?
First of all, let's agree on why you may need to compare directories. There's a few possible reasons:
- identifying if some files are missing from one of the directories – can be useful when you want to make sure two directories with configuration files for a certain package are identical – files can be different, but the same files are present in the same locations for both directories
- confirming if files in two directories are the same – a typical task when comparing your actual data against a backup copy. When something goes wrong, this is one of the first things you do to make sure all the important files are not only present, but are actually the same as they have been when you took the last backup copy
- highlighting textual differences between files in directories – this is a useful exercise when you're looking at two similar directories and expect only minor changes between the files – version numbers, different file or directory names hardcoded in various scripts, etc.
Test setup for comparison exercises
For today's post, I've created a set of directories and files to show how you can compare them. Here is the setup:
ubuntu$ find /tmp/dir1 /tmp/dir2 /tmp/dir1 /tmp/dir1/file1 /tmp/dir1/file2 /tmp/dir1/dir11 /tmp/dir1/dir11/file11 /tmp/dir1/dir11/file12 /tmp/dir2 /tmp/dir2/file1 /tmp/dir2/dir11 /tmp/dir2/dir11/file11 /tmp/dir2/dir11/file12 /tmp/dir2/file3
As you can see, I've got two directories: /tmp/dir1 and /tmp/dir2, with a dir11 subdirectory in each of them. There's also a few files here and there, some of them missing from one of the directories specifically to be highlighted by our comparison exercises.
Basic diff usage for comparing directories
The easiest way to get started is to simply invoke diff command and specify two directories as command line parameters. Here's what you will probably see:
ubuntu$ diff /tmp/dir1 /tmp/dir2 Common subdirectories: /tmp/dir1/dir11 and /tmp/dir2/dir11 diff /tmp/dir1/file1 /tmp/dir2/file1 1d0 < New line Only in /tmp/dir1: file2 Only in /tmp/dir2: file3
This output confirms that /tmp/dir1 and /tmp/dir2 both contain a dir11 directory, and also shows that /tm/dir1/file1 and /tmp/dir2/file1 are actually different files even though they have the same name. By default, diff compares such files and you can see the result of each comparison in the output. Also included are pointers to the files which are present only in one of the compared directories: you can see that file2 can only be found in /tmp/dir1 and file3 was present only in /tmp/dir2.
Find which files are missing in one of the directories
From the example below, it is easy to deduct that the command line for identifying files missing in one of the directories will be this one:
ubuntu$ diff /tmp/dir1 /tmp/dir2 | grep Only Only in /tmp/dir1: file2 Only in /tmp/dir2: file3
Highlight the different files, not the differences
If you're only interested in files which exist in both directory structures, but are different – you can use a special command line option. It will simply point the files out, without getting into any further details. You'll probably notice how this output is very similar to the default one:
ubuntu$ diff --brief /tmp/dir1 /tmp/dir2 Common subdirectories: /tmp/dir1/dir11 and /tmp/dir2/dir11 Files /tmp/dir1/file1 and /tmp/dir2/file1 differ Only in /tmp/dir1: file2 Only in /tmp/dir2: file3
Note how instead of showing the difference between file1 in /tmp/dir1 and /tmp/dir2, this time you only get told that these two files are different.
How to recursively compare directories
If you're dealing with a complex directory structure, you'll be glad to know that –recursive parameter for the diff command compares not only the immediate directories pointed to from the command line, but also walks through the full tree of subdirectories:
ubuntu$ diff --recursive --brief /tmp/dir1 /tmp/dir2 Files /tmp/dir1/dir11/file12 and /tmp/dir2/dir11/file12 differ Files /tmp/dir1/file1 and /tmp/dir2/file1 differ Only in /tmp/dir1: file2 Only in /tmp/dir2: file3
Feel better now? Many directory comparison tasks can be accomplished using the diff command, but if you're stuck with a particular problem which can't be solved using my examples – please leave a commend and I'll come up with a solution.
See also:
Get Username From UID in Unix
Finding out the username by user id (uid) in Unix is not as common a task as determining the uid by a username, but if you need to do it – I'll show you how.
Using variables in Unix shell scripts
May 7th, 2008 — Basic stuff, Scripts, Unix
Any Unix shell script longer than a line will most likely involve using variables. Variables are used to store temporary values to simply using them in various Unix commands of your script. The beauty of using variables is that they can be evaluated and set ones, but then reused as many times as you like without your shell interpreter having to re-evaluate them again.
Defining a variable in Unix shell
To specify a value for a variable, you need to decide on the variable name – can be any word or combination of English alphabet symbols and digits, and specify the value.
Unix filesystem basics: symlink example
February 14th, 2008 — Basic stuff, Unix
I can see some of you have arrived to my Unix file types post looking for an example of using symlinks in Unix. Today I would like to give you a quick introduction into Unix symlinks.
What is symlink?
Symlink is a short name for symbolic link (sometimes also referred as soft link) is a special type of file in Unix, which references another file or directory. Symlink contains the name for another file and contains no actual data. To most commands, symlinks look like a regular file, but all the operations (like reading from a file) are referred to the file the symlink points to.
Continue reading →
How To Find Out a File Type and Permissions in Perl
February 12th, 2008 — Perl
A few months ago, I've given a really simple example of using Perl for parsing directory trees in Unix. If you looked closer at it, you would have noticed that the script was working fine, but showing file modes as strange large numbers which didn't look like the usual file permissions you would expect. Today I'm going to explain why this happens, and show you how to find out a user type in Perl.
lstat and stat functions, return, among other things, the file mode value. While it looks confusing initially, it is in fact quite simply a combination field, which includes both the file type and all the permissions for it. If you print this field as a single decimal number, you will not recognize it, but if you simply convert it to octal, you will immediately start seeing the pattern:
Mysterious mode 33261 from the example below becomes 100755 when converted into octal, and you can easily see then the permission part of it: 0755.