Search Results for: inode

Hard Link

Files in a file system are kept track of by so called "index nodes" or "inodes" which store some metadata about each file such as its type, file, access/change/modification/deletion times, owner, group, and so on (not including file names though). Typically, only one file can be associated with a single inode, but this is where hard links come in.

A hard link allows multiple files to be associated with a single inode where one file is a link, and the other is the original file. The key thing is that the link refers to the same inode pointing to the same data. This is different from a soft link, also known as a "symbolic link" or "symlink" which only contains an abstract path to the original file rather than being associated with the original file's inode.

Because of this hard links don't work across multiple file systems or partitions, and they also cannot link to directories. Their biggest advantage, however, is that they stay linked to the original file wherever on the same file system they are, even if the path location of the original file changes (because the link refers to the inode not the path).

In contrast soft links refer to nothing if the original file changes its location, becoming broken links. Their advantage, however, is that they can link across file systems as well as link to directories (because they refer to the path).

To create a hard link to a file use a command like this:

ln /home/user/file.txt /home/user/file-link.txt

The /home/user/file-link will be a hard link to /home/user/file, and will refer to said file even if you move it to /home/user2/file.txt.

To create a soft link or a symlink run the same command, but with an -s option:

ln -s /home/user/file.txt /home/user/file-link.txt  

That will create file-link.txt as a link to file.txt, but if you move file.txt from /home/user to /home/user2 the link, still pointing to the original path, will be broken.

Share and Enjoy

  • Facebook
  • Twitter
  • Delicious
  • LinkedIn
  • StumbleUpon
  • Add to favorites
  • Email
  • RSS

Useful Options for mount and umount

In UNIX and Linux operating systems everything resides in a tree like structure rooted at /, the root directory. This includes storage devices with partitions and their own file systems. The mount command mounts these somewhere within this tree structure. For instance, a disk partition used for data storage can be mounted on /mnt/data or /media/data or wherever else you find it convenient or prudent.

The mount command accepts options that determine how is the file system mounted. A basic mount command without any options can be as simple as this:

sudo mount /dev/sdb1 /media/usb

Of course the directory you're mounting to (in this example /media/usb) should exist beforehand. To unmount the device from it, thereby removing its file system from the file tree, just use the umount command the same way.

To mount a file system with options pass the -o option first followed by a comma separated list of mounting options, which may look something like this:

sudo mount -o noexec,auto /dev/sdb1 /media/usb/

In this case we've used the noexec and auto options, but we might have as well specified any number of other options depending on our needs and desires. Here is a quick overview of those you might find most useful, depending on the situation. They can be used alone or in combination with each other.

defaults

The defaults option is actually a shortcut to a number of different options that you'll likely want if you have nothing specific in mind, and just want to mount a file system for normal use. They include rw, suid, dev, exec, auto, nouser, and async. In other words running..

sudo mount -o defaults /dev/sdb1 /media/usb  

..is the same as running..

sudo mount -o rw,suid,dev,exec,auto,nouser,async /dev/sdb1 /media/usb 

So what does each of these options mean?

rw – mount with both read and write access

suid – allows giving users who don't own a specific file in a file system temporary permissions for running the file as if they did own it. In other words the owning user can give other users temporary permissions. This makes commands like passwd and ping usable for normal users even though their operation requires actions which normal users typically don't have access to.

dev – allows creating devices nodes such as /dev/sdc1 within a mounted file system. The opposite option, nodev, would disallow this.

exec – allows executing binaries within the mounted file system. The opposite option, noexec, would make it impossible to execute programs from the mounted device.

auto – specifies that this file system will be automatically mounted on system startup or by running the mount -a command.

nouser – disallows the ordinary non-root user to mount this file system.

async – specifies that all input and output to the file system should be asynchronous as opposed to sync which would make them synchronous. The difference is that the asynchronous mode allows processing to run even as I/O operations are still ongoing rather than wait for them to finish, which makes sense most of the time, and is therefore a reasonable default option.

If all of these options sound fine to your specific need then just passing the defaults option will be good. However, if you don't want one or more of these default options you may want to specify your own list, or use defaults with overriding subsequent options.

Here are some of the other useful options you might want to consider, some of which override or are the opposite of the above defaults, but could be useful in certain situations.

ro – mounts the file system as read-only, disallowing any write access to the mounted file system. This could be useful if you want to make sure to preserve the data on the file system as is, and especially prevent overwriting any data, when you are mounting the file system only for the purpose of accessing the files on it.

noexec – disallows executing binaries on the mounted file system, which could also be used for file systems used only as data storage where you don't wish programs to run.

users – makes it possible for every user on the system to mount or unmount the file system.

group – allows non-root users to mount the file system if one of their groups is the same as the group to which the device belongs.

remount – useful when you want to mount an already mounted file system, but with different options than those specified previously or within the /etc/fstab configuration file, and without changing the mount point (the path where it is mounted).

noatime – prevents updating access times on inodes of the file system, which could speed up access on a frequently accessed file system for a specialized purpose.

umount options

Finally, you can pass the same options to the umount command using the -O option first, like this:

umount -O noexec /dev/sdb1 

Options passed to the umount command only mean that the command will apply to file systems which have the specified option within the /etc/fstab configuration file. In the above example it will not apply if /dev/sdb1 has no noexec option specified in /etc/fstab.

Share and Enjoy

  • Facebook
  • Twitter
  • Delicious
  • LinkedIn
  • StumbleUpon
  • Add to favorites
  • Email
  • RSS

ln – make links and symlinks between files or directories

ln is a Unix command for linking files or directories to each other. Essentially, it creates new files with the names you specify, and refer them to already existing files or directories. When you run any Unix command against a symlink, it is first resolved (the original file it points to is confirmed) and the Unix command works with that file to produce desired outcome.

Two types of linking files and directories

There are two common approaches to link a file or directory in Unix: soft linking and hard linking. Soft links are also called symlinks (symbolic links).

What is a soft link?

Soft link (also referred to as symlink - short for symbolic link) is a special type of file in Unix, which references another file or directory. Symlink contains the name for another file and contains no actual data. To most commands, symlinks look like a regular file, but all the operations (like reading from a file) are referred to the file the symlink points to.

When you remove a soft link, you simply remove one of the pointers to the real file. When you remove the original file a soft link points to, your data is lost. Even though your soft link will still exist, it will be pointing to the non-existent file and will therefore be useless (it will probably have to be removed as well).

What is a hard link?

Hard link is a pointer to physical data. Effectively, all standard files are hard links, because they ultimately create an association between a file name and a physical data which corresponds to each file.

In Unix, you can create as many hard links to a file as you like, and there is even a special counter for such references. When you're using the long format of an ls command, you can see this counter.

When you remove a hard link, you decrease this link counter for a data on your storage. If you remove the original file, the data will not be lost as long as there's at least one hard link pointing to it.

Creating soft links (symlinks) with ln

Let's start with a really simple example. We create a text file, and then use soft link to reference it.

This shows how the file is created. It's called file1, and has a line of text data in it which we confirm using cat command:

ubuntu$ echo "Text file #1" > file1
ubuntu$ cat file1
Text file #1

Now let's use ln command to create a soft link. The newly created symlink will be a file called file2, and ls command will show you that it points to file1:

ubuntu$ ls -l file2
lrwxrwxrwx 1 root root 5 Mar 31 03:54 file2 -> file1

If you try accessing the file2, you will ultimately access file1, that's why the following example shows you the contents of file1:

ubuntu$ cat file2
Text file #1

Now, if you remove file1, this will make file2 symlink invalid, and any attempts to use it will return a "file not found" type of error:

ubuntu$ rm file1
ubuntu$ cat file2
cat: file2: No such file or directory

Creating hard links with ln

Now, let's look at creation of hard links in Unix. For this example, we'll recreate the file1:

ubuntu$ echo "Text file #1" > file1
ubuntu$ cat file1
Text file #1

If you use ls to look at file1, you can see that the link counter (second field from the left) is set to 1 – which means that there is only one file name pointing to the data with our "Text file #1" text:

ubuntu$ ls -l file1
-rw-r--r-- 1 root root 13 Mar 31 06:18 file1

And now we use ln command to create a hard link called file3, which points to the same data as file1:

ubuntu$ ln file1 file3

If we use ls command once again, you can see that the link counter has been increased and is now 2:

ubuntu$ ls -l file1 file3
-rw-r--r-- 2 root root 13 Mar 31 06:18 file1
-rw-r--r-- 2 root root 13 Mar 31 06:18 file3

Notice, how the file1 and file3 files look like absolutely normal files, and there's nothing showing a logical link between them.

To confirm that both filenames are actually referring to the same area on the disk, you can use -i option for the ls command, which will show you an i-node value.

i-nodes are data structures of a filesystem used to store all the important properties of each file: size, owner's user id and group id, access permission and more. The important thing is that each named data area on your disk must have an inode, and when you create a new data file this means creating an i-node. But when you're using hard links, you're effectively creating filesystem directory entry, which references an already existing data, so the hard link gets the same i-node number pointing to the same data.

ubuntu$ ls -il file1 file3
655566 -rw-r–r– 2 root root 13 Mar 31 06:18 file1
655566 -rw-r–r– 2 root root 13 Mar 31 06:18 file3

The first number in each line of the output is the i-node number, and since we're referencing the same data, the i-node numbers are also the same.

See also:

  • Unix symlink example

Share and Enjoy

  • Facebook
  • Twitter
  • Delicious
  • LinkedIn
  • StumbleUpon
  • Add to favorites
  • Email
  • RSS

tune2fs – adjust tunable filesystem parameters

tune2fs command is one of the advanced unix commands which allows you to adjust various tunable parameters of the ext2/ext3 filesystems. Naturally, it also helps you confirm the existing parameters configured for your filesystems.

Confirm current filesystem parameters with tune2fs

The tunefs -l command will show you all the information contained in a filesystem's superblock. Here's how it typically looks:

ubuntu# tune2fs -l /dev/sda1
tune2fs 1.40-WIP (14-Nov-2006)
Filesystem volume name:   <none>
Last mounted on:          <not available>
Filesystem UUID:          d2ff8a06-74b7-4877-9d37-1873414e25b3
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal filetype needs_recovery sparse_super
Default mount options:    (none)
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              2490368
Block count:              4980736
Reserved block count:     249036
Free blocks:              3417990
Free inodes:              2401957
First block:              0
Block size:               4096
Fragment size:            4096
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         16384
Inode blocks per group:   512
Filesystem created:       Wed Sep 26 02:30:22 2007
Last mount time:          Tue Apr  1 00:17:16 2008
Last write time:          Tue Apr  1 00:17:16 2008
Mount count:              1
Maximum mount count:      29
Last checked:             Tue Apr  1 00:16:22 2008
Check interval:           15552000 (6 months)
Next check after:         Sun Sep 28 00:16:22 2008
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:               128
Journal inode:            8
Default directory hash:   tea
Directory Hash Seed:      c0c5742c-980a-49b2-ae0b-4e96895376b6
Journal backup:           inode blocks

Reserved space on a Unix filesystem

By default, every filesystem in Unix has some space reserved for the superuser (root). This means that no regular Unix user can fill your filesystem up to 100%, and so it's always going to have enough free space to continue normal function.

As a standard, each filesystem has 5% of space reserved in this way. If you look at the above output, you may notice the following lines there, which regulate the space reservation:

Reserved block count: 249036
Reserved blocks uid: 0 (user root)
Reserved blocks gid: 0 (group root)

Compared to the overall filesystem block count:

Block count: 4980736

this 249036 reserve is exactly 5%. The uid and gid confirm the Unix user id and Unix group id of the user who will be allowed to tap into the reserved space. As I said earlier, it's root.

If you have root access on your system, you can alter this reserved space allocation for any filesystem using tune2fs -m parameter, by specifying the percentage of the space to be reserved.

Here's how we change the default reserve to be 6% of the overall filesystem size:

ubuntu# tune2fs -m 6 /dev/sda1
tune2fs 1.40-WIP (14-Nov-2006)
Setting reserved blocks percentage to 6% (298844 blocks)

And here we change it back. Note how the number of block corresponding to 5% is exactly the figure we've seen earlier – 249036 blocks:

ubuntu# tune2fs -m 5 /dev/sda1
tune2fs 1.40-WIP (14-Nov-2006)
Setting reserved blocks percentage to 5% (249036 blocks)

Default block size for a filesystem

If you ever want to confirm the block size of any filesystem, tune2fs will help you do just that:

ubuntu# tune2fs -l /dev/sda1 | grep Block
Block count:              4980736
Block size:               4096
Blocks per group:         32768

From this example, you can see that the default block size for the filesystem on /dev/sda1 partition is 4096 bytes, or 4k. That's the default block size for ext3 filesystem.

Share and Enjoy

  • Facebook
  • Twitter
  • Delicious
  • LinkedIn
  • StumbleUpon
  • Add to favorites
  • Email
  • RSS

touch – change file timestamps

ouch command is one of these little but extremely useful tools in Unix which you may have used for quite sometime before realizing their full potential. In short, it updates file timestamps – access and modification ones (atime and mtime respectively).

Why modify file timestamps?

There are quite a few legitimate reasons why you may want to update timestamps on a certain file. Ranging from source control approaches to storage usage analysis, there are processes out there which rely on the timestamps associated with each file and directory of yours.

[Read more…]

Share and Enjoy

  • Facebook
  • Twitter
  • Delicious
  • LinkedIn
  • StumbleUpon
  • Add to favorites
  • Email
  • RSS

How To Update atime and mtime for a File in Unix

If you remember, all files and directories in Unix filesystems have three timestamps associated with them – atime, ctime and mtime. Since questions about modifying access time (atime) and modification time (mtime) are quite frequent in my website logs, I thought I'd explain how it is done.

How to view atime, ctime and mtime

Before we go any further, I'd like to remind you that using stat command is probably the easiest way to look at all the three timestamps associated with each file:

[Read more…]

Share and Enjoy

  • Facebook
  • Twitter
  • Delicious
  • LinkedIn
  • StumbleUpon
  • Add to favorites
  • Email
  • RSS

atime, ctime and mtime in Unix filesystems

As you know, Unix filesystems store a number of timestamps for each file. This means that you can use these timestamps to find out when any file or directory was last accessed (read from or written to),  changed (file access permissions were changed) or modified (written to).

File and directory timestamps in Unix

Three times tracked for each file in Unix are these:

  • access time - atime
  • change time – ctime
  • modify time – mtime

[Read more…]

Share and Enjoy

  • Facebook
  • Twitter
  • Delicious
  • LinkedIn
  • StumbleUpon
  • Add to favorites
  • Email
  • RSS

Perl: Searching Through Directory Trees

I had a need to scan a huge directory tree today, identifying the users and Unix groups owning all the files. The problem I faced was too long usernames and group names which meant the

find /directory -ls

command which I normally use for such tasks wasn't terribly useful because there was no space delimiter between a username and a group. Results of such scan of the directory tree will have to later be parsed by other tools, and that's why proper splitting of the output into separate fields is so important.

 

This issue was motivational enough to refresh my Perl skills and sketch the following script (based entirely on this Never Run Unix Find Again article).

[Read more…]

Share and Enjoy

  • Facebook
  • Twitter
  • Delicious
  • LinkedIn
  • StumbleUpon
  • Add to favorites
  • Email
  • RSS