Centralized BASH history with timestamps

For every Unix user, there comes a point where shell history suddenly becomes very relevant. You learn to consult it, then start recovering the last command, then switch to searching past commands history to save precious time normaly taken typing.
Shortly after such a point in your life, you'll probably want to enhance your shell history in two very common ways:
  1. Make sure every terminal window can update AND access your centralized shell history. So you run a command or two in one window, then type "history" anywhere else and see them two commands right there.
  2. Provide meanigful timeline, this is done with timestamps. Very simple and powerful change helps you see exactly when each command occured.

Here's how you achieve both of these massive improvents to your history in BASH. Just add this to /etc/bashrc on your Linux system:

export HISTTIMEFORMAT="%d.%m.%y %T "
export HISTCONTROL=ignoredups:erasedupsshopt -s histappend
export PROMPT_COMMAND="${PROMPT_COMMAND:+$PROMPT_COMMAND$'\n'}history -a; history -c; history -r;"
export HISTCONTROL=ignoreboth

Share and Enjoy

  • Facebook
  • Twitter
  • Delicious
  • LinkedIn
  • StumbleUpon
  • Add to favorites
  • Email
  • RSS

Most popular BSD distributions

BSD stands for Berkeley Software Distribution, and it was a UNIX software compilation released by the Computer Software Research Group at the University of California, Berkeley between 1977 and 1995. The first one was 1BSD (First Berkeley Software Distribution) compiled by Bill Joy, but only as an add-on to Sixth Edition Unix from Bell Labs. In 2BSD Bill Joy added a C Shell and the iconic vi text editor.

As BSD evolved it became a complete UNIX operating system in its own right. The last release from Berkeley was 4.4BSD-Lite Release 2, and it contained no proprietary code from AT&T, and was freely available under the permissive BSD license. Since then other projects, descendants of the original BSD, continued the development and they are what BSD today generally refers to. Most popular of these are FreeBSD, NetBSD, and OpenBSD.

BSD was the first to include support for the Internet Protocol stack in form of its Berkeley sockets, which made it easy to read and write files over the network. It is also easier to natively run software from other operating systems on BSD thanks to its binary compatibility layer. The very permissive nature of the BSD open source license enabled widespread use of its code in various other software projects. Apple's OSX and iOS, for example, are based on BSD code.

FreeBSD

Originally based on top of 386BSD, and since version 2.0 on 4.4BSD-Lite FreeBSD exists with a goal of providing a complete operating system that can serve any purpose without any strings attached, free in every sense of the word. FreeBSD originated the ports collection system for easy download, building, and installation of software packages that continues to be one of the easiest and most sophisticated ways to install software in the UNIX world. This was adopted by NetBSD and OpenBSD as well.

FreeBSD also uses a rather open model of development by letting hundreds of "committers" make changes to FreeBSD at any time as needed. The selection of committers and resolution of any disputes is managed by the elected Core Team.

FreeBSD is the most popular BSD version used, and also served as the basis of many other operating systems such as most notably the Apple's OSX. There is also a number of FreeBSD variants such as the desktop oriented PC-BSD created to be easy for everyone to use.

NetBSD

NetBSD was founded after FreeBSD with major emphasis on portability at a time when FreeBSD was mostly focusing on the x86 architecture. NetBSD runs on so many platforms that they have a slogan saying "Of course it runs BSD". This makes it particularly suitable for computer research because it readily runs on both old and new architectures alike. NetBSD also uses the pkgsrc package management system originally based on the Ports collection system from FreeBSD.

OpenBSD

In 1995 Theo De Radt forked the NetBSD project to create OpenBSD. Particular focus was put on security as well as strong emphasis on great documentation, code correctness, and open source licensing. The project has also spawned a number of key widely used security tools like OpenSSH, OpenNTPD, PF, and most recently the LibreSSL fork of OpenSSL after the Heartbleed bug fiasco. Unlike FreeBSD the project is more strictly managed by Theo De Radt himself. OpenBSD also supports about 20 different hardware architectures.

Share and Enjoy

  • Facebook
  • Twitter
  • Delicious
  • LinkedIn
  • StumbleUpon
  • Add to favorites
  • Email
  • RSS

How to Change Hostname in FreeBSD

Changing the hostname in FreeBSD is very simple. It can be done immediately using the hostname command while logged in as root:

hostname new.hostname

Just replace new.hostname with your own desired name. It doesn't have to have the dot in it, it can be just "myhost", for example.

This changes the hostname for the current session only, which means that if you reboot it will be revert to the previous one. You can make sure the hostname change sticks permanently by editing the /etc/rc.conf file to add the following line:

hostname="new.hostname"

Again, just replace new.hostname with your own. Also, if this line already exists you can just edit it to change the hostname. Once you save this file your hostname change will stick even after you reboot the machine.

That's all there is to it.

Share and Enjoy

  • Facebook
  • Twitter
  • Delicious
  • LinkedIn
  • StumbleUpon
  • Add to favorites
  • Email
  • RSS

How to Find Directories Larger Than 1GB in Linux

Finding out sizes of files and directories in Linux is done using the du command, which estimates their disk space usage. The du command can be used with options that allow you to customize the results you get.

Commands can also be combined with each other so the second command filters the results output by the first command. So to find directories that are larger than 1GB you can run this command:

du -h --max-depth=1 / | grep '[0-9]G\>'

The -h option displays the sizes in a more human readable format, in gigabytes rather than kilobytes. The –max-depth=1 option makes it display the sizes of only the directories immediately within the specified path, in this case the root path /, which in Linux includes directories like /home, /usr, /bin, /var, and so on. If we were to specify 2 it would go a level further and also look into directories like /home/user, /usr/bin, /var/log, etc.

The | sign is the pipe sign that allows combining the first command with the next command, in this case grep, which is used for searching through text output for the specified strings, which can be specified through regular expressions. So the grep '[0-9]G>' searches through the du output and displays only the files that are larger than 1GB.

Here's another example of this command, but with –max-depth=1 being shortened to -d 1, which works the same, and checking the /var directory instead of the root / directory.

du -h -d 1 /var | grep '[0-9]G\>'

This single command is all you need for this task, but there's an even shorter one that works well too. You can use the built in -t or –threshold option of the du command like this:

du -h -d 1 -t 1G /

It will display first level directories larger than 1GB within the root / path, just like the first command. If you want it to display all directories, not just first level, just leave out the -d 1 option.

Finally, as a bonus, you can sort the results from largest to smallest by piping in the sort command:

du -h -d 1 -t 1G / | sort -hr 

You can also sort the first command like this too, in which case you're combining the three commands into one:

du -h --max-depth=1 / | grep '[0-9]G\>' | sort -hr 

´╗┐Those are some of the ways you find directories larger than 1GB, or any other size you desire for that matter. Just switch 1G to 2G or 500M and so on.

Share and Enjoy

  • Facebook
  • Twitter
  • Delicious
  • LinkedIn
  • StumbleUpon
  • Add to favorites
  • Email
  • RSS

How to Use visudo

The visudo command is a safe and secure way of editing the /etc/sudoers file on UNIX and Linux systems. Since the sudoers file determines which users can run administrative tasks, those requiring superuser privileges, it is a good idea to take some precautions when editing it, and that's what visudo does.

It locks the sudoers file so it cannot be edited by anyone else simultaneously. It also checks the syntax of your edits and provides basic sanity checks. If someone else is editing the file you'll get a message to try later, and if there are errors in your edits it wont save them.

Preventing simultaneous editing by someone else is helpful to ensure your edits aren't lost, and saving a sudoers file without errors is important because you could otherwise end up locked out of your system. An unreadable sudoers file will prevent you from running administrative tasks by using the sudo command or becoming root, and editing the sudoers file itself requires those privileges. So you really don't want to screw that one up.

Visudo is basically a wrapper for a text editor such as vi or nano. Vi is traditionally the default unless your distribution or OS has something else set up. For basics on how to use vi for editing check out the vi survival guide.

Visudo has a built in list of supported editors that can be used, and you can change which it will use by setting the "EDITOR" environment variable on the command line like this: export EDITOR=nano. This will set nano as the default editor. To save this permanently add the same line to the .bashrc file in your home directory. On Ubuntu, where nano is actually set as the default, you can also change it by running sudo update-alternatives –config editor and then selecting your preference.

Editing Sudoers

To open up the /etc/sudoers file for editing with visudo simply run sudo visudo.

Before making any edits it's a good idea to check the existing configuration, and understand what everything means. One line you'll definitely encounter is this:

root    ALL=(ALL:ALL) ALL

This gives the root user all of the superuser privileges, as can be expected. The format of the rule set such as this is as follows:

user hosts=(users:groups) commands

What you're doing is specifying which commands can a given user run under which circumstances. In case where all of them are set to ALL, like for root, it means that the user can run all commands on all hosts, as all users and groups.

If all you want is enable another user with the same powers as root, obtainable by issuing the sudo command before the desired command, you can just copy the root line and change "root" with your username, in this example "daniel":

daniel    ALL=(ALL:ALL) ALL

But if you don't want to give all of the privileges you can adjust the rules. For example you can allow "daniel" to only run certain commands:

daniel ALL=(ALL:ALL) mytop,cat,tail

Besides users you can also give superuser permissions to groups using a % indicator:

%admin ALL=(ALL) ALL

This would allow all users in the admin group to run all commands as root.

Aliases

Finally, you can set up aliases to group multiple entries into a single one for use in these statements. There are four types of aliases: User_Alias for listing users, Runas_Alias for listing users a given user can run as, Host_Alias for listing hosts, and Cmnd_Alias for listing commands.

Aliases are useful if you have a more complex set up with multitude of users that should have varying degrees of privileges on the system. To set up an alias just state the alias type, its name, and then the list of users, hosts or commands you want to associate it with. For example to set up a User_Alias you can do this:

User_Alias MANAGERS = steve,bill,james

All the other aliases follow the same format only with the different specified type, and listing different types of things, like users, hosts or commands. If we wanted to put the three commands from the above example with the "daniel" user under an alias we could do this:

Cmnd_Alias READ = mytop,cat,tail

And then instead of listing these two commands in our configuration for daniel we can just specify the READ alias:

daniel ALL=(ALL:ALL) READ

It works the same way for other types of aliases. If we want to give the same privileges to users steve, bill, and james we can say:

MANAGERS ALL=(ALL:ALL) READ

You get the idea.

These are the basics of using visudo and editing the sudoers file with it. We recommend you check out the manual pages if you ever need more detailed reference, like man visudo and man sudoers. You can also see a sample sudoers file with many examples at its web site.

Share and Enjoy

  • Facebook
  • Twitter
  • Delicious
  • LinkedIn
  • StumbleUpon
  • Add to favorites
  • Email
  • RSS

Most Important sshd Configuration Options

SSH, or Secure SHell, allows the user of one computer on the network to connect to and use the shell of another over a secure connection. It consists of two basic components, the SSH client used to connect to a remote server, and the SSH server daemon (sshd) running on the server to accept SSH connections from elsewhere.

Configuration for the sshd server is found in the /etc/ssh/sshd_config file. The client configuration is in /etc/ssh/ssh_config.

Here are some of the most important configuration options for an SSH server:

Port

The default port for SSH is 22, which is typically fine, but it could be changed to some other available port if you want to throw an extra obstacle to would be unauthorized attempts to connect.

PermitRootLogin

This option can be set to either yes or no. If it is set to yes then it will allow using SSH to log in directly as root by running something like ssh root@host.net from the client computer. It may be a good idea to set this to "no" in order to close even the remote possibility of someone cracking through the root password and wreaking havoc. Just a decent precaution.

AllowUsers

With this option you can set to allow only some users on the system to connect via SSH. For multiple users separate them by spaces. For example:

AllowUsers james kevin

That will allow only james and kevin users to connect.

LoginGraceTime

This is the amount of time SSH will wait on the user to authenticate before cutting the connection. By default it is set to 120, or 2 minutes, but it can be reduced if you want to diminish chances of someone successfully attempting a brute force attack.

PasswordAuthentication

Set to yes by default this enable password authentication, which definitely should be enabled unless you have public key authentication enabled, because otherwise basically anyone could connect.

PubkeyAuthentication

An alternative or an addition to PasswordAuthentication setting this to yes could significantly increase security. For it to work you also need an option that specifies where the authorized keys are:

AuthorizedKeysFile ~/.ssh/authorized_keys

TCPKeepAlive

Set to yes by default this option checks the status of your connection by sending keepalive messages to the client. If there are network interruptions it will then close the connection rather than continue to use up resources.

Share and Enjoy

  • Facebook
  • Twitter
  • Delicious
  • LinkedIn
  • StumbleUpon
  • Add to favorites
  • Email
  • RSS

How to Use wget and curl

Both wget and curl are command line tools for transferring files over the network via various network protocols like HTTP or FTP. Wget is a GNU Project by the Free Software Foundation licensed under the GNU GPL whereas Curl is an independent project licensed under a variant of the MIT license.

Curl is also based on a libcurl library, part of the same project, which makes it more suitable for use in programming various applications. It is generally more flexible and featureful whereas wget is simpler.

Simple download

Here's how to initiate a simple download with both wget and curl.

wget ftp://mirrors.kernel.org/gnu/unifont/unifont-7.0.01/unifont_upper-7.0.01.ttf

curl -O ftp://mirrors.kernel.org/gnu/unifont/unifont-7.0.01/unifont_upper-7.0.01.ttf

This will download the file to the current working directory with its original file name (which is what the -O option passed to curl is for). You can run curl without any options as well, but it will dump the file contents on to the screen as it downloads.

Download multiple files at once

You can also download multiple files in one go by specifying multiple URLs:

wget ftp://mirrors.kernel.org/gnu/unifont/unifont-7.0.01/unifont-7.0.01.ttf ftp://mirrors.kernel.org/gnu/unifont/unifont-7.0.01/unifont_upper-7.0.01.ttf

curl -O ftp://mirrors.kernel.org/gnu/unifont/unifont-7.0.01/unifont-7.0.01.ttf -O ftp://mirrors.kernel.org/gnu/unifont/unifont-7.0.01/unifont_upper-7.0.01.ttf

Specifying -O before each URL isn't necessary, but we found that not specifying it results in the second file being downloaded being dumped into the command line.

That said, it is also possible to specify multiple files for download in accordance to a given range of numbers or letters in the file name.

With curl you can also download multiple files with sequential numbers or letters in their names like file1, file2, file3, and so on by specifying the range in the brackets:

curl -O ftp://example.com/file[1-50].txt

Multiple nested sequences are possible too, like this:

curl -O http://example.com/[2002-2014]/file{a-z}.txt

Resume downloads

Both wget and curl allow you to resume partial downloads, which is useful if a network outage in the middle of the download interrupted it, for example, and you want to continue where you left off.

With wget you simply use the -c or –continue option. Some have a habit of always passing this option just in case.

wget -c ftp://mirrors.kernel.org/gnu/unifont/unifont-7.0.01/unifont-7.0.01.ttf

With curl you can use the -C – options:

curl -C - ftp://mirrors.kernel.org/gnu/unifont/unifont-7.0.01/unifont-7.0.01.ttf

That's in a nutshell how to use wget and curl. They're both powerful, curl a bit more powerful than wget, and you can find all about what they can do in their manual pages: man wget, and man curl.

Share and Enjoy

  • Facebook
  • Twitter
  • Delicious
  • LinkedIn
  • StumbleUpon
  • Add to favorites
  • Email
  • RSS

Using nice and renice to Change Process Priority

Every process in UNIX runs with a particular priority assigned to it, which determines how much processing time should it be allowed to use. Priorities are represented as numbers from -20 to 19 where -20 represents the highest priority, because -20 comes before every other number on that scale, and 19 is actually the lowest priority because it comes last.

This is probably part of why this value is called the "nice value", because the greater the number the "nicer" is the process when it comes to demanding resources. A process with the value of 19 is the nicest because it is the least selfish, so to speak.

The nice command allows you to start a process with a particular nice value, setting the process priority, whereas the renice command allows changing the nice value of an already running process.

nice

To start a process with a custom nice value you run the desired command prefixed by the appropriate nice command, like this:

nice -n 10 znc

So the -n option says we want to adjust the nice value, the 10 is the actual value, and znc is the command in this example. This will start the znc process with the nice value of 10, a fairly low priority.

Now if you use the ps command, which lists running processes, to search for a znc process you'll see that its nice (NI) value is 10:

ps -l -C "znc"

F S   UID   PID  PPID  C PRI  NI ADDR SZ WCHAN  TTY          TIME CMD
1 S  1008 13631     1  0  90  10 -  5251 -      ?        00:15:05 znc

The -l option tells it to show the process in the long format, listing all information about it, and the -C option followed by the command just tells it to look for a process started by that command.

renice

If you've got an already running process you can change its nice value, and therefore its priority, without restarting it. Simply run the renice command like this:

 renice -n 7 -p 13631  

So the syntax is the same except instead of specifying a command we're specifying a process ID (PID), with the -p option. We need to have a process ID of the running process before we can renice it. You can find it in various ways, but the simplest is to just use the above mentioned ps command:

ps -l -C "znc"  

Then simply look for the PID field for the value to put after -p and you're set.

When you run the ps command after renicing you'll see the new NI value reflected.

Share and Enjoy

  • Facebook
  • Twitter
  • Delicious
  • LinkedIn
  • StumbleUpon
  • Add to favorites
  • Email
  • RSS

Useful Options for mount and umount

In UNIX and Linux operating systems everything resides in a tree like structure rooted at /, the root directory. This includes storage devices with partitions and their own file systems. The mount command mounts these somewhere within this tree structure. For instance, a disk partition used for data storage can be mounted on /mnt/data or /media/data or wherever else you find it convenient or prudent.

The mount command accepts options that determine how is the file system mounted. A basic mount command without any options can be as simple as this:

sudo mount /dev/sdb1 /media/usb

Of course the directory you're mounting to (in this example /media/usb) should exist beforehand. To unmount the device from it, thereby removing its file system from the file tree, just use the umount command the same way.

To mount a file system with options pass the -o option first followed by a comma separated list of mounting options, which may look something like this:

sudo mount -o noexec,auto /dev/sdb1 /media/usb/

In this case we've used the noexec and auto options, but we might have as well specified any number of other options depending on our needs and desires. Here is a quick overview of those you might find most useful, depending on the situation. They can be used alone or in combination with each other.

defaults

The defaults option is actually a shortcut to a number of different options that you'll likely want if you have nothing specific in mind, and just want to mount a file system for normal use. They include rw, suid, dev, exec, auto, nouser, and async. In other words running..

sudo mount -o defaults /dev/sdb1 /media/usb  

..is the same as running..

sudo mount -o rw,suid,dev,exec,auto,nouser,async /dev/sdb1 /media/usb 

So what does each of these options mean?

rw – mount with both read and write access

suid – allows giving users who don't own a specific file in a file system temporary permissions for running the file as if they did own it. In other words the owning user can give other users temporary permissions. This makes commands like passwd and ping usable for normal users even though their operation requires actions which normal users typically don't have access to.

dev – allows creating devices nodes such as /dev/sdc1 within a mounted file system. The opposite option, nodev, would disallow this.

exec – allows executing binaries within the mounted file system. The opposite option, noexec, would make it impossible to execute programs from the mounted device.

auto – specifies that this file system will be automatically mounted on system startup or by running the mount -a command.

nouser – disallows the ordinary non-root user to mount this file system.

async – specifies that all input and output to the file system should be asynchronous as opposed to sync which would make them synchronous. The difference is that the asynchronous mode allows processing to run even as I/O operations are still ongoing rather than wait for them to finish, which makes sense most of the time, and is therefore a reasonable default option.

If all of these options sound fine to your specific need then just passing the defaults option will be good. However, if you don't want one or more of these default options you may want to specify your own list, or use defaults with overriding subsequent options.

Here are some of the other useful options you might want to consider, some of which override or are the opposite of the above defaults, but could be useful in certain situations.

ro – mounts the file system as read-only, disallowing any write access to the mounted file system. This could be useful if you want to make sure to preserve the data on the file system as is, and especially prevent overwriting any data, when you are mounting the file system only for the purpose of accessing the files on it.

noexec – disallows executing binaries on the mounted file system, which could also be used for file systems used only as data storage where you don't wish programs to run.

users – makes it possible for every user on the system to mount or unmount the file system.

group – allows non-root users to mount the file system if one of their groups is the same as the group to which the device belongs.

remount – useful when you want to mount an already mounted file system, but with different options than those specified previously or within the /etc/fstab configuration file, and without changing the mount point (the path where it is mounted).

noatime – prevents updating access times on inodes of the file system, which could speed up access on a frequently accessed file system for a specialized purpose.

umount options

Finally, you can pass the same options to the umount command using the -O option first, like this:

umount -O noexec /dev/sdb1 

Options passed to the umount command only mean that the command will apply to file systems which have the specified option within the /etc/fstab configuration file. In the above example it will not apply if /dev/sdb1 has no noexec option specified in /etc/fstab.

Share and Enjoy

  • Facebook
  • Twitter
  • Delicious
  • LinkedIn
  • StumbleUpon
  • Add to favorites
  • Email
  • RSS

Difference Between chmod and chown

The chmod and chown commands are used to control access to files in UNIX and Linux systems.

The chmod command stands for "change mode", and allows changing permissions of files and folders, also known as "modes" in UNIX. The chown command stands for "change owner", and allows changing the owner of a given file or folder, which can be a user and a group. That's the difference between them in a nutshell.

They are interrelated in so far as changing ownership of a file changes who the set permissions apply to. The new owner inherits the permissions.

Let's take a quick look at the basic usage of these commands.

chmod

The chmod command can be used in a couple of different ways, with permissions (or modes) set by numbers or by letters. Permissions can be given to a user who owns the file (u = user), group of said user (g = group), everyone else (o = others) or all users (a). And the basic permissions that can be given include read (r), write (w), and execute (x). There are also X, s, and t, but they're less commonly used.

When using numbers you can use a numeric value such as 644 to set permissions. The position of the value represents to whom is the permission given, and the actual value represents which (or how much) permissions are given as a sum total of each permission's unique value.

First position (in the above example 6) refers to the user. Second refers to the group of the user, and the third refers to all others.

Numeric values for permissions are:

4 = read 2 = write 1 = execute

So a value of 4 will only give read rights, but a value of 6 will give read and write rights because it is a sum of 4 and 2. 5 will give only read ane execute rights, and 7 will give all rights. Do this calculation for each numerical position and you'll end up with the desired value. So in the example of 644 we're giving the user who owns the file the permission to read and write (but not execute), the group of that user the permission to read only, and others the right to read only as well.

To set this mode with chmod on a file called important.txt we would simply run this command:

chmod 644 important.txt

Note that making a file executable, if it were a script or a program, amounts to simply giving someone or everyone a permission to execute. If this was an important.sh bash script we could allow the owner to execute, and others to read with the 744 mode, or everyone to execute with 755.

chmod 755 important.sh

Now, we can also use letters to accomplish the same thing, and we've already mentioned the relevant letters above. This is probably easier to remember than using numbers. For example, to accomplish the 644 permissions above we would run this:

chmod u+rw,go+r important.txt

So we're saying file owner user gets read and write permissions, group and others get to read.

The second example, with the important.sh file being made executable we could just run this:

chmod u+rwx,go+rx important.sh

If important.sh already had permissions set to 644 we can add everyone execute rights by simply running:

chmod +x important.sh

Not specifying the letter for anyone is treated as if we said "a", for all.

Finally, if we're setting permissions to a folder we need to specify the -R option (standing for "recursive"):

chmod -R 644 important-files/

chown

Basic usage of chown is pretty straightforward. You just need to remember that first comes the user (owner), and then the group, delimited by a colon.

This command will set the user "daniel", from the group of "admins" as owners of the directory "important-files":

chown -R daniel:admins important-files

Just like with chmod, the -R is there when it's a directory.

Share and Enjoy

  • Facebook
  • Twitter
  • Delicious
  • LinkedIn
  • StumbleUpon
  • Add to favorites
  • Email
  • RSS