bat – syntax highlighter

I’m back online! Have been doing a bit of reading but was too busy to publish anything, so this week should be fun with lots of content.

bat command

bat is a simple tool that lets you show contents of plain text files with syntax highlighting. Using different colours and even colour themese, bat lets you read files much easier because you learn very quickly what to focus on based on the highlight colour.

Here’s how it differs from the standard cat command:

Installing bat in Ubuntu

I simply did this:

greys@xps: ~ $ sudo apt install bat

Basic syntax for bat command is the same as it is for cat command.

That’s it for today! Have fun!

See Also




chown example: recursive update

chown command

chown is a basic Unix command, super useful and very powerful. I have provided a chown example earlier, but would like to show another common way it’s used.

chown: update ownership recursively

chown command uses -R option to apply changes recursively.

For example, if I have a /Users/greys/unixtutorial directory with the following layout:

file1
file2
dir1/file3

… then it’s very easy to show the difference between chown and chown -R commands.

Let’s go to the /Users/greys/unixtutorial directory:

greys@maverick:~ $ cd /Users/greys/unixtutorial
greys@maverick:~/unixtutorial $

Now, let’s look at the files with the ls command:

greys@maverick:~/unixtutorial $ ls -al *
 -rw-r--r--  1 greys  staff  0  8 Oct 22:55 file1
 -rw-r--r--  1 greys  staff  0  8 Oct 22:55 file2
dir1:
 total 0
 drwxr-xr-x  3 greys  staff   96  8 Oct 22:55 .
 drwxr-xr-x  5 greys  staff  160  8 Oct 22:55 ..
 -rw-r--r--  1 greys  staff    0  8 Oct 22:55 file3

Everything belongs to my own user, greys and my primary group: staff.

Time to change ownership of everything in the current directory:

greys@maverick:~/unixtutorial $ sudo chown root:wheel *

Checking files with ls again, I can see that immediate contents of the /Users/greys/unixtutorial directory (where I was at the time of running chown) got updated ownership: file1 and file2, along with dir1, now belong to root:wheel.

But file3 was in a dir1 subdirectory, so it stayed intact and still belongs to greys:staff:

greys@maverick:~/unixtutorial $ ls -al *

-rw-r--r--  1 root  wheel  0  8 Oct 22:55 file1

-rw-r--r--  1 root  wheel  0  8 Oct 22:55 file2
dir1:
 total 0
 drwxr-xr-x  3 root   wheel   96  8 Oct 22:55 .
 drwxr-xr-x  5 root   wheel  160  8 Oct 22:55 ..
 -rw-r--r--  1 greys  staff    0  8 Oct 22:55 file3

That’s because without the -R (recursive) option, chown will only inspect and update files in the current directory, but not in any of its subdirectories (or their subdirectories, and so on).

Running the same command with -R option does the trick:

greys@maverick:~/unixtutorial $ sudo chown -R root:wheel *
greys@maverick:~/unixtutorial $ ls -al *
-rw-r--r--  1 root  wheel  0  8 Oct 22:55 file1
-rw-r--r--  1 root  wheel  0  8 Oct 22:55 file2
dir1:
total 0
drwxr-xr-x  3 root  wheel   96  8 Oct 22:55 .
drwxr-xr-x  5 root  wheel  160  8 Oct 22:55 ..
-rw-r--r--  1 root  wheel    0  8 Oct 22:55 file3

Hope you learned something new, come back for more!

See Also




Manage Settings with dotfiles

Unix Tutorial

I’m finally taking the time to adopt a fascinating concept of organising dotfiles properly: track them for source code changes, store them centrally, deploy them automatically on various systems, etc. Dotfiles are configuration files for Unix-like and Linux systems that help you setup most of the tools: text editors, command line interpreters, git and shell aliases, etc.

I’ve learned enough to share this, but will be certainly revisiting this topic soon enough as I’m completing a Unix Tutorial Project on it.



What is a dotfile?

dotfile is a configuration file in Unix. It starts with a dot symbol (.) to make sure that default ls command output won’t show it. So it’s kind of a hidden file. You can create as many dotfiles as you like, but just like any other file, certain rules (like ownership and permissions) apply. Also, you can’t have dotfiles with the same name in the same location, for obvious reasons.

dotfile locations

Vast majority of dotfiles are located in user homedirectories. That’s mostly because there’s not much need to be hiding system wide settings – they’re usuall kept under /etc directory and have no dots or any other ways hiding the content.

For user-specific dotfiles, you’ll have them under your user homedir. For me, that’s /home/greys in Linux and /Users/greys in macOS.

Many projects create dotfiles upon first start, so that means if a group of users is using the same software, each user will have their own configuration file with individual preferences – stored in their own directory.

Examples of dotfiles

Here’s just a few of the dotfiles I have:

  • .bashrc – one of initialisaiton files for Bash
  • .bash_profile – similar file in macOS
  • .vimrc – this is the ViM editor config file
  • .gitconfig – configuration file for git
  • .tmux.conf – config file for tmux

Are the dot directories as well?

Sure thing! In fact, for many larger projects default to organising their configuration in specific hidden directories, like .vim or .ssh or .vnc

That’s it for today! Will share my dotfiles Unix Tutorial Project shortly!

See Also




Examples of Using ip command

ip command showing IPv4 addresses

You may have seen the ip command page on this website, and even used ip addr show version of it. Here’s a few more really powerful options for ip.

Show Only IPv4 Addresses with ip command

If default ip addr show (or ip a for short) is too much information:

greys@becky:~ $ ip addr show
1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether b8:27:eb:b5:fb:da brd ff:ff:ff:ff:ff:ff
inet 192.168.1.66/24 brd 192.168.1.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::ba27:ebff:feb5:fbda/64 scope link
valid_lft forever preferred_lft forever

…just specify the -4 option (short for IPv4) to only show IPv4 addresses info:

greys@becky:~ $ ip -4 addr show
1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
2: eth0: mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
inet 192.168.1.66/24 brd 192.168.1.255 scope global eth0
valid_lft forever preferred_lft forever

Show Specific Interface with ip command

To further narrow it down and show just info for one of the interfaces, specify it in the command line:

greys@becky:~ $ ip -4 addr show dev eth0
2: eth0: mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
inet 192.168.1.66/24 brd 192.168.1.255 scope global eth0
valid_lft forever preferred_lft forever

Show Brief Summary using ip command

Just specify -br option to see just bare essentials for the specified interfaces (or all of them if you’re not indicating interface name) – you’ll get interface name, status (whether it’s UP or DOWN right now) and the assigned IP address:

greys@becky:~ $ ip -br -4 addr
 lo               UNKNOWN        127.0.0.1/8
 eth0             UP             192.168.1.66/24

If I want to just show this for eth0, here’s how I do it

greys@becky:~ $ ip -br -4 addr show dev eth0
 eth0             UP             192.168.1.66/24

That’s useful enough to learn in case ifconfig command is not found or you simply want to use ip command instead of ifconfig command a bit more.

See Also




Assign Keyboard Shortcut to Screenshot in Ubuntu

I’m finding myself working on Linux laptop with Ubuntu 19.04 more often than I expected – sometimes I spend most of my day research and preparing Unix Tutorial posts in Linux instead of macOS. Today I got an opportunity to improve my screenshotting productivity a bit more.



How To Take Screenshots in Ubuntu

I tried a few screenshot apps in Ubuntu but eventually settled on the default one that comes pre-installed – it’s called Screenshot. The main reason for this choice wasn’t for its functionality (and certainly not for its performance, it’s rather slow) but because most of other options don’t support the HiDPI resolutions properly – so a quick task of screenshotting something becomes a tedious chore that defeats the purpose.

So I settled on using Screenshot app. I usually press the Start (Windows?) key on my keyboard and this brings a view of all the windows and gives me the app search window at the top of the screen:

I can then type Screenshot there and press Enter to run the app:

And then select the type of screenshot action I want to progress:

Adding Keyboard Shortcut to Screenshot

I decided to improve the things by skipping the whole “search for an app named Screenshot” part of the process and started looking into Keyboard Shortcuts settings.

Turns out, there is an even better option: I can select a shortcut to not just start Screenshot but to also select the action (screenshot current window) and save the file into Pictures directory:

The default shortcut Alt+Print is more than adequate, but on my multimedia keyboard from Logitech I don’t have Print button and must press Fn for invoke it. So I decided to change the default shortcut to Alt+S:

That’s it, I now can simply press Alt+S and get the window I’m working with automatically screenshotted (is this a word?) and placed into Pictures:

Hope you like this tip, have a great day!

See Also




Listing Full Timestamps with ls in Linux

I was reading a man page for ls command and noticed a very interesting command line option. It appears that full-time option is available in Linux, but not in MacOS. Need to check later to confirm if it’s a Linux-only or filesystem specific option.



Default ls -al Behavior

Here’s how a typical long form of ls looks:

greys@s2:~/scripts $ ls -al
 total 40
 drwxr-xr-x  4 root  root  4096 Apr 25  2018 .
 drwx------. 9 greys greys 4096 Jul 26 01:09 ..
 -rwxr-xr-x  1 root  root  1344 Apr 25  2018 backup.sh
 -rwxr-xr-x  1 root  root   762 Apr 25  2018 content_jira.sh
 -rwxr-xr-x  1 root  root  1125 Apr 25  2018 copy_content_guest.sh
 drwxr-xr-x  8 root  root  4096 Apr 25  2018 .git
 drwxr-xr-x  2 root  root  4096 Apr 25  2018 lists
 -rw-r--r--  1 root  root   340 Apr 25  2018 README.md
 -rw-r--r--  1 root  root  1732 Apr 25  2018 scanning_ports.sh
 -rwxr-xr-x  1 root  root  1413 Apr 25  2018 server-info.sh

Listing full timestamps

And this is the full timestamps:

greys@s2:~/scripts $ ls --full-time
total 28
-rwxr-xr-x 1 root root 1344 2018-04-25 13:10:21.379844332 +0100 backup.sh
-rwxr-xr-x 1 root root 762 2018-04-25 13:10:21.379844332 +0100 content_jira.sh
-rwxr-xr-x 1 root root 1125 2018-04-25 13:44:04.054711619 +0100 copy_content_guest.sh
drwxr-xr-x 2 root root 4096 2018-04-25 13:31:56.514054659 +0100 lists
-rw-r--r-- 1 root root 340 2018-04-25 13:10:21.379844332 +0100 README.md
-rw-r--r-- 1 root root 1732 2018-04-25 13:10:21.379844332 +0100 scanning_ports.sh
-rwxr-xr-x 1 root root 1413 2018-04-25 13:10:21.379844332 +0100 server-info.sh

Seems pretty cool, right?

See Also




How To Tell What Group Unix User Is In

Have been getting this question asked a lot in the search engine queries for Unix Tutorial lately, and realised in all this time I haven’t created a separate page for the groups command – one of the most basic yet most useful Unix commands ever!

groups Command in Unix

Just like most of Unix/Linux commands, groups does exactly what it says: it shows you a list of Unix groups. Specifically, it shows you groups that your own user belongs to, or lists groups of any other user you specify.

In simplest form, you simply type groups without any parameters:

greys@s2:~ $ groups
greys libvirt

On this s2 server, I’m a member of my own group called greys and also belong to libvirt group which lets me manage kvm virtualization.

Of course, you can specify your own user implicitly as well.

This is what groups I’m in on MacOS:

greys@maverick:~ $ groups greys
 staff com.apple.sharepoint.group.1 everyone localaccounts _appserverusr admin _appserveradm _lpadmin com.apple.sharepoint.group.2 com.apple.sharepoint.group.3 _appstore _lpoperator _developer _analyticsusers com.apple.access_ftp com.apple.access_screensharing com.apple.access_ssh

groups is a basic Unix command

I’ve added groups to the Basic Unix Commands page and also created it a page today: groups command. Can’t believe this truly deserving Unix command escaped the chance to be in the Basic Commands hall of fame for so long!

Other Ways To List User Groups

I’ve actually taken the time ot refresh a really old but great post of mine today: 3 Ways to List Groups for a User in Linux – go read and and let me know what you think. It talks about using id command (quite basic) and getent command (far more advanced) for listing groups.

See Also




Advice: Safely Removing a File or Directory in Unix

Unix Tutorial

Removing files and directories is a very common task, so in some environments support engineers or automation scripts delete hundreds of files per day. That’s why I think it’s important to be familiar with different ways and safety mechanisms you should use when it comes to removing Unix directories. This article has a number of principles that should help you make your day-to-day files and directories operations safer.



DISCLAIMER: one can never be too careful when using a command line, and removing files or directories in Unix is not an exception. That’s why please take extra care and spend time planning and understanding commans and command line options before executing them on production data. I’m sharing my own advice and my approach, but DO YOUR OWN RESEARCH as I accept no responsibility for any possible loss cuased by direct or indirect use of the suggested commands.

Safely Removing Files and Directories

Advice in this article is equally applicable to commands you type and to the automation solutions you create. Be it a single command line or a complex Ansible playbook – safety mindset should be applied whenever you’re creating an actionable plan for working with important data.

If you can think of any more advice related to this topic, please let me know!

1. Double-check Directories Before Removing

I wouldn’t call this out if it hasn’t saved me so many times. No matter who made the request, no matter how urgent the task is, no matter how basic and obvious the directory name seems, always double-check directories before removing!

Basic approach is: replace rm/rmdir with ls -l command.

So instead of

$ rm -rf /etc /bin

you type ls command and review the output:

$ ls -l /etc /bin

Things you’re checking for are:

  1. Is this a user/task specific directory or a global directory?
  2. Does it seem to be part of the core OS?
  3. Will removing these files break any functionality you can think of?
  4. Does the directory contain any files?
  5. Does the number of files seem different from what you expected?

For instance: you’re asked to delete an empty directory. Do a quick ls, and if it has files – double-check if they should be deleted as well. Also, check if it’s one of the common core OS directories like /etc or /bin or /var – it could be that you got the name by mistake but removing directory without checking would become an even bigger mistake.

2. Consider Moving Instead of Removing

In troubleshooting, many requests are made so that you free up a directory or tidy up filesystem structure. But the issues are mostly around file and directory names, rather than the space they take up.

So if filesystem space is not an issue right now, consider moving the directory instead of removing (deleting) it completely:

$ mv /home/greys/dir1 /home/greys/dir1.old

The end result will be that /home/greys/dir1 directory is gone, but you still have time to review and recover files from /home/greys/dir1.old if necessary.

3. Use root Privilege Wisely

Hint: don’t use root unless you absolutely have to. If the request is to remove a subdirectory in some application path – find out what user the application is running as and become that user before removing the directory.

For instance:

# su - javauser
# rm -rf /opt/java/logs/debug

Run as root user, this will let you become a javauser and attempt to remove the /opt/java/logs/debug directory (debug subdirectory in /opt/java/logs directory).

If there’s an issue (like getting permissions denied error) – review and find out what the problem is instead of just becoming root and removing the directory or files anyway. Specifically: permission denied suggests that files belong to another user or group, meaning they are potentially used and needed for something else and not just the application you’re working on.

4. Double-check Any Masks or Variables

If you’re dealing with expanding filename masks, double-check them to have correct and non-zero values.

Consider this:

$ rm -rf /${SOMEDIR}

if you’re not careful validating it, then it’s quite possible that $SOMEDIR is not initialised (or initialised under some other user session), thus resulting in the vastly different command with catastrophic results (yes, I know: this exact example below is NOT that bad, because as regular user it simply won’t work. But run as run will result in OS self-destruct):

$ rm -rf /

Similarly, if there are filenames to be expanded, verify that expansion works as intended. Very important thing to realise is that your filename masks will be expanded as your current user.

$ <SUDO> rm /$(ls /root) 
ls: cannot access '/root/': Permission denied
 rm: cannot remove '/': Is a directory

This example above is using shell expansion: it runs ls /root command that will return valid values if you have enough permissions. But run as regular use this will give an error and also alter the path used for the rm command. It will be as if you tried to run the following with sudo privileges:

$ rm /

Again, I’m not giving you the full commans as it’s all too easy to break your Unix like OS beyond repair when you run full commands as root without double-checking.

5. echo Each Command Before Running

The last principle I find very useful is to prepend any potentially dangerous command with echo. This means your shell will attempt expanding any command line parameters and substitutions, but then show the resulting command line instead of actually executing it:

greys@becky:~ $ echo rm -rf /opt/java/logs/${HOSTNAME}
rm -rf /opt/java/logs/becky

See how it expanded ${HOSTNAME} variable and replaced it with the actual hostname, becky?

Use echo just to be super sure about what you think the Unix shell will execute.

That’s it for today, hope you like this collection of safety principles. Let me know if you want more articles of this kind!

See Also

return nothing




Most Useful Basic Unix Commands

unix-tutorial

Seems the¬†Basic Unix Commands section is gaining popularity online – that’s great to see, means I’m putting focus where it’s actually needed. Today I want to share my view of the most useful basic unix commands – the ones that I use daily.

What Defines a List of Daily Used Commands

The types of commands you will need depends on the tasks and how you go about them. Chances are, most of your work is done remotely – which means one of your most used commands could be git or ssh.

Apart from this, it’s unbelievably awesome how the majority of daily work can be accomplished using very basic unix commands, nothing fancy at all.

If you’re a software developer, you’ll probably have your share of programming language interpreters and compilers along with other components of a DevOps toolchain, but even with them in the picture there’s still quite a chunk of work done using the most beatifully simple commands like cd and ls.

Daily Tasks I Accomplish with Command Line

Here’s the list of things I get to do pretty much every day:

  • navigate around filesystems
    • where am I?
    • go to a specific directory
    • is this the same or separate filesystem?
    • check how much space is there available?
  • check files and directories
    • is there a file named such-and-such?
    • what’s the size of the file?
    • who owns the file (username and group)
    • when was the file last updated?
    • what’s the contents of this file?
  • manage files and directories
    • create a new directory
    • create or edit a file
    • change username or group ownership
    • remove a directory or a file
    • copy or move a file
  • remote access¬†
    • connect to a remote server
    • copy file from my system to remote server
    • transfer files from remote server to my local system

Basic Unix Commands For Daily Use

I’ll probably show examples of command lines for these tasks in some future post, but for now will just map the basic commands to each section:

See Also




sudo chown

Unlike chmod command, chown only becomes useful if run with elevated (root) privileges. In Linux, it is most commonly used with the help of sudo command.

Why sudo is needed for chown

Basic security model in Unix is around users and groups and their ownership of various files and directories. This means that without elevated privileges (becoming root or running commands via sudo) no regular user is meant to have enough privileges to act on behalf of another user.

Example of sudo chown

Here’s how we would run chown to change ownership of my own files (belonging to user greys) to root and back:

[greys@rhel8~]$ touch file1 file2 file
[greys@rhel8~]$ ls -la file*
-rw-rw-r--. 1 greys greys 0 Jun 19 12:43 file1
-rw-rw-r--. 1 greys greys 0 Jun 19 12:43 file2
-rw-rw-r--. 1 greys greys 0 Jun 19 12:43 file3

Without sudo our chown attempt will fail:

[greys@rhel8 ~]$ chown root file*
chown: changing ownership of 'file1': Operation not permitted
chown: changing ownership of 'file2': Operation not permitted
chown: changing ownership of 'file3': Operation not permitted

…but with sudo it does the trick:

[greys@rhel8 ~]$ sudo chown root file*
[sudo] password for greys: 
[greys@rhel8 ~]$ ls -la file*
-rw-rw-r--. 1 root greys 0 Jun 19 12:43 file1
-rw-rw-r--. 1 root greys 0 Jun 19 12:43 file2
-rw-rw-r--. 1 root greys 0 Jun 19 12:43 file3

That’s all I wanted to share today. Have fun!

See Also