I was reading a man page for ls command and noticed a very interesting command line option. It appears that full-time option is available in Linux, but not in MacOS. Need to check later to confirm if it's a Linux-only or filesystem specific option.[Read more…] about Listing Full Timestamps with ls in Linux
I've just started working on the Linux Commands section of Unix Tutorial, and dmidecode is one of the best commands to mention when it comes to recent Linux distros. It's found in most distributions and helps with learning lots of useful facts about your environments – both physical and virtual.
How To Use dmidecode Command
dmidecode command needs to be run as root and shows you hardware information about your system:
- BIOS name and version
- manufacturer of your server, desktop or laptop
- model name and serial number of your system
greys@xps:~ $ sudo dmidecode | less # dmidecode 3.2 Getting SMBIOS data from sysfs. SMBIOS 3.2.1 present. # SMBIOS implementations newer than version 3.2.0 are not # fully supported by this version of dmidecode. Table at 0x000E0000. Handle 0x0000, DMI type 0, 26 bytes BIOS Information Vendor: Dell Inc. Version: 1.2.1 Release Date: 02/14/2019 Address: 0xF0000 Runtime Size: 64 kB ROM Size: 32 MB ...
Browsing dmidecode output and searching through the output are the best ways to use dmidecode command, but once you become familiar with your environment you'll probably get a few dmidecode parameters you can grep for.
Hardware Vendor with dmidecode
This will report the manufacturer of your system. For my XPS laptop, it shows Dell:
greys@xps:~ $ sudo dmidecode | grep Vendor Vendor: Dell Inc. Vendor ID:
Model name with dmidecode
Look for the Product Name to confirm the name of your system. It shows my laptop's model for me:
greys@xps:~ $ sudo dmidecode | grep Product Product Name: XPS 13 9380 Product Name: 0KTW76
Motherboard with dmidecode
Finding the motherboard model will require you to search through the less pager output (press / and start typing word motherboard, then scroll up and down).
Here's what one of my dedicated servers shows:
Handle 0x0002, DMI type 2, 15 bytes Base Board Information Manufacturer: Supermicro Product Name: X11SSE-F Version: 1.01 Serial Number: ZM163S009892 Asset Tag: To be filled by O.E.M. Features: Board is a hosting board Board is replaceable Location In Chassis: To be filled by O.E.M. Chassis Handle: 0x0003 Type: Motherboard Contained Object Handles: 0
Serial numbers with dmidecode
Just grep for the word Serial to find lots of serial numbers of various recognised devices. One of them (the first one in the output) is the Dell's service tag that you usually need for hardware support:
greys@xps:~ $ sudo dmidecode | grep Serial Serial services are supported (int 14h) Serial Number: 50G8V** Serial Number: /50G8V**/CN***00***00F8/ Serial Number: 50G8V** Serial Number: Not Specified Serial Number: Not Specified Serial Number: To Be Filled By O.E.M. Serial Number: 0A3E Debug Use USB(Disabled:Serial)
There's lots of other things dmidecode is useful for – I'll be sure to update the dmidecode command page going forward.
I didn't create any swap during initial Ubuntu 19.04 install on Dell XPS laptop, so had to improvise when compiling a massive opensource project recently. This post shows you how to temporarily add swap memory using a regular file.
Create swap Using Regular File
The usual approach is fairly simple:
- you create a large enough file (the desired size of your swap memory) – 1GB or 4GB or something like that
- you initialise it as swap
- you activated it as swap
Swap Files Support in BTRFS
Since I've used BTRFS filesystem for root in my Ubuntu setup, I discovered that there are additional steps needed: BTRFS filesystem creates and writes files in a way that's been mostly incompatible with swap usage.
But as luck would have it, BTRFS supports swap files with Linux Kernel 5.0.x, so you just need to create file with specific attribute.
Swap Space Using File Procedure
Step 1: Create new empty file
We need to create a file with NOCOW (NO Copy-On-Write) property (no harm anywhere but is a requirement for BTRFS):
root@xps:/ # touch /swapfile root@xps:/ # chattr +C /swapfile
Step 2: Allocate required amount of space to the file
root@xps:/ # fallocate -l 8G /swapfile
This file is 8GB now:
root@xps:/ # ls -ald /swapfile -rw------- 1 root root 8589934592 Jul 17 19:55 /swapfile
Step 3: Update permissions
root@xps:/ # chmod 0600 /swapfile
Step 4: Initialise /swapfile as swap storage
root@xps:/ # mkswap /swapfile Setting up swapspace version 1, size = 8 GiB (8589930496 bytes) no label, UUID=16d35c04-78de-4dd3-aeb0-e2228bb7ce36
Step 5: Activate swap space from /swapfile
root@xps:/ # swapon /swapfile
Step 6: Confirm newly activated swap space
root@xps:/ # free -h total used free shared buff/cache available Mem: 15Gi 6.1Gi 1.7Gi 3.1Gi 7.5Gi 5.8Gi Swap: 8.0Gi 0B 8.0Gi
That's it for today!
AppArmor is a Linux Kernel security module that implements mandatory access control (MAC) security with per-application profiles in Debian based systems. It's possible to confirm if AppArmor is enabled in your Debian or Ubuntu system and to also find out the mode it's running in.
AppArmor Status with aa-status Command
aa-status command will list the currently loaded AppArmor modules.
For instance, here's how it looks on a system where AppArmor is inactive (Debian 9 in my case):
root@debian9:~# aa-status apparmor module is loaded. apparmor filesystem is not mounted.
And here is how AppArmor status is reported on Debian 10 system where it's activated by default:
root@debian10:~# aa-status apparmor module is loaded. 20 profiles are loaded. 18 profiles are in enforce mode. /usr/bin/evince /usr/bin/evince-previewer /usr/bin/evince-previewer//sanitized_helper /usr/bin/evince-thumbnailer /usr/bin/evince//sanitized_helper /usr/bin/man /usr/lib/telepathy/mission-control-5 /usr/lib/telepathy/telepathy-* /usr/lib/telepathy/telepathy-*//pxgsettings /usr/lib/telepathy/telepathy-*//sanitized_helper /usr/lib/telepathy/telepathy-ofono libreoffice-senddoc libreoffice-soffice//gpg libreoffice-xpdfimport man_filter man_groff nvidia_modprobe nvidia_modprobe//kmod 2 profiles are in complain mode. libreoffice-oopslash libreoffice-soffice 0 processes have profiles defined. 0 processes are in enforce mode. 0 processes are in complain mode. 0 processes are unconfined but have a profile defined.
Debian 10 "Buster" got released ealier this week, culminating work of the past 2 years. It's a massive upgrade to Debian 9 and a solid release available for 10 architectures (x86/x64/AMD64 and arm64/amrhf plus MIPS, PowerPC and IBM System Z.
I'm away from home for the past 2 weeks but will attempt distro upgrade in my Debian 9 VM, so stay tuned.
Notable Improvements in Debian 10 Buster
Secure Boot is finally here in Debian 10!
This means Debian 10 comes with signed and fully trusted version of GRUB boot loader that is activated by the Shim loader.
Secure Boot is a cool security feature found on most modern PCs and laptops, basically to ensure that trusted (secure) code is loaded by UEFI (Unified Extensible Firmware Interface) before Operating System.
AppArmor is Activated by Default
This means at least the core OS functionality will run with improved security. You'll need to disable AppArmor using GRUB options if you want to continue running OS without AppArmor.
nftables is the Default Packet Filter
nftables tool will help with managing both IPv4 and IPv6 packets. I think iptables compatibility is there, meaning you can run iptables commands with common syntax – but implementation will actually be done using nftables.
Linux Kernel Updated to 4.19
Not quite the bleeding edge (I hear Linux Kernel 5.2 released recently), but that's exactly the point: Debian always prides itself on being one of the most stable and reliable distros, meaning it won't have latest features but will ensure that everything avaialbe will work as expected.
Now and then you may notice that apt-get upgrade command keeps a few packages back, meaning they don't get upgraded. This quick post shows what you can do about it and how to get all the packages upgraded.
How apt-get Keeps Packages Back
This is how keeping packages back will look like:
greys@xps:~ $ sudo apt-get upgrade Reading package lists... Done Building dependency tree Reading state information... Done Calculating upgrade... Done The following packages have been kept back: linux-generic linux-headers-generic linux-image-generic 0 to upgrade, 0 to newly install, 0 to remove and 3 not to upgrade.
Why Packages Are Kept Back by apt-get
Most likely reason for keeping packages back is that upgrading them means installing new packages or removing existing ones. apt-get upgrade strictly upgrades existing packages, without removing or installing anything.
Since the command you're giving to apt-get is upgrade and not to install new packages, packages that require some old packages removed or new packages installed are kept back.
How To Upgrade Packages That Were Kept Back
Simply use the dist-upgrade option of apt-get, which will resolve dependencies and install/remove dependent package as needed:
greys@xps:~ $ sudo apt-get dist-upgrade Reading package lists... Done Building dependency tree Reading state information... Done Calculating upgrade... Done The following NEW packages will be installed linux-headers-5.0.0-20 linux-headers-5.0.0-20-generic linux-image-5.0.0-20-generic linux-modules-5.0.0-20-generic linux-modules-extra-5.0.0-20-generic The following packages will be upgraded: linux-generic linux-headers-generic linux-image-generic 3 to upgrade, 5 to newly install, 0 to remove and 0 not to upgrade. Need to get 67.0 MB of archives. After this operation, 334 MB of additional disk space will be used. Do you want to continue? [Y/n] y
That's it for todat, enjoy!
It appears there's a long-standing malfunction of various microSD card readers running Linux. In my particular case, the issue happens on XPS 13 9380 laptop running latest Ubuntu 19.04 with all the updates as of early July 2019. I'll update this post once I confirm the fix.
Card Reader Device on Dell XPS 13 9380
I believe this is the device I have have:
root@xps:~ # lspci | grep -i reader 01:00.0 Unassigned class [ff00]: Realtek Semiconductor Co., Ltd. RTS525A PCI Express Card Reader (rev 01)
mmc0: error -110 whilest initialising SD card
The error message is a bit strange: I'm not trying to initialise my SD card, but instead want to read it. It's a pretty standard 128GB microSD by SanDisk, but I think part of the problem is that it's a high-speed SDXC card and the issue is that card reader can't support the card because it's running on slower speeds by default.
Here's how the error looks:
Jul 2 14:02:43 xps kernel: [18743.768947] mmc0: error -110 whilst initialising SD card Jul 2 14:02:44 xps kernel: [18745.108865] mmc0: error -110 whilst initialising SD card Jul 2 14:02:46 xps kernel: [18746.452902] mmc0: error -110 whilst initialising SD card
Reloading SDHCI Kernel Module with debug_quirks
One of the common fixes for the problem is to reload kernel module sdhci with debug parameters that assist with improved voltage required for higher speeds.
Unfortunately, this fix didn't work for me:
$ sudo modprobe sdhci debug_quirks2="0x80000000"
Syslog reports that module has been reloaded:
Jul 06 12:22:01 xps kernel: sdhci: Secure Digital Host Controller Interface driver Jul 06 12:22:01 xps kernel: sdhci: Copyright(c) Pierre Ossman
… but when I insert the code I still get the same error:
Jul 06 12:24:43 xps kernel: mmc0: error -110 whilst initialising SD card
Jul 06 12:24:45 xps kernel: mmc0: error -110 whilst initialising SD card
Jul 06 12:24:46 xps kernel: mmc0: error -110 whilst initialising SD card
I'm glad I also have an external card reader with USB-C interface, it works just fine with perfect access to the same microSD card. But ideally I want to fix this issue for the build-in card reader.
- Unix Tutorial Projects
- Projects: Install Ubuntu 19.04 on Dell XPS 13 9380
- Keyboard Backlight in Linux on Dell XPS laptop
CentOS and RedHat Linux are still the majority of my Linux servers and so now and then I have a RedHat specific question to investigate. This time around, I've explored getting the list of installed packages using yum command.
yum list installed
As hard as it may be to believe it, the actual command I needed is this:
[greys@rhel8 ~]$ yum list installed
That's right – type it word for word and yum will report the full list of packages installed in your system along with package versions and package group names.
Here's what Red Hat Enterprise Linux 8 beta VM reports:
[greys@rhel8 ~]$ yum list installed | more Not root, Subscription Management repositories not updated 2018-10-28 13:33:38,137 [WARNING] yum:31323:MainThread @logutil.py:141 - logging already initialized Not root, Subscription Management repositories not updated Installed Packages GConf2.x86_64 3.2.6-22.el8 @rhel-8-for-x86_64-appstream-beta-rpms ModemManager.x86_64 1.8.0-1.el8 @rhel-8-for-x86_64-baseos-beta-rpms ModemManager-glib.x86_64 1.8.0-1.el8 @rhel-8-for-x86_64-baseos-beta-rpms NetworkManager.x86_64 1:1.14.0-5.el8 @anaconda NetworkManager-adsl.x86_64 1:1.14.0-5.el8 @rhel-8-for-x86_64-baseos-beta-rpms NetworkManager-bluetooth.x86_64 1:1.14.0-5.el8 @rhel-8-for-x86_64-baseos-beta-rpms NetworkManager-libnm.x86_64 1:1.14.0-5.el8 @anaconda NetworkManager-ovs.x86_64 1:1.14.0-5.el8 @rhel-8-for-x86_64-baseos-beta-rpms NetworkManager-team.x86_64 1:1.14.0-5.el8 @anaconda NetworkManager-tui.x86_64 1:1.14.0-5.el8 @anaconda NetworkManager-wifi.x86_64 1:1.14.0-5.el8 @rhel-8-for-x86_64-baseos-beta-rpms NetworkManager-wwan.x86_64 1:1.14.0-5.el8 @rhel-8-for-x86_64-baseos-beta-rpms PackageKit.x86_64 1.1.10-6.el8 @rhel-8-for-x86_64-appstream-beta-rpms PackageKit-command-not-found.x86_64 1.1.10-6.el8 @rhel-8-for-x86_64-appstream-beta-rpms PackageKit-glib.x86_64 1.1.10-6.el8 @rhel-8-for-x86_64-appstream-beta-rpms PackageKit-gstreamer-plugin.x86_64 1.1.10-6.el8 @rhel-8-for-x86_64-appstream-beta-rpms PackageKit-gtk3-module.x86_64 1.1.10-6.el8 @rhel-8-for-x86_64-appstream-beta-rpms abattis-cantarell-fonts.noarch 0.0.25-4.el8 @rhel-8-for-x86_64-appstream-beta-rpms accountsservice.x86_64 0.6.50-5.el8 @rhel-8-for-x86_64-appstream-beta-rpms accountsservice-libs.x86_64 0.6.50-5.el8 @rhel-8-for-x86_64-appstream-beta-rpms acl.x86_64 2.2.53-1.el8 @anaconda adcli.x86_64 0.8.2-2.el8 @rhel-8-for-x86_64-baseos-beta-rpms adobe-mappings-cmap.noarch 20171205-3.el8 @rhel-8-for-x86_64-appstream-beta-rpms adobe-mappings-cmap-deprecated.noarch 20171205-3.el8 @rhel-8-for-x86_64-appstream-beta-rpms ...
Grep yum list installed using group name
The output makes is very easy to grep for packages that belong to the same software group, like rhel-8-for-x86_64-baseos-beta-rpms in this example:
[greys@rhel8 ~]$ yum list installed | grep rhel-8-for-x86_64-baseos-beta-rpms | more 2018-10-28 13:40:14,740 [WARNING] yum:31405:MainThread @logutil.py:141 - logging already initialized ModemManager.x86_64 1.8.0-1.el8 @rhel-8-for-x86_64-baseos-beta-rpms ModemManager-glib.x86_64 1.8.0-1.el8 @rhel-8-for-x86_64-baseos-beta-rpms NetworkManager-adsl.x86_64 1:1.14.0-5.el8 @rhel-8-for-x86_64-baseos-beta-rpms NetworkManager-bluetooth.x86_64 1:1.14.0-5.el8 @rhel-8-for-x86_64-baseos-beta-rpms NetworkManager-ovs.x86_64 1:1.14.0-5.el8 @rhel-8-for-x86_64-baseos-beta-rpms NetworkManager-wifi.x86_64 1:1.14.0-5.el8 @rhel-8-for-x86_64-baseos-beta-rpms NetworkManager-wwan.x86_64 1:1.14.0-5.el8 @rhel-8-for-x86_64-baseos-beta-rpms adcli.x86_64 0.8.2-2.el8 @rhel-8-for-x86_64-baseos-beta-rpms at.x86_64 3.1.20-11.el8 @rhel-8-for-x86_64-baseos-beta-rpms attr.x86_64 2.4.48-3.el8 @rhel-8-for-x86_64-baseos-beta-rpms augeas-libs.x86_64 1.10.1-3.el8 @rhel-8-for-x86_64-baseos-beta-rpms avahi.x86_64 0.7-18.el8 @rhel-8-for-x86_64-baseos-beta-rpms avahi-glib.x86_64 0.7-18.el8 @rhel-8-for-x86_64-baseos-beta-rpms avahi-libs.x86_64 0.7-18.el8 @rhel-8-for-x86_64-baseos-beta-rpms bash-completion.noarch 1:2.7-5.el8 @rhel-8-for-x86_64-baseos-beta-rpms bc.x86_64 1.07.1-5.el8 @rhel-8-for-x86_64-baseos-beta-rpms binutils.x86_64 2.30-49.el8 @rhel-8-for-x86_64-baseos-beta-rpms blktrace.x86_64 1.2.0-9.el8 @rhel-8-for-x86_64-baseos-beta-rpms bluez.x86_64 5.50-1.el8 @rhel-8-for-x86_64-baseos-beta-rpms bluez-libs.x86_64 5.50-1.el8 @rhel-8-for-x86_64-baseos-beta-rpms bluez-obexd.x86_64 5.50-1.el8 @rhel-8-for-x86_64-baseos-beta-rpms bolt.x86_64 0.4-1.el8 @rhel-8-for-x86_64-baseos-beta-rpms bpftool.x86_64 4.18.0-32.el8 @rhel-8-for-x86_64-baseos-beta-rpms bubblewrap.x86_64 0.3.0-1.el8 @rhel-8-for-x86_64-baseos-beta-rpms
That's it for today!
This week's Unix Tutorial Project is super geeky and fun: I'm setting up text-based email archive system using Mutt (NeoMutt, actually), OfflineIMAP and hopefully NotMuch. Will publish a project summary on the weekend.
Why use OfflineIMAP
OfflineIMAP tool is an open-source tool for downloading your email messages and storing them locally in a Maildir format (meaning each email message is stored in a separate file, each folder/GMail tag is a separate directory).
As the name suggests, this tool's primary objective is to let you read your emails offline. Contrary to the other part of the name, offlineimap is NOT an IMAP server implementation.
I'd like to explore OfflineIMAP/Neomutt setup as a backup/archive solution for my cloud email accounts. I used to be with Fastmail but switched to gSuite email last year. I think it's very important to keep local copies of any information you have in any cloud – no matter how big/reliable the service provider is, there are many scenarios where your data could be completely lost, and responsibility for keeping local backups is always with you.
Both gMail and Fastmail solutions are perfect for web browser use but any local email software is invariably bulkier and slower compared to web interface. I'm not giving up on finding the acceptably performance and reliable solution though.
This is one of the most recent attempts to download all emails and to have them easily searchable on my local PCs and laptops.
OfflineIMAP Configuration Steps
I'm only learning this tool, so this is probably the most basic usage:
- Confirm your mail server details (IMAP)
- Confirm your mailbox credentials (for Google, gSuite and even Fastmail you need to generate an app password – it's separate and different from your primary email password)
- Create .offlineimaprc file in your home directory as shown below
- If necessary, create credentials file (for now – with cleartext app password for email access) – mine is /home/greys/.creds/techstack.pass
- Run offlineimap (first time and every time you want your email refreshed)
My .offlineimaprc file
Here's what I have in my .offlineimaprc file for this experiment:
[general] ui = ttyui accounts = techstack [Account techstack] localrepository = techstack-local remoterepository = techstack-remote [Repository techstack-local] type = Maildir localfolders = ~/Mail/techstack/ [Repository techstack-remote] type = Gmail remoteuser = firstname.lastname@example.org remotepassfile = ~/.creds/techstack.pass maxconnections = 5 ssl = yes sslcacertfile = /etc/ssl/certs/ca-certificates.crt folderfilder = lambda foldername: foldername not in ['Archive'] expunge = no
You can have multiple accounts in this one config file, they'll be listed in the accounts section (accounts = techstack, unixtutorial would mean 2 accounts: techstack one and one for my Unix Tutorial email).
localfolders parameter specifies that I want OfflineIMAP to create a Mail directory in my homedir (so ) and then techstack subdirectory there – meaning you can have account subidrectories there like /home/greys/Mail/techstack and /home/greys/Mail/personal, etc.
You define two repositories, local and remote one. The task of OfflineIMAP is to sync the two.
IMPORTANT: The really important parameter is maxconnections one. Default is 3 and I've changed it to 5 for quicker email sync. Setting it to a higher value resulted in failures – probably because Google servers rate limit my connection.
CRITICAL: expunge parameter is set to yes by default, so you must set it to no if your plan is to keep emails on the mail server after you sync them. By default they will be removed from the server as soon as they are downloaded, meaning Gmail app won't see any messages. Once deleted, it will be rather tricky to restore all the emails – so it's important to get this setting right from the very start. Since my primary usage is still web and Gmail app based, I certainly want all my emails to stay in Google cloud even after I download them using OfflineIMAP – that's why I configured it as expunge = no.
As you can see, this config references the /home/greys/.creds/techstack.pass file. This file has an clear-text application password I generated for my email address in gSuite admin panel. My understanding is that this can be improved, so I'll do a follow-up post later.
How To Use OfflineIMAP
Simply run the offlineimap command and you should see something like this:
greys@xps:~ $ offlineimap OfflineIMAP 7.2.2 Licensed under the GNU GPL v2 or any later version (with an OpenSSL exception) imaplib2 v2.57 (system), Python v2.7.16, OpenSSL 1.1.1b 26 Feb 2019 Account sync techstack: *** Processing account techstack Establishing connection to imap.gmail.com:993 (techstack-remote) Folder 2016 [acc: techstack]: Syncing 2016: Gmail -> Maildir Folder 2016/01-January [acc: techstack]: Syncing 2016/01-January: Gmail -> Maildir Folder 2016/02-February [acc: techstack]: Syncing 2016/02-February: Gmail -> Maildir Folder 2016/01-January [acc: techstack]:
As you can see, it processes account techstack, connects to gmail and starts processing remote folders (gmail tags) like 2016, 2016/01-January, 2016-02-February etc – these are the tags I have in my gSuite account.
Initial download would take a while. My 150K messages took almost 3 days to download.
That's all for today, hope you give OfflineIMAP a try!
One of the remaining things to fix on my new Dell XPS 13 laptop has been the graphics mode in GRUB bootloader that got activated with my Ubuntu 19.04 install. Somehow GRUB is smart enough to recognise 4K resolution on the laptop, so the GRUB boot menu looks so tiny that I can't read any text (there's no scaling applied to fonts). I finally decided to fix this.
Graphics modes in GRUB bootloader
GRUB is a simple enough software solution that traditionally used text console for presenting boot menu. In the last few years it introduced graphics mode: you still see a text menu with boot options, but they're rendered in a graphics mode rather than shown in text mode.
Turns out, there's a special option in /boot/grub/grub.cfg file that allows you to select a graphics resolution:
Change graphics mode for GRUB
To update this value properly, I suggest you edit the GRUB_GFXMODE in /etc/default/grub file:
IMPORTANT: 1920×1080 mode is NOT supported, so don't specify it. 1024×768 is a safe resolution that should be available on most hardware systems. I'll write another post soon expanding on GRUB bootloader graphics resolutions topic.
Once this is done, re-build all the grub configuration files:
$ sudo grub-update
To verify that our resolution of 1024×768 made it into the config, grep for it:
greys@xps:~ $ grep 1024 /boot/grub/grub.cfg set gfxmode=1024x768
That's it, you can reboot your PC or laptop now to enjoy a different resolution.