One of the things still left to be done on my Ubuntu laptop is to get the full resolution output on my LG 5K monitor. This is still work in progress, but one of the most useful tools for getting there is xrandr.[Read more…] about Show Monitor Resolutions with xrandr
I'm finding myself working on Linux laptop with Ubuntu 19.04 more often than I expected – sometimes I spend most of my day research and preparing Unix Tutorial posts in Linux instead of macOS. Today I got an opportunity to improve my screenshotting productivity a bit more.[Read more…] about Assign Keyboard Shortcut to Screenshot in Ubuntu
Quite often you're clear on what your Ansible playbook should be doing and perhaps even know the syntax is correct, but there are still concerns about the implications. In such cases it's best to run your Ansible playbook in dry run mode: preview changes without actually making them.
How To Preview Ansible Playbook Run
Simply add the –check parameter to the end of your ansible-playbook command:
$ ansible-playbook try.yaml --check
This will show you exactly the changes the playbook would have made, but without actually making them.
Ansible would even go as far as computing the necessary changes as if you made them, lining up variable values etc. It would also show you what will get changed as if it's already changed – but without any risks.
That's it for today, this is a very short note but I'll come back and revisit it soon to provide examples.
This is not the most obvious functionality, hence I decided to share it as a separate post. It's quite easy and perfectly acceptable to specify more than one SSH port for your sshd daemon – useful for debugging or added security (when bound to separate IP addresses).
Adding Extra SSH ports
Simply edit the /etc/ssh/sshd_config file and add more port numbers under the existing default port (it's commented out because 22 is used by default):
greys@server:~$ sudo vi /etc/ssh/sshd_config
IMPORTANT: you must uncomment Port 22, otherwise new ports will be the only SSH ports listened on (so SSH port 22 will stop working).
Now restart ssh:
greys@server:~$ sudo systemctl restart ssh
Confirm each new SSH port
netstat command with grep confirms that all 3 ports are being listened on now:
greys@server:~$ netstat -nal | grep 22
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:221 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:222 0.0.0.0:* LISTEN
If we want to, we can even try connecting to a non-standard ssh port like 221 or 222 as per our changes.
Don't be alarmed about warning:
root@server:~# ssh greys@localhost -p 222
The authenticity of host '[localhost]:222 ([127.0.0.1]:222)' can't be established.
ECDSA key fingerprint is SHA256:12efZx1MOEmlxQOWKhM5eaxDwJr4vUlLhcpElkGHTow.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '[localhost]:222' (ECDSA) to the list of known hosts.
Welcome to Ubuntu 19.04 (GNU/Linux 4.18.0-20-generic x86_64)
Hope you enjoy this advice, stay tuned for more!
You're probably familiar with tune2fs command, it's great for reporting filesystem layout and configuration options. It's also helpful when you want to free up some of the reserved space.
Reserved Space in Unix/Linux Filesystems
Reserved space is one of the more mysterious things in Linux/Unix filesystems. A rather old tradition, it is simply a practice of automatically reserving some filesystem space for system (superuser) use. When you're formatting a new filesytem, it automatically reserves 5% of space for superuser access – meaning regular users and processes won't be able to use this space (filesystem will report to be 100% full), but root user can still write and troubleshoot.
Reserved Space meant a lot more convenience when hard disks and filesystems were small – about 20 years ago your / or /var filesystem could be 1GB in total, and this meant it was absolutely crucial to reserve some space so that regular users could not cause an outage by generating some temporary file and filling up a key filesystem. These days sizes of filesystem are vastly larger and this means two things:
- You probably don't benefit from reserved space as much as someone used to
- You may well never even know you had space reserved because you simply don't run out of space
Here's how this would look:
root@xps:/storage # df -h /storage Filesystem Size Used Avail Use% Mounted on /dev/nvme0n1p8 95G 90G 232M 100% /storage
Doesn't quite add up, does it? It's a 95GB filesystem with only 90GB used, yet filesystem is 100% full.
Change Reserved Space Percentage with tune2fs
Let's use tune2fs command to reduce the percentage of reserved space (-m 1 means set reserve to 1% of total capacity):
root@xps:/storage # tune2fs -m 1 /dev/nvme0n1p8 tune2fs 1.44.6 (5-Mar-2019) Setting reserved blocks percentage to 1% (254279 blocks)
… and enjoy the results:
root@xps:/storage # df -h /storage Filesystem Size Used Avail Use% Mounted on /dev/nvme0n1p8 95G 90G 4.2G 96% /storage
That's it for today, hope you learned something new!
- Advanced Unix commands
- tune2fs command
- Unix Commands
- Basic Unix commands
- How To: change filesystem label with tune2fs
AppArmor is a Linux Kernel security module that implements mandatory access control (MAC) security with per-application profiles in Debian based systems. It's possible to confirm if AppArmor is enabled in your Debian or Ubuntu system and to also find out the mode it's running in.
AppArmor Status with aa-status Command
aa-status command will list the currently loaded AppArmor modules.
For instance, here's how it looks on a system where AppArmor is inactive (Debian 9 in my case):
root@debian9:~# aa-status apparmor module is loaded. apparmor filesystem is not mounted.
And here is how AppArmor status is reported on Debian 10 system where it's activated by default:
root@debian10:~# aa-status apparmor module is loaded. 20 profiles are loaded. 18 profiles are in enforce mode. /usr/bin/evince /usr/bin/evince-previewer /usr/bin/evince-previewer//sanitized_helper /usr/bin/evince-thumbnailer /usr/bin/evince//sanitized_helper /usr/bin/man /usr/lib/telepathy/mission-control-5 /usr/lib/telepathy/telepathy-* /usr/lib/telepathy/telepathy-*//pxgsettings /usr/lib/telepathy/telepathy-*//sanitized_helper /usr/lib/telepathy/telepathy-ofono libreoffice-senddoc libreoffice-soffice//gpg libreoffice-xpdfimport man_filter man_groff nvidia_modprobe nvidia_modprobe//kmod 2 profiles are in complain mode. libreoffice-oopslash libreoffice-soffice 0 processes have profiles defined. 0 processes are in enforce mode. 0 processes are in complain mode. 0 processes are unconfined but have a profile defined.
CentOS and RedHat Linux are still the majority of my Linux servers and so now and then I have a RedHat specific question to investigate. This time around, I've explored getting the list of installed packages using yum command.
yum list installed
As hard as it may be to believe it, the actual command I needed is this:
[greys@rhel8 ~]$ yum list installed
That's right – type it word for word and yum will report the full list of packages installed in your system along with package versions and package group names.
Here's what Red Hat Enterprise Linux 8 beta VM reports:
[greys@rhel8 ~]$ yum list installed | more Not root, Subscription Management repositories not updated 2018-10-28 13:33:38,137 [WARNING] yum:31323:MainThread @logutil.py:141 - logging already initialized Not root, Subscription Management repositories not updated Installed Packages GConf2.x86_64 3.2.6-22.el8 @rhel-8-for-x86_64-appstream-beta-rpms ModemManager.x86_64 1.8.0-1.el8 @rhel-8-for-x86_64-baseos-beta-rpms ModemManager-glib.x86_64 1.8.0-1.el8 @rhel-8-for-x86_64-baseos-beta-rpms NetworkManager.x86_64 1:1.14.0-5.el8 @anaconda NetworkManager-adsl.x86_64 1:1.14.0-5.el8 @rhel-8-for-x86_64-baseos-beta-rpms NetworkManager-bluetooth.x86_64 1:1.14.0-5.el8 @rhel-8-for-x86_64-baseos-beta-rpms NetworkManager-libnm.x86_64 1:1.14.0-5.el8 @anaconda NetworkManager-ovs.x86_64 1:1.14.0-5.el8 @rhel-8-for-x86_64-baseos-beta-rpms NetworkManager-team.x86_64 1:1.14.0-5.el8 @anaconda NetworkManager-tui.x86_64 1:1.14.0-5.el8 @anaconda NetworkManager-wifi.x86_64 1:1.14.0-5.el8 @rhel-8-for-x86_64-baseos-beta-rpms NetworkManager-wwan.x86_64 1:1.14.0-5.el8 @rhel-8-for-x86_64-baseos-beta-rpms PackageKit.x86_64 1.1.10-6.el8 @rhel-8-for-x86_64-appstream-beta-rpms PackageKit-command-not-found.x86_64 1.1.10-6.el8 @rhel-8-for-x86_64-appstream-beta-rpms PackageKit-glib.x86_64 1.1.10-6.el8 @rhel-8-for-x86_64-appstream-beta-rpms PackageKit-gstreamer-plugin.x86_64 1.1.10-6.el8 @rhel-8-for-x86_64-appstream-beta-rpms PackageKit-gtk3-module.x86_64 1.1.10-6.el8 @rhel-8-for-x86_64-appstream-beta-rpms abattis-cantarell-fonts.noarch 0.0.25-4.el8 @rhel-8-for-x86_64-appstream-beta-rpms accountsservice.x86_64 0.6.50-5.el8 @rhel-8-for-x86_64-appstream-beta-rpms accountsservice-libs.x86_64 0.6.50-5.el8 @rhel-8-for-x86_64-appstream-beta-rpms acl.x86_64 2.2.53-1.el8 @anaconda adcli.x86_64 0.8.2-2.el8 @rhel-8-for-x86_64-baseos-beta-rpms adobe-mappings-cmap.noarch 20171205-3.el8 @rhel-8-for-x86_64-appstream-beta-rpms adobe-mappings-cmap-deprecated.noarch 20171205-3.el8 @rhel-8-for-x86_64-appstream-beta-rpms ...
Grep yum list installed using group name
The output makes is very easy to grep for packages that belong to the same software group, like rhel-8-for-x86_64-baseos-beta-rpms in this example:
[greys@rhel8 ~]$ yum list installed | grep rhel-8-for-x86_64-baseos-beta-rpms | more 2018-10-28 13:40:14,740 [WARNING] yum:31405:MainThread @logutil.py:141 - logging already initialized ModemManager.x86_64 1.8.0-1.el8 @rhel-8-for-x86_64-baseos-beta-rpms ModemManager-glib.x86_64 1.8.0-1.el8 @rhel-8-for-x86_64-baseos-beta-rpms NetworkManager-adsl.x86_64 1:1.14.0-5.el8 @rhel-8-for-x86_64-baseos-beta-rpms NetworkManager-bluetooth.x86_64 1:1.14.0-5.el8 @rhel-8-for-x86_64-baseos-beta-rpms NetworkManager-ovs.x86_64 1:1.14.0-5.el8 @rhel-8-for-x86_64-baseos-beta-rpms NetworkManager-wifi.x86_64 1:1.14.0-5.el8 @rhel-8-for-x86_64-baseos-beta-rpms NetworkManager-wwan.x86_64 1:1.14.0-5.el8 @rhel-8-for-x86_64-baseos-beta-rpms adcli.x86_64 0.8.2-2.el8 @rhel-8-for-x86_64-baseos-beta-rpms at.x86_64 3.1.20-11.el8 @rhel-8-for-x86_64-baseos-beta-rpms attr.x86_64 2.4.48-3.el8 @rhel-8-for-x86_64-baseos-beta-rpms augeas-libs.x86_64 1.10.1-3.el8 @rhel-8-for-x86_64-baseos-beta-rpms avahi.x86_64 0.7-18.el8 @rhel-8-for-x86_64-baseos-beta-rpms avahi-glib.x86_64 0.7-18.el8 @rhel-8-for-x86_64-baseos-beta-rpms avahi-libs.x86_64 0.7-18.el8 @rhel-8-for-x86_64-baseos-beta-rpms bash-completion.noarch 1:2.7-5.el8 @rhel-8-for-x86_64-baseos-beta-rpms bc.x86_64 1.07.1-5.el8 @rhel-8-for-x86_64-baseos-beta-rpms binutils.x86_64 2.30-49.el8 @rhel-8-for-x86_64-baseos-beta-rpms blktrace.x86_64 1.2.0-9.el8 @rhel-8-for-x86_64-baseos-beta-rpms bluez.x86_64 5.50-1.el8 @rhel-8-for-x86_64-baseos-beta-rpms bluez-libs.x86_64 5.50-1.el8 @rhel-8-for-x86_64-baseos-beta-rpms bluez-obexd.x86_64 5.50-1.el8 @rhel-8-for-x86_64-baseos-beta-rpms bolt.x86_64 0.4-1.el8 @rhel-8-for-x86_64-baseos-beta-rpms bpftool.x86_64 4.18.0-32.el8 @rhel-8-for-x86_64-baseos-beta-rpms bubblewrap.x86_64 0.3.0-1.el8 @rhel-8-for-x86_64-baseos-beta-rpms
That's it for today!
I noticed that it's been a while since I upgraded my Raspberry Pi systems. I have updated Raspberry Pi firmware on all of them recently enough, but now decided to upgrade distro. Since Raspbian OS is based on Debian released, it meant I would have to upgrade Raspbian from Jessie base to Stretch.
Step 1: Update/upgrade existing distribution
This simply means we want to upgrade all existing packages before we'll be moving to the next releast.
Refresh package info:
greys@s7:~ $ sudo apt-get update
Upgrade all the Raspbian OS packages
greys@s7:~ $ sudo apt-get upgrade ... greys@s7:~ $ sudo apt dist-upgrade Reading package lists... Done Building dependency tree Reading state information... Done Calculating upgrade... The following packages were automatically installed and are no longer required: libcwiid1 libjs-prettify libllvm3.7 libqscintilla2-11 libqscintilla2-l10n libqt4-network libqt4-xmlpatterns libqtwebkit4 libruby1.9.1 libruby1.9.1-dbg libtcl8.5 libtcltk-ruby1.9.1 libtk8.5 pix-icons pix-plym-splash pixel-wallpaper ri1.9.1 ruby1.9.1 ruby1.9.1-dev ruby1.9.1-examples ruby1.9.1-full ruby1.9.3 supercollider supercollider-common supercollider-ide supercollider-language supercollider-supernova tcl8.5 tk8.5 Use 'apt-get autoremove' to remove them. Done The following NEW packages will be installed: bc coinor-libipopt1 colord colord-data cups cups-browsed cups-core-drivers cups-daemon cups-filters cups-filters-core-drivers cups-ppdc cups-server-common ghostscript libcolorhug2 libcupscgi1 libcupsmime1 libcupsppdc1 libfile-copy-recursive-perl libfontembed1 libgmime-2.6-0 libgs9 libgs9-common libgusb2 libgutenprint2 libieee1284-3 libijs-0.35 libinput-bin libinput10 libjbig2dec0 libllvm3.9 libmumps-seq-4.10.0 liboauth0 libopenjp2-7 libpam-chksshpwd libpaper-utils libpaper1 libqpdf13 libqt5concurrent5 libqt5core5a libqt5dbus5 libqt5gui5 libqt5network5 libqt5opengl5 libqt5printsupport5 libqt5svg5 libqt5widgets5 libraw10 libsane libsane-common libsane-extras libsane-extras-common libwacom-bin libwacom-common libwacom2 libxcb-icccm4 libxcb-image0 libxcb-keysyms1 libxcb-randr0 libxcb-render-util0 libxcb-xkb1 libxkbcommon-x11-0 printer-driver-gutenprint python-cffi python-cryptography python-ply python-pycparser python3-cffi python3-cryptography python3-ply python3-pycparser qpdf qttranslations5-l10n rpd-icons rpd-plym-splash rpd-wallpaper sane-utils ssl-cert update-inetd wolframscript xserver-xorg-input-libinput The following packages will be upgraded: alacarte apt apt-utils bash bind9-host binutils bluez-firmware ca-certificates chromium-browser cpp-4.9 cron cups-bsd cups-client cups-common curl dbus dbus-x11 debconf debconf-i18n debconf-utils e2fslibs e2fsprogs file firmware-atheros firmware-brcm80211 firmware-libertas firmware-ralink firmware-realtek fuse g++-4.9 gcc-4.9 gcc-4.9-base gir1.2-gdkpixbuf-2.0 git git-core git-man gnupg gnupg-agent gnupg2 gpgv gpicview gstreamer0.10-alsa gstreamer0.10-plugins-base gstreamer1.0-alsa gstreamer1.0-plugins-bad gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-x gtk2-engines-pixbuf idle-python2.7 idle-python3.4 initramfs-tools isc-dhcp-client isc-dhcp-common libapt-inst1.5 libapt-pkg4.12 libarchive13 libasan1 libasound2 libasound2-data libatomic1 libaudiofile1 libavcodec56 libavformat56 libavresample2 libavutil54 libbind9-90 libblas-common libblas3 libbluray1 libc-ares2 libc-bin libc-dev-bin libc6 libc6-dbg libc6-dev libcaca0 libcairo-gobject2 libcairo2 libcomerr2 libcups2 libcupsfilters1 libcupsimage2 libcurl3 libcurl3-gnutls libdb5.3 libdbus-1-3 libdns-export100 libdns100 libdrm-amdgpu1 libdrm-freedreno1 libdrm-nouveau2 libdrm-radeon1 libdrm2 libdvdnav4 libegl1-mesa libelf1 libevent-2.0-5 libexpat1 libexpat1-dev libfaad2 libfcgi-perl libffi6 libfm-data libfm-extra4 libfm-gtk-data libfm-gtk4 libfm-modules libfm4 libfuse2 libgbm1 libgcc-4.9-dev libgcc1 libgcrypt20 libgd3 libgdk-pixbuf2.0-0 libgdk-pixbuf2.0-common libgfortran3 libgl1-mesa-dri libgl1-mesa-glx libglapi-mesa libgles1-mesa libgles2-mesa libgme0 libgnutls-deb0-28 libgnutls-openssl27 libgomp1 libgraphite2-3 libgssapi-krb5-2 libgstreamer-plugins-bad1.0-0 libgstreamer-plugins-base0.10-0 libgstreamer-plugins-base1.0-0 libgstreamer1.0-0 libgtk2.0-0 libgtk2.0-bin libgtk2.0-common libgudev-1.0-0 libhogweed2 libicu52 libidn11 libio-socket-ssl-perl libirs-export91 libisc-export95 libisc95 libisccc90 libisccfg-export90 libisccfg90 libjasper1 libjpeg62-turbo libjs-jquery libk5crypto3 libkrb5-3 libkrb5support0 liblapack3 liblcms2-2 libldap-2.4-2 libldb1 liblwres90 libmad0 libmagic1 libmp3lame0 libmpg123-0 libmwaw-0.3-3 libmysqlclient18 libncurses5 libncursesw5 libnettle4 libnss3 libobrender29 libobt2 libopencv-calib3d2.4 libopencv-contrib2.4 libopencv-core2.4 libopencv-features2d2.4 libopencv-flann2.4 libopencv-highgui2.4 libopencv-imgproc2.4 libopencv-legacy2.4 libopencv-ml2.4 libopencv-objdetect2.4 libopencv-video2.4 libpam-modules libpam-modules-bin libpam-runtime libpam-systemd libpam0g libpng12-0 libpng12-dev libpolkit-agent-1-0 libpolkit-backend-1-0 libpolkit-gobject-1-0 libpoppler46 libprocps3 libpython2.7 libpython2.7-minimal libpython2.7-stdlib libpython3.4 libpython3.4-dev libpython3.4-minimal libpython3.4-stdlib libraspberrypi-bin libraspberrypi-dev libraspberrypi-doc libraspberrypi0 librtmp1 libruby2.1 libsdl-image1.2 libsmbclient libsndfile1 libsoundtouch0 libsoup-gnome2.4-1 libsoup2.4-1 libsqlite3-0 libss2 libssh-4 libssh2-1 libssl-dev libssl-doc libssl1.0.0 libstdc++-4.9-dev libstdc++6 libswscale3 libsystemd0 libtag1-vanilla libtag1c2a libtasn1-6 libtevent0 libtiff5 libtinfo5 libtirpc1 libubsan0 libudev1 libva1 libvorbis0a libvorbisenc2 libvorbisfile3 libwayland-client0 libwayland-cursor0 libwayland-egl1-mesa libwayland-server0 libwbclient0 libwpd-0.10-10 libx11-6 libx11-data libx11-xcb1 libxcursor1 libxfixes3 libxfont1 libxi6 libxml2 libxpm4 libxrandr2 libxslt1.1 libxtst6 libxv1 locales login lxpanel lxpanel-data man-db multiarch-support mysql-common ncurses-base ncurses-bin ncurses-term nodered nuscratch openbox openssh-client openssh-server openssh-sftp-server openssl passwd patch pcmanfm perl perl-base perl-modules pi-bluetooth pi-greeter piclone pigpio pipanel pishutdown pix-icons pix-plym-splash pixel-wallpaper policykit-1 poppler-utils pprompt procps python-gpiozero python-lxml python-openssl python-picamera python-picraft python-pigpio python-werkzeug python2.7 python2.7-minimal python3-gpiozero python3-openssl python3-picamera python3-picraft python3-pigpio python3-werkzeug python3.4 python3.4-dev python3.4-minimal raspberrypi-bootloader raspberrypi-kernel raspberrypi-sys-mods raspberrypi-ui-mods raspi-config raspi-gpio rc-gui realvnc-vnc-server realvnc-vnc-viewer rpcbind rpi-chromium-mods rsync ruby2.1 samba-common samba-libs sed sensible-utils sonic-pi ssh sudo systemd systemd-sysv tar tzdata udev unzip va-driver-all wget wireless-regdb wiringpi wolfram-engine wpasupplicant x11-common xarchiver xdg-utils xserver-common xserver-xorg xserver-xorg-core xserver-xorg-input-all xserver-xorg-input-evdev xserver-xorg-input-synaptics xserver-xorg-video-fbdev xserver-xorg-video-fbturbo 352 upgraded, 80 newly installed, 0 to remove and 0 not upgraded. Need to get 363 MB/729 MB of archives. After this operation, 245 MB of additional disk space will be used. Do you want to continue? [Y/n] y
Double-check Upgrade Was Complete
If we re-run the same command, there should be no more updates pending:
greys@s7:~ $ sudo apt dist-upgrade Reading package lists... Done Building dependency tree Reading state information... Done Calculating upgrade... The following packages were automatically installed and are no longer required: libcwiid1 libjs-prettify libllvm3.7 libqscintilla2-11 libqscintilla2-l10n libqt4-network libqt4-xmlpatterns libqtwebkit4 libruby1.9.1 libruby1.9.1-dbg libtcl8.5 libtcltk-ruby1.9.1 libtk8.5 pix-icons pix-plym-splash pixel-wallpaper ri1.9.1 ruby1.9.1 ruby1.9.1-dev ruby1.9.1-examples ruby1.9.1-full ruby1.9.3 supercollider supercollider-common supercollider-ide supercollider-language supercollider-supernova tcl8.5 tk8.5 Use 'apt-get autoremove' to remove them. Done 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Perfect! Just a few packages to be tidied up with apt-get autoremove.
Tidy Up with apt-get autoremove
greys@s7:~ $ sudo apt-get autoremove Reading package lists... Done Building dependency tree Reading state information... Done The following packages will be REMOVED: libcwiid1 libjs-prettify libllvm3.7 libqscintilla2-11 libqscintilla2-l10n libqt4-network libqt4-xmlpatterns libqtwebkit4 libruby1.9.1 libruby1.9.1-dbg libtcl8.5 libtcltk-ruby1.9.1 libtk8.5 pix-icons pix-plym-splash pixel-wallpaper ri1.9.1 ruby1.9.1 ruby1.9.1-dev ruby1.9.1-examples ruby1.9.1-full ruby1.9.3 supercollider supercollider-common supercollider-ide supercollider-language supercollider-supernova tcl8.5 tk8.5 0 upgraded, 0 newly installed, 29 to remove and 0 not upgraded. After this operation, 129 MB disk space will be freed. Do you want to continue? [Y/n] y (Reading database ... 138805 files and directories currently installed.) Removing supercollider (1:3.6.6~repack-2-1) ... Removing supercollider-ide (1:3.6.6~repack-2-1) ... Removing supercollider-language (1:3.6.6~repack-2-1) ... Removing libcwiid1 (0.6.00+svn201-3.1) ... Removing supercollider-common (1:3.6.6~repack-2-1) ... Removing libjs-prettify (2013.03.04+dfsg-4) ... Removing libllvm3.7:armhf (1:3.7-4~bpo8+1) ... Removing libqscintilla2-11 (2.8.4+dfsg-1) ... Removing libqscintilla2-l10n (2.8.4+dfsg-1) ... Removing libqtwebkit4:armhf (2.3.4.dfsg-3) ... Removing libqt4-xmlpatterns:armhf (4:4.8.6+git64-g5dc8b2b+dfsg-3+deb8u1+rpi1) ... Removing libqt4-network:armhf (4:4.8.6+git64-g5dc8b2b+dfsg-3+deb8u1+rpi1) ... Removing ruby1.9.1-full (18.104.22.1684-2) ... Removing ruby1.9.1-dev (22.214.171.1244-2) ... Removing ruby1.9.3 (126.96.36.1994-2) ... Removing libruby1.9.1-dbg (188.8.131.524-2) ... Removing libtcltk-ruby1.9.1 (184.108.40.2064-2) ... Removing tk8.5 (8.5.17-1) ... Removing tcl8.5 (8.5.17-1) ... Removing libtk8.5:armhf (8.5.17-1) ... Removing pix-icons (0.8) ... Removing pix-plym-splash (0.11) ... Removing pixel-wallpaper (0.4) ... Removing ri1.9.1 (220.127.116.114-2) ... Removing ruby1.9.1-examples (18.104.22.1684-2) ... Removing supercollider-supernova (1:3.6.6~repack-2-1) ... Removing libtcl8.5:armhf (8.5.17-1) ... Removing ruby1.9.1 (22.214.171.1244-2) ... Removing libruby1.9.1 (126.96.36.1994-2) ... Processing triggers for gnome-menus (3.13.3-6) ... Processing triggers for desktop-file-utils (0.22-1) ... Processing triggers for mime-support (3.58) ... Processing triggers for man-db (2.7.5-1~bpo8+1) ... Processing triggers for libc-bin (2.19-18+deb8u10) ... Processing triggers for ruby (1:2.1.5+deb8u2) ...
All clean now!
Step 2: Upgrade Raspbian OS to the next Release
You need to do pretty much the same steps as in previous section, but AFTER you update apt sources links to use the next Raspbian OS release.
As you know, Raspbian OS is based on Debian so it uses Debian's naming conventions.
Meaning my old release was this:
… and next one (the one I'm upgrading to) is
Update apt sources.lst with new release name
So let's update /etc/apt/sources.list file to reference stretch, which is the Debian release we want:
greys@s7:~ $ sudo vi /etc/apt/sources.list
edit this line:
deb http://raspbian.raspberrypi.org/raspbian/ jessie main contrib non-free rpi
so it looks like this:
deb http://raspbian.raspberrypi.org/raspbian/ stretch main contrib non-free rpi
Refresh apt information
greys@s7:~ $ sudo apt-get update
Upgrade all the packages once again
This time they'll be upgraded to the latest versions available for the next Raspbian OS release:
greys@s7:~ $ sudo apt-get upgrade ... greys@s7:~ $ sudo apt dist-upgrade
That should be it. Now let's reboot and Raspberry Pi should come back online sporting a brand new Raspbian OS release:
greys@s7:~ $ sudo shutdown -r now
Took around 1 minute for it to reboot… Then:
greys@s7:~ $ cat /etc/debian_version 9.9
This week's Unix Tutorial Project is super geeky and fun: I'm setting up text-based email archive system using Mutt (NeoMutt, actually), OfflineIMAP and hopefully NotMuch. Will publish a project summary on the weekend.
Why use OfflineIMAP
OfflineIMAP tool is an open-source tool for downloading your email messages and storing them locally in a Maildir format (meaning each email message is stored in a separate file, each folder/GMail tag is a separate directory).
As the name suggests, this tool's primary objective is to let you read your emails offline. Contrary to the other part of the name, offlineimap is NOT an IMAP server implementation.
I'd like to explore OfflineIMAP/Neomutt setup as a backup/archive solution for my cloud email accounts. I used to be with Fastmail but switched to gSuite email last year. I think it's very important to keep local copies of any information you have in any cloud – no matter how big/reliable the service provider is, there are many scenarios where your data could be completely lost, and responsibility for keeping local backups is always with you.
Both gMail and Fastmail solutions are perfect for web browser use but any local email software is invariably bulkier and slower compared to web interface. I'm not giving up on finding the acceptably performance and reliable solution though.
This is one of the most recent attempts to download all emails and to have them easily searchable on my local PCs and laptops.
OfflineIMAP Configuration Steps
I'm only learning this tool, so this is probably the most basic usage:
- Confirm your mail server details (IMAP)
- Confirm your mailbox credentials (for Google, gSuite and even Fastmail you need to generate an app password – it's separate and different from your primary email password)
- Create .offlineimaprc file in your home directory as shown below
- If necessary, create credentials file (for now – with cleartext app password for email access) – mine is /home/greys/.creds/techstack.pass
- Run offlineimap (first time and every time you want your email refreshed)
My .offlineimaprc file
Here's what I have in my .offlineimaprc file for this experiment:
[general] ui = ttyui accounts = techstack [Account techstack] localrepository = techstack-local remoterepository = techstack-remote [Repository techstack-local] type = Maildir localfolders = ~/Mail/techstack/ [Repository techstack-remote] type = Gmail remoteuser = email@example.com remotepassfile = ~/.creds/techstack.pass maxconnections = 5 ssl = yes sslcacertfile = /etc/ssl/certs/ca-certificates.crt folderfilder = lambda foldername: foldername not in ['Archive'] expunge = no
You can have multiple accounts in this one config file, they'll be listed in the accounts section (accounts = techstack, unixtutorial would mean 2 accounts: techstack one and one for my Unix Tutorial email).
localfolders parameter specifies that I want OfflineIMAP to create a Mail directory in my homedir (so ) and then techstack subdirectory there – meaning you can have account subidrectories there like /home/greys/Mail/techstack and /home/greys/Mail/personal, etc.
You define two repositories, local and remote one. The task of OfflineIMAP is to sync the two.
IMPORTANT: The really important parameter is maxconnections one. Default is 3 and I've changed it to 5 for quicker email sync. Setting it to a higher value resulted in failures – probably because Google servers rate limit my connection.
CRITICAL: expunge parameter is set to yes by default, so you must set it to no if your plan is to keep emails on the mail server after you sync them. By default they will be removed from the server as soon as they are downloaded, meaning Gmail app won't see any messages. Once deleted, it will be rather tricky to restore all the emails – so it's important to get this setting right from the very start. Since my primary usage is still web and Gmail app based, I certainly want all my emails to stay in Google cloud even after I download them using OfflineIMAP – that's why I configured it as expunge = no.
As you can see, this config references the /home/greys/.creds/techstack.pass file. This file has an clear-text application password I generated for my email address in gSuite admin panel. My understanding is that this can be improved, so I'll do a follow-up post later.
How To Use OfflineIMAP
Simply run the offlineimap command and you should see something like this:
greys@xps:~ $ offlineimap OfflineIMAP 7.2.2 Licensed under the GNU GPL v2 or any later version (with an OpenSSL exception) imaplib2 v2.57 (system), Python v2.7.16, OpenSSL 1.1.1b 26 Feb 2019 Account sync techstack: *** Processing account techstack Establishing connection to imap.gmail.com:993 (techstack-remote) Folder 2016 [acc: techstack]: Syncing 2016: Gmail -> Maildir Folder 2016/01-January [acc: techstack]: Syncing 2016/01-January: Gmail -> Maildir Folder 2016/02-February [acc: techstack]: Syncing 2016/02-February: Gmail -> Maildir Folder 2016/01-January [acc: techstack]:
As you can see, it processes account techstack, connects to gmail and starts processing remote folders (gmail tags) like 2016, 2016/01-January, 2016-02-February etc – these are the tags I have in my gSuite account.
Initial download would take a while. My 150K messages took almost 3 days to download.
That's all for today, hope you give OfflineIMAP a try!
I have just installed ZSH shell using Homebrew to learn some new Unix shell tricks on my Macbook Pro. Next up is Oh My ZSH. Since this is the official Oh My ZSH installer, it should work on most Linux distros, but I tested it on my Macbook Pro running MacOS Mojave.
Install Oh My ZSH using curl
This command assumes you have curl command installed on your system, curl comes pre-installed with macOS but you may need to install it separately in Linux:
greys@maverick:/ $ sh -c "$(curl -fsSL https://raw.github.com/robbyrussell/oh-my-zsh/master/tools/install.sh)" Cloning Oh My Zsh... Cloning into '/Users/greys/.oh-my-zsh'... remote: Enumerating objects: 1042, done. remote: Counting objects: 100% (1042/1042), done. remote: Compressing objects: 100% (959/959), done. remote: Total 1042 (delta 23), reused 848 (delta 20), pack-reused 0 Receiving objects: 100% (1042/1042), 692.46 KiB | 1.67 MiB/s, done. Resolving deltas: 100% (23/23), done. Looking for an existing zsh config... Using the Oh My Zsh template file and adding it to ~/.zshrc. Time to change your default shell to zsh: Do you want to change your default shell to zsh? [Y/n] n Shell change skipped.
And just like that, we're done: