How To Resume wget Downloads

Resume wget download

I’m downloading CentOS 8 Stream as we speak, and it’s a large enough ISO file – standard 8GB DVD image. I stopped download because I wanted to restart it in a tmux session, and realised that it’s a good opportunity to explain how this works.

How To Start Wget Download

Assuming you go to the CentOS Stream Download page and pick an image to download, you’ll end up with a download URL like this:

http://ftp.heanet.ie/pub/centos/8-stream/isos/x86_64/CentOS-Stream-x86_64-dvd1.iso

Using wget, here’s how you can start downloading it:

greys@redhat:/dist/iso $ wget http://ftp.heanet.ie/pub/centos/8-stream/isos/x86_64/CentOS-Stream-x86_64-dvd1.iso
--2019-09-30 14:35:36--  http://ftp.heanet.ie/pub/centos/8-stream/isos/x86_64/CentOS-Stream-x86_64-dvd1.iso
Resolving ftp.heanet.ie (ftp.heanet.ie)… 87.44.34.235
Connecting to ftp.heanet.ie (ftp.heanet.ie)|87.44.34.235|:80… connected.
HTTP request sent, awaiting response… 200 OK
Length: 8572108800 (8.0G) [application/octet-stream]
Saving to: ‘CentOS-Stream-x86_64-dvd1.iso’
CentOS-Stream-x86_64-dvd1.iso  1%[+      ] 100.83M  1.77MB/s    eta 73m 43s
^C

At any time you can press Ctlr+C to stop the process. A file with unfinished download will be left in the current directory:

greys@redhat:/dist/iso $ ls -ald *iso
-rw-rw-r--. 1 greys 118453636 Sep 30 14:37 CentOS-Stream-x86_64-dvd1.iso

And if you just re-run the same wget command, your download will actually restart:

greys@redhat:/dist/iso $ wget http://ftp.heanet.ie/pub/centos/8-stream/isos/x86_64/CentOS-Stream-x86_64-dvd1.iso
--2019-09-30 14:44:08--  http://ftp.heanet.ie/pub/centos/8-stream/isos/x86_64/CentOS-Stream-x86_64-dvd1.iso
Resolving ftp.heanet.ie (ftp.heanet.ie)… 87.44.34.235
Connecting to ftp.heanet.ie (ftp.heanet.ie)|87.44.34.235|:80… connected.
HTTP request sent, awaiting response… 200 OK
Length: 8572108800 (8.0G) [application/octet-stream]
Saving to: ‘CentOS-Stream-x86_64-dvd1.iso.1CentOS-Stream-x86_64-dvd1.iso.1  0%[+      ]  24.88M   775KB/s    eta 2h 13m
^C

If you notice though, the download restarts with a different filename: CentOS-Stream-x86_64-dvd1.iso.1 in my case. So wget kept your previous download attempt and started a new one:

greys@redhat:/dist/iso $ ls -ald iso
-rw-rw-r--. 1 greys 118453636 Sep 30 14:37 CentOS-Stream-x86_64-dvd1.iso
-rw-rw-r--. 1 greys  26088163 Sep 30 14:44 CentOS-Stream-x86_64-dvd1.iso.1

How To Resume wget Download

Assuming we want to resume download, you need to use the wget -c option:

greys@redhat:/dist/iso $ wget -c http://ftp.heanet.ie/pub/centos/8-stream/isos/x86_64/CentOS-Stream-x86_64-dvd1.iso
--2019-09-30 14:45:27--  http://ftp.heanet.ie/pub/centos/8-stream/isos/x86_64/CentOS-Stream-x86_64-dvd1.iso
Resolving ftp.heanet.ie (ftp.heanet.ie)… 87.44.34.235
Connecting to ftp.heanet.ie (ftp.heanet.ie)|87.44.34.235|:80… connected.
HTTP request sent, awaiting response… 206 Partial Content
Length: 8572108800 (8.0G), 8453655164 (7.9G) remaining [application/octet-stream]
Saving to: ‘CentOS-Stream-x86_64-dvd1.iso’
CentOS-Stream-x86_64-dvd1.iso  1%[+      ] 120.68M  1.49MB/s    eta 91m 20s
^C

So it picked up the original 100MB or so downloaded file and resumed downloading from 100MB+ location.

If we stop this with Ctlr+C again, we’ll see the following:

greys@redhat:/dist/iso $ ls -ald iso
-rw-rw-r--. 1 greys 126668812 Sep 30 14:45 CentOS-Stream-x86_64-dvd1.iso
-rw-rw-r--. 1 greys  26088163 Sep 30 14:44 CentOS-Stream-x86_64-dvd1.iso.1
greys@redhat:/dist/iso $ du -sh iso
121M    CentOS-Stream-x86_64-dvd1.iso
25M    CentOS-Stream-x86_64-dvd1.iso.1

So yes, it’s clearly the first file that is slightly larger down – meaning if we continue downloading, it will start from about 126MB location:

greys@redhat:/dist/iso $ wget -c http://ftp.heanet.ie/pub/centos/8-stream/isos/x86_64/CentOS-Stream-x86_64-dvd1.iso
--2019-09-30 14:47:08--  http://ftp.heanet.ie/pub/centos/8-stream/isos/x86_64/CentOS-Stream-x86_64-dvd1.iso
Resolving ftp.heanet.ie (ftp.heanet.ie)… 87.44.34.235
Connecting to ftp.heanet.ie (ftp.heanet.ie)|87.44.34.235|:80… connected.
HTTP request sent, awaiting response… 206 Partial Content
Length: 8572108800 (8.0G), 8445439988 (7.9G) remaining [application/octet-stream]
Saving to: ‘CentOS-Stream-x86_64-dvd1.iso’

CentOS-Stream-x86_64-dvd1.iso  1%[+      ] 127.76M  1.63MB/s    eta 98m 41s

And that’s how you resume wget downloads and continue when previous wget download left off.

See Also




Upgrading to CentOS 7.7

CentOS

CentOS 7.7 has just been released a few days ago and with any luck CentOS 8 will be released next week. I decided to quickly upgrade one of my dedicated servers from CentOS 7.6 to CentOS 7.7.

Confirm CentOS Version

Just a couple of steps to check what CentOS release we’re running:

root@s2:~ # cat /etc/redhat-release
CentOS Linux release 7.6.1810 (Core)

There should also be a package reflecting CentOS release already installed on your system. This package will be upgraded along with the rest of the OS when we’re stepping up to CentOS 7.7:

root@s2:~ # rpm -qa | grep centos-release
 centos-release-7-6.1810.2.el7.centos.x86_64

Check Available CentOS Upgrades

yum command has the check-update option for verifying if any packages are available for updating:

root@s2:~ # yum check-update

This will return a rather long list. To be super-sure we’ll actually get the CentOS 7.7 upgrade, look for the same centos-release package:

root@s2:~ # yum check-update | grep centos-release
 centos-release.x86_64                    7-7.1908.0.el7.centos           base

Upgrade CentOS 7.6 to CentOS 7.7

We need the yum update command here. After you run it it will resolve dependencies and report something like this, prompting for your confirmation:

Transaction Summary
===============================================================
Install    3 Packages (+25 Dependent packages)
Upgrade  406 Packages
Total download size: 577 M
Is this ok [y/d/N]:

Sounds about right! We’ll press Y and let the server download and apply all the updates…

On my hosting it took about 1min to download all the packages!

Transaction Summary
===============================================================
Install    3 Packages (+25 Dependent packages)
Upgrade  406 Packages
Total download size: 577 M
Is this ok [y/d/N]:
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
 Updating   : libgcc-4.8.5-39.el7.x86_64                                                                                                       1/841
 Installing : urw-base35-fonts-common-20170801-10.el7.noarch                                                                                   2/841
 Updating   : 1:grub2-common-2.02-0.80.el7.centos.noarch                                                                                       3/841
 Updating   : centos-release-7-7.1908.0.el7.centos.x86_64                                                                                      4/841
 Updating   : langtable-0.0.31-4.el7.noarch                                                                                                    5/841
 Updating   : libreport-filesystem-2.1.11-43.el7.centos.x86_64                                                                                 6/841
...
yum-plugin-fastestmirror.noarch 0:1.1.31-52.el7                           yum-utils.noarch 0:1.1.31-52.el7
 Replaced:
   urw-fonts.noarch 0:2.4-16.el7
 Complete!

It took less than 10 min to apply all the package updates, so the only things left are to capture current kernel version before and after the reboot:

root@s2:/ # uname -a
Linux s2 3.10.0-957.5.1.el7.x86_64 #1 SMP Fri Feb 1 14:54:57 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Now let’s reboot:

root@s2:/ # shutdown -r now

… and confirm that we’re running newer Linux Kernel and sporting the CentOS 7.7 release now:

greys@s2:~ $ uname -a
Linux s2 3.10.0-1062.1.1.el7.x86_64 #1 SMP Fri Sep 13 22:55:44 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Hmmm. This does look a bit conservative! It’s 3.10 branch of Linux kernel whereby desktop releases are sporting Linux Kernel 5.x already.

Anyway, that’s server s2 upgraded to CentOS 7.7 successfully!

We’re certainly running the CentOS 7.7 release now:

greys@s2:~ $ cat /etc/redhat-release
CentOS Linux release 7.7.1908 (Core)

See Also




How To: Remove Old Kernels in CentOS

CentOS-linux-logo.pngFor dedicated servers and virtual machines that you keep upgrading in-place, you will eventually reach the situation where there’s a number of old kernel packages installed. That’s because when you’re updating OS packages and get new kernel installed, the old ones are not auto-removed – allowing you to fall back if there are issues with the latest kernel.

How To List Old Kernels in CentOS/Red Hat Linux

rpm -q command comes to the resque! just run it for the kernel packages:

root@centos:~ # rpm -q kernel
kernel-3.10.0-327.28.3.el7.x86_64
kernel-3.10.0-327.36.3.el7.x86_64
kernel-3.10.0-693.21.1.el7.x86_64
kernel-3.10.0-957.5.1.el7.x86_64

You can use the uname command to verify the current kernel you’re running:

root@centos:~ # uname -a
Linux centos.ts.fm 3.10.0-957.5.1.el7.x86_64 #1 SMP Fri Feb 1 14:54:57 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

How To Remove Old Linux Kernels in CentOS

There’s actually a special command for doing this, but it’s probably not installed by default. It’s part of the yum-utils package that you may have to install like this first:

root@centos:~ # yum install yum-utils

Now that it’s installed, we’ll use the package-cleanup command. It takes the number of most recent kernels that you want to keep. So if you want to keep just the currently used kernel, the number should be 1. I recommend you keep 2 kernels – current and the one before it, so the count should be 2.

Just to be super sure, the package-cleanup -oldkernels command will ask you if you’re positive about removing the listed kernel packages before progressing:

root@centos:~ # package-cleanup --oldkernels --count=2
Loaded plugins: fastestmirror, langpacks
--> Running transaction check
---> Package kernel.x86_64 0:3.10.0-327.28.3.el7 will be erased
---> Package kernel.x86_64 0:3.10.0-327.36.3.el7 will be erased
--> Finished Dependency Resolution
epel/x86_64/metalink | 22 kB 00:00:00

Dependencies Resolved

===============================================================
Package Arch Version Repository Size
=============================================================== 
Removing:
kernel x86_64 3.10.0-327.28.3.el7 @centos-updates 136 M
kernel x86_64 3.10.0-327.36.3.el7 @updates 136 M

Transaction Summary
=============================================================== 
Remove 2 Packages

Installed size: 272 M
Is this ok [y/N]: y
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Erasing : kernel.x86_64 1/2
Erasing : kernel.x86_64 2/2
Verifying : kernel-3.10.0-327.36.3.el7.x86_64 1/2
Verifying : kernel-3.10.0-327.28.3.el7.x86_64 2/2

Removed:
kernel.x86_64 0:3.10.0-327.28.3.el7 kernel.x86_64 0:3.10.0-327.36.3.el7

Complete!

… and yes, don’t worry to be left without any Linux kernels! I checked, and specifying count=0 will not result in the package-cleanup killing your operating system:

root@centos:~ # package-cleanup --oldkernels --count=0
Loaded plugins: fastestmirror, langpacks
Error should keep at least 1 kernel!

That’s it for today. Hope you enjoyed the article!

See Also




Disable portmapper in CentOS 7

If you don’t have any other network services running on your Linux system, you probably don’t need portmapper running. Here are the steps to check and to disable portmap.

What portmapper does

Portm apper is a special Unix/Linux service that runs on networked systems that provide RPC (Remote Procedure Call) based services, like NFS.

Port mapper service is called portmapper and always runs on TCP and UDP ports 111.

IMPORTANT: back in 2015 portmapper was confirmed as vulnerable for Distributed Denial of Service attacks (DDoS) – so it’s considered a good practice to disable it or at least protect using firewall.

List RPC services

You can use rpcinfo command to list currently active RPC services on your system.

In my example below there’s nothing else running RPC, just the portmapper itself:

root@s5:~ # rpcinfo -p
program vers proto port service
100000 4 tcp 111 portmapper
100000 3 tcp 111 portmapper
100000 2 tcp 111 portmapper
100000 4 udp 111 portmapper
100000 3 udp 111 portmapper
100000 2 udp 111 portmapper

Stop portmapper in CentOS 7

Somewhat confusing, the service providing portmapper functionality is always called rpcbind.

First, let’s stop the portmapper service:

root@s5:~ # systemctl stop rpcbind
Warning: Stopping rpcbind.service, but it can still be activated by:
rpcbind.socket
root@s5:~ # systemctl stop rpcbind.socket

Prevent portmapper from restarting upon reboot

Now, let’s make sure the service is also disabled:

root@s5:~ # systemctl disable rpcbind
Removed symlink /etc/systemd/system/multi-user.target.wants/rpcbind.service.

And just to confirm it’s all done correctly, let’s run rpcinfo again, it will return an error now:

root@s5:~ # rpcinfo -p
rpcinfo: can't contact portmapper: RPC: Remote system error - Connection refused

See Also




How To: Disable SElinux

Screen Shot 2019-01-16 at 16.22.57.png
SElinux – Security Enhanced Linux

If you’re using RedHat or CentOS Linux distros (or sporting a Fedora Linux desktop), you probably have SELinux enabled by default. SELinux is a Security-Enhanced Linux – a framework for securely managing processes, users and files on your RedHat OS.

Confirm current SElinux mode

Just run the getenforce command to see what the story is. Most likely it will say “Enforcing” which is really good – means your OS is under solid protection:

[root@rhel8 ~]# getenforce
Enforcing

Temporarily Disable SELinux

If you need to disable SELinux just for a few minutes to debug some issue (mind you, there are better ways to debug than disabling SELinux!), you should use the setenforce command:

[root@rhel8 ~]# setenforce 0

As you can see, getenfore will now report that your system is running in a Permissive mode – not very safe:

[root@rhel8 ~]# getenforce
Permissive

IMPORTANT: This change won’t survive a reboot, so next time you restart your system it will come back with SELinux enabled and enforcing again.

Permanently Disable SELinux

If you’re serious about disabling SELinux altogether, you’ll have to do two things:

  1. Update /etc/selinux.png/config file (change SELINUX=enforcing to SELINUX=disabled)
  2. Reboot your Linux system

See Also




How To: 5 Ways to Check CentOS Version

2018-12-10_12-25-49.png

One of the very first questions a Linux user asks is about confirming the release (OS version) in use. Knowing release helps with highlighting software dependencies and compatibilities, confirms availability of certain features in your OS and simplifies the process of system administration – certain releases have a preferred set of commands for day-to-day management.

With CentOS being a rather popular server grade Linux distro, I can see that many visitors of my blog look for  the same guidance quite regularly: check CentOS version. This article introduces 5 of the most common ways to do just that.

1. Inspect /etc/system-release

Just to be super sure that you’re actually looking at a CentOS distribution of Linux, I suggest you start with the /etc/os-release file. As shown below, it will help you with confirming your Linux distro and its major release version (CentOS and 7 in my case):

greys@s5:~ $ cat /etc/os-release
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

Now that we’re sure it’s CentOS, let’s look into the /etc/centos-release file – this will show you the full release version of your operating system:

greys@s5:~ $ cat /etc/centos-release
CentOS Linux release 7.4.1708 (Core)

Interesting: if you’re coming from RedHat infrastructure, you’d normally be looking for /etc/redhat-release file. That’s okay and the good news is this will still work in CentOS:

greys@s5:~ $ cat /etc/redhat-release
CentOS Linux release 7.4.1708 (Core)

In fact, if you look at the /etc/redhat-release file on a CentOS server closely, you’ll notice that it is a symbolic link to /etc/centos-release:

greys@s5:~ $ ls -ald /etc/redhat-release
lrwxrwxrwx. 1 root root 14 Sep 18 2017 /etc/redhat-release -> centos-release

2. Use hostnamectl to confirm Linux release

Provided that you’re running a recent enough version of Linux, you should have the hostnamectl command installed.

Among other things, hostnamectl provides easy access to the OS release information and Linux kernel version:

greys@s5:~ $ hostnamectl
Static hostname: s5.ts.im
Icon name: computer-desktop
Chassis: desktop
Machine ID: 5f7e36c18a974f06ae94ddaaf11d71e8
Boot ID: 337e48b00fed4abe9ab929fed5aa6018
Operating System: CentOS Linux 7 (Core)
CPE OS Name: cpe:/o:centos:centos:7
Kernel: Linux 3.10.0-693.11.6.el7.x86_64
Architecture: x86-64

3. Confirm CentOS version with rpm

Next option you have is to use the RPM package manager to query a special package named centos-release. It will include the exact CentOS release version right in its full package name:

greys@s5:~ $ rpm -qa centos-release
centos-release-7-4.1708.el7.centos.x86_64

if you’re using RedHat, just do the same for the redhat-release package.

4. Confirm CentOS version using Linux kernel version

There are many ways to confirm your Linux kernel version, like uname command:

greys@s5:~ $ uname -r
3.10.0-693.11.6.el7.x86_64

Using the kernel version number – 3.10.0-693 in my example – you can confirm the CentOS release using one of the public version information pages, like the CentOS wikipedia page.

Once you browse to the CentOS wikipedia page, just search for the kernel version number and it should find something like this for you, confirming CentOS version to be 7.4-1708:

2018-12-10_11-50-19.png

5. Use lsb_release command to confirm Lunux release

Linux Standard Base (LSB) is a joint project by major Linux vendors to standardise configuration and usage of Linux distros. Amonth other things, it provides the lsb_release command that can help you check CentOS version.

Most likely, you’ll have to install it first:

greys@s5:~ $ yum install redhat-lsb-core
...
Once installed, run the lsb_release command with the -d option:
greys@s5:~ $ lsb_release -d
Description: CentOS Linux release 7.4.1708 (Core)

See Also




How To: Use yum behind proxy

I’ve got a few RedHat and CentOS VMs running on internal network of my server, and since updating them quite regularly is a habit I thought this little piece of advice will help all of you who run VMs in a similar scenario.

To make yum command use proxy, your best best is to edit /etc/yum.conf and add your proxy server reference:

proxy=http://192.168.3.1:3128

You don’t have to restart anything but it may be a good idea to do yum clean all and then yum check-update:

[root@testvm1 ~]# yum check-update
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: be.mirror.eurid.eu
* epel: epel.uni-oldenburg.de
* extras: be.mirror.eurid.eu
* updates: centosa5-msync-dvd.centos.org
base                                         | 3.7 kB 00:00
base/primary_db                              | 4.5 MB 00:01
cr                                           | 3.0 kB 00:00
cr/primary_db                                | 1.2 kB 00:00
epel                                         | 3.4 kB 00:00
epel/primary_db                              | 3.7 MB 00:00
extras                                       | 3.5 kB 00:00
extras/primary_db                            | 6.3 kB 00:00
updates                                      | 3.5 kB 00:00
updates/primary_db                           | 1.8 MB 00:01