Using LXC
While the mainstream tendency was migrating to clouds, at my past work we continued using bare metal servers. So I dug into virtualization quite lazily, occasionally using VirtualBox and giving Qubes OS a try (yuck!). However, during the recent overhaul of my home network I finally moved to Linux containers. This blog post summarizes my experience, focusing on Armbian.
References
- https://wiki.debian.org/LXC
- https://stgraber.org/2013/12/20/lxc-1-0-blog-post-series/
- https://blog.benoitblanchon.fr/lxc-unprivileged-container/
For setting up Armbian refer to my blog post From Sid down to Bullseye: installing Armbian on NanoPi M4 v2.
Installing LXC
apt install lxc lxc-templates libvirt0 libpam-cgfs bridge-utils uidmap debootstrap distro-info
I don't use default bridge, I prefer to configure it manually in /etc/network/interfaces. So I don't need lxc-net service:
systemctl stop lxc-net systemctl disable lxc-net
Make change to /etc/default/lxc-net:
USE_LXC_BRIDGE="false"
If you prefer to setup a bridge interface manually, run the following commands:
brctl addbr br0 brctl addif br0 eth0 ip link set br0 up
Given that the bridge interface is br0, /etc/lxc/default.conf should contain
lxc.net.0.link = br0
Change apparmor profile, the reason is running unprivileged containers:
lxc.apparmor.profile = unconfined
For privileged containers I used `unconfigured`. Actually, I don't know what's the difference.
lxc.apparmor.profile = unconfigured
Creating LXC container
For the first time I played with privileged containers but soon started using unprivileged ones. Unprivileged containers have some limitations, for example, it's impossible to run nfs-kernel-server or mount a block device, even if you give access to that device and even if you are able to read the data from it. Mounting file systems is allowed for root only.
Let's create container `test`:
lxc-create -n test -t debian -- -r bullseye
Before starting the container a few tweaks are necessary.
If the container is unprivileged, add the following to the configuration file /var/lib/lxc/test/config:
# Map user and group ids lxc.include = /usr/share/lxc/config/debian.userns.conf lxc.idmap = u 0 100000 65536 lxc.idmap = g 0 100000 65536
In addition to the above change, make sure both /etc/subuid and /etc/subgid on the host system contain
root:100000:65536
Then, configure networking in /var/lib/lxc/test/rootfs/etc/network/interfaces.
Also, I configure SSH server. First, make the following changes to /var/lib/lxc/test/rootfs/etc/ssh/sshd_config:
PermitRootLogin yes PubkeyAuthentication yes PasswordAuthentication no
And then,
mkdir /var/lib/lxc/test/rootfs/root/.ssh chmod 700 /var/lib/lxc/test/rootfs/root/.ssh cp -a .ssh/authorized_keys /var/lib/lxc/test/rootfs/root/.ssh/
i.e. I use the same public keys from the host system. You can use different ones.
For unprivileged containers change the owner of rootfs:
chown -R 100000:100000 /var/lib/lxc/test/rootfs
By default dmesg output is available from unprivileged container. Here's a security tweak to /etc/sysctl.conf:
kernel.dmesg_restrict = 1
Running LXC container
If the new container is successfuly started with
lxc-start test
you can either login via SSH or attach from the host system: lxc-attach test
.
If something went wrong, you can start the container in foreground:
lxc-start -F -n test
however, in all my cases the output was useless and I had to play with configuration file.
So, if everything went okay and you're in the container, a minimal usable system is necessary:
apt install less nano psutils rsync apt-utils iputils-tracepath inetutils-ping \ dnsutils nftables curl gnupg2 ca-certificates lsb-release debian-archive-keyring apt install --no-install-recommends rsyslog logrotate cron
The second command needs --no-install-recommends because all those packages want Exim.
Using block devices and mounting file systems
Actually, I did not try that from a privileged container. This section contains instructions for unprivileged containers only.
To use a block device in a container, change group of the block device to 100000 The owner may remain root. To automate this, create /etc/udev/rules.d/90-sda-permissions.rules with the following line (assuming the device is sda):
KERNEL=="sda", ACTION=="add", GROUP="100000"
Next, allow use of block device in the container. Add the following lines to /var/lib/lxc/test/config:
lxc.cgroup.devices.allow = b 8:0 rwm lxc.mount.entry = /dev/sda dev/sda none bind,create=file
Open question is how to make this by UUID? Namely, sda name and 8:0 which are major:minor numbers may change, UUID is stable.
As I already told, you can read-write block device from a container, but you can't mount a file system. For filesystems, they should be mounted on the host system and then you can use bind mounts in the configuration file:
lxc.mount.entry = /mnt/filestore mnt/filestore none bind 0 0
VPN
I tried Wireguard as a client in unprivileged containers. It works without any tweaks. Installation wants too much and needs a command line option:
apt install --no-install-recommends wireguard wireguard-tools
I tried OpenVPN client from a privileged container. It needs the following lines to the configuration file:
lxc.hook.autodev = /var/lib/lxc/test/autodev lxc.cgroup2.devices.allow = c 10:200 rwm
where autodev hook contains the following:
#!/bin/bash pushd ${LXC_ROOTFS_MOUNT}/dev mkdir net mknod net/tun c 10 200 chmod 666 net/tun popd
That's all. Bye for now.