Libvirtd conffiles should be less misleading and document tcp/tls usage

Bug #1960937 reported by Michael OReilly
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
libvirt (Ubuntu)
Fix Released
Medium
Unassigned

Bug Description

I was testing out libvirtd on Ubuntu 20.04

Actually testing out this - https://techviewleo.com/install-webvirtcloud-kvm-web-dashboard-on-ubuntu/

I get the error 16509 which I can replicate with this command

# virsh -c qemu+tcp://host/system
error: unable to connect to server at 'host:16509': Connection refused
error: failed to connect to the hypervisor

https://wiki.libvirt.org/page/Libvirt_daemon_is_not_listening_on_tcp_ports_although_configured_to

The libvirt systemd service starts the libvirt process with $libvirt_opts as a parameter to the executable.

if I update the libvirtd config file /etc/default/libvirt

# options passed to libvirtd, add "-l" to listen on tcp
libvirtd_opts="-l -d"

Adding any option in libvirtd_opts causes the service to fail on restart without the listener running on port 16509.

Seems like the config behaviour changed since this bug https://bugs.launchpad.net/mos/+bug/1473569 was closed.

Tags: bitesize

Related branches

Revision history for this message
Michael OReilly (oreillymj) wrote :

Digging into this a bit more, it looks like the libvirtd.service file needs to be changed to enable the tcp listener instead of using the /etc/default/libvirtd config file. Would be nice if the service had the tcp option (commented out by default)

My changes were to
Remove $libvirtd_opts because any setting here causes the service to fail at startup.
Add
Wants=libvirtd-tcp.socket
and
Also=libvirtd-tcp.socket

[Unit]
Description=Virtualization daemon
Requires=virtlogd.socket
Requires=virtlockd.socket
# Use Wants instead of Requires so that users
# can disable these three .socket units to revert
# to a traditional non-activation deployment setup
Wants=libvirtd.socket
Wants=libvirtd-ro.socket
Wants=libvirtd-tcp.socket
Wants=libvirtd-admin.socket
Wants=systemd-machined.service
Before=libvirt-guests.service
After=network.target
After=dbus.service
After=iscsid.service
After=apparmor.service
After=local-fs.target
After=remote-fs.target
After=systemd-logind.service
After=systemd-machined.service
After=xencommons.service
Conflicts=xendomains.service
Documentation=man:libvirtd(8)
Documentation=https://libvirt.org

[Service]
Type=notify
EnvironmentFile=-/etc/default/libvirtd
ExecStart=/usr/sbin/libvirtd $libvirtd_opts
#ExecStart=/usr/sbin/libvirtd -l -d
ExecReload=/bin/kill -HUP $MAINPID
KillMode=process
Restart=on-failure
# At least 1 FD per guest, often 2 (eg qemu monitor + qemu agent).
# eg if we want to support 4096 guests, we'll typically need 8192 FDs
# If changing this, also consider virtlogd.service & virtlockd.service
# limits which are also related to number of guests
LimitNOFILE=8192
# The cgroups pids controller can limit the number of tasks started by
# the daemon, which can limit the number of domains for some hypervisors.
# A conservative default of 8 tasks per guest results in a TasksMax of
# 32k to support 4096 guests.
TasksMax=32768
# With cgroups v2 there is no devices controller anymore, we have to use
# eBPF to control access to devices. In order to do that we create a eBPF
# hash MAP which locks memory. The default map size for 64 devices together
# with program takes 12k per guest. After rounding up we will get 64M to
# support 4096 guests.
LimitMEMLOCK=64M

[Install]
WantedBy=multi-user.target
Also=virtlockd.socket
Also=virtlogd.socket
Also=libvirtd.socket
Also=libvirtd-ro.socket
Also=libvirtd-tcp.socket

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Hi,
thank you for your report!

The config in /etc/default/libvirtd is pretty much there for compatibility reasons with older setups and (on Debian) people using still sys-V init.
And even with systemd some other options than "-l" can still be sued that way e.g. "-v" for extra verbosity.

If one sets that option the error message gladly is pretty clear:
libvirtd[2725397]: --listen parameter not permitted with systemd activation sockets, see 'man libvirtd' for further guidance

In that man page it explains (as that isn't a Ubuntu only problem):

```
...
       In socket activation mode, it will rely on systemd to create and listen on the UNIX, and optionally TCP/IP, sockets and pass them as pre-opened file descriptors. In this mode, it
       is not permitted to pass the --listen parameter, and most of the socket related config options in /etc/libvirt/libvirtd.conf will no longer have any effect. To enable TCP or TLS
       sockets use either

          $ systemctl start libvirtd-tls.socket
       Or
          $ systemctl start libvirtd-tcp.socket
...
```

There is no need to change the .service file as you did, and in fact any subsequent update would eliminate that change.

As a TL;DR and I hope IIRC it would be like:
$ sudo systemctl stop libvirtd
$ sudo systemctl start libvirtd-tcp.socket
# in any real setup you'd now setup SASL, but for this to
# be short I set auth_tcp = "none" in /etc/libvirt/libvirtd.conf

With that in place
$ virsh -c qemu+tcp://127.0.0.1/system list

Will start libvirtd and it accepts tcp connections.

In addition there also is TLS which is more complex there is:

#setup CA and issue/config certificates for libvirtd
# see TLS x509 certificate configuration and any entry related to
# *tls* and *certificates* in /etc/libvirt/libvirtd.conf
$ sudo systemctl stop libvirtd
$ sudo systemctl start libvirtd-tls.socket

What to do from here:
- I agree that a hint in /etc/default/libvirtd would be great to have as its current
  form is misleading.
- This bug demonstrates that it is unclear how to be used.
  Therefore a documentation entry in the Ubuntu server guide would be very helpful.
  It should show how this can be done (for TCP) and also include an example of a
  cert setup and end by accessing via TLS

Triaging the bug with that in mind, but please speak up for a discussion on this if you want.

summary: - Libvirtd on 20.04 does not listen on port 16509
+ Libvirtd conffiles should be less misleading and document tcp/tls usage
tags: added: server-todo
Changed in libvirt (Ubuntu):
importance: Undecided → Medium
tags: added: bitesize
tags: removed: server-todo
Changed in libvirt (Ubuntu):
assignee: nobody → Michał Małoszewski (michal-maloszewski99)
Changed in libvirt (Ubuntu):
status: New → In Progress
Revision history for this message
Michał Małoszewski (michal-maloszewski99) wrote :

Work in progress.

tags: added: server-todo
Revision history for this message
Michał Małoszewski (michal-maloszewski99) wrote :
Revision history for this message
Michał Małoszewski (michal-maloszewski99) wrote :
Robie Basak (racb)
Changed in libvirt (Ubuntu):
assignee: Michał Małoszewski (michal-maloszewski99) → nobody
Revision history for this message
Paride Legovini (paride) wrote :

That salsa MR is now merged. Hopefully this bug will auto-close when a new Debian revision including that commit will reach Ubuntu. Commit [1] includes appropriate gbp meta tags.

[1] https://salsa.debian.org/libvirt-team/libvirt/-/merge_requests/207/diffs?commit_id=03d707d6b33fedf5a199a4a6d282d9257ed2c40d

Revision history for this message
Robie Basak (racb) wrote :

Thanks Paride! This looks like it'll be fixed in the next package merge, so there's no special action needed in the server team now.

Changed in libvirt (Ubuntu):
status: In Progress → Triaged
tags: removed: server-todo
Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (5.4 KiB)

This bug was fixed in the package libvirt - 10.0.0-1ubuntu1

---------------
libvirt (10.0.0-1ubuntu1) noble; urgency=medium

  * Merge with Debian unstable (LP: #2040393, #2037606). Remaining changes:
    - Disable libssh2 support (universe dependency)
    - d/control: add libzfslinux-dev to build-deps
    - d/control: drop libvirt-lxc, vbox and xen drivers to suggest
    - debian/patches/ubuntu/ovmf_paths.patch: adjust paths to secboot.fd UEFI
      Secure Boot enabled variants of the OVMF firmware and variable store for
      the paths where we ship these files in Ubuntu.
    - Set qemu-group to kvm (for compat with older ubuntu)
    - Additional apport package-hook
    - Autostart default bridged network (As upstream does, but not Debian).
      In addition to just enabling it our solution provides:
      + do not autostart if subnet is already taken (e.g. in guests).
      + iterate some alternative subnets before giving up
    - d/p/ubuntu/Allow-libvirt-group-to-access-the-socket.patch: This is
      the group based access to libvirt functions as it was used in Ubuntu
      for quite a long time.
      + d/p/ubuntu/daemon-augeas-fix-expected.patch fix some related tests
        due to the group access change.
      + d/libvirt-daemon-system.postinst: add users in sudo to the libvirt
        group.
    - Update README.Debian with Ubuntu changes
    - d/p/ubuntu/ubuntu_machine_type.patch: accept ubuntu types as pci440fx
    - fix autopkgtests (LP 1899180)
      + d/t/control, d/t/smoke-qemu-session: fixup smoke-qemu-session by making
        vmlinuz available and accessible (Debian bug 848314)
      + d/t/control: fix smoke-qemu-session by ensuring the service will run
        installing libvirt-daemon-system
      + d/t/smoke-lxc: fix smoke-lxc by ignoring potential issues on destroy as
        long as the following undefine succeeds
      + d/t/smoke-lxc: use systemd instead of sysV to restart the service
      + d/t/control, d/t/smoke-lxc: retry service restart and skip test if
        failing; This was flaky on some release/architectures
      + d/t/smoke-lxc: retry check_domain being flaky on arm64
    - dnsmasq related enhancements
      + run dnsmasq as libvirt-dnsmasq (LP 1743718)
      + d/libvirt-daemon-system.postinst: add libvirt-dnsmasq user and group
      + d/libvirt-daemon-system.postrm: remove libvirt-dnsmasq user and group
        on purge
      + d/p/ubuntu/dnsmasq-as-priv-user: write dnsmasq config with user
        libvirt-dnsmasq and adapt the self tests to expect that config
      + d/libvirt-daemon-system.postinst: fix old libvirt-dnsmasq users group
      + Add dnsmasq configuration to work with system wide dnsmasq-base
    - d/p/ubuntu/set-default-machine-to-ubuntu.patch: to select default
      machine type correctly with newer qemu/libvirt
    - d/p/ubuntu/lp-1861125-ubuntu-models: recognize Ubuntu models for
      (LP 1861125) fixups
    - d/p/ubuntu/wait-for-qemu-kvm.patch - avoid hangs on startup (LP 1887592)
    - d/libvirt-daemon-system.libvirt-guests.default: shut guests down
      in parallel
    - Apparmor Delta that is Ubuntu specific or yet to be upstreamed
      split into logical pieces. File names ...

Read more...

Changed in libvirt (Ubuntu):
status: Triaged → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.