apport general hooks break on type annotations

Bug #2025376 reported by Chad Smith
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
cloud-init (Ubuntu)
Fix Released
Critical
Chad Smith
Focal
Fix Released
Undecided
Unassigned

Bug Description

[Impact]
On Focal systems, inability to report using apport CLI.

apport-bug cloud-init will Traceback preventing apport report collection.
This would force folks to manually file bugs against cloud-init and attach supporting evidence at https://bugs.launchpad.net/ubuntu/+source/cloud-init/bugs.

[ Test Plan ]
Assert `apport-bug cloud-init` doesn't immediately traceback on the commandline and includes expected Cloud* report fields for openstack and ec2.

test script (attached):
#!/bin/bash
set -ex

cat > setup_proposed.sh <<EOF
#/bin/bash
mirror=http://archive.ubuntu.com/ubuntu
echo deb \$mirror \$(lsb_release -sc)-proposed main | tee /etc/apt/sources.list.d/proposed.list
apt-get update -q
apt-get install -qy cloud-init
EOF

cat > openstack.json <<EOF
{
  "ds" : {},
  "v1": {
        "cloud_id": "openstack",
        "cloud_name": "openstack",
        "machine": "CloudArchitecture",
        "platform": "openstack",
        "region": null,
        "subplatform": "metadata (http://169.254.169.254)"
  }
}
EOF

cat > ec2.json << EOF
{
  "ds": {
    "imageId": "ami-123",
    "instanceType": "m1.tiny",
    "region": "us-east-1"
  },
  "v1": {
    "cloud_name": "aws",
    "distro": "ubuntu",
    "distro_release": "jammy",
    "distro_version": "22.04",
    "instance_id": "i-06b5687b4d7b8595d",
    "machine": "x86_64",
    "platform": "ec2",
    "python_version": "3.10.4",
    "region": "us-east-2",
    "variant": "ubuntu"
  }
}
EOF

for release in focal jammy kinetic lunar; do
 VM=sru-$release
 lxc launch ubuntu-daily:$release $VM
 while ! lxc exec $VM -- cloud-init status --wait --long; do
    sleep 5
 done
 echo --- 1. Generate current apport report, selecting Openstack as cloud.
 echo --- step through prompts and select 'K' to keep report
 lxc exec $VM -- apport-bug cloud-init
 APPORT_FILE=$(lxc exec $VM ls /tmp | grep apport)
 lxc file pull $VM/tmp/$APPORT_FILE apport-$VM.orig
 lxc exec $VM rm /tmp/$APPORT_FILE

 lxc file push setup_proposed.sh $VM/
 lxc exec $VM -- bash /setup_proposed.sh | grep cloud-init

 lxc file push openstack.json $VM/run/cloud-init/instance-data.json
 echo --- 2. Generate -proposed apport report which sources openstack instance-data.json, selecting Openstack as cloud.
 lxc exec $VM -- apport-bug cloud-init
 APPORT_FILE=$(lxc exec $VM ls /tmp | grep apport)
 lxc file pull $VM/tmp/$APPORT_FILE apport-$VM.openstack-proposed
 lxc exec $VM rm /tmp/$APPORT_FILE

 echo --- 3. Generate -proposed apport report which sources ec2 instance-data, selecting Ec2 as cloud.
 lxc file push ec2.json $VM/run/cloud-init/instance-data.json
 lxc exec $VM -- apport-bug cloud-init
 APPORT_FILE=$(lxc exec $VM ls /tmp | grep apport)
 lxc file pull $VM/tmp/$APPORT_FILE apport-$VM.ec2-proposed
 lxc exec $VM rm /tmp/$APPORT_FILE

 # redact logs lines for easy diffs
 for file in `ls apport*`; do
     sed -i '1,/logs.tgz/!d' $file
 done
 echo --- 4. Inspect diff tags of orig to openstack-proposed report
 diff -urN apport-$VM.orig apport-$VM.openstack-proposed || true
 echo --- 5. Inspect diff tags of openstack-proposed to ec2-proposed report
 diff -urN apport-$VM.openstack-proposed apport-$VM.ec2-proposed || true

done

[ Where problems could occur ]
apport Report could omit new Cloud* keys if instance-data.json is not processed correctly.

More likely, if python code/syntax or logic is invalid apport command will quickly generate Tracebacks on unexpected values or data types.

This would prevent someone from creating cloud-init specific bugs on the apport CLI.

[ Other Info ]

This bug represents a SRU regression found during testing cloud-init 23.2 in -proposed and only affects Focal. It doesn't regress Jammy, Kinetic or Lunar series.

[ Original Description ]

SRU verification regression found during testing of https://bugs.launchpad.net/ubuntu/+source/apport/+bug/1724623.

cloud-init 23.2 introduced a couple new functions to extract cloud-specific values automatically for apport bug reports in apport/general-hooks/cloud_init.py.

The functions _get_azure_data and _get_ec2_data added type hints which were incompatible with Focal's python3.8.

Attempts to run apport-bug cloud-init on focal results in immediate tracebacks which collecting apport info:
*** Collecting problem information

The collected information can be sent to the developers to improve the
application. This might take a few minutes.
......ERROR: hook /usr/share/apport/general-hooks/cloud-init.py crashed:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/apport/report.py", line 226, in _run_hook
    exec(compile(fd.read(), hook, 'exec'), symb)
  File "/usr/share/apport/general-hooks/cloud-init.py", line 7, in <module>
    def _get_azure_data(ds_data) -> dict[str, str]:
TypeError: 'type' object is not subscriptable
.
*** Your device details (lshw) may be useful to developers when addressing this bug, but gathering it requires admin privileges. Would you like to include this info?

This regression will block SRU to focal. It does not affect Jammy or later series

Chad Smith (chad.smith)
Changed in cloud-init (Ubuntu):
status: Triaged → Fix Committed
Revision history for this message
Chad Smith (chad.smith) wrote (last edit ):
Revision history for this message
Chad Smith (chad.smith) wrote :

SRU manual verification script

Chad Smith (chad.smith)
description: updated
Revision history for this message
Andreas Hasenack (ahasenack) wrote : Please test proposed package

Hello Chad, or anyone else affected,

Accepted cloud-init into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/23.2.1-0ubuntu0~20.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in cloud-init (Ubuntu Focal):
status: New → Fix Committed
tags: added: verification-needed verification-needed-focal
Revision history for this message
Chad Smith (chad.smith) wrote :

verification logs for focal and all other series (also attached to LP:# 1724623)

tags: added: verification-done verification-done-focal
removed: verification-needed verification-needed-focal
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cloud-init - 23.2.1-0ubuntu0~20.04.2

---------------
cloud-init (23.2.1-0ubuntu0~20.04.2) focal; urgency=medium

  * d/apport-general-hook: correct type hints on Dict for py3.8
    (LP: #2025376)

cloud-init (23.2.1-0ubuntu0~20.04.1) focal; urgency=medium

  * Upstream snapshot based on 23.2.1. (LP: #2025180).
    List of changes from upstream can be found at
    https://raw.githubusercontent.com/canonical/cloud-init/23.2.1/ChangeLog

cloud-init (23.2-0ubuntu0~20.04.1) focal; urgency=medium

  * d/control: Remove pep8 dependency. It is no longer used
  * d/patches/netplan99-cannot-use-default.patch:
    - Retain routes' definitions compatible with netplan 0.99 (Fixes: #4133)
  * d/apport-general-hook.py: Add general apport hook to append cloud type,
    image and instance size information to bug reports (LP: #1724623)
  * d/cloud-init.preinst: Oracle to remove vestigial /etc/cloud.cloud.cfg.d/
    99-disable-network-config.cfg because system config is now honored before
    datasource config (LP: #1956788)
  * d/cloud-init.preinst: Clean up vestigial
    /etc/cloud/cloud.cfg.d/99-disable-network-activation.cfg on GCE instances
    after fix in upstream google-guest-agent.
  * Refresh patches against upstream/main:
    - d/p/expire-on-hashed-users.patch
    - d/p/retain-netplan-world-readable.patch
  * Upstream snapshot based on 23.2. (LP: #2023110).
    List of changes from upstream can be found at
    https://raw.githubusercontent.com/canonical/cloud-init/23.2/ChangeLog

 -- Chad Smith <email address hidden> Thu, 29 Jun 2023 09:41:13 -0600

Changed in cloud-init (Ubuntu Focal):
status: Fix Committed → Fix Released
Revision history for this message
Andreas Hasenack (ahasenack) wrote : Update Released

The verification of the Stable Release Update for cloud-init has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Chad Smith (chad.smith)
Changed in cloud-init (Ubuntu):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.