Unable to boot/install Impish daily in UEFI boot mode

This bug has been reported on the Ubuntu ISO testing tracker.

A list of all reports related to this bug can be found here:
http://iso.qa.ubuntu.com/qatracker/reports/bugs/1937115

This also occurring on Lubuntu Impish daily 2021-07-21
Machine tested Dell [ Optiplex] MT 7060, i7-8700 booting in UEFI+secure boot mode

Further tests on 2 Lenovo machines and cannot reproduce the bug.
Strange but seems to only affect Dell machines -- also tested a Dell laptop 7280 aand also could not boot into efi modes.

Lubuntu impish QA-test install to
- sony vaio svp112a1cw (i5-9400u, 4gb, intel haswell-ULT)
testcase: full disk, uEFI, ... (fails to boot so doesn't matter)

"INVALID IMAGE"
"FAILED TO BOOT"

then quickly boots to "Attempting to decrypt master key..." of the prior QA-test install located on ssd

the actual messages are more than I typed; but they only flash before switching to the ssd boot which boots the prior QA-test install to the box (an encrypted lubuntu install).

Xubuntu & Ubuntu impish dailies also fail to boot in the same/identical manner. The dailies boot though in other boxes tested.

Status changed to 'Confirmed' because the bug affects multiple users.

Another user has reported issue (yesterday) on askubu

https://askubuntu.com/questions/1353382/any-ubuntu-21-10-iso-download-that-isnt-corrupted

Picture of screen end-user reported can be seen at https://i.stack.imgur.com/oeYYg.jpg

The picture fits what @Leok describes I believe; which is different to the sony vaio I used - however as the sony quickly moved to the next boot device (ssd) it booted a prior installed image meaning I never got a good look at it.

Details relating to the machine involved in the previous post by guiverc
========================================================================

Make/Model: Dell Inspiron 5567
CPU: Intel i5-7200U @ 2.5 GHz x 4
RAM: 16GB
GPU: AMD Radeon R7 M445 AMD� Iceland/Mesa; Intel� HD Graphics 620 (KBL GT2)
Desktop Env: Gnome 3.36.8
Windowing System: X11
Storage: 2 x SSD 250 GB + 2 x USB 15/30 GB
Booting: UEFI

Daily 20210721 .iso . This Bug was reported in QA Test Live Session.
Just installed .iso in Virtual Box from a download file,
Tested as Live Session, Success. Loaded good everything worked.
Except for some outdated files that apport would not collect.
Most likely files being used until developed ones finish.
Please check your media and .iso checksum.
Could have a corrupt download or bad USB.

My media and the .iso checksum are good.

Please note that my success in Virtual Box is due to the fact that it is running in BIOS mode.

This bug has been confirmed by other users as well.

Today's boot contains 4 lines (Lubuntu impish) on
- sony vaio svp112a1cw (i5-9400u, 4gb, intel haswell-ULT)

invalid image
failed to load
failed to load
... returns unsupported

the message is still very quick; the screen quickly being cleared and "Attempting to decrypt master key..." from the last successful QA-test install of Lubuntu impish on this box.

The thumb-drive boots normally on other boxes I've tested as usual...

Daily 20210721 .iso . This Bug was reported in QA Test Live Session.
Flashed .iso to USB with Etcher. The image failed due to the Linux Header.
Image did not attempt to start, Showed a warning sign.
UEFI boot mode on computer attempting to do live Session.
Lenovo Think Center M78 / 2 core AMD A6-5400B APU / Radeon HD 7540D Graphics
Display: X11
Daily build 20210731 UEFI failed to boot on USB live. Linux Header Failed.

@bernard010 Bernard Stafford

Please also mark this bug as affecting you and also advise what machine and model this occurred on.

Thank you for testing.

Ubuntu impish ISO fails to boot again on
- sony vaio svp112a1cw (i5-9400u, 4gb, intel haswell-ULT)

It fails to boot if ISO is written using mkusb CLONE OR LIVE modes.

(as noted earlier; applies to Xubuntu & Lubuntu too)

Testing Ubuntu Impish daily ISO 25-07-2021

Tested 8 machines and had 3 boot failures - all machines that failed were Dell machines,and all tests were in UEFI and/or UEFI with secure boot modes.

The boot media was created with "Startup Disk Creator".

The error messages are as follows (see also attached image):

Failed to open \EFI\BOOT\? - Invalid Parameter
Failed to load image \EFI\BOOT\?: Invalid Parameter
start_image() returned Invalid Parameter

Test results posted on the QA testing site -
http://iso.qa.ubuntu.com/qatracker/milestones/424/builds/234198/testcases/1303/results

Well this is a bit strange. Does anyone know when this started? The way the ISO is put together hasn't changed since the hirsute release, so this is probably related to a change in grub2 or shim. The bug was reported less than 24 hours after a new version of grub2 landed which is definitely something that makes one go "hmm". I'll reassign the package there -- this certainly isn't a syslinux problem as that hasn't been used at all to boot ISOs since focal, and afaik never ever when booting via UEFI.

@mwhudson Michael Hudson Doyle

This bug first appeared with the Ubuntu Impish daily on 21 July 2021 as in the original report -
I test the daily builds every day and can confirm that this was the first instance of the bug.

Seems to be an issue with shim / the UEFI firmware on those machines. Could be a regression from the load option parsing, but confused as to how we end up trying to boot a non existing file instead of falling back.

Could also be memory corruption (shim pull #365).

Need to mokutil --set-verbosity true and record the log during boot with a camera.

Please also include the output of efibootmgr -v on those systems. There is likely garbage in the boot loader entries the UEFI generated for those devices.

@juliank Julian Andres Klode

Sorry for any confusion -- how can I run the commands if I cannot boot the machines?

I have posted a work around for this issue upstream: https://github.com/rhboot/shim/pull/393

This will add a 2s delay on invalid filenames or not found files, and then fallback to the default loader. Hence it's still a good idea to parse the load options properly to avoid the errors in the first place :)

Thanks Julian - will check this out when available.

The timing of the bug report matches the rebuild of cd-boot-images-amd64 against shim-signed 1.50

Status changed to 'Confirmed' because the bug affects multiple users.

This bug affects my Dell Latitude E7240 in UEFI mode (but booting works in BIOS mode).

I'd still like to see efibootmgr -v output from affected machines, for the boot entry that failed to boot.

Seeing that this affects multiple machines, bumping the priority. Also, looks like we might want to get this fixed before 20.04.3 is released.

I can report about a workaround:

The current Lubuntu Impish file, impish-desktop-amd64.iso, dated 2021-07-28 17:22 worked for me this way:

- Create a persistent live drive with mkusb-dus.

- Verify that it can boot both in UEFI and legacy BIOS mode.

- Install Lubuntu in UEFI mode (there was no problem for cloned systems to boot in BIOS mode).

But it did not work, when there was a system in the target drive. After wiping the first mibibyte, it was straightforward to install Lubuntu, and I have a working system now :-)

I should add, that I installed in a Toshiba,

http://www.toshiba.se/laptops/satellite-pro/c850/satellite-pro-c850-19w/

and then moved the [internal] SSD to a USB3 to SATA adapter and booted my Dell Latitude E7240 and it works in UEFI mode. I don't want to overwrite the internal M2 drive of my Latitude.

Someone else, who has a computer that is affected by this bug may want to try via mkusb-dus and Lubuntu directly in the affected computer. (But it seems to me, that the problem is booting the live drive, not installing and booting after installation.)

Tested latest Lubuntu Impish ISO dated 29-07-2021 on 2 Dell computers Optiplex 7060 and Optiplex 7040
if I used "startup disk creator" to create the boot media I had 2 failures as in previous tests.

After that I tested the same 2 machines using the suggestion from sudodus(nio-wiklund) and created boot media persistent live media with mkusb-dus - and can confirm that I could boot up in both UEFI and UEFI+secure boot modes.I have not tested booting in BIOS (Legacy) as this has not been a problem.

Hope this is helpful.

Further - here is a link to my test results on the QA site:

http://iso.qa.ubuntu.com/qatracker/milestones/424/builds/234402/testcases/1303/results

@leok,

Thanks for testing :-)

I think it could be interesting if you have time to test in one of your Dell Optiplex computers, if it is possible to *install Lubuntu Impish* from a persistent live session, and that the installed system can boot.

@sudodus (nio-wiklund)

Tested Dell 7060 Optiplex box in UEFI+secure boot mode and installed Lubuntu - rebooted and applied updates and rebooted again - all good -

@leok,

Thanks again :-)

Now we can conclude that the booting problem is only affecting the live system, maybe only a cloned live system (we have not yet tested systems made with Rufus; probably such systems will work too).

@sudodus

I can run a quick check with Rufus created media - will report back later this evening.

Hi folks,

we don't need reports of what works and what doesn't.

We already know that only live media are affected - installed systems usually don't boot via removable media path - but if they do, they go via fallback loader binary which works around the issue as it does not interpret its arguments.

We even have two approaches to work around the issue that are under discussion upstream.

Feedback we're looking for for fixing this better would be output of efibootmgr -v with a tarball of /sys/firmware/efi/efivars/Boot* files, on those systems where you can't boot live media. And which of those boot entries did not work.

What happens here is that the bootloader entries in UEFI contain arguments that get passed to shim. Shim tries to parse them, as it wants to treat the first argument as the bootloader to boot next - whcih is used for fwupd. Some firmware simply contains garbage in there, causing things to fail. We'd like to know what this garbage this such that we can safely ignore it and avoid even the workarounds.

A USB drive with Impish made by Rufus 'UEFI only' & GPT fails in my Dell Latitude E7240 with the same message as a cloned drive.

@juliank,

Where (in which operating system) and when (at what stage) should we run

efibootmgr -v
tar -cvf boot-debug,tar /sys/firmware/efi/efivars/Boot*

Maybe I'm stupid, but I ask because what we want to run does not boot.

Also tested Rufus "UEFI only" & GPT and fails in Dell Optiplex 7060

I also have no idea (asked about this earlier) as to how to run commands on a machine that does not boot?

What am I missing?

output of `efibootmgr -v` on the sony crapbook
- sony vaio svp112a1cw (i5-9400u, 4gb, intel haswell-ULT)
https://paste.ubuntu.com/p/z9rn6Kg625/
FYI: the only OS on it is lubuntu; last focal.3 daily I installed
(these commands were operated from that installed focal.3 system)
---
BootNext: 0005
BootCurrent: 0007
Timeout: 0 seconds
BootOrder: 0005,0006,0007
Boot0005* Sony Original VenHw(99e275e7-75a0-4b37-a2e6-c5385e6c00cb)
Boot0006* Sony Original VenHw(99e275e7-75a0-4b37-a2e6-c5385e6c00cb)
Boot0007* Windows Boot Manager HD(1,GPT,6936f223-1e91-eb4e-8498-0ab5af7b9347,0x1000,0x96000)/File(\EFI\Boot\bootx64.efi)
---

sudo tar -zcvf sony_crap.gz /sys/firmware/efi/efivars/Boot*

got the attached results (hopefully no typos)

This is another "what works, what doesn't" Sorry (made b/c of focal status at top)

But the
- sony vaio svp112a1cw (i5-9400u, 4gb, intel haswell-ULT)
boots the focal daily without issue, even current (20210729).

I've have only had issues with the impish daily.
(the install the prior comment was made from looks like daily 20210716 according to https://phab.lubuntu.me/w/release-team/testing-checklist/ though I'll likely change that shortly)

@sudodos, @leok You can do that in an installed Ubuntu system, on a non-impish live media, heck in any other Linux distro.

We need the new shim for 20.04.3, and this is the sorta blocker, hence the focal tag.

Requested debug data for Dell Latitude E7240:

---
$ efibootmgr -v
BootCurrent: 0012
Timeout: 0 seconds
BootOrder: 0007,0001,000E,0006,000A,0003,0000,0010,0012,0013,0002,0004,0011,0005
Boot0000* ubuntu HD(1,GPT,b3276a58-ea15-4cea-8c74-095b13ea7aa6,0xffff,0xefff1)/File(\EFI\ubuntu\shimx64.efi)
Boot0001* ubuntu HD(3,GPT,3b439a55-6c39-4b8d-a2e4-a0846ab9b467,0xf43,0x7a120)/File(\EFI\ubuntu\shimx64.efi)
Boot0002* Internal HDD BBS(HD,P1: LITEONIT LMT-128M6M mSATA ,0x0)AMBO
Boot0003* ubuntu HD(1,GPT,3dd2bedb-9379-49e0-ab83-52981c20f241,0xffff,0xefff1)/File(\EFI\ubuntu\shimx64.efi)
Boot0004* CD/DVD/CD-RW Drive BBS(CDROM,Generic AutoRun Disk 8.00,0x0)AMBO
Boot0005* Onboard NIC BBS(Network,IBA GE Slot 00C8 v1538,0x0)AMBO
Boot0006* ubuntu HD(3,GPT,0a3953ae-538e-4fd1-aaad-146e05234fdc,0x1000,0x7a000)/File(\EFI\ubuntu\shimx64.efi)
Boot0007* ubuntu HD(3,GPT,e7402217-d036-46d6-9068-db14b58e7449,0x6d7000,0x20000)/File(\EFI\ubuntu\shimx64.efi)
Boot000A* ubuntu HD(1,GPT,4f7d6397-0a47-4cd2-8b7d-897fc6754a99,0xffff,0xefff1)/File(\EFI\ubuntu\shimx64.efi)
Boot000E* Diskette Drive BBS(Floppy,Diskette Drive,0x0)AMBO
Boot0010* UEFI: KINGSTON SUV400S37120G 0 PciRoot(0x0)/Pci(0x1d,0x0)/USB(1,0)/USB(2,0)/HD(1,GPT,db5d8712-381b-48d1-816b-d016596812ec,0x70d8800,0x6ebc000)AMBO
Boot0011* USB Storage Device BBS(USB,JetFlashTranscend 16GB 8.01,0x0)AMBO
Boot0012* UEFI: KINGSTON SUV400S37120G 0 PciRoot(0x0)/Pci(0x1d,0x0)/USB(1,0)/USB(2,0)/HD(3,GPT,b67a20b6-c46d-4979-93eb-222e03f1c50d,0x1000,0x7a000)AMBO
Boot0013* UEFI: KINGSTON SUV400S37120G 0 PciRoot(0x0)/Pci(0x1d,0x0)/USB(1,0)/USB(2,0)/HD(4,GPT,5419b702-448d-4f8d-a230-60b3e10925a3,0x7b000,0x1a2000)/HD(1,MBR,0x18c263b4,0x380,0x1f00)AMBO
---

and an attached tarball.

Requested data for Dell Optiplex 7060:

& efibootmgr -v

BootCurrent: 0000
Timeout: 0 seconds
BootOrder: 0000
Boot0000* ubuntu HD(1,GPT,24470b55-3e6c-c149-bc92-f07e5c4c3940,0x1000,0x96000)/File(\EFI\ubuntu\shimx64.efi)

Attached is tar

On Fri, Jul 30, 2021 at 07:57:26AM -0000, Julian Andres Klode wrote:
> @sudodos, @leok You can do that in an installed Ubuntu system, on a non-
> impish live media, heck in any other Linux distro.

Preferably from live media, because some firmwares synthesize boot entries
on the fly when booting from external media and it's important to see
exactly what that boot entry is to verify why it fails with impish shim.

focal daily [20210731] BOOTS on
- sony vaio svp112a1cw (i5-9400u, 4gb, intel haswell-ULT)

Comment as this is the first daily I've noted with 5.11 HWE kernel (impish daily 20210731 still fails)

Status changed to 'Confirmed' because the bug affects multiple users.

Same boot issue on a Dell Latitude 5410:

From a 20.10 live image the outputs are:

ubuntu@ubuntu:~$ efibootmgr -v
BootCurrent: 0005
Timeout: 2 seconds
BootOrder: 0001,0003,0004,0005
Boot0001* Windows Boot Manager HD(1,GPT,2e375888-8cac-43f2-9558-c9005ab9eb9d,0x800,0xfa000)/File(\EFI\Microsoft\Boot\bootmgfw.efi)WINDOWS.........x...B.C.D.O.B.J.E.C.T.=.{.9.d.e.a.8.6.2.c.-.5.c.d.d.-.4.e.7.0.-.a.c.c.1.-.f.3.2.b.3.4.4.d.4.7.9.5.}...f................
Boot0003* Onboard NIC(IPV4) PciRoot(0x0)/Pci(0x1f,0x6)/MAC(747827081eea,0)/IPv4(0.0.0.00.0.0.0,0,0)..BO
Boot0004* Onboard NIC(IPV6) PciRoot(0x0)/Pci(0x1f,0x6)/MAC(747827081eea,0)/IPv6([::]:<->[::]:,0,0)..BO
Boot0005* UEFI: General USB Flash Disk 1.00, Partition 2 PciRoot(0x0)/Pci(0x14,0x0)/USB(2,0)/HD(2,GPT,7ee1ffac-4072-46b8-885c-a7ea3f9c70cf,0x57843c,0x26e0)..BO

tarball attached.

When will the bugfixed impish iso file be uploaded and available for us to test?

First we need to submit a shim for review and signing, then upload it, so later next week presumably. I just did the paperwork here for now :)

I am subscribing as my boot issue seems similar; I wonder if this is affecting some other Hairy Hippo users?

My boot-info detail: https://paste.ubuntu.com/p/ZYWPgTJJPG/
My boot-repair info: https://paste.ubuntu.com/p/YcSFTnxjVZ/

The laptop is an HPP455 G2 with a 1TB SSD partitioned to include a separate Data partition. Secure boot is disabled. I have installed a dual boot Windows 10 and Ubuntu 21.04 system and (with some difficulty) had it working well.

Expected behaviour: On switch on or restart the system boots into the Ubuntu dual boot screen, allowing user to select Ubuntu or Windows; if no selection within ten seconds boots into Ubuntu.

Observed behaviour: On switch on or restart the system fails to boot with the error messages as below; looping through the initial post screen and into the error message. Selecting "Esc" at the post screen allows selection of efi boot options; the third of which is "ubuntu" - this takes me to the dual boot screen after which boot proceeds into the selected (or default) OS as expected.

On first installing the SDD I installed Windows then Ubuntu. Since the system did not boot properly I used 'Boot-Repair' Advanced options and followed the directions to "change the default boot entry of your Windows bootloader ... boot into Windows then type the following in an admin command prompt: bcdedit /set {bootmgr} path \EFI\ubuntu\shimx64.efi . The system then booted normally into the standard Ubuntu dual boot screen. It has been ok for the past three months or so.

Today I did a restart and I was unable to boot as normal, however could enter the EFI boot options screen and select Ubuntu, which then went to the dual boot screen from where boot is normal into Ubuntu or Windows.

I again used Boot-Repair Advanced options and again followed the directions to "change the default boot entry of your Windows bootloader ... boot into Windows then type the following in an admin command prompt: bcdedit /set {bootmgr} path \EFI\ubuntu\shimx64.efi . This did not sort the problem out.

Full details of error message:

  Failed to open \efi\ubuntu\S - Invalid Parameter

  Failed to load image \efi\ubuntu\S: - Invalid Parameter

  start_image() returned Invalid Parameter

I would be grateful for any advice.

Springnuts:

So if I understand you correctly, the ubuntu boot entry works for you. I think what happens is that Windows changes the boot order during boot, or the firmware forgets it, which is a bit outside our ability to fix.

Unfortunately, I don't think I'm able to help you here. I don't see a boot entry containing S outside WINDOWS in your boot entry.

IIRC, Boot-Repair is known to break boot, and we can't really provide any support for it or Windows boot loader editing tools.

In either case, the next shim should boot for you in those situations, albeit with a 2s delay if it's booting via shimx64.efi (0s if it's booting via bootx64.efi).

I believe the bcdedit command causes the Window's equivalent of shim (the bootx64.efi) to install to then run shim as the second stage, and hence it might pass windows specific arguments to it that shim does not understand, but I can't say for sure, it's a Windows question

Ubuntu Impish AMD64 daily live 20210806 image for QA Testing failed to boot. Unsupported Linux header , posted this warning. Used Etcher to make USB with Linux OS. UEFI failed to boot flash drive.

I've attached a tarball of my Boot efivars as requested by Julian and the output of efibootmgr -v below.

This is from an affected Dell XPS 9343 machine

BootCurrent: 0004
Timeout: 1 seconds
BootOrder: 0004,0006
Boot0004* elementary OS 6 PciRoot(0x0)/Pci(0x1f,0x2)/Sata(3,65535,0)/HD(1,GPT,d20333f5-d8c1-4a44-af39-21ace0ec620d,0x1000,0x838f7)/File(\EFI\ubuntu\shimx64.efi)
Boot0006* UEFI: SanDisk Extreme 0001 PciRoot(0x0)/Pci(0x14,0x0)/USB(12,0)/HD(1,MBR,0x38b1c112,0x6a4,0x1f40)..BO

Ubuntu Impish doesn't boot on Dell Precision 7530

Failed to open \EFI\BOOT\? - Invalid Parameter
Failed to load image \EFI\BOOT\?: Invalid Parameter
start_image() returned Invalid Parameter

Same problem with new shim on Fedora 34
https://bugzilla.redhat.com/show_bug.cgi?id=1954245

It gave me this error:

Failed to open \EFI\BOOT\? - Invalid Parameter
Failed to load image \EFI\BOOT\?: Invalid Parameter
start_image() returned Invalid Parameter

Hello Leó, or anyone else affected,

Accepted shim into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/shim/15.4-0ubuntu9 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

This bug was fixed in the package shim - 15.4-0ubuntu9

---------------
shim (15.4-0ubuntu9) hirsute; urgency=medium

  * Fix booting installer media on some machines (LP: #1937115)
    - Always fallback to the default loader (PR #393)
    - Dump load options parsed (PR #393)
    - Disable load option parsing on removable media path (PR #399)
  * trivial: Fix a minor overflow in the mok importing code (PR #365)
  * Fix fall back loader to find the correct boot entry, avoiding potential
    corruption of firmware (PR #396).

 -- Julian Andres Klode <email address hidden> Fri, 06 Aug 2021 13:16:33 +0200

@brian-murray

When will Focal and Impish ISO's be available for QA testing?

Hello Leó, or anyone else affected,

Accepted shim-signed into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/shim-signed/1.40.7 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

On Fri, Aug 13, 2021 at 08:28:31PM -0000, Leó Kolbeinsson wrote:
> @brian-murray
>
> When will Focal and Impish ISO's be available for QA testing?

Tomorrow's daily build of Impish should pickup the new version of shim.
You could check the manifest file before downloading the ISO to be sure.

http://cdimage.ubuntu.com/ubuntu/daily-live/current/impish-desktop-amd64.manifest

--
Brian

This bug was fixed in the package shim-signed - 1.51

---------------
shim-signed (1.51) impish; urgency=medium

  * Update to shim 15.4-0ubuntu9
    - Fix booting installer media on some machines (LP: #1937115)
      + Always fallback to the default loader (PR #393)
      + Dump load options parsed (PR #393)
      + Disable load option parsing on removable media path (PR #399)
    - trivial: Fix a minor overflow in the mok importing code (PR #365)
    - Fix fall back loader to find the correct boot entry, avoiding potential
      corruption of firmware (PR #396).

 -- Julian Andres Klode <email address hidden> Fri, 13 Aug 2021 18:00:15 +0200

Tested Kubuntu Impish daily ISO - http://cdimage.ubuntu.com/kubuntu/daily-live/20210814/impish-desktop-amd64.iso which included ""shim-signed 1.51+15.4-0ubuntu9""

and 2 failures on Dell Optiplex machines bootin UEFI and UEFI+ secure boot

Test results on QA test site
http://iso.qa.ubuntu.com/qatracker/milestones/424/builds/235250/testcases/1303/results

will test other flavours as they become available.

Failure for Kubuntu Impish daily (2021-08-14) on
- sony vaio svp112a1cw (i5-9400u, 4gb, intel haswell-ULT)

Manifest says "shim-signed 1.51+15.4-0ubuntu9"

(QA-test notes can be found on iso.qa.ubu provided in prior comment #63)

this requires a respin of the cd-boot-images-amd64 package before it will actually be picked up for use in the image boot.

Tested Impish daily ISO's

Kubuntu = http://cdimage.ubuntu.com/kubuntu/daily-live/20210815/impish-desktop-amd64.iso

Xubuntu = http://cdimage.ubuntu.com/xubuntu/daily-live/20210815/impish-desktop-amd64.iso

Manifests for both Kubuntu and Xubuntu state "shim-signed 1.51+15.4-0ubuntu9"

Again 2 Dell Optiplex boxes fail to boot in UEFI+secure boot and UEFI modes

See test results on QA test site

Kubuntu = http://iso.qa.ubuntu.com/qatracker/milestones/424/builds/235299/testcases/1303/results
Xubuntu = http://iso.qa.ubuntu.com/qatracker/milestones/424/builds/235295/testcases/1303/results

Please read Steve's comment, Leo. The images are not ready yet, they contain the old shim (the new one is inside the live system only).

Sorry ---I saw Steve´s comment but assumed that the manifests were correct.
Would appreciate being notified when the new shim (images) are available for testing.

The manifest is accurate, it just doesn't say what you think. It is the list of packages and versions included within the squashfs - it says nothing about the bootloader on the image.

I intend to update cd-boot-images on Monday if no one beats me to it

@vorlon Steve Langasek

Thanks for the clarification! Looks like one can learn something everyday!

While the updated Impish images are not yet available the Focal ones are and given that this is an issue with a failure to boot it should be quite easy to test the Focal image too. Would you mind doing that in the meantime? Thanks in advance.

@brian-murray Brian Murray

Yes - more than willing to test Focal images.

I see that the latest Focal manifest for Ubuntu http://cdimage.ubuntu.com/focal/daily-live/current/focal-desktop-amd64.manifest does not list the new shim. Re comment nr 69 - How can I determine that the bootloader has been updated?

Assume it is better to check the "file listing" i.e. http://cdimage.ubuntu.com/focal/daily-live/current/focal-desktop-amd64.list

is that not correct?

This bug was fixed in the package cd-boot-images-amd64 - 16

---------------
cd-boot-images-amd64 (16) impish; urgency=medium

  [ Brian Murray ]
  * Fix typos in debian/control.

  [ Julian Andres Klode ]
  * Rebuild against shim 1.51 (LP: #1937115)

 -- Julian Andres Klode <email address hidden> Mon, 16 Aug 2021 13:00:56 +0200

Tested Focal daily ISO Ubuntu http://cdimage.ubuntu.com/focal/daily-live/20210816/focal-desktop-amd64.iso

Tested 4 Dell machines 2 Lenovo and 1 AWOW - all passed and booted successfully - all tests UEFI+secure boot and UEFI modes - total 7 boxes

Test results here : http://iso.qa.ubuntu.com/qatracker/milestones/408/builds/235363/testcases/1303/results

Correction FOCAL tested

Apparently shim is not pre-installed in focal, but installed from pool, so yes, file list is correct:

/pool/main/s/shim-signed/shim-signed_1.40.7+15.4-0ubuntu9_amd64.deb

So this is good, marking as verified then

Booted the current Ubuntu iso files from the iso testing web-site

-rw------- 2,9G 2021-08-16 08:21 "focal-desktop-amd64.iso"
-rw------- 2,9G 2021-08-16 08:30 "impish-desktop-amd64.iso"

in a Dell Latitude E7240 in UEFI mode.

The impish iso file fails :-(

Then I read the latest posts by you developers, so I zsynced the focal iso file and it boots :-)

Daily build 20210816.iso REJECT The Header, Kernel, Modules & Extra Modules UNSUPPORTED! For UEFI Install using USB.. Latest FIX is a FAILURE. COMPLETE FAIL< WILL NOT BOOT...
Latest Kernel I have tested that works is 5.13.11 @ kernel.org. Works Awesome for this Version of Ubuntu impish.
Tested 4 times.

KERNEL FAILED ON UEFI BOOT Not Fixed...!!!
UPDATE KERNEL Please!
Daily build 20210816.iso REJECT The Header, Kernel, Modules & Extra Modules UNSUPPORTED! For UEFI Install using USB.. Latest FIX is a FAILURE. COMPLETE FAIL< WILL NOT BOOT...
Latest Kernel I have tested that works is 5.10.60 @ kernel.org. Works Awesome for this Version of Ubuntu Impish.
Tested 4 times. With new current Kernel

Both Lubuntu & Ubuntu(desktop) focal dailies (2021-08-16) boot on
- sony vaio svp112a1cw (i5-9400u, 4gb, intel haswell-ULT)

(the only box I have that was impacted by this bug)

The daily images from 20210817 should have the latest version of shim on them now so feel free to test that.

Tested Kubuntu daily ISO 170821 no boot errors

see: http://iso.qa.ubuntu.com/qatracker/milestones/424/builds/235402/testcases/1300/results

for details.

Tested the current Lubuntu Impish iso file dated

2021-08-17 17:22 "impish-desktop-amd64.iso"

Booting a cloned USB drive works [not only in BIOS mode but also] in UEFI mode in a Dell Latitude E7240 :-)

So I can confirm that the bugfix works also to boot Lubuntu Impish in my Dell.

I tested the current Lubuntu Impish live also in a Dell Precision M4800. It works in UEFI mode too.

Another confirm that bux fix good in Lubuntu

Tested Lubuntu Impish daily ISO 17-08-2021

Results here = http://iso.qa.ubuntu.com/qatracker/milestones/424/builds/235442/testcases/1701/results

InWin BL641 BIOS mode in VirtBox - success
Dell optiplex 7040 in UEFI mode - success
Dell optiplex 7060 in UEFI+ secure boot mode - success

Thanks, this is a generic fix that has been confirmed, verified for focal and released for impish. No further tests/reports needed.

Thank you for all the verification! I also confirmed this shim is working correctly on physical hardware - let me release it into focal-updates in a few moments.

This bug was fixed in the package shim - 15.4-0ubuntu9

---------------
shim (15.4-0ubuntu9) hirsute; urgency=medium

  * Fix booting installer media on some machines (LP: #1937115)
    - Always fallback to the default loader (PR #393)
    - Dump load options parsed (PR #393)
    - Disable load option parsing on removable media path (PR #399)
  * trivial: Fix a minor overflow in the mok importing code (PR #365)
  * Fix fall back loader to find the correct boot entry, avoiding potential
    corruption of firmware (PR #396).

 -- Julian Andres Klode <email address hidden> Fri, 06 Aug 2021 13:16:33 +0200

This bug was fixed in the package shim-signed - 1.40.7

---------------
shim-signed (1.40.7) focal; urgency=medium

  * Update to shim 15.4-0ubuntu9
    - Fix booting installer media on some machines (LP: #1937115)
      + Always fallback to the default loader (PR #393)
      + Dump load options parsed (PR #393)
      + Disable load option parsing on removable media path (PR #399)
    - trivial: Fix a minor overflow in the mok importing code (PR #365)
    - Fix fall back loader to find the correct boot entry, avoiding potential
      corruption of firmware (PR #396).

 -- Julian Andres Klode <email address hidden> Fri, 13 Aug 2021 18:07:24 +0200

The verification of the Stable Release Update for shim has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

QA Testing of Live daily build 20210819. UEFI boots properly flashed to USB key with Etcher.
This Bug Report is sufficiently satisfied.

@ Julian Andres Klode (juliank),

Please explain:

How does this bug affect the old versions Xenial and Bionic?

And how should we test that our computers are affected?

recently... while booting the ISO image for the Impish live build
the following error lines are displayed:

[ 1.501001] gpio gpiochip: (gpio_aaeon): tried to insert a GPIO chip with zero lines
[ 1.501013] gpiochip_add_data_with_key: GPIOs O..-1 (gpio_aaeon) failed to register. -22
[ 0.197133] gpio gpiochip: (gpio_aaeon): tried to insert a GPIO chip with zero lines
[ 0.197161] gpiochip_add_data_with_key: GPIOs O..-1 (gpio_aaeon) failed to register. -22

After a pause the iso then continues to boot and a successful installation proceeds without any signs of further error.

this first occurred at 20210820 (since fix released)

I have an own build PC with nVidia graphics and use 'safe graphics' to boot the iso
Prior to this I was not experiencing the problems that had been occurring with Dell laptops.

The matter reported in post #92 appears to be in process of being dealt with in Bug#1937897

Hello Leó, or anyone else affected,

Accepted shim into hirsute-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/shim/15.4-0ubuntu9 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-hirsute to verification-done-hirsute. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-hirsute. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello Leó, or anyone else affected,

Accepted shim-signed into hirsute-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/shim-signed/1.51 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-hirsute to verification-done-hirsute. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-hirsute. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello Leó, or anyone else affected,

Accepted shim into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/shim/15.4-0ubuntu9 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello Leó, or anyone else affected,

Accepted shim-signed into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/shim-signed/1.37~18.04.11 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Marking as verified as per

"Verification is not necessary for individual SRU releases."

We did not touch any code paths involving shim-grub interaction, the fix here for disabling boot option parsing on removable media paths is valid across all releases, since the binary is the same.

This bug was fixed in the package shim - 15.4-0ubuntu9

---------------
shim (15.4-0ubuntu9) hirsute; urgency=medium

  * Fix booting installer media on some machines (LP: #1937115)
    - Always fallback to the default loader (PR #393)
    - Dump load options parsed (PR #393)
    - Disable load option parsing on removable media path (PR #399)
  * trivial: Fix a minor overflow in the mok importing code (PR #365)
  * Fix fall back loader to find the correct boot entry, avoiding potential
    corruption of firmware (PR #396).

 -- Julian Andres Klode <email address hidden> Fri, 06 Aug 2021 13:16:33 +0200

This bug was fixed in the package shim-signed - 1.51

---------------
shim-signed (1.51) impish; urgency=medium

  * Update to shim 15.4-0ubuntu9
    - Fix booting installer media on some machines (LP: #1937115)
      + Always fallback to the default loader (PR #393)
      + Dump load options parsed (PR #393)
      + Disable load option parsing on removable media path (PR #399)
    - trivial: Fix a minor overflow in the mok importing code (PR #365)
    - Fix fall back loader to find the correct boot entry, avoiding potential
      corruption of firmware (PR #396).

 -- Julian Andres Klode <email address hidden> Fri, 13 Aug 2021 18:00:15 +0200

This bug was fixed in the package shim-signed - 1.37~18.04.11

---------------
shim-signed (1.37~18.04.11) bionic; urgency=medium

  * Update to shim 15.4-0ubuntu9
    - Fix booting installer media on some machines (LP: #1937115)
      + Always fallback to the default loader (PR #393)
      + Dump load options parsed (PR #393)
      + Disable load option parsing on removable media path (PR #399)
    - trivial: Fix a minor overflow in the mok importing code (PR #365)
    - Fix fall back loader to find the correct boot entry, avoiding potential
      corruption of firmware (PR #396).

 -- Julian Andres Klode <email address hidden> Tue, 07 Sep 2021 12:03:11 +0200

This bug was fixed in the package shim - 15.4-0ubuntu9

---------------
shim (15.4-0ubuntu9) hirsute; urgency=medium

  * Fix booting installer media on some machines (LP: #1937115)
    - Always fallback to the default loader (PR #393)
    - Dump load options parsed (PR #393)
    - Disable load option parsing on removable media path (PR #399)
  * trivial: Fix a minor overflow in the mok importing code (PR #365)
  * Fix fall back loader to find the correct boot entry, avoiding potential
    corruption of firmware (PR #396).

 -- Julian Andres Klode <email address hidden> Fri, 06 Aug 2021 13:16:33 +0200

This has been reported in the ISO tracker as affecting the 18.04.6 candidate images (20210913):

http://iso.qa.ubuntu.com/qatracker/milestones/426/builds/236713/testcases

The reported issue happens on bare metal. In QEMU/KVM the 20210913 bionic-live-server-amd64 images work fine with secureboot enabled (checked via `mokutil --sb-state` in both the installer and installed systems).

This has also been reported in the ISO tracker as affecting the 18.04.6 candidate images Ubuntu Desktop (20210913):

http://iso.qa.ubuntu.com/qatracker/milestones/426/builds/236704/testcases/1300/results

These tests on bare metal machines as were the server tests in comment # 103 above.

This is because debian-installer somehow was uploaded for 18.04.6 built against shim-signed 1.37~18.04.10+15.4-0ubuntu7 instead of shim-signed 1.37~18.04.11+15.4-0ubuntu9. Not sure how or why this happened, but shim-signed has been released to bionic-updates now, so I'll rebuild debian-installer for the fix.

Hello Leó, or anyone else affected,

Accepted debian-installer into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/debian-installer/20101020ubuntu543.19 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

failure to boot 18.04.6 ISO (20210913.1) on
- sony vaio svp112a1cw (i5-9400u, 4gb, intel haswell-ULT)

Can't add -proposed (easily) to a system that fails to boot :(

4 error lines briefly appear on screen before box moves to next boot device; which is SSD & lubuntu impish GPG key is requested..

@Steve I'm still a bit puzzled by the reported failure [1], which is against the live-server (subiquity) image, and not against the d-i image, but according to the problem description the failure mode is similar (image fails to boot).

Could it be that some other component in the live-server image was built against the old shim, as it happened for debian-installer?

[1] http://iso.qa.ubuntu.com/qatracker/milestones/426/builds/236713/testcases

The boot bits of all the hybrid images are laid down by debian-cd, which for bionic takes those bits from the debian-installer build, and not from the archive directly.

That's reassuring, thanks Steve. Now waiting for a respin to proceed with the tests.

@Paride So I don't know these 'legacy' boot bits, but quoting Steve from yesterday:

"vorlon: yes, the boot bits for both images are copied from debian-installer output. In later releases this is done via xnox's cd-boot-images instead"

So possibly this might also cause issues for all isos?

This bug was fixed in the package debian-installer - 20101020ubuntu543.19

---------------
debian-installer (20101020ubuntu543.19) bionic; urgency=medium

  * Fix Vcs-Bzr branch target.
  * Rebuild against shim-signed 1.37~18.04.11+15.4-0ubuntu9. LP: #1937115.

 -- Steve Langasek <email address hidden> Tue, 14 Sep 2021 14:36:54 -0700

Ubuntu 18.04.6 Desktop ISO (20210915) now boots on
- sony vaio svp112a1cw (i5-9400u, 4gb, intel haswell-ULT)

(failure in comment #107)

I can also confirm that Ubuntu 18.04.06 desktop and server ISO's (29210915) boot on boxes that previously failed - see my comments # 103 and 104 above.

Test results on QA tracking site here:

1. Desktop tests = http://iso.qa.ubuntu.com/qatracker/milestones/426/builds/236832/testcases/1303/results

2. Server tests = http://iso.qa.ubuntu.com/qatracker/milestones/426/builds/236831/testcases/1697/results/

I assume we're not going to fix this for Xenial?