[regression] cryptsetup does not work on raw encrypted drives

Attached is the proposed fix.

Once jaunty opens, I can get the cryptsetup patch uploaded so the SRU can move forward.

The vol_id call was introduced to parse LUKS data? I see that removing it unbreaks raw devices, but doesn't vol_id also make sure that the device is fully settled, as opposed to a RAID just being halfway up or so? Or is that handled later?

You don't need to wait until Jaunty opens. We can upload to intrepid-proposed, let it build, and copy the source+binary to jaunty (I'll do that, it's quite common right after releases, where the new one isn't open yet and we have plenty of SRUs).

If there's still a race remaining, it would be solved by the time the user went to enter their password for the device. I'm not sure there is a better way to test that the device is ready since it can be a raw-encrypted drive.

True that, I actually meant "ready" for a block layer level. But that should be checked at a different place anyway.

Has this check been there in hardy?

There was no device check loop/timeout in Hardy. Just a single udevsettle
and a -e test which would immediately fail if the device was missing.
The intrepid changes are certain much better, but the use of "vol_id"
is not valid.

The diff in the description for bug 164044 shows the changes pretty well
(i.e. replacing a single -e test with a loop using -e and vol_id). It
is basically just a think-o when doing the copy/paste from local's
version of this loop.

Kees Cook <email address hidden> writes:

> If there's still a race remaining, it would be solved by the time the
> user went to enter their password for the device.

For the passphrase case that would be indeed the ideal
solution. Problems with that approach:

 - the user might use a keyfile on a removable media (usb, etc.)
 - we need to name the drive the user is about to enter his password for

TBH, I thought we decided to drop the non-LUKS case ages ago. Probably
it happened to start working after my last merge, but I have to admit
that while integrating the proposed patch, I didn't really consider
the non-LUKS case.

Having said this, I'd really welcome a proper solution for
cryptset/initramfs integration. kirkland told me that the work he did on
the degraided-raid spec would be very helpful for the use case we are
talking here about.

I suggest that a bof is scheduled with at least kirkland, pitti, kees at
the next UDS. If time is permitting, I'll try to join via VOIP.

--
Gruesse/greetings,
Reinhard Tartler, KeyID 945348A4

Hi Kees,

Kees Cook [2008-10-31 21:45 -0000]:
> The diff in the description for bug 164044 shows the changes pretty well
> (i.e. replacing a single -e test with a loop using -e and vol_id). It
> is basically just a think-o when doing the copy/paste from local's
> version of this loop.

Ah, that makes sense. Please upload then.

Accepted into intrepid-proposed, please test and give feedback here. Please see https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

intrepid-proposed package copied to jaunty.

Kees, anyone, please test the version in -proposed. It has sat there for three months, time to either pull it or verify it.

Sorry, I didn't think my opinion counted on this one since I uploaded it originally. Yes, this works for me without problems. I will try to get soren to comment.

Kees Cook [2009-01-28 19:01 -0000]:
> Sorry, I didn't think my opinion counted on this one since I uploaded it
> originally.

Well, if you make sure that you use the actual intrepid-proposed
package, as opposed to a local build, that's ok.

Thanks!

I've installed cryptsetup from intrepid-proposed on my laptop. The system has been restarted and things are working as before.

This bug was fixed in the package cryptsetup - 2:1.0.6-6ubuntu2.1

---------------
cryptsetup (2:1.0.6-6ubuntu2.1) intrepid-proposed; urgency=low

  * debian/initramfs/cryptroot-script: do not require that vol_id
    can parse the encrypted device as valid (LP: #291752).

 -- Kees Cook <email address hidden> Fri, 31 Oct 2008 13:10:06 -0700

Hi,

My intrepid system was updated with cryptsetup 2:1.0.6-6ubuntu2.1 a short while ago (via intrepid-updates).

Ever since, it will not boot with my encrypted root file system. It asks for the password as usual, and when I enter it (correctly), it reports: "cryptsetup: cryptsetup failed, bad password or options?".

When I boot in recovery mode, I see that something is reported about /dev/md1 not being a block device.

I've restored the cryptoroot file in /usr/share/initramfs-tools/scripts/local-top to what it was before (using grub to choose an initrd that does not contain this version of cryptsetup yet), and ran update-initramfs -u. This has solved my problem. It seems that the vol_id call actually does something that is required for my setup.

My setup is as follows:

- 5 disks of equal size.
- First part of each disk is part of a RAID1 (using a Linux raid autodetect partition) that I use as /boot (ext3).
- Remaining space on each drive is part of a RAID5 (using a Linux raid autodetect partition).
- The RAID5 is encrypted using LUKS.
- The encrypted RAID5 contains LVM with five logical volumes of varying sizes.

If I can do anything to help sorting this out, please let me know.

With kind regards,
Sven

Forgot to mention that / is one of the LVM logical volumes.

Sven

Sven,

Thanks for the follow-up. I'm reopening this bug and marking it as 'critical' since it appears to be a regression in a published update.

I believe the developer who prepared the update is away this week, but we'll see what we can do about getting this resolved - if nothing else, by reverting the change in this update.

Comments from Scott James Remnant, resident udev expert:

18:05 < Keybuk> device node can exist, but have no filesystem inside it, because other things need to do magic
18:05 < Keybuk> e.g. md and devmapper devices
18:06 < Keybuk> the device node shows up when you create the name
18:06 < Keybuk> before you load the table
18:07 < Keybuk> so you need to check
18:07 < Keybuk> 1. does the device node exist? [ -e ]
18:07 < Keybuk> 2. has udev finished with it? udevadm settle
18:07 < Keybuk> 3. has it got a filesystem inside it? vol_id

We actually probably *don't* need 3) as long as we have 2); but in any case this explains why the preceding update was wrong (though it surprises me that anyone is hitting this race in practice). I'll prepare an emergency update to revert this change, then we can re-evaluate how to fix this for everyone in SRU.

cryptsetup 1.0.6-6ubuntu2.2 uploaded. Martin/Colin, please review and accept.

Accepted into intrepid-proposed; please test and give feedback here. Please see https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

This needs to be fixed in Jaunty then, too.

Sven, if you could test this new package in intrepid-proposed that would be great (it should be available on archive.ubuntu.com in about 1.5 hours). Since this reverts the package to the exact state of intrepid final, we can waive the 7 day maturing period here.

I propose to set the original bug to "wontfix" in Intrepid then, at least until we found a rock solid solution in Jaunty. Or decide to entirely drop support for non-LUKS partitions.

On Tue, Feb 24, 2009 at 07:28:34AM -0000, Martin Pitt wrote:
> I propose to set the original bug to "wontfix" in Intrepid then, at
> least until we found a rock solid solution in Jaunty. Or decide to
> entirely drop support for non-LUKS partitions.

I believe using 'udevadm settle' in place of the vol_id call gives us that;
was waiting until after alpha-5 to test it since it's not a milestone
blocker and I'm nervous around udev.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
<email address hidden> <email address hidden>

Hello,

I've upgraded cryptsetup to the version in intrepid-proposed and it has indeed solved my problem. I can reboot now and it simply works :-)

Thank you all for the very quick solution!

Sven

This bug was fixed in the package cryptsetup - 2:1.0.6-6ubuntu2.2

---------------
cryptsetup (2:1.0.6-6ubuntu2.2) intrepid-proposed; urgency=low

  * Revert previous update; this introduces a race condition when the
    underlying device is a md device or similar, whose device node
    appears before udev is done with it. LP: #291752.

 -- Steve Langasek <email address hidden> Tue, 24 Feb 2009 03:58:11 +0000

Thank you for quick testing, copied to -updates.

So with that we are now back to the original bug filed by Kees. I reset the tags and bug status.

A new revision of cryptsetup has been uploaded to jaunty that should work for both use cases. Changelog:

cryptsetup (2:1.0.6-7ubuntu6) jaunty; urgency=low

  * debian/initramfs/cryptroot-script: we don't require vol_id to understand
    the encrypted device, but we should check the device is fully up first
    before continuing by calling udevadm settle. LP: #291752.

Accepted cryptsetup into intrepid-proposed; please test and give feedback here. Please see https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Sven, it would be particularly helpful if you could test this new version of cryptsetup in intrepid-proposed, since you were reporting the regression before. This new upload *should* enable cryptsetup on raw encrypted drives while also working for crypted RAID devices.

Steve,

I've tested the version in intrepid-proposed on the computer where the cryptsetup package previously failed, and the new version works as expected!

Thanks for looking into this!
Sven

This bug was fixed in the package cryptsetup - 2:1.0.6-6ubuntu2.3

---------------
cryptsetup (2:1.0.6-6ubuntu2.3) intrepid-proposed; urgency=low

  * debian/initramfs/cryptroot-script: we don't require vol_id to understand
    the encrypted device, but we should check the device is fully up first
    before continuing by calling udevadm settle. LP: #291752.

 -- Steve Langasek <email address hidden> Sun, 08 Mar 2009 16:07:57 -0700