Wrong SHA256-value computed on kinetic

Status changed to 'Confirmed' because the bug affects multiple users.

The attachment "openldap-contrib-sha2.patch" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

lunar, kinetic, and jammy all return the first result, while focal provides the second:

triage-lunar+23.04: ~$ slappasswd -s secret -h '{SHA256}' -o module-load=pw-sha2
{SHA256}WIrrpN3OjEVOUf6yrH1j+o+ODuUuNBo979Od4UXnu54=
triage-lunar+23.04: ~$ slapd -VV
@(#) $OpenLDAP: slapd 2.6.3+dfsg-1~exp1ubuntu1 (Nov 18 2022 21:07:45) $

triage-kinetic+22.10: ~$ slappasswd -s secret -h '{SHA256}' -o module-load=pw-sha2
{SHA256}WIrrpN3OjEVOUf6yrH1j+o+ODuUuNBo979Od4UXnu54=
triage-kinetic+22.10: ~$ slapd -VV
@(#) $OpenLDAP: slapd 2.5.13+dfsg-1ubuntu1 (Sep 20 2022 19:30:47) $

triage-jammy+22.04: ~$ slappasswd -s secret -h '{SHA256}' -o module-load=pw-sha2
{SHA256}WIrrpN3OjEVOUf6yrH1j+o+ODuUuNBo979Od4UXnu54=
triage-jammy+22.04: ~$ slapd -VV
@(#) $OpenLDAP: slapd 2.5.13+dfsg-0ubuntu0.22.04.1 (Aug 5 2022 14:51:52) $

triage-focal+20.04: ~$ slappasswd -s secret -h '{SHA256}' -o module-load=pw-sha2
{SHA256}K7gNU3sdo+OL0wNhqoVWhr3g6s1xYv72ol/pe/Unols=
triage-focal+20.04: ~$ slapd -VV
@(#) $OpenLDAP: slapd (Ubuntu) (May 12 2022 13:11:05) $
triage-focal+20.04: ~$ apt-cache policy slapd
slapd:
  Installed: 2.4.49+dfsg-2ubuntu1.9

On all releases, the openssl dgst call produces the same result, K7gNU3sdo+OL0wNhqoVWhr3g6s1xYv72ol/pe/Unols=

Here's two other references mentioning the same problem, and same suggested workaround:

* https://www.mail-archive.com/search?<email address hidden>&q=subject:%22%22&o=newest&f=1
* https://stackoverflow.com/questions/74928752/slappasswd-generating-a-strange-password-hash-sha256-only

I don't know whether there might be side effects from adding "-fno-strict-aliasing". However, the patch's compilation modifications looks like it'll affect the performance of only just the sha2 module, so for SRU policy this seems a narrow enough fix. Since this is described in the first link as a contrib module, that may explain why this issue hasn't come to light earlier.

Writing down some options from a quick handover discussion I had with Sergio, in no particular order:
- rebuild openldap with the attached patch, disabling strict aliasing just for that module
- check if LTO is having an effect on this: maybe disabling LTO also fixes it, and there is precedence for such a change in an SRU
- maybe the module code can be changed to not trigger this bug
- future-land: make the module use gnutls for such functions

This bug was fixed in the package openldap - 2.6.3+dfsg-1~exp1ubuntu2

---------------
openldap (2.6.3+dfsg-1~exp1ubuntu2) lunar; urgency=medium

  * Build the passwd/sha2 contrib module with -fno-strict-aliasing to
    avoid computing an incorrect SHA256 hash with some versions of the
    compiler (LP: #2000817):
    - d/t/{control,sha2-contrib}: test to verify the SHA256 hash
      produced by passwd/sha2
    - d/rules: set -fno-strict-aliasing only when building the
      passwd/sha2 contrib module
  * d/t/smbk5pwd: Allow the openldap user to read the Heimdal master key in the
    smbk5pwd DEP8 test (LP: #2004560)

 -- Andreas Hasenack <email address hidden> Fri, 03 Feb 2023 09:33:14 -0300

Hello Christian, or anyone else affected,

Accepted openldap into kinetic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/openldap/2.5.14+dfsg-0ubuntu0.22.10.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-kinetic to verification-done-kinetic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-kinetic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

All autopkgtests for the newly accepted openldap (2.5.14+dfsg-0ubuntu0.22.10.2) for kinetic have finished running.
The following regressions have been reported in tests triggered by the package:

cyrus-sasl2/2.1.28+dfsg-6ubuntu2 (armhf)
exim4/4.96-3ubuntu1.1 (ppc64el)
kopanocore/8.7.0-7.1ubuntu11 (armhf)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/kinetic/update_excuses.html#openldap

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Kinetic verification

Confirming the problem with the unfixed slapd package:

$ apt-cache policy slapd
slapd:
  Installed: 2.5.14+dfsg-0ubuntu0.22.10.1
  Candidate: 2.5.14+dfsg-0ubuntu0.22.10.1
  Version table:
 *** 2.5.14+dfsg-0ubuntu0.22.10.1 500
        500 http://br.archive.ubuntu.com/ubuntu kinetic-updates/main amd64 Packages
        100 /var/lib/dpkg/status
     2.5.13+dfsg-1ubuntu1 500
        500 http://br.archive.ubuntu.com/ubuntu kinetic/main amd64 Packages

$ ./test.sh
Reference hash of "secret" (openssl): {SHA256}K7gNU3sdo+OL0wNhqoVWhr3g6s1xYv72ol/pe/Unols=
slapd's pw-sha2 hash: {SHA256}WIrrpN3OjEVOUf6yrH1j+o+ODuUuNBo979Od4UXnu54=
ERROR: hashes differ

Updating to proposed:
$ apt-cache policy slapd
slapd:
  Installed: 2.5.14+dfsg-0ubuntu0.22.10.2
  Candidate: 2.5.14+dfsg-0ubuntu0.22.10.2
  Version table:
 *** 2.5.14+dfsg-0ubuntu0.22.10.2 500
        500 http://br.archive.ubuntu.com/ubuntu kinetic-proposed/main amd64 Packages
        100 /var/lib/dpkg/status
     2.5.14+dfsg-0ubuntu0.22.10.1 500
        500 http://br.archive.ubuntu.com/ubuntu kinetic-updates/main amd64 Packages
     2.5.13+dfsg-1ubuntu1 500
        500 http://br.archive.ubuntu.com/ubuntu kinetic/main amd64 Packages

Problem fixed:
$ ./test.sh
Reference hash of "secret" (openssl): {SHA256}K7gNU3sdo+OL0wNhqoVWhr3g6s1xYv72ol/pe/Unols=
slapd's pw-sha2 hash: {SHA256}K7gNU3sdo+OL0wNhqoVWhr3g6s1xYv72ol/pe/Unols=
PASS: hashes are identical

Kinetic verification succeeded.

I'm checking the exim4 dep8 failures in kinetic.

I'm able to reproduce the exim4 dep8 error on kinetic ppc64el with the updated libldap packages, investigating...

And also without the updated packages. It's a bit random. When exim4 is failing, it stays failing. Restart it, and it might or might not be in failing mode. Sometimes it works, and keeps working. It's not related to the openldap update. I think the migration-reference/0 run that succeeded was a fluke.

https://bugs.launchpad.net/ubuntu/+source/exim4/+bug/1983605 is the exim4 bug. I would ask the SRU team to ignore the exim4 DEP8 errors on basis of that.

Hello Christian, or anyone else affected,

Accepted openldap into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/openldap/2.5.14+dfsg-0ubuntu0.22.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Jammy verification

Reproducing the problem with the release packages:

$ cat > test.sh
#!/bin/bash

reference_hash="{SHA256}$(echo -n secret | openssl dgst -sha256 -binary | openssl enc -base64)"
test_hash=$(slappasswd -s secret -h '{SHA256}' -o module-load=pw-sha2)

echo "Reference hash of \"secret\" (openssl): ${reference_hash}"
echo "slapd's pw-sha2 hash: ${test_hash}"

if [ "${reference_hash}" != "${test_hash}" ]; then
    echo "ERROR: hashes differ"
    exit 1
else
    echo "PASS: hashes are identical"
fi
^D

# apt-cache policy slapd
slapd:
  Installed: 2.5.14+dfsg-0ubuntu0.22.04.1
  Candidate: 2.5.14+dfsg-0ubuntu0.22.04.1
  Version table:
 *** 2.5.14+dfsg-0ubuntu0.22.04.1 500
        500 http://br.archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages
        100 /var/lib/dpkg/status
(...)

root@j-slapd-sha2:~# ./test.sh
Reference hash of "secret" (openssl): {SHA256}K7gNU3sdo+OL0wNhqoVWhr3g6s1xYv72ol/pe/Unols=
slapd's pw-sha2 hash: {SHA256}WIrrpN3OjEVOUf6yrH1j+o+ODuUuNBo979Od4UXnu54=
ERROR: hashes differ

Installing slapd from proposed:
root@j-slapd-sha2:~# apt-cache policy slapd
slapd:
  Installed: 2.5.14+dfsg-0ubuntu0.22.04.2
  Candidate: 2.5.14+dfsg-0ubuntu0.22.04.2
  Version table:
 *** 2.5.14+dfsg-0ubuntu0.22.04.2 500
        500 http://br.archive.ubuntu.com/ubuntu jammy-proposed/main amd64 Packages
        100 /var/lib/dpkg/status

Test now passes:
root@j-slapd-sha2:~# ./test.sh
Reference hash of "secret" (openssl): {SHA256}K7gNU3sdo+OL0wNhqoVWhr3g6s1xYv72ol/pe/Unols=
slapd's pw-sha2 hash: {SHA256}K7gNU3sdo+OL0wNhqoVWhr3g6s1xYv72ol/pe/Unols=
PASS: hashes are identical

Jammy verification succeeded.

Another migration-reference/0 attempt for kinetic[1] confirmed that the previous "success" was a fluke, and exim4 on plain kinetic is already failing[2]. That's bug #1983605

1. https://autopkgtest.ubuntu.com/packages/e/exim4/kinetic/ppc64el
2. https://autopkgtest.ubuntu.com/results/autopkgtest-kinetic/kinetic/ppc64el/e/exim4/20230403_151428_4e639@/log.gz

This bug was fixed in the package openldap - 2.5.14+dfsg-0ubuntu0.22.10.2

---------------
openldap (2.5.14+dfsg-0ubuntu0.22.10.2) kinetic; urgency=medium

  * Build the passwd/sha2 contrib module with -fno-strict-aliasing to
    avoid computing an incorrect SHA256 hash with some versions of the
    compiler (LP: #2000817):
    - d/t/{control,sha2-contrib}: test to verify the SHA256 hash
      produced by passwd/sha2
    - d/rules: set -fno-strict-aliasing only when building the
      passwd/sha2 contrib module

 -- Andreas Hasenack <email address hidden> Sun, 12 Mar 2023 14:09:05 -0300

The verification of the Stable Release Update for openldap has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

This bug was fixed in the package openldap - 2.5.14+dfsg-0ubuntu0.22.04.2

---------------
openldap (2.5.14+dfsg-0ubuntu0.22.04.2) jammy; urgency=medium

  * Build the passwd/sha2 contrib module with -fno-strict-aliasing to
    avoid computing an incorrect SHA256 hash with some versions of the
    compiler (LP: #2000817):
    - d/t/{control,sha2-contrib}: test to verify the SHA256 hash
      produced by passwd/sha2
    - d/rules: set -fno-strict-aliasing only when building the
      passwd/sha2 contrib module

 -- Andreas Hasenack <email address hidden> Sun, 12 Mar 2023 14:11:53 -0300