failing autopkgtest due to password issue by nss

Subscribing tjaalton to get his opinion on this.

It also seems to have effect on bug 1746948 with corosync postinst.

I only thought "hey nss minor update, why not pull it in", but without help to understand and resolve this I might be forced to go to a 3.35+really3.34 update to go back as I'm not enough of an nss expert to resolve :-/

How do you expect this to be fixed in 4.4.4, while proposed already has 4.6.2? The test should be ignored....

Hi Timo,
according to my sniff tests it will fail later on in 4.6.2 as well.
It seems the new nss makes the crypto/passwords no more behave the way as expected.

Of course the autopkgtest for 4.6.2 won't fail (as there is no old cert8.db, so the call is skipped), but if there would be one (e.g. on an upgrade) then it would fail (at least according to my tests in some containers).

So as soon as I bad-test 4.4.4 things will go on as the autopkgtest won't test the upgrade path. but it will still be "broken under the hood".

I didn't expect a fix in 4.4.x but instead wondered if you might be able to help to understand why it fails at all. And then depending on that insight we can work on a fix for either nss or freeipa as needed.

I want to decouple things a bit to make this bug less blocking, so I will:
1. prep a nss upload 2:3.35-2ubuntu1+really3.34.1-1ubuntu1
   That will (for now) skip the nss merge that I meant to do to help while coming by, but seems to
   cause issues.
2. test that in a ppa if it would test correctly where nss 2:3.35-2ubuntu1 currently fails in regard to freeipa

That will allow me to get through for nss what I need for other things, without the yet unclear impact on the nss password handling.

P.S. the actual change we wanted was the opening of freebl3 as you know, I really need to test #1 if that change might have been the trigger for these failure :-/

I'll report back here after the tests

For a better overview and to make a decision (as a +really version always sucks to some extend) I did some tests:
- built nss 3.34 with the freebl3 change in ppa [1] as 2:3.35-2ubuntu1+really3.34-1ubuntu2
- set up some containers to test
- ran the sequence of installs/commands that freeipa tests would do

I did so in different combinations:
1. freeipa 4.4.4 + nss 3.34-1ubuntu1 (as bionic is)
2. freeipa 4.6.3 + nss 3.35-1ubuntu1 (full bionic proposed)
3. freeipa 4.4.4 + nss 3.35-1ubuntu1 (as tested by autopkgtest by pinning)
4. freeipa 4.4.4 + nss 3.35-2ubuntu1+really3.34-1ubuntu2 (ppa)
5. freeipa 4.6.3 + nss 3.35-2ubuntu1+really3.34-1ubuntu2 (proposed + ppa)

I tested:
- the install that fails in the autopkgtest
  $ apt install freeipa-server freeipa-server-dns freeipa-server-trust-ad freeipa-common
    freeipa-client freeipa-admintools freeipa-tests python-ipaclient python-ipalib
    python-ipaserver python-ipatests
- the python call that fails (old & new form of it as it needs an additional import in 4.6.2)
  python2 -c 'from ipapython.certdb import update_ipa_nssdb; update_ipa_nssdb()'
  python2 -c 'from ipaclient.install.client import update_ipa_nssdb; update_ipa_nssdb()'

     #1 install #2 old python #3 new python
1. ok ok fail (4.4 has only old import)
2. ok (skip) fail (4.6 need new import) ok
3. fail fail (nss format) fail (4.4 has only old import)
4. ok ok fail (4.4 has only old import)
5. ok (skip) fail (4.6 need new import) ok

So an nss upload should work as planned with both verserions:
- freeipa 4.4 (case 4. #2)
- freeipa 4.6 (case 5. #3)
- and both cases would install (4./5. #1).

Due to the hint by Timo (thanks) I found [1] which explains a bit what is going on.
That is a nice change to be made in nss, but not unplanned and unprepared.
Some consuming packages need to be adapted first, and that was not what I intended by picking a new minor version. So that as well points to an upload reverting the move to 3.35.

Get me right - the move to 3.35 and the new file format should be done at some point, but not now unplanned (it accidentally slipped in by the merge) - so I'm uploading 2:3.35-2ubuntu1+really3.34-1ubuntu2 to un-break it for now.

[1]: https://fedoraproject.org/wiki/Changes/NSSDefaultFileFormatSql

I was discussing this with Timo and he correctly pointed out that reverting [1] might be enough.
This would allow to get all fixes of 3.35, but at the same time not run into this bug here all over the place.

Building with that for some test runs.

[1]: https://github.com/nss-dev/nss/commit/33b114e38278c4ffbb6b244a0ebc9910e5245cd3

Built with said change - and now testing against the nss in the ppa (should all work):
- install 4.4 (ppa) -> OK
- install 4.6 (proposed + ppa) -> OK
- old python call against 4.4 (ppa) -> OK
- new python call against 4.6 (proposed + ppa) -> OK

I dupped Corosync on here, so lets verify it fixes that as well
- install corosync-qnetd as in proposed (should fail) -> FAIL
- install corosync-qnetd with ppa (should work) -> OK

This was a bit of a panic exercise for me :-)
After breaking things with nss which isn't my home turf I wanted to get rid of it asap.
Thanks to Timo to keep me in line with our discussions so it ended up as a much more reasonable fix.
Uploading the fixed 3.35 version now ...

This bug was fixed in the package nss - 2:3.35-2ubuntu2

---------------
nss (2:3.35-2ubuntu2) bionic; urgency=medium

  * d/p/lp1746947-revert-switch-default-to-sql.patch: the switch of the
    default is still causing too much issues in consumers of nss.
    So until resolved revert the switched default (LP: #1746947)

nss (2:3.35-2ubuntu1) bionic; urgency=medium

  * Merge with Debian unstable. Remaining changes:
    - When building with -O3, build with -Wno-error=maybe-uninitialized.
  * Added Changes:
    - d/libnss3.links: make freebl3 available as library (LP: #1744328)
      + d/control: add dh-exec to Build-Depends
      + d/rules: make mkdir tolerate debian/tmp existing (due to dh-exec)

nss (2:3.35-2) unstable; urgency=medium

  * nss/lib/freebl/Makefile: Build Hacl_Poly1305_64.o on arm64.

nss (2:3.35-1) unstable; urgency=medium

  * New upstream release.

nss (2:3.34.1-1) unstable; urgency=medium

  * New upstream release.

 -- Christian Ehrhardt <email address hidden> Mon, 05 Feb 2018 11:36:07 +0100