gnome-shell crashed (out of memory) with SIGSEGV in crocus_begin_query() from crocus_begin_query() from crocus_end_query() from crocus_end_query() from tc_call_end_query()

Bug #1972977 reported by errors.ubuntu.com bug bridge
26
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Mesa
Fix Released
Unknown
mesa (Ubuntu)
Fix Released
High
Unassigned
Jammy
Fix Released
High
Unassigned

Bug Description

[Impact]
Gnome-shell is crashing on older gen Intel hw due to a memory leak.

[Fix]
An upstream commit that fixes the memleak in the crocus driver (which is only used on Intel 4th gen chips and older)

[Test case]
Either test this on an old Intel or see if there's a dent in the reported crashes.

[Where things could go wrong]
Mainly if this would not fix the crashes.

--
The Ubuntu Error Tracker has been receiving reports about a problem regarding gnome-shell. This problem was most recently seen with package version 42.0-2ubuntu1, the problem page at https://errors.ubuntu.com/problem/d8aa1211f7e8b219a4ee6dcae294ac16decd7fe3 contains more details, including versions of packages affected, stacktrace or traceback, and individual crash reports.
If you do not have access to the Ubuntu Error Tracker and are a software developer, you can request it at http://forms.canonical.com/reports/.

summary: - /usr/bin/gnome-
- shell:11:crocus_begin_query:crocus_begin_query:crocus_end_query:crocus_end_query:tc_call_end_query
+ gnome-shell crashed with SIGSEGV in crocus_begin_query() from
+ crocus_begin_query() from crocus_end_query() from crocus_end_query()
+ from tc_call_end_query()
affects: gnome-shell (Ubuntu) → mesa (Ubuntu)
tags: added: crocus
Revision history for this message
Launchpad Janitor (janitor) wrote : Re: gnome-shell crashed with SIGSEGV in crocus_begin_query() from crocus_begin_query() from crocus_end_query() from crocus_end_query() from tc_call_end_query()

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in mesa (Ubuntu):
status: New → Confirmed
Changed in mesa:
status: Unknown → New
Revision history for this message
Daniel van Vugt (vanvugt) wrote :

Upstream says the crash is caused by a leak and reviewing the Mesa code seems to support that theory. The fix is:

https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17731

tags: added: gnome-shell-leak
Changed in mesa (Ubuntu):
importance: Undecided → High
status: Confirmed → Fix Committed
tags: added: fixed-in-23 fixed-upstream
Revision history for this message
Daniel van Vugt (vanvugt) wrote :

Only fixed in Mesa 23 right now but I would expect to see it in 22.x

Revision history for this message
Timo Aaltonen (tjaalton) wrote :

I find it confusing to move a bug 'fix committed' when it's only in upstream master. Kinetic will get 22.2.x, and it'll have this (hasn't branched yet).

Revision history for this message
Daniel van Vugt (vanvugt) wrote :

I agree... it's just desktop team policy requested by seb128.

Changed in mesa (Ubuntu):
status: Fix Committed → Triaged
Revision history for this message
Daniel van Vugt (vanvugt) wrote :

I misinterpreted mesa's branching structure. The commit is indeed in 22.2 (not tagged yet).

tags: added: fixed-in-22.2
removed: fixed-in-23
summary: - gnome-shell crashed with SIGSEGV in crocus_begin_query() from
- crocus_begin_query() from crocus_end_query() from crocus_end_query()
- from tc_call_end_query()
+ gnome-shell crashed (out of memory) with SIGSEGV in crocus_begin_query()
+ from crocus_begin_query() from crocus_end_query() from
+ crocus_end_query() from tc_call_end_query()
tags: added: fixed-in-22.1.5
Changed in mesa (Ubuntu):
status: Triaged → Fix Released
tags: added: rls-jj-incoming
Changed in mesa:
status: New → Fix Released
Revision history for this message
Daniel van Vugt (vanvugt) wrote :

This is now the top gnome-shell crasher in jammy, which is impressive for the quality of gnome-shell because the crash only occurs on Intel 4th gen and older.

tags: added: dt-881
Timo Aaltonen (tjaalton)
description: updated
description: updated
Timo Aaltonen (tjaalton)
Changed in mesa (Ubuntu Jammy):
status: New → In Progress
Revision history for this message
Chris Halse Rogers (raof) wrote : Proposed package upload rejected

An upload of mesa to jammy-proposed has been rejected from the upload queue for the following reason: "Looks like there's unintentional changes in src/mesa/formats.csv".

Revision history for this message
Chris Halse Rogers (raof) wrote : Please test proposed package

Hello errors.ubuntu.com, or anyone else affected,

Accepted mesa into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/mesa/22.0.5-0ubuntu0.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in mesa (Ubuntu Jammy):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-jammy
description: updated
Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (mesa/22.0.5-0ubuntu0.2)

All autopkgtests for the newly accepted mesa (22.0.5-0ubuntu0.2) for jammy have finished running.
The following regressions have been reported in tests triggered by the package:

mutter/42.5-0ubuntu1 (arm64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/jammy/update_excuses.html#mesa

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Revision history for this message
Timo Aaltonen (tjaalton) wrote :

vanvugt: any way to verify this fix?

Revision history for this message
Daniel van Vugt (vanvugt) wrote :

I've only been monitoring crash reports to verify the fix worked in kinetic and later. Although anecdotally I recall one or two users of Haswell or older saying it happened frequently for them. I guess we could try such an older system and monitor for the gnome-shell memory leak that should precede the crash.

(Sorry for the delay - it was a long end of year break)

Revision history for this message
Timo Aaltonen (tjaalton) wrote :

or just skip this and accept the kinetic backport which has the fix via upstream

Revision history for this message
Daniel van Vugt (vanvugt) wrote :

I have promised to actually test this on some older hardware but I want to get through the past month's bug mail first...

Revision history for this message
Daniel van Vugt (vanvugt) wrote :

Tested jammy with crocus on an i5-4300U, both 22.0.5-0ubuntu0.1 and 22.0.5-0ubuntu0.2

gnome-shell's RSS grows but seems to plateau similarly with both.

Based on upstream's comments, I started counting i915.gem instances in /proc/PID/smaps and found that both 22.0.5-0ubuntu0.1 and 22.0.5-0ubuntu0.2 seem to leak them equally. Never reached the point of crashing but the evidence is that mesa 22.0.5-0ubuntu0.2 makes gnome-shell leak almost as much as 22.0.5-0ubuntu0.1 ... That said, I expect most graphics leaks to be hardware independent and the fault of gnome-shell/mutter, not mesa.

Looking at the upstream fix again I realize it's not fixing all the leaks that's important here, but avoiding crashes when they occur. And for that we are missing the correct patch:

https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17615/diffs

It's not in mesa_22.0.5-0ubuntu0.2.diff.gz

tags: added: verification-failed-jammy
removed: verification-needed-jammy
Revision history for this message
Timo Aaltonen (tjaalton) wrote :

but it is in 22.2.x which we need in proposed asap anyway, so I'll drop this from the current upload so that it can be fast-tracked through, as #1998893 is even more urgent.

Revision history for this message
Timo Aaltonen (tjaalton) wrote :

or, just accept the current upload and revisit it again with the 22.2 backport if needed, but in any case I don't think this bug should block either one from getting into updates

Revision history for this message
Daniel van Vugt (vanvugt) wrote (last edit ):

Yeah kinetic doesn't have this bug so we can probably fix jammy via the backport without knowing which commits are most important.

Revision history for this message
Andreas Hasenack (ahasenack) wrote : Please test proposed package

Hello errors.ubuntu.com, or anyone else affected,

Accepted mesa into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/mesa/22.0.5-0ubuntu0.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

tags: added: verification-needed-jammy
removed: verification-failed-jammy
Revision history for this message
Andreas Hasenack (ahasenack) wrote :

I accepted 22.0.5-0ubuntu0.3 which does NOT fix this bug, so I'm reopening it.

Changed in mesa (Ubuntu Jammy):
status: Fix Committed → Confirmed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package mesa - 22.0.5-0ubuntu0.3

---------------
mesa (22.0.5-0ubuntu0.3) jammy; urgency=medium

  * crocus-fix-leak-in-query-code.patch: Drop the patch so the other fix
    can get through to updates.

mesa (22.0.5-0ubuntu0.2) jammy; urgency=medium

  * crocus-fix-leak-in-query-code.patch: Fix gnome-shell crashing on
    older Intel hw. (LP: #1972977)
  * patches: Update pci-id's for ADL-S/RPL-S. (LP: #1998893)

 -- Timo Aaltonen <email address hidden> Thu, 05 Jan 2023 10:45:40 +0200

Changed in mesa (Ubuntu Jammy):
status: Confirmed → Fix Released
Revision history for this message
Timo Aaltonen (tjaalton) wrote :

nope, reopening so that this will be fixed via the kinetic backport

Changed in mesa (Ubuntu Jammy):
status: Fix Released → Triaged
Revision history for this message
Daniel van Vugt (vanvugt) wrote :

While 22.04 isn't fixed yet, please try editing /etc/environment and add:

  MESA_LOADER_DRIVER_OVERRIDE=i965

then reboot. It might be a suitable workaround.

Changed in mesa (Ubuntu Jammy):
importance: Undecided → High
Revision history for this message
Timo Aaltonen (tjaalton) wrote :

is fixed for real now

Changed in mesa (Ubuntu Jammy):
status: Triaged → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.