How to debug NetworkManager issue after suspend/resume in Ubuntu 22.04
****Since upgrading to Ubuntu 22.04 recently, immediately after a reboot I have WiFi to an internet hotspot as I did prior to upgrading.
After suspend then resume, the network is no longer working although the NetworkManager process is running. I've tried various nmcli commands including nmcli networking reload but they time out.
After a while (>15m), sometimes this rectified itself, but generally not until after reboot.
When I shutdown the system I find that it is waiting 90s for a 'stop job' on NetworkManager, so maybe the earlier system suspend is not successful leading the resume to time out while trying to re-enable networking. That is just speculation, I'm not sure how to test that. EDIT: see 'workaround' below for more on this.
Any clues on how to look deeper into this, or ideas as to why suspend/resume should cause problems with NetworkManager?
This is a Dell laptop which has been through at least one earlier full Ubuntu upgrade without this kind of issue.
Updates:
NOT SOLVED: The issue went away for a few weeks, possibly after an update but then returned and has been forcing me to reboot after each resume for the past 10ish days, except for one occasion when I had network after resume.
WORKAROUND: If I stop NetworkManager before sleep and start it after resume everything seems ok. Early days but so far so good. Commands I use are:
sudo service NetworkManager stop
...
sudo service NetworkManager start
So as I suspected the issue seems to be that NetworkManager is not stopping properly when the system is going to sleep (see note above about waiting for a stop job).
UPDATE: After using the above workaround for a few days I forgot, but on restarting everything worked again and has for several days since, much like before. So it is an intermittent issue and I'm not sure of the usefulness of the workaround until it fails again.
QUESTIONS:
- is there a way to modify when/how NetworkManager is stopped/started? I haven't been able to find this myself.
- is there a way to insert my own commands to stop/start on sleep/resume?
2 answers
Begin with the logs
I'd begin with the logs: journalctl -u NetworkManager
You may want to create a log config file for this:
vim /etc/NetworkManager/conf.d/95-nm-debug.conf # perhaps sudo, to have access to /etc
File contents:
[logging]
domains=ALL:TRACE
systemctl restart NetworkManager to make NM notice the changes.
Logs incomplete?
If logging output is incomplete, disable rate limiting in the journal:
vim /etc/systemd/journald.conf- find
RateLimitBurst - set to
0 systemctl restart systemd-journald
Python helper scripts
I hear there are some, but haven't used them.
sudo /usr/lib/NetworkManager/debug-helper.py --nm debug # or info or trace if not debug
Testcase for debugging
This stops NM, unloads and loads the DRIVER, then starts NM again. Should provide good logs.
sudo service network-manager stop
sudo modprobe -r DRIVER
sudo modprobe DRIVER
sudo service network-manager start
Sources
For more, see:
- https://wiki.ubuntu.com/DebuggingNetworkManager
- https://wiki.gnome.org/Projects/NetworkManager/Debugging
- https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/configuring_and_managing_networking/introduction-to-networkmanager-debugging_configuring-and-managing-networking
0 comment threads
Although you have left a post saying it is solved, I think it's not uncommon for this kind of problem to show up. I think it can happen due to misconfiguration of either NM or the network drivers/hardware. It would be more useful to have a general troubleshooting question for dealing with NM timeouts.
In your case, I suspect that some changes were made to either NM config or system networking configs by the update. Since normally updates do not change user configs, probably what happened is that the upstream changed defaults you were not aware of and pulled the rug out of you, so to speak.
What I would do is:
- Look for a detailed log from the package manager about what files exactly were modified. Figure out what version of each package I used to have and find the old version of those files. Check what actually changed and see if that reveals anything obvious.
- Downgrade the NM package to what it was before. If a change in NM caused this, it should start working again. If not, we know the problem is outside NM.
- Examine detailed log output from NM regarding what it is doing. There may be a
--debugor--verboseoption that can be enabled for this. - Try purging NM, including configs, and installing from scratch and following most recent instructions.
0 comment threads
1 comment thread