bin/helpers: check for unexpected LXD warnings/errors during cleanup()

[simondeziel]

This brings the question of should we keep warning about this:

time="2024-04-09T19:46:28-04:00" level=warning msg=" - Couldn't find the CGroup network priority controller, per-instance network priority will be ignored. Please use per-device limits.priority instead"

I have this warning locally despite not making conscious use of any network priority. I've also ACK'ed the corresponding lxc warning but it doesn't silence that message.

[simondeziel]

FYI, this is expected to fail on many tests until canonical/lxd#13299 is merged.

[mihalicyn]

LGTM! Thanks!

[simondeziel]

This turned up many more errors than I expected, some of which I think shouldn't be errors like those:

time="2024-04-10T14:14:50Z" level=error msg="SFTP server stopped with error" err="signal: terminated" instance=cluster-2783-1 instanceType=container project=default stderr=
time="2024-04-10T14:10:39Z" level=error msg="Failed to statfs" err="no such file or directory" instance=c1 instanceType=container path=tmpfs project=default

Should I postpone this PR and open bugs in LXD to have those silenced or reported at a debug/info level maybe?

Also, I'll have to rethink how problems are ignored because many of them are legitimate. What would you think of having a list of messages regex to ignore at the top of scripts where some errors are to be expected? This way, if we take the storage-vm btrfs test for example, we'd probably want to ignore those:

 time="2024-04-10T14:22:28Z" level=warning msg="Unable to use virtio-fs for config drive, using 9p as a fallback" err="Stateful migration unsupported" instance=v1 instanceType=virtual-machine project=default
time="2024-04-10T14:22:28Z" level=warning msg="Using writeback cache I/O" devPath=/var/snap/lxd/common/lxd/storage-pools/vmpool-btrfs-3267/virtual-machines/v1/root.img device=root fsType=btrfs instance=v1 instanceType=virtual-machine project=default

Some other errors like in https://github.com/canonical/lxd-ci/actions/runs/8632671674/job/23663945355?pr=135 are a bit more unclear to me. Maybe they are just noise due to opportunistic cleanup done by LXD.

[tomponline]

Should I postpone this PR and open bugs in LXD to have those silenced or reported at a debug/info level maybe?

I think WARN level would be better for these