What happened?
I switched agent pools to use newly created VMSS to switch from Azure _v5 gen VMs to _v7 gen VMs.
After the switch, at ~23:01 UTC every day we started to see that agent gets offline. We are Using Ubuntu 24.04
Message looks exactly like in #5023, so 401 and trying to reauthenticate. Authentication even looks like it successfully retry at some point, but agent shutdowns.
After diffing old and new VMSS, the only visible changes are expected
- VM Size (v5 to v7)
- OS disk settings (switch from SCSI to NVMe because of VM size change)
Old VMSS was created years ago and new ones about 30 March 2026 - 10 April 2026.
Both VMSSes old and new of course uses exactly same OS image with Ubuntu 24.04, unattended updates and needsrestart disabled and removed.
After switching agent pool back to VMSSes created years ago, but using same OS image as new pools, issue disappears.
Devops agent we are using is 3.236.0. This is forced by devops platform in installed extension, we have no control over it.
Versions
dev.azure.com one
Environment type (Please select at least one enviroment where you face this issue)
Azure DevOps Server type
dev.azure.com (formerly visualstudio.com)
Azure DevOps Server Version (if applicable)
No response
Operation system
Ubuntu 24.04
Version controll system
No response
Relevant log output
(We are GMT+2, so it shows ~1:05)
What happened?
I switched agent pools to use newly created VMSS to switch from Azure _v5 gen VMs to _v7 gen VMs.
After the switch, at ~23:01 UTC every day we started to see that agent gets offline. We are Using Ubuntu 24.04
Message looks exactly like in #5023, so 401 and trying to reauthenticate. Authentication even looks like it successfully retry at some point, but agent shutdowns.
After diffing old and new VMSS, the only visible changes are expected
Old VMSS was created years ago and new ones about 30 March 2026 - 10 April 2026.
Both VMSSes old and new of course uses exactly same OS image with Ubuntu 24.04, unattended updates and needsrestart disabled and removed.
After switching agent pool back to VMSSes created years ago, but using same OS image as new pools, issue disappears.
Devops agent we are using is 3.236.0. This is forced by devops platform in installed extension, we have no control over it.
Versions
dev.azure.com one
Environment type (Please select at least one enviroment where you face this issue)
Azure DevOps Server type
dev.azure.com (formerly visualstudio.com)
Azure DevOps Server Version (if applicable)
No response
Operation system
Ubuntu 24.04
Version controll system
No response
Relevant log output