Alert is active for an-presto1016, an-presto1017, an-presto1019, an-presto1020.

It's been two weeks since the fix was deployed. Please reopen this task if you notice any further issues.

doc2002 is a bit lower but follows a similar growth pattern: https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&viewPanel=12&var-server=doc2002&refresh=5m&from=now-6M&to=now&var-datasource=thanos&var-cluster=ci

Since the change improved the situation we will not be subscribing to the Validity RBL and will leave it disabled for now. Please let us know if you notice any issues that would warrant re-enabling the RBL.

@Platonides could you help us verify this?

Zuul workers post-migration are also in scope.

The potential fix was merged in https://gitlab.com/mailman/mailman/-/issues/1151 and is included in Mailman version 3.3.10.

This is not something we'd work on in the foreseeable future.

Once the T348734 is resolved Gerrit will be able to use the global blocklist.

@jhathaway do you think this is a VRTS problem or an MX problem?

Auto-recovered within 5 minutes, resolving.

The alert was silenced for three days so the silence should be removed if the optimize finishes sooner than that.

We're in touch with ITS and will be doing follow up testing with them next week.

Same alert is firing for ml-serve-eqiad and ml-staging-codfw.

Same alert is firing for ml-serve-eqiad and ml-staging-codfw.

Know race condition, self-recovered.

Host is up now, see T376881: elastic1064 is down