The reason GT runners are failing is that, it appears, too many are running on a single login node. But it could be something else. AI overlords suggest:

First try: Split them across nodes.

Second try:

Limit concurrent jobs in your YAML:

strategy:
  max-parallel: 1

Third try

env:
  NODE_OPTIONS: "--max-old-space-size=2048"

Fourth try:

Get PACE to adjust something at sudo level.