Description
Issue description
We are facing intermittent issues resolving dynamic variables—especially SSM/SecretsManager-based secrets and stage/environment-dependent values—when using Serverless Framework versions beyond v4.4.3.
These failures occur both during serverless deploy -s dev
and serverless offline start -s dev
. About 70% of the time, variable resolution fails, while the remaining attempts succeed without any changes to configuration or credentials.
We're now suspecting the issue is not limited to SSM resolution alone, but could be linked to dynamic variable interpolation (e.g., ${self:custom.ssm.${self:provider.stage}.openai-api-key}) or stage/environment usage across files.
Error:
✖ Error: Cannot resolve '${self:custom.ssm.dev.openai-api-key}' variable at 'provider.environment.OPENAI_API_KEY'. No value is available for this variable, and no default value was provided. Please check your variable definitions or provide a default value.
at _ResolverManager.resolve (file:///home/user/.serverless/releases/4.4.4/package/dist/sf-core.js:655:55913)
at async #handlePlaceholderNode (file:///home/user/.serverless/releases/4.4.4/package/dist/sf-core.js:655:54343)
at async file:///home/user/.serverless/releases/4.4.4/package/dist/sf-core.js:655:52335
at async processNodeAndHandleCompletion (file:///home/user/.serverless/releases/4.4.4/package/dist/sf-core.js:655:43086)
Environment:
- Node.js: v22.15.0
- Serverless Framework: Affected on 4.4.4, 4.14.3, 4.17.0, 4.17.1 (works on 4.4.2)
- AWS CLI v2, using AWS SSO login via aws-sso-credential-getter
Project Configuration Overview:
// serverless.yml:
custom: ${file(serverless_custom.yml)}
provider:
name: aws
profile: ${env:my-aws-profile}
runtime: nodejs20.x
stage: ${opt:stage, 'dev'}
region: ${opt:region, 'us-east-1'}
environment: ${file(serverless_environment.yml)}
// serverless_environment.yml:
OPENAI_API_KEY: ${self:custom.ssm.${self:provider.stage}.openai-api-key}
MOMENTO_API_KEY: ${self:custom.ssm.${self:provider.stage}.momento-api-key}
S3_UPLOAD_BUCKET: spicychat-assets-${self:provider.stage}
FIREHOSE_STREAM_NAME: ${self:custom.firehoseStreamName.${self:provider.stage}}
// serverless_custom.yml:
ssm:
prod: ${ssm:/aws/reference/secretsmanager/spicychat/${sls:stage}}
dev: ${ssm:/aws/reference/secretsmanager/spicychat/${sls:stage}}
uat: ${ssm:/aws/reference/secretsmanager/spicychat/${sls:stage}}
Steps to Reproduce:
- Authenticate using:
export AWS_PROFILE=<my-aws-profile>
aws sso login --profile <my-aws-profile>
ssocred <my-aws-profile>
- Run
serverless deploy -s dev
OR
serverless offline start -s dev
- Observe that sometimes all environment variables resolve correctly, but most times critical ones like OPENAI_API_KEY, MOMENTO_API_KEY, etc., are resolved as null.
What we've tried
- Verified all secrets exist in AWS Secrets Manager.
- Cleared
.serverless
cache directory. - Confirmed valid SSO session and that AWS profile environment variables are set.
- Reverted to v4.4.2 — the last known stable version where secrets resolve consistently.
Expected Behavior:
Serverless should consistently resolve SSM secrets when valid AWS credentials are provided and referenced correctly in the configuration.
Actual Behavior:
SSM secrets intermittently fail to resolve, leading to deployment and local development failures. Variables resolve to null
, and placeholders like ${self:custom.ssm.dev.openai-api-key}
trigger fatal errors due to unresolved values.
Debug Logs:
s:core: { serverless_framework: '4.4.4' } s:main: Initializing s:core:router: { command: [ 'offline', 'start' ], options: { s: 'dev', debug: '*' }, versions: { serverless_framework: '4.4.4' }, customConfigFilePath: null, composeParams: undefined, composeResolverProviders: undefined, isWithinCompose: false, composeOrgName: null, composeServiceName: null } ......... ......... ......... s:core:resolver:manager: adding resolver provider opt with config { type: 'opt' } s:core:resolver:manager: provider 'opt', resolver 'options' resolved 'region' to 'null' - skipping to next fallback. s:core:resolver:manager: resolved✖ Error: Cannot resolve '${self:custom.ssm.dev.openai-api-key}' variable at 'provider.environment.OPENAI_API_KEY'. No value is available for this variable, and no default value was provided. Please check your variable definitions or provide a default value.
at _ResolverManager.resolve (/home/user/.serverless/releases/4.4.4/sf-core/src/lib/resolvers/manager.js:667:11)
at async _ResolverManager.#handlePlaceholderNode (/home/user/.serverless/releases/4.4.4/sf-core/src/lib/resolvers/manager.js:584:9)
at async (/home/user/.serverless/releases/4.4.4/sf-core/src/lib/resolvers/manager.js:477:11)
at async processNodeAndHandleCompletion (/home/user/.serverless/releases/4.4.4/sf-core/src/lib/resolvers/graph.js:20:5)