Skip to content

Add support for environment variable resolution in httpGet probe paths #128900

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
bachmanity1 opened this issue Nov 21, 2024 · 22 comments
Closed
Assignees
Labels
kind/feature Categorizes issue or PR as related to a new feature. lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. sig/node Categorizes an issue or PR as relevant to SIG Node.

Comments

@bachmanity1
Copy link

bachmanity1 commented Nov 21, 2024

Currently, Kubernetes does not support resolving environment variables in httpGet probe paths, which limits configuration flexibility.

Expected behavior:

  • Allow environment variable interpolation like path: "$(CONTEXT_PATH)/health/"
  • Support standard shell-like variable substitution syntax

Sample configuration:

       readinessProbe:
          httpGet:
            path: "$(CONTEXT_PATH)/actuator/health/readiness"
            port: 8080
            scheme: HTTP
@k8s-ci-robot k8s-ci-robot added needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Nov 21, 2024
@k8s-ci-robot
Copy link
Contributor

This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@bachmanity1
Copy link
Author

/sig node

@k8s-ci-robot k8s-ci-robot added sig/node Categorizes an issue or PR as relevant to SIG Node. and removed needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Nov 21, 2024
@bachmanity1
Copy link
Author

If the relevant SIG believes this feature request is worthwhile, I can begin working on it.

@kannon92
Copy link
Contributor

/kind feature

@k8s-ci-robot k8s-ci-robot added the kind/feature Categorizes issue or PR as related to a new feature. label Nov 21, 2024
@pacoxu
Copy link
Member

pacoxu commented Nov 21, 2024

IIUC, a simple workaround is using exec probe.

  • There are alternative solutions, but I am not opposed to this requirement.

@bachmanity1
Copy link
Author

IIUC, a simple workaround is using exec probe.

Yes, that's correct, but it would mean the image needs to have curl (or a similar tool) included, which isn’t always the case. Plus, using exec would make the probe configuration a bit more verbose.

@saschagrunert
Copy link
Member

Exec probes are also a bit more resource intensive compared to the other ones. I kinda like the idea of supporting the predefined environment variables, but it needs to apply to all other types of probes as well.

@bachmanity1
Copy link
Author

bachmanity1 commented Nov 21, 2024

I’m considering an approach like the one below to resolve environment variables in the path:

exec := exec.New()
cmd := exec.Command("echo", pathToProcess)
buff := bytes.Buffer{}
cmd.SetStdout(&buff)
cmd.Run()
processedPath := buff.String()

However, I’m concerned this might not be safe, as it could open the door to remote shell execution attacks (though exec probes have a similar risk). Additionally, running a shell before using httpGet could end up being just as resource-intensive as exec probes. An alternative might be to fetch environment variables directly from the container object, but there’s a chance that some variables in the path might not be present in the container spec. Additionally, values for certain variables, like the pod name, cannot be derived from the container spec.

@saschagrunert
Copy link
Member

An alternative might be to fetch environment variables directly from the container object

That's how we should do it from my point of view.

@bachmanity1
Copy link
Author

That's how we should do it from my point of view.

I think I’m leaning toward the first approach since, as mentioned, fetching environment variables directly from the container object wouldn’t work for all cases.

@sftim
Copy link
Contributor

sftim commented Nov 22, 2024

We should use Kubernetes-style environment variable interpolation, I think

See https://kubernetes.io/docs/tasks/inject-data-application/define-environment-variable-container/#using-environment-variables-inside-of-your-config for an example.

@saschagrunert
Copy link
Member

@bachmanity1 the issue is that the kubelet calls the http probes directly, while exec'ing into the container comes with a reasonable overhead. Keeping those constraints allows us only to work with the data we have at the manifest level.

@bachmanity1
Copy link
Author

Thanks for the feedback! I’m planning to open a PR in a couple of days using Kubernetes-style environment variable interpolation.

@bachmanity1
Copy link
Author

/assign @bachmanity1

@bachmanity1
Copy link
Author

Hi @pacoxu, @saschagrunert, and @sftim, thank you for your feedback! I've opened a PR and linked it above. Could you take a look when you have a moment? Thanks!

@aojea
Copy link
Member

aojea commented Dec 16, 2024

Exec probes are also a bit more resource intensive compared to the other ones. I kinda like the idea of supporting the predefined environment variables, but it needs to apply to all other types of probes as well.

@saschagrunert they are not for the kubelet, http and tcp probes steal resources from the kubelet that is more problematic, actually for constrained kubelet that fails network probes the workaround is to move to use exec probes #89898 (comment)

@liggitt
Copy link
Member

liggitt commented Dec 17, 2024

reinterpreting existing fields in new ways isn't really ok... existing httpGet probes with environment-looking substrings would behave differently

see kubernetes/enhancements#559 / #71351 for a similar addition to support envvars in subpath mounts, which required a new field that explicitly supported envvar expansion

@liggitt
Copy link
Member

liggitt commented Dec 17, 2024

if this was added, I would expect the same env values and format used to expand other aspects of the pod (container args, etc)

@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue as fresh with /remove-lifecycle stale
  • Close this issue with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Mar 18, 2025
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue as fresh with /remove-lifecycle rotten
  • Close this issue with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

@k8s-ci-robot k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Apr 17, 2025
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Reopen this issue with /reopen
  • Mark this issue as fresh with /remove-lifecycle rotten
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

@k8s-ci-robot
Copy link
Contributor

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Reopen this issue with /reopen
  • Mark this issue as fresh with /remove-lifecycle rotten
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot closed this as not planned Won't fix, can't repro, duplicate, stale May 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/feature Categorizes issue or PR as related to a new feature. lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. sig/node Categorizes an issue or PR as relevant to SIG Node.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

9 participants
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy