Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Implement system packages option for --image-type #1555

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

feat: Implement system packages option for --image-type #1555

wants to merge 2 commits into from

Conversation

booxter
Copy link
Contributor

@booxter booxter commented Mar 11, 2025 •
edited
Loading

What does this PR do?

This is a second attempt to implement system packages option for run. Now
it's selected explicitly by passing --image-type system to run command. The
previous attempt implicitly assumed that the default behavior is to use system
packages; and it broke existing conda users.

Test Plan

Confirmed that this starts the server:

$ . ./venv/bin/activate
$ llama stack run --image-type system $CONFIG

(venv was pre-initialized with --image-type venv.)

@booxter
Revert "revert: feat(server): Use system packages for execution (meta…
67b7644 
…-llama#1551)"

This reverts commit e13c92f.

Co-Authored-By: Sébastien Han <seb@redhat.com>

Signed-off-by: Ihar Hrachyshka <ihar.hrachyshka@gmail.com>
@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Meta Open Source bot. label Mar 11, 2025
@booxter
Copy link
Contributor Author

booxter commented Mar 11, 2025

@leseb what do you think of this interface?

@booxter booxter force-pushed the system-packages-again branch from a1b7b78 to 14b6eb5 Compare March 11, 2025 17:11
@booxter
fix: Introduce new system option for --image-type
40fa729 
It will enforce using the current environment packages.

Note: this patch makes the original implementation not touch the default
`conda` value for the argument.

Signed-off-by: Ihar Hrachyshka <ihar.hrachyshka@gmail.com>
@booxter booxter force-pushed the system-packages-again branch from 14b6eb5 to 40fa729 Compare March 11, 2025 17:11
@booxter booxter marked this pull request as ready for review March 11, 2025 22:01
leseb
leseb suggested changes Mar 12, 2025
View reviewed changes
Copy link
Contributor

@leseb leseb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don’t see why we need to introduce a new image type. My patch only changes things for users who were using Conda - they now need to explicitly call it in the CLI. So essentially, this used to work fine if you had conda:

llama stack run <config> --image-name foo

Or without --image-name if CONDA_DEFAULT_ENV was present.

Now you need to do:

llama stack run --image-type=conda --image-name=foo <config>

Or without --image-name if CONDA_DEFAULT_ENV is present.

@booxter @ashwinb @dineshyv Does it make sense?

@leseb
Copy link
Contributor

leseb commented Mar 12, 2025

If we want to ease the experience for conda users, perhaps we can set image_type=conda if CONDA_DEFAULT_ENV is present?

@booxter
Copy link
Contributor Author

booxter commented Mar 12, 2025

I think @leseb is making a good point; conda being the default may not be the optimal choice after all. (That said, I believe some docs would have to be updated if we change this behavior - some assume conda used when no argument passed I think.)

@ashwinb what's the exact reason conda is the default? Seems like an unusual choice for a project. Is it just historically the way it was? Should we proliferate this?

@ashwinb
Copy link
Contributor

ashwinb commented Mar 13, 2025 •
edited
Loading

conda was the historical choice just because we started with that and due to our internal environments at Meta. We are happy to change the default to venv -- I think that's more natural. We'd probably need to switch a bit carefully since folks might be depending on the default?

@leseb
Copy link
Contributor

leseb commented Mar 13, 2025

@booxter @ashwinb how about going with just this #1555 (comment) for now? It's simple, doesn't break conda users, and allows to run without a new system flag? Thanks!

@booxter
Copy link
Contributor Author

booxter commented Mar 13, 2025

@leseb AFAIU when you call run, your target conda environment is not, yet, necessarily activated, no? If so, the variable is not, yet, set. (I admit I have little knowledge about how conda workflow is actually used since I don't use it, so I may be off.) Otherwise, if the variable is always set when run is executed, then yes, it could be an option.

@booxter
Copy link
Contributor Author

booxter commented Mar 15, 2025

I think the CONDA_DEFAULT_ENV variable MAY be present but is not REQUIRED. (It's set as long as conda config is injected in the current shell rc file - e.g. .bashrc.) On the other hand, if conda is installed and the variable is initialized, it doesn't mean llama-stack run must execute in conda mode (I can imagine a user has conda installed but still wants to use uv or venv for llama-stack development.)

I think the ultimate path is to make conda selection non-default. Whether it means that we make "system" choice the default, or we from now on require the choice to be made - I don't have a preference. But regardless I'd rather not keep conda as the tool of choice for the project.

@ashwinb I know we reverted the original patch that defaulted to "system packages" mode before back to conda. I wonder we should proliferate conda default position though; and if not, perhaps the original patch was correct? Do you think it's time to start thinking of gradually removing conda from the tree?

(It's a bit unfortunate that llama-stack run combines both deployment and execution goals in a single tool. Ideally, virtual environment / container preparation would be handled separately; leaving run to assume the environment is ready to execute. Perhaps we should do this separation more explicit at some point.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@leseb leseb leseb requested changes

@ashwinb ashwinb Awaiting requested review from ashwinb ashwinb is a code owner

@yanxi0830 yanxi0830 Awaiting requested review from yanxi0830 yanxi0830 is a code owner

@hardikjshah hardikjshah Awaiting requested review from hardikjshah hardikjshah is a code owner

@dltn dltn Awaiting requested review from dltn dltn is a code owner

@raghotham raghotham Awaiting requested review from raghotham raghotham is a code owner

@dineshyv dineshyv Awaiting requested review from dineshyv dineshyv is a code owner

@vladimirivic vladimirivic Awaiting requested review from vladimirivic vladimirivic is a code owner

@sixianyi0721 sixianyi0721 Awaiting requested review from sixianyi0721 sixianyi0721 is a code owner

@ehhuang ehhuang Awaiting requested review from ehhuang ehhuang is a code owner

@terrytangyuan terrytangyuan Awaiting requested review from terrytangyuan terrytangyuan is a code owner

@SLR722 SLR722 Awaiting requested review from SLR722 SLR722 is a code owner

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
CLA Signed This label is managed by the Meta Open Source bot.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@booxter @leseb @ashwinb @facebook-github-bot
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy