I see this as an advantage of the multi-tenant system, not a challenge. Multiple organizations guarantee provisioner isolation, and, within an org, we will have "provisioner groups" that provide isolation. In an enterprise or SaaS setting, two organizations, even with malicious intent for the other, can safely co-exist on the deployment.

Of course, two orgs that want to unify provisioning infrastrcture can both deploy provisioners alongside eachother.

Multiple organizations guarantee provisioner isolation, and, within an org, we will have "provisioner groups" that provide isolation.

I don't fully understand this sentence. What are the technical means that provide the isolation?

Like, if the isolation mechanism is separate deployments, I don't get why we're saying separate Coder deployments is a significant administrative burden, but separate provisioner deployments is not.

I don't fully understand this sentence. What are the technical means that provide the isolation?

Templates in one organization cannot use the same provisioner as templates in another organization.

Like, if the isolation mechanism is separate deployments, I don't get why we're saying separate Coder deployments is a significant administrative burden, but separate provisioner deployments is not.

I'm thinking about the pains as many of the same ones that would afflict us in SaaS. While there is tenant isolation, we would want to unify operations around the control plane, esp. geo-distribution, security, disaster recovery / backups, and deployability. It would be quite cumbersome to run different instances of the coder process and different databases for each organization.

There is also the licensing benefit. It's very difficult for us to enforce seat limits when an enterprise is using multiple deployments.

@szab100 If we were to add a LoginTypeNone on a user by user basis that prevents logging in via any means (oidc, password, etc), that would satisfy what you are looking for.

A follow up question would be, once this user is given an api key/token (via some other means like an admin making them one), should that user then be able to generate more tokens/keys? This is not a hard problem to solve, just curious on your take.

Hey @Emyrk, thanks for the reply. Yes, that would solve the problem (not a huge one, really, since passwords are never given to users).

The token generation is indeed an interesting question. If the token's scope is all and the user's Role has permissions on ResourceAPIKey, then probably yes, it should allow generating new tokens/keys. So this looks like an RBAC / AuthZ question rather than whether login is allowed or not.

This does not really affect us, since we run a reverse-proxy in front of the Coder deployments, which selects & decrypts the user's session token (scope='all') from our app's combined Cookie value (having a b64-encoded JSON map of the user's encrypted tokens for multiple Coder deployments). After decrypting the token (with a key unique to the Coder deployment), the rev-proxy sets the cleartext token as Coder-Session-Token header, but only for a set of whitelisted "safe" request patterns needed for accessing workspaces, eg. various GET Coder API endpoints, like user info, workspace agents & connections (/coordinate, /pty etc).. The only POST we allow for user requests is /api/v2/authcheck 🙈 And we always issue user tokens through our API (returns encrypted tokens) using an admin user token, so similarly to user passwords, we never handout cleartext Coder tokens to our users either.

Speaking of tokens, we are working on a feature called "workspace sharing" (multi-tenancy on workspace level 😄, primarily for pair-programming / troubleshooting) where users can share a single workspace with guests (connect using ssh + access workspace apps). We can do this on our end by handing out the workspace owner's regular (scope=full), encrypted session token with some extra fields in our Cookie's JSON to restrict guest access to a given workspace name / enforce shorter expiration or revocation (+ a signature to avoid user-tampering). But in order to limit access to a single workspace, our rev-proxy needs to read into Coder's DB (with heavy caching, since workspace & agent IDs are immutable) using Coder's internal api (eg. coderd's database.Store & workspaceapps.Provider) to look up workspace details (name / id) from ws-agent or ws-apps type of requests. We have a working POC for this, but thought it would be both easier & more efficient to simply pass the token through with the requests to Coder, so it would limit the access to the single workspace the token is related to. We actually thought of contributing this small feature. The problem is that even with such workspace-scoped tokens, we can't avoid looking up workspace details for incoming requests, since when UserA (guest) wants to access UserB's shared workspace, we need to know which workspace the requests are related to, eg whether to set the guest token (to access UserB's shared workspace) OR UserA's normal access token (to access his own workspaces) on the outgoing Coder requests.

Maybe a better approach to support "workspace sharing" would be letting users explicitly grant temporary access to their workspaces to other Coder users, eg. UserA authorizes UserB to access its Workspace-X for 30 mins (or until revoked), so UserB will be granted to access UserA's Workspace-X using UserB's own (full) session token. This is trickier than the workspace-scoped user-tokens explained above, so probably only worth if you want to build out a complete Workspace Sharing feature for Coder. But then guest workspace access can be properly handled, like displaying shared workspaces on the UI (similarly to GSuite's "Shared with me" tab), proper Audit log entries, etc. Maybe in case of SSH, even spawn guest shells with a differentiated guest_user (configured with agent) in the Workspace, eg. guest on Linux K8S Pods, so customers can set up workspace permissions accordingly, like denying 'sudo' / set restricted / jailed shells and prevent guests reading the owner's home dir (we use /workspace as PVC mount for work area, but /home can still contain temp credentials, etc), while giving write access to the work area to both the owner & guest and running web-based workspace apps with dedicated users instead of the owner's.

Or a third (simplest) option would be to further improve "Application Scoped API Keys", so that those api-keys can be requested to limit access to a single workspace's single application, as well as the /api/v2/applications/auth-redirect endpoint to support the required additional query params (eg workspace_id and app_name), so the redirect URI with the encrypted token it returns with will only allow the guest user(s) to access that particular app / workspace. In this case, we could simply give out these redirect URLs to guest users, giving them access to VSCode only (which we can run with a restricted user instead of the main workspace user). This endpoint should also return with a HTTP header containing the created app-scoped token's ID, which callers can store in their DB and delete the token when the user wants to "unshare" the workspace, which will kill the guest access (except guest who were already issued a "SignedToken", unless Coder keeps a list of unvalidated JWTs!). This would equal to Gitpod's link-based "live workspace sharing" feature, which fulfills our requirements.

Hmm, actually, I just realized that this 3rd option (only giving guests access to VSCode Web) is something we can already solve with the rev-proxy solution without any DB lookups, by adding a "guest_tokens" JSON map to our session Cookie, where the keys are the FQDN application subdomains of the shared workspaces' VSCode apps. When we process requests, if it's a subdomain (host != coder access url) request, we simply just need to see if the user has a "guest token" for that subdomain (it's actually the owner's token & is still encrypted + the subdomain / expiration signed so users can't modify) and if so, just pass the "guest token" for those requests and use the user's regular token for all other (permitted) requests, so guests can still fully access their own workspaces. No problem with SignedTokens either, as we can maintain a global guest token revocation list, so as soon as our rev-proxy finds out that the guest token was revoked, it can simply omit the app's JWT Cookie from downstream requests ==> the guest access is killed immediately.

So sorry for the super lengthy comment (and thanks for being my rubber duck.. 🤣)! Hope at least some of this is useful for you to further improve Coder, we should be good without any changes actually, while being able to narrow APIKeys' scope down to either a single workspace and/or a single [workspace + app] would probably be beneficial to many users, but all up to you. ✌️

Yes, that would solve the problem

Not a huge problem, but it is now possible to set LoginTypeNone!
#8009

The token generation is indeed an interesting question. If the token's scope is all and the user's Role has permissions on ResourceAPIKey, then probably yes, it should allow generating new tokens/keys. So this looks like an RBAC / AuthZ question rather than whether login is allowed or not.

Exactly what I was thinking. I mainly asked this to check if your situation required a new scope to be added, and I could get that in for you quickly. If this is not a blocker though, then we do not need to do this now.

... shared workspaces ...

This is a feature we have considered. I personally think it would be pretty neat, and coming up with the implementation is tricky from all the things you have mentioned. I imagine we might need to pair it well with something in the agent/terraform to pass the logged in "user" into the workspace at some controlled permission.

while being able to narrow APIKeys' scope down to either a single workspace and/or a single [workspace + app] would probably be beneficial to many users, but all up to you.

This is interesting because we actually had this in the initial version of the RBAC.

#3426

The description is a more high level why it was done, but there was some technical reasons as well. In short, we can always answer this question "List me all workspaces I have access to, 50 at a time for pagination" quickly because of this simplification. Essentially we take the rego policy (our authz language) and convert it to SQL and then we can use LIMIT and OFFSET to paginate. It's clean and great.

If we add resource specific permissions, we have the problem that the SQL query now grows without bounds. If you have 200 workspaces shared with you, the query has to include: AND id = ANY(1,2,3,4...,200). This is probably solvable since our "without bounds" is still likely some reasonable number that postgres can handle, but it is unfortunate that our query size for list calls could blow up.

The second technical reason assigning perms to a single workspace is difficult, is that it makes discoverability of who can access a workspace exceptionally hard. To answer the question "who can access workspace x", I now need to query all users and check their roles. Then maybe in your case also query all api keys since the "user" might be able too, but they have no api keys that can.

This discoverability is bad from a UX POV, but also from a management POV. How do we ensure credentials are not hanging around and your workspace actually has some backdoor access you forgot to revoke?

Now workspace sharing is still something we want and we need a solution. The current proposed idea by yours truly (me 😄) is to use ACL's on the workspace itself.

We actually do this for templates.

This solves the two technical problems above.

1. The SQL query is bounded because the query says AND user_id = ANY(template.user_acl) (deal with jsonb, but that's the idea). This is a bounded query because the number of shared users does not affect the SQL query length. There is another benefit in how we convert rego -> sql, but not worth getting into here.
2. Discoverability is easy as it is tied to the workspace. If you can fetch the workspace, you have the ACL list of anyone aside from the owner.

So I think how workspace sharing would work is an ACL list on the workspace itself. How that lends itself to your application means you need both the workspace and the user to know if the access can be granted.

So sorry for the super lengthy comment

Love long comments. Seems you are doing some really cool stuff ontop of Coder!