Skip to main content

Internal User Self-Serve

Allow users to create their own keys on Proxy UI.​

  1. Add user with permissions to a team on proxy

Go to Internal Users -> +New User

  1. Share invitation link with user

Copy the invitation link with the user

info

Use Email Notifications to email users onboarding links

  1. User logs in via email + password auth
info

LiteLLM Enterprise: Enable SSO login

  1. User can now create their own keys

Allow users to View Usage, Caching Analytics​

  1. Go to Internal Users -> +Invite User

Set their role to Admin Viewer - this means they can only view usage, caching analytics


  1. Share invitation link with user

  1. User logs in via email + password auth

  1. User can now view Usage, Caching Analytics

Available Roles​

Here's the available UI roles for a LiteLLM Internal User:

Admin Roles:

  • proxy_admin: admin over the platform
  • proxy_admin_viewer: can login, view all keys, view all spend. Cannot create/delete keys, add new users.

Internal User Roles:

  • internal_user: can login, view/create/delete their own keys, view their spend. Cannot add new users.
  • internal_user_viewer: can login, view their own keys, view their own spend. Cannot create/delete keys, add new users.

Advanced​

Setting custom logout URLs​

Set PROXY_LOGOUT_URL in your .env if you want users to get redirected to a specific URL when they click logout

export PROXY_LOGOUT_URL="https://www.google.com"

Set max budget for internal users​

Automatically apply budget per internal user when they sign up. By default the table will be checked every 10 minutes, for users to reset. To modify this, see this

litellm_settings:
  max_internal_user_budget: 10
  internal_user_budget_duration: "1mo" # reset every month

This sets a max budget of $10 USD for internal users when they sign up.

This budget only applies to personal keys created by that user - seen under Default Team on the UI.

This budget does not apply to keys created under non-default teams.

Set max budget for teams​

Go Here

Auto-add SSO users to teams​

  1. Specify the JWT field that contains the team ids, that the user belongs to. 
general_settings:
  master_key: sk-1234
  litellm_jwtauth:
    team_ids_jwt_field: "groups" # 👈 CAN BE ANY FIELD

This is assuming your SSO token looks like this:

{
  ...,
  "groups": ["team_id_1", "team_id_2"]
}
  1. Create the teams on LiteLLM 
curl -X POST '<PROXY_BASE_URL>/team/new' \
-H 'Authorization: Bearer <PROXY_MASTER_KEY>' \
-H 'Content-Type: application/json' \
-D '{
    "team_alias": "team_1",
    "team_id": "team_id_1" # 👈 MUST BE THE SAME AS THE SSO GROUP ID
}'
  1. Test the SSO flow

Here's a walkthrough of how it works

Restrict Users from creating personal keys​

This is useful if you only want users to create keys under a specific team.

This will also prevent users from using their session tokens on the test keys chat pane.

👉 See this

All Settings for Self Serve / SSO Flow​

litellm_settings:
  max_internal_user_budget: 10        # max budget for internal users
  internal_user_budget_duration: "1mo" # reset every month

  default_internal_user_params:    # Default Params used when a new user signs in Via SSO
    user_role: "internal_user"     # one of "internal_user", "internal_user_viewer", "proxy_admin", "proxy_admin_viewer". New SSO users not in litellm will be created as this user
    max_budget: 100                # Optional[float], optional): $100 budget for a new SSO sign in user
    budget_duration: 30d           # Optional[str], optional): 30 days budget_duration for a new SSO sign in user
    models: ["gpt-3.5-turbo"]      # Optional[List[str]], optional): models to be used by a new SSO sign in user


  upperbound_key_generate_params:    # Upperbound for /key/generate requests when self-serve flow is on
    max_budget: 100 # Optional[float], optional): upperbound of $100, for all /key/generate requests
    budget_duration: "10d" # Optional[str], optional): upperbound of 10 days for budget_duration values
    duration: "30d" # Optional[str], optional): upperbound of 30 days for all /key/generate requests
    max_parallel_requests: 1000 # (Optional[int], optional): Max number of requests that can be made in parallel. Defaults to None.
    tpm_limit: 1000 #(Optional[int], optional): Tpm limit. Defaults to None.
    rpm_limit: 1000 #(Optional[int], optional): Rpm limit. Defaults to None.

  key_generation_settings: # Restricts who can generate keys. [Further docs](./virtual_keys.md#restricting-key-generation)
    team_key_generation:
      allowed_team_member_roles: ["admin"]
    personal_key_generation: # maps to 'Default Team' on UI 
      allowed_user_roles: ["proxy_admin"]