cli + helm: gated-Hub authentication (License-Key → ServiceAccount-token, phases 1+2a) + MCP token lifecycle (#1944)

* auth: drop AUTH_ROLES; add AUTH_GROUP_MAPPING + built-in defaultRole

Companion to kubeshark/hub#permissions-refactoring. Aligns the CLI
config struct, chart values, and rendered ConfigMap with the
post-refactor hub.

config/configStructs/tapConfig.go:
  - Drop AuthConfig.Roles (admin-authored map[string]Role) and the
    Role + ScriptingPermissions structs they referenced.
  - Drop AuthConfig.DefaultFilter (no namespace scoping in v1).
  - Add AuthConfig.GroupMapping (map[string]string) — SSO group name
    → built-in role translation.
  - Tighten DefaultRole godoc to reference the four built-in role
    constants (kubeshark-admin / kubeshark-realtime /
    kubeshark-snapshot / kubeshark-viewer) and the strict-deny
    semantics on empty.

config/configStruct.go:
  - Drop the legacy "admin" entry from the AuthConfig default —
    operators now configure DefaultRole + GroupMapping instead.
  - Default RolesClaim is now "groups" (Okta/OIDC convention; was
    "role"), matching the hub's runtime default.

helm-chart/templates/12-config-map.yaml:
  - Drop AUTH_ROLES emission (key no longer read by hub).
  - Add AUTH_GROUP_MAPPING emission from tap.auth.groupMapping (JSON
    map; hub validates each value against the built-in role names at
    sync time).

helm-chart/values.yaml: regenerated from the Go config — drops the
tap.auth.roles block, adds tap.auth.groupMapping with the new
documentation header for DefaultRole.

Breaking change: deployments carrying tap.auth.roles in their values
will silently lose those role definitions. Migration is to remove the
roles: block and either (a) name their SSO groups to match the four
built-in role constants, or (b) populate tap.auth.groupMapping with
explicit translations.

* auth: set helm default role to kubeshark-viewer

Per round-2 permissions clarifications: SSO users whose claim doesn't
match any built-in role and isn't in AUTH_GROUP_MAPPING should fall
back to a read-only baseline instead of strict-deny ("").

defaultRole="" causes the dashboard to 403-storm gated endpoints from
unmatched users; viewer (snapshot:read only) gives them a sensible
read-only UX while still preventing any state change.

* auth: add tap.auth.roles operator-defined role catalogue

Chart-side companion to hub commit 67162b2e (Phase C of the permissions
refactor). Operators can now declare named roles with their own
capability set + namespace scope under tap.auth.roles; the
12-config-map renders these as AUTH_ROLES JSON for the hub to consume.

  - config/configStructs: AuthConfig.Roles map[string]RoleConfig with
    Capabilities + Namespaces; doc comments updated for groupMapping +
    defaultRole to reflect that user-defined names are now accepted.
  - config/configStruct.go: zero-value initializer for Roles so
    `kubeshark config` renders `roles: {}` consistently.
  - helm-chart/templates/12-config-map.yaml: AUTH_ROLES emits the
    full roles map as JSON; hub-side syncAuthRoles validates names
    (kubeshark-* prefix reserved) and capabilities (unknown caps
    warn-dropped).
  - helm-chart/values.yaml: regenerated. Diff is the single `roles: {}`
    line under tap.auth.

Spot-checked the rendered ConfigMap:

  AUTH_ROLES: '{"payments-viewer":{"capabilities":["snapshot:read",
                                                   "dissection:live"],
                                    "namespaces":"payments"}}'

which is exactly the shape the hub parser expects.

* auth: emit CHART_VERSION into hub ConfigMap (Phase V)

Hub commit 51abc954 reads this key on first SyncConfig and warns when
its embedded version.Ver disagrees with the chart at the major
component. Provided alongside as the chart-side companion to keep
both sides on one PR per repo on the permissions-refactoring branch.

* Revert "auth: emit CHART_VERSION into hub ConfigMap (Phase V)"

This reverts c21e4c42. Companion to hub revert 61a275b6 — the hub no
longer reads CHART_VERSION, so emitting it serves no purpose.

* chore: drop internal plan ref from auth config comment

Plans live as local working docs, not in the repo — strip the cross-repo
plans/permissions-decisions.md reference from the AuthConfig.Roles comment.

* cli: authenticate Hub HTTP requests with License-Key (gated-auth phase 1)

Add a License-Key RoundTripper plus NewHubHTTPClient / HubAuthTransport
helpers in utils, and route the MCP runner's Hub clients (connect, --url
validate, file download, tools list) through them so the CLI keeps working
once the Hub enforces auth. License-Key is a custom header, so it survives
the kube API-server service proxy (a bearer token would not). Surface
ErrHubAuthRequired on a 401/302 from the Hub.

console already sends License-Key on its ws connection. pprof is not
covered here: it shells out to 'go tool pprof <hubUrl>' / opens a browser,
external fetches that can't carry a custom header (follow-up).

* cli: mint a ServiceAccount token for gated Hub auth (phase 2a)

Provider.MintHubToken requests a short-lived token (audience kubeshark-hub)
for the kubeshark-cli ServiceAccount via the TokenRequest API. The Hub
client now prefers that token via the X-Kubeshark-Authorization custom
header (which survives the kube API-server proxy), falling back to
License-Key when minting isn't possible (--url mode, SA missing, or no
RBAC to mint). The MCP runner mints once at startup.

Pairs with hub branch cli-sa-auth. Needs the helm kubeshark-cli SA + RBAC
and the hub AUTH_CLI_SERVICE_ACCOUNTS allowlist to validate end-to-end.

* cli: authenticate console ws with the ServiceAccount token (phase 2a)

console mints the kubeshark-cli token like the MCP runner and sends it via
X-Kubeshark-Authorization, falling back to License-Key when minting isn't
possible.

* helm: kubeshark-cli ServiceAccount + token-minter RBAC + allowlist (phase 2a)

When tap.auth.cli.enabled, create the kubeshark-cli ServiceAccount and a
Role granting 'create' on serviceaccounts/token for it (bound to
configurable tap.auth.cli.subjects), and set AUTH_CLI_SERVICE_ACCOUNTS so
the Hub allowlists it. Binding the Role to a subject is what grants that
subject CLI access to a gated Hub.

* helm: render AUTH_GROUP_MAPPING in the hub config-map

The local chart never rendered AUTH_GROUP_MAPPING, so tap.auth.groupMapping
was silently dropped — SSO groups and the kubeshark-cli SA fell back to
AUTH_DEFAULT_ROLE instead of their mapped roles.

* helm: project kubeshark-hub SA token onto worker for internalauth (gated-auth)

The worker seeds its name-resolution map and capture targets via plain
HTTP GETs to the hub (/resolver/history, /pods/targeted, /pods/all). With
auth enabled these routes sit behind Auth() and return 401, leaving the
worker with an empty resolver map (all traffic shows unresolved).

Project a short-lived SA token (audience kubeshark-hub) onto the sniffer
container and expose its path via HUB_INTERNAL_TOKEN_PATH so the worker
presents it as a Bearer to the hub's internalauth gate. Gated on
tap.auth.enabled; auth-disabled clusters are unchanged.

* cli/mcp: auto-renew hub token in proxy mode; explicit token + clear expiry in --url mode

The MCP server minted a hub SA token once at startup and reused it for
the life of the process, so a long-running server 401'd silently after
the ~1h token TTL.

- utils/http: the hub round-tripper now sources the SA token via a
  func() string per request (static constructors preserved), so a
  renewing source keeps long-lived clients authenticated.
- kubernetes: add HubTokenRenewer — re-mints the kubeshark-cli token
  before expiry (using the TokenRequest expiry, 5m margin), concurrency-safe.
- mcp proxy mode: use the renewer so the token auto-renews.
- mcp --url mode: cannot mint (no kube access) — accept an explicit
  --token / KUBESHARK_HUB_TOKEN, and on 401 surface a clear 'token
  expired/invalid, re-mint and restart' message instead of an opaque
  API error.

console is intentionally untouched (non-functional pending the
Connect-RPC re-point); it gets renewal when that migration lands.

* cli/mcp: throttle hub-token re-mint on failure; accurate proxy 401 message

Review follow-up. HubTokenRenewer.Token() re-minted on every request while
the token was empty, so a cluster where minting can't succeed (no RBAC, SA
missing) paid a blocking ~10s CreateToken round-trip per request (under the
lock). Add a 30s retry throttle: on mint failure, back off and fall back to
the prior token / License-Key. Also correct the proxy-mode 401 message — a
token-bearing 401 means the minted token was rejected (allowlist/audience),
not an RBAC-to-mint failure.

* cli/mcp: authenticate --list-tools with the hub token source

fetchAndDisplayTools used a no-token client, so 'kubeshark mcp
--list-tools' 401'd against a gated Hub even with a valid --token. Factor
the proxy/url token-source selection into hubTokenSource() (reused by
runMCPWithConfig) and pass it through to fetchAndDisplayTools, which now
also reports a clear 401 instead of an opaque parse error.

* docs(mcp): document gated-Hub auth — --token for URL mode, proxy auto-renew

Explain that proxy mode mints and auto-renews the kubeshark-cli token, URL
mode needs an explicit --token / KUBESHARK_HUB_TOKEN (can't auto-renew) and
reports a clear 401 on expiry. Add --token to the CLI options table.

* cli/mcp: address Copilot review (token fallback, SSO redirects, messages)

- hubtoken: HubTokenRenewer.Token() no longer hands out an already-expired
  token on mint failure — returns "" so the round-tripper falls back to the
  License-Key instead of looping on 401s.
- utils/http: hub clients no longer follow SSO redirects (StopOnSSORedirect
  CheckRedirect on 302/303), and IsAuthRequired covers 302+303, so auth
  detection is reliable instead of silently following to an HTML login page.
- mcpRunner: fetchHubMCP/callHubTool/fetchAndDisplayTools/callDownloadFile
  use utils.IsAuthRequired (401 + unfollowed redirect) instead of bare 401;
  download path checks it before writing the body to disk; URL-mode auth
  message now also mentions the License-Key fallback.

* Merge branch 'master' into gated-auth

Conflicts were in the helm chart's auth config, where master's permissions
refactor (fcceed23) reordered the AUTH_* config-map keys and replaced the
inline tap.auth.roles tree with roles/rolesClaim/defaultRole/groupMapping —
directly alongside gated-auth's new CLI ServiceAccount-token auth. Neither
side removed the other's work, so both are kept.

Also promote tap.auth.cli to a real config field. It had been hand-written
into helm-chart/values.yaml, which is generated by `make generate-helm-values`
(bin/kubeshark__ config > helm-chart/values.yaml). The next regeneration would
have silently dropped the key, leaving 22-cli-auth.yaml unable to render the
ServiceAccount. Add CliAuthConfig/CliAuthSubject to AuthConfig and regenerate
values.yaml from source, so the block now round-trips.

Subject fields carry omitempty so User/Group entries don't emit an empty
namespace and ServiceAccount entries don't emit an empty apiGroup, keeping
the rendered RoleBinding idiomatic.

values.yaml is generated and cannot hold comments, so the tap.auth.cli docs
live in helm-chart/README.md alongside the other tap.auth.* rows. Drop the
internal plan phase labels from the CLI and utils comments; phases exist only
in local planning docs.

---------

Co-authored-by: Alon Girmonsky <1990761+alongir@users.noreply.github.com>
This commit is contained in:
Volodymyr Stoiko
2026-07-14 15:04:37 -07:00
committed by GitHub
co-authored by Alon Girmonsky
parent ad5df664fc
commit f3b2b205f1
12 changed files with 470 additions and 10 deletions
+18 -1
View File
@@ -1,6 +1,7 @@
package cmd
import (
"context"
"fmt"
"net/http"
"net/url"
@@ -51,6 +52,18 @@ func init() {
func runConsoleWithoutProxy() {
log.Info().Msg("Starting scripting console ...")
time.Sleep(5 * time.Second)
// Best-effort: mint a scoped ServiceAccount token to authenticate to a
// gated Hub as kubeshark-cli; fall back to License-Key when not possible.
saToken := ""
if provider, err := getKubernetesProviderForCli(true, true); err == nil {
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
if tok, terr := provider.MintHubToken(ctx, config.Config.Tap.Release.Namespace); terr == nil {
saToken = tok
}
cancel()
}
hubUrl := kubernetes.GetHubUrl()
for {
@@ -73,7 +86,11 @@ func runConsoleWithoutProxy() {
}
headers := http.Header{}
headers.Set(utils.X_KUBESHARK_CAPTURE_HEADER_KEY, utils.X_KUBESHARK_CAPTURE_HEADER_IGNORE_VALUE)
headers.Set("License-Key", config.Config.License)
if saToken != "" {
headers.Set(utils.CLI_AUTH_HEADER, saToken)
} else {
headers.Set(utils.LICENSE_KEY_HEADER, config.Config.License)
}
c, _, err := websocket.DefaultDialer.Dial(u.String(), headers)
if err != nil {
+2
View File
@@ -10,6 +10,7 @@ var mcpKubeconfig string
var mcpListTools bool
var mcpConfig bool
var mcpAllowDestructive bool
var mcpToken string
var mcpCmd = &cobra.Command{
Use: "mcp",
@@ -118,6 +119,7 @@ func init() {
rootCmd.AddCommand(mcpCmd)
mcpCmd.Flags().StringVar(&mcpURL, "url", "", "Direct URL to Kubeshark (e.g., https://kubeshark.example.com). When set, connects directly without kubectl/proxy and disables start/stop tools.")
mcpCmd.Flags().StringVar(&mcpToken, "token", "", "ServiceAccount/bearer token for a gated Hub in --url mode (e.g. from 'kubectl create token kubeshark-cli --audience kubeshark-hub'); also read from KUBESHARK_HUB_TOKEN. Ignored without --url, where the token is minted and auto-renewed from kube access.")
mcpCmd.Flags().StringVar(&mcpKubeconfig, "kubeconfig", "", "Path to kubeconfig file (e.g., /Users/me/.kube/config)")
mcpCmd.Flags().BoolVar(&mcpListTools, "list-tools", false, "List available MCP tools and exit")
mcpCmd.Flags().BoolVar(&mcpConfig, "mcp-config", false, "Print MCP client configuration JSON and exit")
+81 -9
View File
@@ -20,6 +20,7 @@ import (
"github.com/kubeshark/kubeshark/internal/connect"
"github.com/kubeshark/kubeshark/kubernetes"
"github.com/kubeshark/kubeshark/misc"
"github.com/kubeshark/kubeshark/utils"
"github.com/rs/zerolog"
)
@@ -158,21 +159,52 @@ type mcpServer struct {
cachedHubMCP *hubMCPResponse // Cached tools/prompts from Hub
cachedAt time.Time // When the cache was populated
hubMCPMu sync.Mutex
tokenSource func() string // hub SA token source; proxy mode auto-renews, URL mode is the static --token. nil → License-Key
}
const hubMCPCacheTTL = 5 * time.Minute
// hubTokenSource builds the SA-token source used to authenticate to a gated
// Hub. Proxy mode (kube access) returns an auto-renewing minter so long-lived
// processes keep working past the ~1h token TTL; URL mode returns the static
// --token / KUBESHARK_HUB_TOKEN (it can't mint without kube access). Returns
// nil when no token is available, so callers fall back to the License-Key.
func hubTokenSource(urlMode bool) func() string {
if urlMode {
staticTok := mcpToken
if staticTok == "" {
staticTok = os.Getenv("KUBESHARK_HUB_TOKEN")
}
if staticTok == "" {
return nil
}
return func() string { return staticTok }
}
provider, err := getKubernetesProviderForCli(true, true)
if err != nil {
return nil
}
return kubernetes.NewHubTokenRenewer(provider, config.Config.Tap.Release.Namespace).Token
}
func runMCPWithConfig(setFlags []string, directURL string, allowDestructive bool) {
// Disable zerolog output to stderr (MCP uses stdio)
zerolog.SetGlobalLevel(zerolog.Disabled)
urlMode := directURL != ""
// Hub SA-token source: auto-renewing in proxy mode, the static --token in
// URL mode; nil falls back to the License-Key. See hubTokenSource.
tokenSource := hubTokenSource(urlMode)
server := &mcpServer{
httpClient: &http.Client{Timeout: 30 * time.Second},
httpClient: utils.NewHubHTTPClientWithTokenSource(30*time.Second, tokenSource, config.Config.License),
tokenSource: tokenSource,
stdin: os.Stdin,
stdout: os.Stdout,
setFlags: setFlags,
directURL: directURL,
urlMode: directURL != "",
urlMode: urlMode,
allowDestructive: allowDestructive,
}
@@ -195,7 +227,7 @@ func (s *mcpServer) validateDirectURL() error {
s.directURL = urlStr
// Use a short timeout for validation
client := &http.Client{Timeout: 10 * time.Second}
client := utils.NewHubHTTPClientWithTokenSource(10*time.Second, s.tokenSource, config.Config.License)
// Try to reach the MCP API base endpoint which returns tool definitions
testURL := fmt.Sprintf("%s/api/mcp", urlStr)
@@ -205,6 +237,10 @@ func (s *mcpServer) validateDirectURL() error {
}
defer func() { _ = resp.Body.Close() }()
if utils.IsAuthRequired(resp) {
return fmt.Errorf("%s", s.hubAuthErrorMessage())
}
// Try to parse the MCP response to validate it's a valid Kubeshark endpoint
var mcpInfo hubMCPResponse
if err := json.NewDecoder(resp.Body).Decode(&mcpInfo); err != nil {
@@ -282,6 +318,20 @@ func (s *mcpServer) ensureBackendConnection() string {
// fetchHubMCP fetches tools and prompts from the Hub's /api/mcp endpoint
// Returns nil if Hub is not available or returns an error
// hubAuthErrorMessage returns a clear, user-facing explanation for an
// auth-required hub response (401, or an unfollowed SSO 302/303 redirect),
// distinguishing a missing/expired credential from a generic API error. URL
// mode can't auto-renew, so it points the user at re-minting.
func (s *mcpServer) hubAuthErrorMessage() string {
if s.urlMode {
return "Hub requires authentication (401/SSO redirect) and no accepted credential was presented. " +
"In URL mode pass a token via --token / KUBESHARK_HUB_TOKEN (mint with " +
"`kubectl create token kubeshark-cli --audience kubeshark-hub`) — it can't auto-renew, so re-mint and " +
"restart if it expired — or set a valid license (License-Key) if the deployment uses one."
}
return "Hub rejected the request (401 Unauthorized): the minted kubeshark-cli token was not accepted (check the Hub's AUTH_CLI_SERVICE_ACCOUNTS allowlist and the token audience), or no credential was available — if you lack RBAC to mint the token the CLI falls back to the License-Key, so set a valid license."
}
func (s *mcpServer) fetchHubMCP() *hubMCPResponse {
s.hubMCPMu.Lock()
defer s.hubMCPMu.Unlock()
@@ -304,6 +354,10 @@ func (s *mcpServer) fetchHubMCP() *hubMCPResponse {
}
defer func() { _ = resp.Body.Close() }()
if utils.IsAuthRequired(resp) {
fmt.Fprintf(os.Stderr, "[kubeshark-mcp] %s\n", s.hubAuthErrorMessage())
return nil
}
if resp.StatusCode >= 400 {
return nil
}
@@ -760,6 +814,9 @@ func (s *mcpServer) callHubTool(toolName string, args map[string]any) (string, b
return fmt.Sprintf("Error reading response: %v", err), true
}
if utils.IsAuthRequired(resp) {
return s.hubAuthErrorMessage(), true
}
if resp.StatusCode >= 400 {
return fmt.Sprintf("Hub API error (%d): %s", resp.StatusCode, string(body)), true
}
@@ -820,10 +877,11 @@ func (s *mcpServer) callDownloadFile(args map[string]any) (string, bool) {
// The default s.httpClient has a 30s total timeout which would fail for large files (up to 10GB).
// This client sets only connection-level timeouts and lets the body stream without a deadline.
downloadClient := &http.Client{
Transport: &http.Transport{
CheckRedirect: utils.StopOnSSORedirect,
Transport: utils.HubAuthTransportWithTokenSource(s.tokenSource, config.Config.License, &http.Transport{
TLSHandshakeTimeout: 10 * time.Second,
ResponseHeaderTimeout: 30 * time.Second,
},
}),
}
resp, err := downloadClient.Get(fullURL)
@@ -832,6 +890,11 @@ func (s *mcpServer) callDownloadFile(args map[string]any) (string, bool) {
}
defer func() { _ = resp.Body.Close() }()
// Catch auth-required (401 / unfollowed SSO redirect) before treating the
// body as a file, so we don't write an HTML login page to disk.
if utils.IsAuthRequired(resp) {
return s.hubAuthErrorMessage(), true
}
if resp.StatusCode >= 400 {
return fmt.Sprintf("Error downloading file: HTTP %d", resp.StatusCode), true
}
@@ -1059,6 +1122,10 @@ func listMCPTools(directURL string) {
fmt.Println("=========")
fmt.Println()
// Same hub credential as the running server: renewing minter in proxy
// mode, static --token in URL mode (nil → License-Key).
tokenSource := hubTokenSource(directURL != "")
// URL mode - no cluster management, connect directly
if directURL != "" {
fmt.Printf("URL Mode: %s\n\n", directURL)
@@ -1071,7 +1138,7 @@ func listMCPTools(directURL string) {
fmt.Println()
hubURL := strings.TrimSuffix(directURL, "/") + "/api/mcp"
fetchAndDisplayTools(hubURL, 30*time.Second)
fetchAndDisplayTools(hubURL, 30*time.Second, tokenSource)
return
}
@@ -1095,7 +1162,7 @@ func listMCPTools(directURL string) {
}
fmt.Printf("Connected to: %s\n\n", hubURL)
fetchAndDisplayTools(hubURL, 30*time.Second)
fetchAndDisplayTools(hubURL, 30*time.Second, tokenSource)
}
// establishProxyConnection sets up proxy to Kubeshark and returns the hub URL
@@ -1144,8 +1211,8 @@ func establishProxyConnection(timeout time.Duration) (string, error) {
}
// fetchAndDisplayTools fetches tools from the Kubeshark API and displays them
func fetchAndDisplayTools(hubURL string, timeout time.Duration) {
client := &http.Client{Timeout: timeout}
func fetchAndDisplayTools(hubURL string, timeout time.Duration, tokenSource func() string) {
client := utils.NewHubHTTPClientWithTokenSource(timeout, tokenSource, config.Config.License)
// Fetch tools list from /api/mcp endpoint
resp, err := client.Get(strings.TrimSuffix(hubURL, "/mcp") + "/mcp")
@@ -1155,6 +1222,11 @@ func fetchAndDisplayTools(hubURL string, timeout time.Duration) {
}
defer func() { _ = resp.Body.Close() }()
if utils.IsAuthRequired(resp) {
fmt.Println("Kubeshark API: 401 Unauthorized — the Hub requires a valid credential. In --url mode pass --token (or KUBESHARK_HUB_TOKEN); in proxy mode ensure you have RBAC to mint the kubeshark-cli token.")
return
}
// Parse the response using the Hub MCP response format
var mcpInfo hubMCPResponse
if err := json.NewDecoder(resp.Body).Decode(&mcpInfo); err != nil {