Replace the legacy "redacted placeholder + Clear* boolean" pattern with
dedicated per-resource credential subresources across MCP services,
Models, Web search providers, and Data sources.
Why
---
The previous design had three problems:
1. Main PUT body carried secret fields. The frontend echoed back a
redacted "***" placeholder, and a fragile MergeUpdate / IsRedactedOrEmpty
defense in the service layer tried to detect "user did not change this"
vs "user wants to clear this". A regression in that defense (or a new
frontend forgetting it) silently overwrites the stored secret with the
placeholder.
2. The "remove this credential" UX was a red checkbox under a pre-filled
password input. Three intents (preserve / replace / clear) collapsed
onto one field, and credential changes were bundled with unrelated
config edits in the same submit. Users wiped working keys by mistake.
3. Secret presence was inferred from "did the response come back with a
'***' placeholder", which couples the contract to a magic string.
Design
------
Each of the four resources now exposes:
PUT /{resource}/{id}/credentials # write one or more fields
DELETE /{resource}/{id}/credentials/{field} # clear a single field
"Is this configured?" metadata travels on the main resource response as
a typed map (dto.MCPServiceResponse.Credentials etc.) — no separate GET
endpoint. The frontend reads the configured boolean from the main GET
and never sees secret values at all.
Main PUT endpoints now ignore any secret fields in the request body and
log a deprecation warning if they appear, so legacy clients fail loudly
rather than overwriting silently.
Frontend
--------
- New reusable <CredentialResource> component renders a three-state card
(unconfigured / configured / editing) and drives the dedicated
endpoints. Used by MCP, Model, Web search; DataSource has a bespoke
card with the same behaviour because its credentials are a single
atomic per-connector map.
- Cancel from edit mode now restores state synchronously from the meta
prop. The previous async refresh() was a no-op while state was still
'editing', leaving the input frozen open.
- Remove is single-click + toast. The danger-themed button is the
deterrent; a modal confirm adds friction without adding safety (the
plaintext is irrecoverable client-side either way — recovery means
re-typing).
DTOs (internal/handler/dto/) are deliberately separate from the GORM
models so "no secret in response" is a compile-time invariant: a future
contributor cannot leak a secret without explicitly adding the field to
the DTO, which is review-able in a single diff.
Storage is unchanged — credentials still live in the existing jsonb /
parameters columns. No schema migration.
Cleanup
-------
- types.MCPService / types.Model / types.WebSearchProviderEntity /
types.DataSourceConfig: drop ClearAPIKey / ClearToken / ClearAppSecret
/ ClearCredentials boolean fields, MergeUpdate(), RedactSensitiveData().
- utils/types/secret.go: drop PreserveIfRedacted / IsRedactedOrEmpty.
RedactedSecretPlaceholder constant is retained because VectorStore
still uses the old pattern and is out of scope here.
- Frontend hasExistingApiKey / clearApiKey / convertToLegacyFormat
redaction handling removed; i18n keys renamed secret -> credential.
docreader fails to start on a fresh `uv sync` of main:
$ uv sync --frozen
$ uv run python -m docreader.main
Traceback (most recent call last):
...
File ".../docreader/proto/docreader_pb2_grpc.py", line 19, in <module>
raise RuntimeError(...)
RuntimeError: The grpc package installed is at version 1.76.0, but the
generated code in docreader_pb2_grpc.py depends on grpcio>=1.78.0.
Please upgrade your grpc module to grpcio>=1.78.0 or downgrade your
generated code using grpcio-tools<=1.76.0.
`docreader_pb2_grpc.py:8` was regenerated with grpcio-tools 1.78+ and
hardcodes `GRPC_GENERATED_VERSION = '1.78.0'`, then at import time runs
`first_version_is_lower(GRPC_VERSION, GRPC_GENERATED_VERSION)` and
raises if the installed runtime is older. But `pyproject.toml` still
pins:
"grpcio>=1.76.0",
"grpcio-health-checking>=1.76.0",
"grpcio-tools>=1.76.0",
…and `uv.lock` resolves to grpcio 1.76.0. So every fresh clone +
`uv sync` ends up with a docreader that crashes on import, and a
WeKnora backend that thinks docreader is "not connected" — silently
falling back to the Go-side static engine registry only.
Downstream user-visible symptom: `/api/v1/system/parser-engines`
returns just the Go-side engines. Python-only engines (e.g.
markitdown, plus any custom Python parsers) never appear in the KB
create/edit form. Document parsing falls back to mineru/builtin even
when the user intended to use docreader-backed engines.
Reproduce on main (before this commit):
cd docreader
uv sync --frozen
uv run python -m docreader.main
# → RuntimeError at import, server never starts
After this commit:
Resolved 79 packages
Updated grpcio v1.76.0 -> v1.80.0
Updated grpcio-health-checking v1.76.0 -> v1.80.0
Updated grpcio-tools v1.76.0 -> v1.80.0
...
$ uv run python -c "from docreader.proto import docreader_pb2_grpc; print('ok')"
ok
Fix is the minimum-floor bump on the three grpcio packages, plus the
lockfile update uv emits as a result (all 1.76.0 → 1.80.0, newest
stable matching the proto's floor). No proto regen, no code change —
just aligning the dep declaration with what the generated client
already requires.
The next regen of docreader_pb2_grpc.py (e.g. with grpcio-tools 1.80+)
would raise the floor again; pinning at the proto's declared floor is
the conservative choice that keeps this in sync.
Closes#988
Unify the "write-only secrets" pattern across MCP services, Models,
WebSearch providers, and DataSources. Fix two related defects:
1. GET responses leaked plaintext credentials — GetMCPServiceByID
while List masked, all Model / WebSearchProvider / DataSource GET
responses unconditionally.
2. When the frontend echoed a masked placeholder back on save, the
update path overwrote the stored secret with the mask — silently
wiping the real credential. MCP surfaced this most visibly:
ListMCPServices masked Token to "first4****last4" and
McpServiceDialog round-tripped that masked string back into the DB.
- internal/types/secret.go introduces the shared sentinel
RedactedSecretPlaceholder = "***" and two helpers:
- IsRedactedOrEmpty(s): true for "" or the sentinel
- PreserveIfRedacted(incoming, existing): merge predicate
- Each resource adds a RedactSensitiveData (mutates-in-place) or
Redacted (returns-a-copy) method and a MergeUpdate contract:
empty or placeholder preserves, any other value replaces, and a
write-only Clear* bool flag (gorm:"-", json omitempty) wipes with
precedence over preserve / replace. Clear* never persists.
- Every GET / List / Create / Update response path now redacts.
VectorStore's pre-existing "***" literal is refactored onto the
shared constant for consistency.
All four dialogs — McpServiceDialog, ModelEditorDialog,
WebSearchSettings, DataSourceEditorDialog — never pre-fill secret
inputs. The stored-state signal is collapsed into the input
placeholder: "•••••••• (Enter new value to replace)" when the server
reports a stored value, falling back to the connector-specific
placeholder otherwise. The input itself carries the "something is
there" cue, matching how password dots render.
An opt-in "Remove this credential" checkbox appears in edit mode when
a value is stored; checking it disables the input and triggers a
danger-themed confirmation dialog before submit. DataSource rolls the
clear up to a connector-level "Remove all credentials" flag since its
credentials are per-connector atomic bundles (e.g. Feishu app_id +
app_secret).
i18n updates across en-US, zh-CN, ko-KR, ru-RU:
secret.storedPlaceholder / clearHint / confirmClearTitle /
confirmClearBody.
- MCP update handler's manual map-to-struct projection was silently
dropping clear_api_key / clear_token / custom_headers on partial
updates. Pull them off the map explicitly.
- MCP configChanged detection now snapshots URL / StdioConfig /
TransportType before the merge so connection recycling still
triggers on real transport changes (not just whenever AuthConfig
is present).
- MCP CustomHeaders: nil preserves existing, non-nil replaces. Was
always overwriting, silently wiping headers on any auth-bearing
update.
- GetMCPServiceByID now redacts (was plaintext while List masked).
- WebSearchProvider update preserves Name / Description when empty
instead of clobbering them.
- WebSearchProvider TestProviderRaw rejects the "***" placeholder up
front with an actionable message rather than forwarding it to the
upstream provider API as a literal key.
- Model builtin hideSensitiveInfo now preserves non-secret metadata
(SupportsVision, Provider, ExtraConfig, InterfaceType,
ParameterSize, EmbeddingParameters) that were previously zeroed
out for builtin model cards.
- Replace local equalStringSlices / equalStringMaps helpers with
stdlib slices.Equal / maps.Equal (Go 1.21+).
- internal/types/{secret,mcp,model,web_search_provider,datasource}_test.go:
table-driven coverage of merge, redact, clear, and write-only flag
invariants for every resource.
- internal/application/service/mcp_service_test.go: full service layer
with a fake repo; covers ListMCPServices / GetMCPServiceByID
redaction, UpdateMCPService partial-update honoring of Clear*,
CustomHeaders preserve-on-nil / replace-on-non-nil, and
clear-flag precedence over submitted values.
Additive only. No changes to request or response JSON shape beyond
the write-only Clear* flags (omitted by default via omitempty). No DB
schema changes. No SDK struct field removals. Existing clients that
don't know about the new flags continue to function unchanged.
- DataSource.Config jsonb at-rest encryption. Pre-existing residual
risk; tracked as a follow-up.
- Renaming VectorStore.MaskSensitiveFields to match the new
RedactSensitiveData / Redacted naming convention. Left alone to
minimize diff; a follow-up rename PR can unify.
Follow-up to #1359. Addresses a set of correctness and security gaps in
the initial docreader auth implementation.
- docker-compose: inject GRPC_TLS_*/GRPC_TLS_SERVER_NAME/GRPC_AUTH_TOKEN
into the WeKnora-app service. Without this the Go client never saw the
knobs, so enabling token auth on the server broke every RPC.
- client: bind tokenAuth.RequireTransportSecurity() to TLSEnabled so a
bearer token cannot be sent over an insecure channel once TLS is on.
- server: load_tls_credentials now raises TLSConfigError on misconfig
(cert/key missing, file unreadable, mTLS without CA); main.py exits 1
instead of silently downgrading to insecure.
- server: replace endswith("/Check"|"/Watch") health bypass with exact
match against /grpc.health.v1.Health/{Check,Watch}.
- server: compare tokens with hmac.compare_digest, warn on tokens < 16B.
- server: AuthInterceptor now returns an abort handler matching the
original RPC kind (unary/stream) and uses context.abort, so streaming
RPCs surface UNAUTHENTICATED instead of INTERNAL.
- internal/infrastructure/docparser/grpc_parser.go: drop the duplicated
TLS/tokenAuth block and reuse docreader/client.LoadAuthConfigFromEnv +
BuildDialOptions. Single source of truth for client-side auth.
- Add GRPC_TLS_SERVER_NAME (client SNI override) and
GRPC_MTLS_REQUIRE_CLIENT_CERT (server explicit mTLS toggle); document
the differing CA semantics between client and server in .env*.example.
- Reject half-configured client mTLS (cert XOR key) loudly.
- Fix missing trailing newline in .env.lite.example.
Verified locally: go build ./... and go vet ./... clean; auth.py
fail-fast / token paths smoke-tested.
Code-reuse polish (post-implementation review pass):
- Extract text.OneLine(maxWidth, s) helper combining preview-row
normalization (newline/CR/tab → space) with text.Truncate's
UTF-8-safe truncation. Replaces agent/view.go truncate1Line (ASCII
'...' + byte-slice CJK-unsafe) and chunk/list.go singleLine.
- Lift cmdutil.OpenInput(path) for the '-' = stdin / else os.Open
pattern shared across agent create/edit and the api command.
Replaces agent/create.go's private openInput.
- Strip inline doc-spec parentheticals from source comments — those
belong in commit messages and project docs, not in source where
they rot.
Pre-PR audit fixes:
- doc upload: reject `--metadata` paired with `--from-url` as
input.invalid_argument up-front (the URL-ingest request type has
no metadata field server-side, so the pair would otherwise silently
drop). Long help and CHANGELOG updated to call out the asymmetry.
- doc upload (file path): map sdk.ErrDuplicateFile sentinel to
resource.already_exists. The sentinel arrives with no "HTTP error <n>:"
prefix because the SDK short-circuits on file-hash before reading the
HTTP status, so the previous WrapHTTP fall-through misclassified it
as network.error with a misleading "check base URL reachability" hint.
The --from-url branch already handled ErrDuplicateURL this way; this
closes the asymmetry. Caught by e2e re-upload of an already-ingested
file; regression test added.
- README exit-10 enumeration adds `agent delete` and `chunk delete`
(these were missing alongside the v0.5 destructive verbs they were
meant to gate).
Docs sync:
- cli/README.md: command tree now includes the chunk subtree; adds
agent / chunk lines to the 5-minute quickstart; adds a "Contributing
/ Reporting issues" section pointing at the repo's SECURITY.md and
AGENTS.md; drops third-party CLI parallels from the surface
description.
- cli/AGENTS.md: "Command surface design SOP" gains the
flag-vs-escape-hatch step. "CRUD command flag canon" renamed to the
hard-required-flags pattern with the contrast (TTY-prompts-fill)
defined inline rather than via opaque shorthand.
- cli/CHANGELOG.md: search docs case-sensitivity shift promoted to its
own #### Breaking changes subsection. MCP doc_list filter count
corrected from 5 to 6. Drops the bogus go.mod yaml.v3 entry (yaml.v3
was already a dependency on main; v0.5 added zero go.mod lines).
Replaces internal-Go identifiers (fuzzyTime, NoOptDefVal) with
user-language and drops the § section-symbol jargon.
Brings search docs and search sessions to v0.4 pagination canon
(--limit / --page-size / --all-pages, matching session list / doc list).
Both default --all-pages=true to preserve prior silent walk-all
behavior; explicit knobs added for users who want one-page fetch.
cli/AGENTS.md gains two new sections:
- Command surface design SOP — a 5-step SDK-schema-first pre-design
checklist for future contributors. Earlier spec drafts produced
schema-error classes (missing/mismatched fields, missing pagination
flags) when commands were designed from convention rather than from
the SDK; the SOP makes the SDK the ground truth.
- CRUD command flag canon — Mode A (hard-required + flag error, no
interactive prompts), the established pattern for non-auth CRUD.
Also fixes the agent invoke rationale source: the CLI-layer precedent
for invoke being a separate verb (not a chat mode) is documented
inline rather than referencing other vendor CLI behavior.
Four unrelated shipped-code drifts found during v0.5 audit cycles:
1. MCP search_chunks tool omitted MatchCount from SearchParams. Server
fell back to its default cap; agents asking for limit:50 silently
got fewer results. Adds MatchCount: limit to the struct literal.
2. search sessions printed UpdatedAt as raw RFC3339 while session list
used a fuzzy "X hours ago" render — same SDK field, two human
renderings. Switches to the shared text.FuzzyAgoStr helper for
parity.
3. auth status --json omitted three operationally-meaningful AuthUser
fields (username, is_active, can_access_all_tenants). Agents
branching on can_access_all_tenants previously needed a second
round-trip.
4. session view Long help claimed the SDK doesn't wrap session_messages;
it does (LoadMessages / GetMessagesBefore / GetRecentMessages all
exist in client/message.go). Rewrites the comment to be accurate.
New subtree (chunk list / view / delete) exposes RAG retrieval
debugging primitives with SDK-grounded field set (23 Chunk fields).
Pagination follows v0.4 canon: --limit / --page-size (1..1000) /
--all-pages.
- chunk list --doc <id>: enumerate by ChunkIndex (separate from
search chunks which is hybrid retrieval; Long help documents the
distinction)
- chunk view <id>: scope-less render via /chunks/by-id route; full
content verbatim
- chunk delete <id> --doc <id>: scope-flag + scope-id; L-13
destructive; 404 NOT idempotent; resource.not_found /
auth.forbidden / input.confirmation_required typed exit codes
documented in Long help
MCP server gains chunk_list as 10th curated tool. Schema deliberately
exposes only doc_id + limit (no pagination workflow on MCP); response
includes truncated_at_limit flag when total > limit.
cli/AGENTS.md MCP curation rationale rewritten: curated read-only is
a deliberate product call because the server side does not yet
enforce per-token scope. When server scope ships, mutation tools can
land in the MCP surface.
Shared helper cli/internal/text/timeago_string.go (FuzzyAgoStr)
extracted from session list during the C2 quality-review pass.
The knowledge base file list infinite scroll had a race condition:
- page++ executed before the condition check, causing page numbers
to be skipped when scroll events fired rapidly
- No loading guard meant concurrent API requests could overlap,
with later responses overwriting earlier ones
This caused documents to not load when scrolling to the bottom
of the file list, especially noticeable with 100+ files where
only the first page would reliably appear.
Changes:
- Add scrollLoading flag to prevent concurrent scroll-triggered fetches
- Move page++ inside the condition check (was unconditional before)
- Use resetPage() helper to consistently reset both page and loading state
- Change scroll trigger threshold from >= scrollHeight to >= scrollHeight - 10
for better trigger sensitivity
`gofmt -w cli/` — trailing newlines, minor whitespace alignment. Caught
during PR-readiness review (`gofmt -l .` had been non-clean since the
em-dash → ASCII bulk replace earlier in the v0.4 series).
Survey of 10 mainstream CLIs (gh, lark, stripe, vercel, supabase, aws,
azure, gcloud, openai/codex, github-copilot-cli) showed env-gated
per-command --help blurbs are a Stripe-only pattern; gh uses env detect
for telemetry only, and lark relies on installed agent Skills + MCP.
Our cmd/mcp/serve already covers the dominant 2025/26 path, so
internal/aiclient/ (136 LOC + 38 callsites) is net maintenance burden
without precedent.
- Drop internal/aiclient/ entirely (annotations + detect + tests)
- Remove 38 SetAgentHelp callsites + agentAwareHelpFunc / SetHelpFunc
wiring in cmd/root.go
- Migrate 4 command-level rules to standard Long help (visible to all,
not env-gated): doc upload mode mutex, kb edit at-least-one,
kb pin idempotent, search chunks channel mutex
- Rewrite AGENTS.md as a developer guide (gh-style 6 H2 / 167 lines):
audience preamble + Build / Architecture / Command Structure /
Testing / Code Style / Error Handling. Drops sections absent in
surveyed projects (Commit & PR Conventions, Who Uses This CLI)
- Clean 14 internal doc refs (ADR-N, spec §X, v0.X) in source comments
and docs that pointed at docs/superpowers/ — that directory is
local-only / uncommitted, so refs are dead for outside readers
- Drop forward-looking "once v0.2 ships" from README
Compared the auth subtree (login/logout/list/status/refresh/token)
against gh CLI's auth implementation. Three gaps closed:
1. `auth login --with-token` validates the API key against `/auth/me`
before persisting (mirrors gh's pre-persist GetCurrentLogin probe).
A typo'd / expired / wrong-host key fails fast with
`auth.bad_credential` (exit 3) and nothing is written to the
keyring. Side benefit: api-key contexts now carry the resolved
`user` + `tenant_id` at rest, so `auth list` reflects who owns
the key — previously these columns were blank for `--with-token`
contexts because we never queried the server.
2. `auth login` prints a stderr advisory when the secrets store falls
back to the 0600 plaintext file (keychain unavailable — typical on
headless CI, WSL without DBus, agent containers). `weknora doctor`
carried the same info in its credential_storage check, but users
who go straight to `auth login` could miss it. gh has the same
silent-fallback gap; we're stricter here.
3. AGENTS.md adds an "Auth security contract" section documenting:
- Credential storage (keychain primary, 0600 file fallback)
- `--with-token` reads stdin (not flag value), pre-validated
- No env-var token bypass — by design, to avoid the
`/proc/<pid>/environ` / `ps -E` leak surface that
`GH_TOKEN`-style env vars expose
- `auth status` / `auth list` never emit token values
- `auth refresh --json` returns only `{context}` (never the
new tokens)
- `auth token` stdout has no trailing newline + TTY stderr hint
- `auth logout` is local-only (no server-side revocation)
Verified against gh CLI behavior (cli.github.com manual + cli/cli
trunk source):
| dimension | gh | weknora v0.4 |
|---------------------------------|----------------|--------------|
| pre-persist token validation | ✓ | ✓ (new) |
| OS keychain primary | go-keyring | go-keyring |
| stderr warning on file fallback | ✗ silent | ✓ (new) |
| `auth status` default token | masked prefix | not shown |
| `auth token` TTY warning | ✗ | ✓ |
| env-var token bypass | ✓ (GH_TOKEN) | ✗ by design |
| process-args / `ps` leak surface| ✗ stdin only | ✗ stdin only |
Removes the entire envelope machinery now that every success path
emits bare JSON:
- cli/internal/format/envelope.go (Envelope, Success, Failure,
SuccessWithRisk, WriteEnvelope, Meta, Notice, UpdateNotice,
VersionSkewNotice, Risk, RiskLevel, ErrorBody) + tests.
- cli/internal/format/filter.go envelope-specific helpers
(WriteEnvelopeFiltered, marshalEnvelope, applyFieldFilter,
filterDataPayload, filterObjectData); the reusable
filterArrayItems / filterObjectKeys / writeJQ stay for bare.go.
- cli/internal/cmdutil/exporter.go + tests (envelope-only).
- cli/internal/cmdutil/PrintErrorEnvelope + ToErrorBody +
operationRiskOf + Error.OperationRisk field + OperationRisk struct.
Error path: all errors now go to stderr via cmdutil.PrintError in
`code: message\nhint: ...` form, regardless of --json. Stdout stays
empty (or holds the partial-success the command already wrote) so
downstream `--json | jq` pipelines never have to filter error shapes
out of the success stream. Typed exit codes (3 auth.* / 4
resource.not_found / 5 input.* / 6 server.rate_limited / 7 server.*
+ network.* / 10 input.confirmation_required) carry the failure
class for agents that branch on it.
Acceptance contract:
- envelope_test.go → wire_test.go (TestEnvelopeGolden → TestWireGolden).
- testdata/envelopes/ → testdata/wire/.
- Error-path cases assert the typed code substring on stderr.
- Orphan whoami.*.json goldens deleted.
AGENTS.md + README.md rewritten for the bare-data contract:
- Drop envelope schema section + dry-run rule.
- Document bare JSON on stdout + `code: msg\nhint: …` on stderr.
- ADR-3 reframed around bare data and why error separation matters
for `--json | jq` pipelines.
WriteJSONFiltered short-circuits to WriteJSON when both filters are
empty (skip the marshal-buffer round-trip for the common case).
Final review pass:
- Fix wire-contract bug: `--json id,name` (space form) is broken by
pflag's NoOptDefVal; AGENTS.md / README.md / SetAgentHelp + the
field-discovery help text all switched to `--json=id,name`.
- Fix `weknora api --jq` silently ignored: api.go now routes through
WriteJSONFiltered with jopts.JQ.
- AGENTS.md: drop the false claim that `auth logout` honors `-y`
(logout is local-only with no ConfirmDestructive guard); list the
actual destructive commands instead.
- Rewrite cli/acceptance/e2e/e2e_test.go for the bare-data wire shape
(was still parsing `out["data"]` / `env["ok"]`).
- Add `JSONOptions.Emit(w, v)` helper; collapse ~33 repeated
`format.WriteJSONFiltered(iostreams.IO.Out, X, jopts.Fields,
jopts.JQ)` sites to `jopts.Emit(iostreams.IO.Out, X)` — drops the
format import from 22 cmd/* files.
- Delete single-caller `cmdutil.MustRequireFlag`; inline as
`_ = cmd.MarkFlagRequired(...)` everywhere.
- Add `_ = cmd.MarkFlagRequired("name")` to `kb create`; it was the
only write command relying on runtime --name validation while
`context add` already used the cobra-level mark.
- `context use`: register `--json` / `--jq` (was always emitting JSON
unconditionally with no human path and no flag — diverged from
every other write command); human mode now prints
`✓ Switched context to X (was Y)`.
- Replace per-package `confirmPrompter` / `scriptedConfirm` /
`errPrompter` test doubles with `testutil.ConfirmPrompter`.
- Rename `chatService` → `ChatService` (export to match siblings
`ListService` / `ViewService`); rename `printUploadSuccess` →
`renderUploadSuccess` (siblings use `render*`).
- `defaultHint(CodeResourceNotFound)`: drop the hardcoded
"list available with `weknora kb list`" — misleading on agent /
doc / session 404. Replaced with "verify the resource ID and try
again".
- Strip stale `v0.2/v0.3` / "envelope" / "v0.0/v0.1 supports only"
historical tags from production comments and a few test
descriptions.
Two intertwined mainstream-alignment moves bundled because they share
the migration target (every command's --json path):
1. Drop --dry-run entirely. Survey of comparable API-wrapper CLIs
(gh, aws, stripe, lark): none expose --dry-run. The mainstream that
does (kubectl/git/helm/ansible) operates on declarative manifests
or local state where the preview is materially different from the
executed action. WeKnora's CLI just echoed the same parameters
that would have gone on the wire — the preview added no real
signal over `--help` + reading the call site. Removes:
- root --dry-run persistent flag + cmdutil/dryrun.go
- DryRun fields + EmitDryRun calls in 12 write commands
- format.Envelope.DryRun field
- 8 corresponding *_test.go cases
- --dry-run mention from README.md and CHANGELOG.md
- "dry_run":false from 16 golden envelopes
2. Migrate every --json output to bare data:
- New format.WriteJSON / WriteJSONFiltered helpers
(cli/internal/format/bare.go) share filterArrayItems /
filterObjectKeys / writeJQ with the (still-live for now) envelope
filter helpers.
- Read commands (kb/doc/session list+view, search chunks/docs/
sessions/kb, auth list/status, agent list/view, context list,
doctor) emit bare arrays / objects on stdout.
- Write commands (kb create/edit/delete/pin/empty, doc upload/
upload_recursive/delete, session delete, auth login/logout/
refresh/token, link/unlink, context add/use/remove, agent
invoke, chat, api, version) emit bare result objects. Risk
classification dropped — the resource + exit code already
convey the action.
Per-command shape changes:
list / search → []T (was {ok, data:{items:[…]}})
view → T (was {ok, data:T, _meta:…})
create / edit → T
delete / pin / etc. → {id, …action result…}
doctor → {summary, checks}
api → {status, headers, body}
_meta dropped on the read path:
pagination (page/page_size/total/has_more) — agents iterate with
--all-pages or accept --limit (gh CLI parity);
kb_id / context echo — caller already knows what it asked for.
Acceptance contract goldens regenerated for the new bare shape.
Error envelope on stdout (PrintErrorEnvelope) stays live for now —
the envelope-infra deletion lands in the next commit.
Cross-cutting cleanup that lands alongside the new feature surface:
- `--limit / -L` and `--all-pages` on every list command. Default
--limit 30 (gh-parity); --all-pages drains every server page
client-side, capped by --limit. Closes the audit finding that the
old "1000 max per call" implicit cap was undiscoverable.
- `auth token` emits a TTY-only stderr advisory when stdout is a
terminal (the credential just got displayed in scrollback) plus an
api-key-mode rotation hint.
- Comment + doc discipline pass: drop external project name
references from in-code comments (we reference them in design notes,
not inline).
- Bump `go` directive to 1.26.0 and CI matrix to 1.26.x to align with
the main module's go.mod.
- Rename `cli/internal/agent` → `cli/internal/aiclient` to
disambiguate from the new `cli/cmd/agent` resource subtree. The
package handles AI coding-agent env detection + per-command --help
annotations; the new name reflects that more precisely.
`weknora mcp serve` — long-lived stdio MCP (Model Context Protocol)
transport that exposes a fixed, curated tool surface to MCP-aware
agents (Claude Desktop, Claude Code, custom MCP clients).
Curated tool set (readonly by default):
- whoami — active context + tenant
- search (hybrid retrieval against a KB)
- kb list / view
- doc list / view
- agent list / view / invoke
- session list / view
The list is intentionally narrow to the read + agent-invoke surface;
destructive verbs (`delete` / `empty` / `upload`) are gated behind
`--write`. Schema is built from each leaf cobra command's flags so
adding a new tool is a single registry entry plus a Service interface.
Includes the simplify post-review polish + a second simplify pass to
fold the resulting feedback (typed schemas, agent_help wording, unify
chat / agent invoke option names).
Manages WeKnora's first-class Custom Agent resources — server-side
records (system prompt + model + allowed tools + KB scope) that the
user authored in the web UI.
Commands:
- `weknora agent list` — tenant-visible agents (built-in + custom),
sorted updated_at desc; `--limit`/`-L` caps the slice client-side.
- `weknora agent view <id>` — full sdk.Agent including nested
AgentConfig (mode / model / allowed_tools / KB scope). Human mode
prints a compact KV layout + Config: block.
- `weknora agent invoke <agent-id> "<text>"` — streams the agent's
configured workflow against a query over SSE. Auto-creates a fresh
session unless `--session` is passed. Streaming defaults to TTY +
no-stream/no-json; agent-friendly buffered single-object output
with `--json` (or `--no-stream`).
Decoupled from the existing `chat` subtree: agents bring their own
system prompt / tool surface / KB selection, so the chat / agent split
matches the server-side resource boundary.
Adds the filter flags users were reaching for via `--jq` post-filter:
- `kb list --pinned` — client-side filter to KBs with `IsPinned`.
- `doc list --status <pending|processing|completed|failed>` —
server-side query-param filter; `failed` surfaces ingestion errors
immediately for triage.
- `session list --since <duration>` — client-side filter to sessions
updated within the past duration. Accepts time.ParseDuration forms
(24h, 1h30m, 30m) plus a `<N>d` suffix for whole days (7d, 0.5d).
Server-side filters are forwarded as query params (where the API
supports them) to avoid pulling the full list into memory; client-side
filters apply after the fetch so they compose with `--limit`.
Output ergonomics:
- `--json` accepts a comma-separated field list (gh-parity); selects
named keys from the per-command payload. Bare `--json` keeps the
full shape.
- `--jq <expr>` evaluates a gojq expression over the JSON; pairs with
`--json field-list` so projection runs before jq.
- `--version` is a global cobra flag in addition to the `version`
subcommand; both render the same line.
- Per-command `--help` now renders the available JSON field list under
"JSON fields available via `--json id,name,...`" (field-discovery
parity with gh / kubectl `-o jsonpath`).
New commands:
- `auth token` — print the active context's credential to stdout for
shell command substitution (`WEKNORA_TOKEN=$(weknora auth token)`).
Default: raw secret, no trailing newline. `--json` emits
`{token, mode, context}`.
- `doc upload --from-url <URL>` — ingest a remote URL via the SDK
`CreateKnowledgeFromURL`. `--name` forwarded as `FileName` so the
server's known-extension heuristic upgrades crawl-mode to
file-download-mode where appropriate.
Includes the simplify post-review polish pass (field-filter unit
tests, --json/--jq compose check, agent_help copy fixes).
Five handler helpers (validateAndGetKnowledgeBase in knowledgebase.go,
validateKnowledgeBaseAccessWithKBID in knowledge.go, the kbService
guards in faq.go and tag.go, and getKnowledgeBaseForInitialization +
the per-kb config getter in initialization.go) wrapped every
GetKnowledgeBaseByID error — including the well-known
repository.ErrKnowledgeBaseNotFound sentinel — as
NewInternalServerError. The result was that every probe of a stale or
cross-tenant kb id surfaced as a 500 instead of the 404 it should have
been, both confusing clients ("real 5xx vs. wrong URL") and burning
ops attention on monitoring alerts.
The mapping pattern is the same as PR #1336 for sessions: detect the
sentinel via stderrors.Is and emit NewNotFoundError; everything else
still surfaces as a 500 so genuine DB / repo failures keep firing the
alerts that matter. Caught during the RBAC e2e smoke run on
feat/rbac, where a deliberate cross-tenant kb-id probe produced
HTTP 500 + body "knowledge base not found" — the smoking gun.
Tests: new internal/handler/knowledgebase_not_found_test.go covers
three cases — bare sentinel, fmt.Errorf("%w") wrapped sentinel
(regression guard against a future revert to `==`), and a non-sentinel
infrastructure error that must still 500. All three pass.
The full handler test package is green.
Migration 000002 creates the pg_trgm extension only inside the conditional
embeddings block guarded by app.skip_embedding. Deployments that use a
non-postgres retrieve driver (Qdrant, Elasticsearch, Milvus, etc.) skip
that block entirely, so pg_trgm is never installed. When migration 000041
then runs CREATE INDEX ... USING GIN (lower(title) gin_trgm_ops) the
statement fails and the whole migration is rolled back, leaving
task_pending_ops and task_dead_letters uncreated. The application keeps
starting (migrations are best-effort), but wiki ingest enqueue silently
fails and wiki pages are never produced.
Re-issue CREATE EXTENSION IF NOT EXISTS pg_trgm at the top of 000041 so
the extension is guaranteed present at the moment the trigram index is
created. The statement is idempotent on environments where 000002 already
installed it, and surfaces a clear, early error on environments where the
extension genuinely cannot be loaded.
Fixes#1319.
Co-authored-by: Cursor <cursoragent@cursor.com>
PR #1309 plumbed user-scope into the session/message service layer so
non-owner / wrong-tenant lookups now surface as ErrSessionNotFound at
every entry point that goes through sessionRepo.Get. The handler-side
mapping, however, only existed in 4 of the 8 affected sites:
- handler/session/handler.go:GetSession ✓ already mapped (with `==`)
- handler/session/handler.go:UpdateSession ✓ already mapped (with `==`)
- handler/session/handler.go:DeleteSession ✓ already mapped (with `==`)
- handler/session/handler.go:BatchDeleteSessions ✓ already mapped (with `==`)
- handler/session/handler.go:ClearSessionMessages ✗ always 500
- handler/session/stream.go:ContinueStream session lookup ✓ (with `==`)
- handler/session/stream.go:ContinueStream message lookup ✗ always 500
- handler/message.go:LoadMessages (recent + before-time) ✗ always 500
- handler/message.go:DeleteMessage ✗ always 500
Result: a Contributor in tenant A asking for a Contributor B's session
messages got a 500 instead of 404 — the wire response leaked
"something is broken" rather than "you can't see this URL", which
breaks SDK / frontend axios interceptor error handling that keys on
HTTP status.
Two changes:
1. Add ErrSessionNotFound → 404 mapping to the 4 sites that lacked it
(ClearSessionMessages, ContinueStream's GetMessage, LoadMessages's
two paths, DeleteMessage).
2. Replace `err == errors.ErrSessionNotFound` with
`stderrors.Is(err, errors.ErrSessionNotFound)` everywhere. Sentinel
error comparison must use errors.Is so wrapped errors
(`fmt.Errorf("...: %w", ErrSessionNotFound)`) still match. Today no
service-layer caller wraps, but a future refactor that does would
silently turn 404s into 500s with no test catching it.
Tests:
- internal/handler/message_session_not_found_test.go covers the four
new mapping sites in handler/message.go (LoadMessages-recent,
LoadMessages-before-time, DeleteMessage) and pins the wrapped-error
behaviour with a dedicated regression test that fmt.Errorf("%w") a
sentinel must still be detected as ErrSessionNotFound. The bare
handler is hit through gin.Engine + middleware.ErrorHandler so the
response shape matches production exactly.
- PR #1309's existing service-level tests (session_user_scope_test,
session_test on the repository) still pass — this PR is a strictly
handler-layer follow-up.
When a startup database migration fails (e.g. issue #1319: pg_trgm not
available so 000041 cannot build its trigram index), the application
intentionally keeps booting so operators can reach the UI to diagnose.
However, before this change the system info page silently dropped the
"DB Version" row because the value was empty:
- migration.go only cached the version after a successful m.Up(); the
error path returned early and left migrationVersionSet=false.
- system.go used CachedMigrationVersion's ok=false to skip emitting
db_version, and the JSON tag was already omitempty.
- SystemInfo.vue gated the entire row on v-if="systemInfo?.db_version".
The end result was the most useful diagnostic surface vanishing in the
exact failure mode that needs it most — Wiki ingest and KG features
would silently produce nothing with no UI hint.
Changes:
- migration.go: replace sync.Once-based setter with an RWMutex-guarded
state struct holding {version, dirty, err}. Every failure path now
calls captureMigrationFailure(m, err), which best-effort reads
m.Version() so the cached value still reflects the partial state.
- system.go (handler): always emit db_version (falling back to
"unknown" when no version could be read), append " (failed)" when an
error is recorded, and add db_migration_error to the response.
- swagger / client SDK: keep the API contract in sync with the new
response field.
- SystemInfo.vue: render the DB version row whenever either field is
present, show a "Migration failed" danger tag, and add a full-width
alert below the row carrying the error message plus two links:
1. View troubleshooting guide -> new docs/migration-troubleshooting.md
2. Report an issue -> github.com/Tencent/WeKnora/issues/new,
prefilled with the captured error and environment metadata.
- docs/migration-troubleshooting.md: new self-service guide covering
the common failure modes (missing extension, dirty state, privileges,
out of disk, schema drift) with concrete psql / make commands.
- i18n: add the new keys to zh-CN, en-US, ko-KR, ru-RU.
Refs #1319.
Co-authored-by: Cursor <cursoragent@cursor.com>
Extracts the 25 repeated NewCompositeRetrieveEngine call sites across
seven services into two factory functions with tenant-ownership
verification: CreateRetrieveEngineForKB (synchronous) and
CreateRetrieveEngineFromPayload (async task handlers). Promotes
GetByStoreID from concrete-only to interfaces.RetrieveEngineRegistry.
Extends KBDeletePayload / IndexDeletePayload with an omitempty
VectorStoreID snapshot.
Design highlights
- Factory verifies tenant ownership of the resolved store
(defense-in-depth) so a gap in PR3 validation or a tampered Asynq
payload cannot cross tenants. Cross-tenant attempts return
ErrVectorStoreForbidden with a structured log entry.
- Sentinel errors (ErrTenantInfoMissing, ErrVectorStoreNotFound,
ErrVectorStoreForbidden) let async handlers classify non-retryable
failures as asynq.SkipRetry.
- nil and empty-string pointers for vectorStoreID normalize to
"unbound" so callers never send an empty UUID to GetByStoreID.
- resolveBoundEngine constructs engineInfos directly (with
slices.Clone of Support()) instead of going through
NewCompositeRetrieveEngine. The latter only reads from the
byEngineType map and cannot reach DB stores in byStoreID -- multiple
stores can share the same engine type. Unbound fallbacks still reuse
NewCompositeRetrieveEngine so the tenant effective-engines path is
byte-for-byte unchanged.
- DeleteKnowledgeBase now reads the KB before soft-delete so the
enqueued KBDeletePayload carries a VectorStoreID snapshot; once the
row is soft-deleted GORM's default scope hides it. Cost: +1 SELECT
per KB-delete request (a rare admin operation).
- enqueueIndexDeleteTask gains a vectorStoreID *string parameter that
the single caller (tag.go) populates from the owning KB. The worker
(ProcessIndexDelete) validates ownership via the factory.
- cleanupKnowledgeResources loads the KB so bound-store cleanup routes
correctly. If the load fails it warns and falls back to tenant
effective engines; orphan vectors become observable in logs instead
of disappearing silently.
- DeleteKnowledges (batch) stays on the tenant effective-engines path
-- batches can span stores and multi-store fan-out is PR4 scope.
Noted inline.
Merge safety
All knowledge bases have VectorStoreID = NULL at merge time (introduced
nullable in PR1, Tencent/WeKnora#994). Every factory call falls back to
NewCompositeRetrieveEngine(registry, tenantInfo.GetEffectiveEngines()),
which is the existing code path for these KBs. Async payloads from
before this change decode to VectorStoreID = nil (omitempty) and take
the same fallback.
Testing
- go build ./...
- go vet ./...
- go test -race -count=1 ./internal/application/service/retriever/...
covers unbound (nil/empty), bound, cross-tenant, unregistered store,
ownership infra error, legacy payload shapes (missing field/null/
empty string/nil), tampered payload, and parallel invocation.
- grep -r NewCompositeRetrieveEngine internal/application/service
returns zero results outside factory.go/composite.go.
Refs: Part of Tencent/WeKnora#993. Depends on Tencent/WeKnora#994.
Final design-pass audit on the v0.3 surface flagged two real gaps.
(A) doc view <id> was missing. Every other resource subtree exposes
a view verb (kb view, session view) for inspecting a single record,
but doc — which has the richest metadata of the three (title, file
name, type, size, parse_status, embedding_model, processed_at,
error_message) — had no single-doc surface. Users wanting one
doc's metadata had to `doc list | grep`.
Implementation mirrors kb view: narrow ViewService(GetKnowledge)
interface, --json envelope path, human KEY: VALUE renderer. Optional
fields are omitted rather than rendered as "-" so the panel is
dense. Tested: human renderer, title fallback when FileName empty,
omit-empty contract, JSON envelope shape, 404 classification.
(B) link had no counterpart. Once .weknora/project.yaml is written,
the only way to clear it was `rm` by hand. Both vercel and netlify
ship `unlink` as a top-level verb; not having one was a
discoverability gap. Top-level rather than `link --clear` follows
the verb-noun convention of the rest of the surface — the verb
stands alone and the operation isn't parameterised.
unlink walks up from cwd via projectlink.Discover (the same
parent-chain logic Factory.ResolveKB uses on the read side), so a
user in a subdirectory of a linked project can unlink without
cd-ing up. Errors with input.invalid_argument when no link is
found anywhere in the chain. Idempotent under racy concurrent
removal: os.ErrNotExist on os.Remove falls through to a Success
envelope since the post-condition holds either way.
projectlink package gained Remove() alongside Save / Load /
Discover so unlink doesn't reimplement the idempotent-remove
pattern inline.
Top-level registration in cmd/root.go, alongside link.
cli/AGENTS.md verb canon line adds unlink to the locally-introduced
list. cli/CHANGELOG.md gains an Added entry for each.
5 unit tests for view + 4 for unlink (cwd / walk-up / no-link
error / JSON envelope). Full suite green.
Intentionally deferred:
- session edit (rename a session): sessions auto-name from the
first prompt; polish rather than a gap.
- link --clear as an alternative to unlink: top-level unlink is
the documented form; aliases would just multiply the surface.
Seven bugs surfaced via two audit rounds — parallel reviewer agents
plus a real-server end-to-end demo. Each fix arrives with a
regression test.
1. doc upload --recursive --json corrupted the envelope stream.
Per-file FAIL/OK plain lines printed unconditionally to stdout,
then a Success envelope, then on partial failure a typed error
that the root handler turned into a SECOND Failure envelope —
three outputs where one was expected. Fix: gate the plain lines
behind !opts.JSONOut, and add cmdutil.Error.Silent so the JSON-
path partial-failure preserves its typed exit code without
triggering PrintErrorEnvelope's default Failure-envelope write.
2. auth refresh / AuthRetryTransport misclassified HTTP failures as
network.error. RefreshAndPersist wrapped every refresher error
with CodeNetworkError, but the SDK emits "HTTP error 401: ..."
for a rejected refresh token — which should surface as
auth.token_expired. Switched to WrapHTTP for proper status-
derived classification. Affects both `auth refresh` and the
transport's refresh closure.
3. doc download accepted ".." as a server-suggested filename. The
rejection list covered "" / "." / filepath.Separator but not
bare ".." — filepath.Base("..") is "..", which slipped through
to os.Create and produced a confusing local.file_io wrap. Added
to the rejection set.
4. search chunks / docs / kb / sessions had no lower bound on
--limit. `-L 0` / `-L -1` was forwarded to the server with
undefined behavior. Added a 1..1000 bound at the RunE boundary
across all four (matching doc list / session list page-size
bounds). Internal callers in tests can still pass Limit==0 for
the "no client-side cap" runChunks path — the bound only applies
at the user-input layer.
5. cli/AGENTS.md ADR-3 verb-canon summary listed only v0.2 verbs as
"gh-canonical" and missed v0.3 additions (edit, pin, unpin,
download — all gh-canonical) plus locally-introduced ones
(empty, refresh, add, remove, link). Rewritten as an explicit
gh-canonical / locally-introduced split.
6. kb pin returned 404. Server registers /knowledge-bases/{id}/pin
as PUT (router.go:292); SDK was using POST. gin's router silently
404s on method-mismatch (treats it as path-not-found, not 405),
so the CLI classified the response as resource.not_found and
masked the real failure mode. Switched the SDK to http.MethodPut.
The asymmetry that hid this past round 1: kb unpin on a freshly-
created KB hits the no-op branch in cmd/kb/pin.go that skips the
SDK call entirely, so unpin "worked" without ever exercising the
broken path. Only the real-server demo, where kb pin actually
fires, surfaced it.
7. kb edit clobbered current Name when only --description was
passed. EditOptions used *string to distinguish "unset" from
"set to empty", but sdk.UpdateKnowledgeBaseRequest declares both
fields as plain string (no omitempty), so the JSON body always
carried `"name": ""`. Server requires Name → 400. Fix: runEdit
does fetch-then-update — GetKnowledgeBase first, build the PUT
body with current values, then overlay user-set fields. Same
TOCTOU window as kb pin / unpin.
Audit-flagged items intentionally NOT changed:
- kb pin / unpin check-then-toggle TOCTOU: documented; the clean
fix would be a server-side setter and belongs in a separate API
change.
- AuthRetryTransport singleflight test gap for one concurrency
scenario; v0.4 polish.
- cli/README.md:50 "once v0.2 ships" and CHANGELOG.md:8
"10 top-level commands": v0.2-PR artifacts, not v0.3-introduced.
- kb edit / kb pin are v0.3-new commands, so neither bug needs a
cli/CHANGELOG.md Fixed entry — the v0.3 release ships them
working as the Added bullets advertise.
The `link` subcommand declared a local `--context` StringVar that
shadowed the root-level persistent `--context` flag at the cobra layer.
Two different semantics under one name:
- root global `--context <name>`: "override the active context for
THIS invocation only, no disk write" (single-shot connection
override, applied via Factory.ContextOverride).
- link local `--context <name>`: "the context name to record in
.weknora/project.yaml" (persisted state, written to disk).
The shadow meant `weknora --context staging link` (intent: link runs
against staging) silently did NOT propagate the override into link's
runtime; instead link's local "" beat the global. `weknora link
--context staging` (intent: record staging in the file) did work, but
shared a name with the unrelated global behavior, which is a usability
trap.
Resolution: drop the local flag entirely. The active context at link
time is what gets recorded; users who want to bind under a different
context use the global `--context X link --kb my-kb` form, which now
propagates correctly (no local shadow). This matches the bind-command
patterns surveyed across mainstream CLIs:
- lark-cli `config bind` — uses domain-specific flags (--source /
--app-id / --identity); the global --profile is named distinctly.
- gh `repo set-default` — uses a positional for the bind target;
the global -R/--repo is the only flag-form path.
- netlify `link` — uses --id/--name for the bind target; no
--site global flag at all (env var only).
- vercel `link` — reuses --project for both global and link, but
only works because vercel ships a custom parser that merges
flag/env/file precedence; cobra's persistent-flag shadowing is
silent-override, not graceful merge.
The lark-cli / gh / netlify pattern of "bind command's target flag
must not share a name with the global override flag" is the cobra-
friendly choice; dropping the flag is the simplest form of that.
No behavior change for the common path (`weknora link --kb my-kb`
without --context still records the active context). The "record
under a specific non-current context" use case is now expressed via
the global flag, which is what it was designed for.
link_test.go untouched (no test referenced the dropped flag).
v0.3 feature commits didn't update the docs alongside; this commit
syncs them and introduces a CLI-local changelog so v0.3+ release
notes stop crowding the project root file.
cli/CHANGELOG.md (new):
- Subsystem-local pattern, mirroring mcp-server/CHANGELOG.md. CLI
versions independently from server / frontend cadence; reduces
merge-conflict surface on the shared root file.
- Scope: Added + SDK additions only. v0.3-internal dev churn
(--top-k → --limit, kb clear-contents → kb empty, link --context
introduce-then-drop, internal Go type-name leaks) never reached a
shipped release so it doesn't belong in Changed / Fixed sections.
mcp-server's v1.0.0 changelog is Added-only for the same reason.
- v0.0–v0.2 history stays in the project root CHANGELOG.md;
cross-referenced from the top of cli/CHANGELOG.md.
Stale --help / quickstart examples fixed in cli/cmd/root.go,
cli/README.md, and cli/AGENTS.md — all three showed the dropped bare
`weknora search "<q>" --kb=...` form; updated to `search chunks ...`.
AGENTS.md updates:
- Verb canon table gained edit / empty / download / pin / unpin /
add / remove.
- `auth` subtree description gained `refresh` and the transparent
401-retry transport (replacing the now-inverted "deferred to v0.3"
sentence).
- `search` and `session` subtree paragraphs added; top-level
verb list gained `context` and `session`.
cli/README.md top-level command list gained `session`; `search`
short retitled to the parent description ("Search across chunks,
knowledge bases, documents, or sessions") since search is now a
pure dispatcher.
Pre-existing stale e2e refs swept up while syncing:
- cli/acceptance/doc.go listed e2e/ under "Future v0.2+:" — moved
into the present-tense Sub-packages block.
- envelope_test.go preamble "Deferred to v0.2 e2e" rephrased to
"Deferred to the e2e harness" so it isn't pinned to a past version.
Not changed (out of scope, flagged for future PRs):
- envelope_test.go "Implemented count: 16" vs the actual 14 named
entries — could be a different counting rule; verify with PR-8
author before editing.
- envelope_test.go context_use deferred-cases narrative is loose
(context_use.success IS golden-pinned today) but rewriting needs
careful re-derivation of which error scenarios are still deferred.
- cli/README.md:50 "once v0.2 ships" — v0.2-PR-original wording;
not load-bearing once a release tag exists.
No project-root CHANGELOG.md change in this commit.
Cross-cutting findings surfaced by the branch-completion review.
Perf bug:
- Factory.Client closure was not memoized. Factory.ResolveKB internally
calls f.Client() to resolve --kb name → id, then the command's RunE
calls f.Client() again. Two SDK clients, two keyring reads, two
AuthRetryTransport allocations per name-resolved invocation, with
*independent* token state (a refresh in one was invisible to the
other). Switched to sync.Once like Secrets already does.
Silent bug bait:
- cmdutil.NormalizeHost docstring claimed CodeInputMissingFlag for the
empty case; code returned CodeInputInvalidArgument. Aligned doc to
code (present-but-empty is a bad value, not a missing flag).
Agent contract gaps:
- Five user-facing subcommands lacked SetAgentHelp: auth login /
logout / list / status and chat. Added concise strings with error-
code call-outs so agents can branch without parsing human strings.
Helper extraction (≥3 callers):
- text.KnowledgeDisplayName(fileName, title, id) — byte-identical
formatter that was in both cmd/doc/list.go and cmd/search/docs.go.
Takes fields directly so internal/text stays SDK-free.
- cmdutil.WrapHTTP(cause, fmt, args...) *Error — replaces the
`Wrapf(ClassifyHTTPError(err), err, ...)` pattern across 24 SDK
call sites. Sed-driven migration; off-pattern shapes in chat.go
(used streamErr) and cmdutil/kb.go (in-package) hand-edited.
Contract test gains a comment update: post-migration the dominant
pattern is WrapHTTP which the AST scanner skips entirely (only
NewError/Wrapf selectors inspected); ClassifyHTTPErrorOutputs()
bridge still covers the dynamic codes those paths can yield.
UX consistency:
- cmd/doc/list.go --page-size help now reads "Items per page
(1..1000)" matching cmd/session/list.go. The bounds validation
already enforced 1..1000; the help text was the last drift.
Comment-discipline sweep:
- Deleted the WHAT-only "*Options captures `weknora ...` flag state"
docstring across 23 files (context, kb, auth, doc, session, search,
chat, doctor, link). Where the line carried a real WHY clause
(kb/delete, doc/delete, session/delete, kb/edit), kept the WHY and
dropped only the leading WHAT phrase.
- Stripped third-party project-name attribution from inline comments
and one user-visible flag-help string across ~40 files in cli/cmd
and cli/internal (plus 4 test-file comments). Removed phrases like
"Mirrors `gh X`", "borrowed from lark-cli", "kubectl-style",
"gcloud `--project`", "Stripe pattern", and the embedded GitHub
URLs pointing at those projects. Behavioral descriptions and the
WHY behind each comment are preserved; only the upstream-name
attribution is gone. Inspiration / north-star references belong in
cli/AGENTS.md (the design doc) and commit messages, not scattered
through every file.
Triggered by an audit round that surfaced several false / fragile
parity claims (e.g. "Mirrors `gh repo edit`" — gh repo edit has no
--name flag; "matches gcloud `--project` id-or-name" — gcloud's
--project accepts ID only). Rather than fix them one by one, the
whole category of in-comment external-project references was
stripped uniformly.
api (3-11):
- `--input <file>` reads the request body from disk; `--input -` reads
from stdin. Matches gh CLI canonical naming verified against the gh
manual ("The file to use as body for the HTTP request — use \"-\" to
read from standard input"). `--data` / `--input` are mutually
exclusive.
- Options.StdinReader (defaults to iostreams.IO.In) for test injection.
completion (3-13 smoke only — release-artifact ship deferred to release
milestone):
- Smoke test asserts cobra's auto-registered bash/zsh/fish/powershell
scripts produce non-trivially-sized output with the per-shell
signature (#compdef / complete -c weknora / etc.). Guards against
cobra bumps silently breaking completion for one shell.
3-14 doctor --no-cache: already implemented (factory.go:297) with
TestDoctor_NoCache_BypassesCache covering it — verified, no change
needed.
Roadmap: 3-11, 3-13 (smoke), 3-14 (verified).
Roadmap 3-1. Verb-noun shape borrowed from gh search (gh search repos
/ code / commits / issues / prs verified against the gh manual).
Subcommands:
- `search chunks "<q>" --kb X` — hybrid retrieval (RAG search).
- `search kb "<q>"` — case-insensitive substring match across KB names
and descriptions; sorted by name length (shortest hits first).
- `search docs "<q>" --kb X` — pages through ListKnowledge filtering by
title / file_name; stops once --limit matches are found.
- `search sessions "<q>"` — pages through GetSessionsByTenant filtering
by title / description.
kb / docs / sessions are client-side filters because the server has no
fuzzy search endpoint for any of them. ListKnowledgeBases returns the
full tenant catalog in one call; the doc/session walkers chunk at 200
per request and stop early on limit.
The parent `search` command is a pure dispatcher — there is no bare-
positional form (no `weknora search "<q>"`).
Cleanups surfaced by the post-commit reviewer round:
- UX consistency: search docs's displayDocName ordered Title →
FileName → "-", while doc list's displayName uses FileName → Title
→ ID. Same Knowledge rendered differently across commands. Aligned
search docs on doc list's existing FileName-first convention.
- cmdutil.ResolveKBFlag(ctx, lister, raw) — extracted the
`IsKBID ? raw : ResolveKBNameToID` block duplicated across chunks
and docs.
- text.ContainsFold(needle, fields...) — replaces inline
`strings.Contains(strings.ToLower(field), needle)` patterns.
37 unit tests across chunks/kb/docs/sessions plus the parent
registration smoke-test.
Roadmap: 3-1.
Roadmap items 3-9 (download) and 3-10 (recursive upload).
SDK addition (additive, non-breaking):
- OpenKnowledgeFile(ctx, id) (filename, body io.ReadCloser, err) —
the new primitive that returns the body as a stream plus the
server-suggested Content-Disposition filename. The existing path-
form DownloadKnowledgeFile is now a thin wrapper (also gained
partial-file-on-error cleanup, a pre-existing bug exposed by the
reshape).
doc download <id>:
Borrows shape from `gh release download` (positional id, output flag,
`-` sentinel for stdout). Flag names match gh canon verified against
the gh manual: `-O, --output <file>` for destination; `--clobber` for
overwrite control.
- Default: writes to cwd under the server-suggested filename. If the
server didn't send one, errors with input.missing_flag.
- --output FILE / -O FILE: writes to FILE. Refuses overwrite without
--clobber.
- --output -: stream to stdout (binary-safe).
- Partial writes on error are cleaned up.
doc upload --recursive <dir> --glob '*.pdf':
NOTE on upstream parity: `gh release upload` does NOT support
--recursive (verified — it takes individual file args only). `aws s3
cp --recursive` does, but uses `--include`/`--exclude` glob pattern
pairs rather than a single `--glob`. weknora's single positive `--glob`
is a deliberate simplification, not a direct mirror of either tool.
- Walks the tree, filters by base-name glob, uploads each match
sequentially. Per-file line output: OK / FAIL with the underlying
error. Exit 0 only on full success; on partial failure returns the
first failure's typed code so callers can branch. Rejects --name
with --recursive.
- --dry-run lists matches without uploading.
- --json emits {kb_id, uploaded[], failed[]} envelope at completion.
Bugs caught in the post-commit reviewer round:
- SECURITY: server-supplied filename was used in os.Rename without
sanitization. A malicious / buggy server returning
"../../etc/shadow" could escape cwd. Now filepath.Base'd; "." / "/"
/ "" rejected. Regression test added.
- Wasted-bytes path eliminated via the SDK reshape: the CLI now
inspects filename and applies refuseIfExists BEFORE streaming.
Two-phase temp+rename gone.
- refuseIfExists(path, clobber) helper extracted.
- --json honored in --recursive (uploadOutcome was JSON-tagged but
the envelope was never emitted).
7 + 7 unit tests for download (+ path-traversal regression) and
recursive upload (+ JSON envelope regression).
Roadmap: 3-9, 3-10.
Roadmap items 3-5 (session) and 3-6/7/8 (kb manage).
cli/cmd/session/ (new package; sessioncmd to avoid shadowing stdlib):
- session list: paginated table (ID/TITLE/UPDATED). --page / --page-size
with 1..1000 validation. _meta.has_more from page*size < total.
- session view <id>: prints metadata; non-empty fields only. Server
timestamps arrive as strings; parsed best-effort as RFC3339.
- session delete <id>: high-risk-write; exit-10 confirmation in non-
TTY/--json paths; --dry-run emits envelope.risk + dry_run:true.
cli/cmd/kb (extended):
- kb edit <id> [--name N] [--description D]: at least one flag required;
*string options so unset fields stay unset in the PUT body. SDK
UpdateKnowledgeBaseRequest has no embedding_model field, so the
roadmap's --embedding-model dropped.
- kb pin <id> / kb unpin <id>: direct parity with gh issue pin /
gh issue unpin (verified against gh manual). Idempotent: GetKnowledgeBase
reads IsPinned, TogglePinKnowledgeBase fires only on state change.
SDK KnowledgeBase struct gained the IsPinned field (server already
returned it; SDK just hadn't modeled it — non-breaking additive).
- kb empty <id>: high-risk-write; exit-10 confirmation;
--dry-run. Returns deleted_count from the async clear response.
weknora-specific operation; no mainstream parallel.
Golden envelopes for kb_list and kb_view updated to include the new
is_pinned field — strict-additive change.
Cleanups surfaced by the post-commit reviewer round:
- ConfirmPrompter promoted to cli/internal/testutil/ (4-copy threshold
reached: context/remove, kb/delete, kb/empty, session/delete).
kb/delete_test.go's pre-existing local copy left untouched per the
upstream-respect convention.
- kb pin/unpin idempotent no-op path no longer emits a write-class
envelope. Added _meta.warnings "already {un}pinned — no server
call made" and dropped the risk classification on the no-op branch.
- doc list --page-size was unbounded while session list enforces
1..1000. Same validation added to doc list.
18 + 18 unit tests; e2e exit codes verified.
Roadmap: 3-5, 3-6, 3-7, 3-8.
Two halves of v0.3 roadmap item 3-2.
(1) `weknora auth refresh` — explicit token renewal:
Reads the stored refresh_token, spends it via POST /api/v1/auth/refresh
(OAuth refresh-token grant), and persists both new tokens. API-key
contexts rejected with input.invalid_argument (no refresh semantic).
NOTE: gh CLI has `gh auth refresh` but with different semantics —
gh's variant is an OAuth scope expansion / re-prompt via the browser
(verified against the gh manual). The two share a name but solve
different problems; there's no direct gh parallel for refresh-token
grant because gh's PAT/OAuth-app model doesn't expose a short-lived
access_token + refresh_token pair to clients.
Error mapping:
- no current context → auth.unauthenticated
- --name unknown → local.context_not_found
- missing refresh in keyring → auth.token_expired (hint: re-login)
- server Success=false → auth.token_expired
- network → network.error
Envelope omits the token values (would leak into agent transcripts).
(2) AuthRetryTransport — transparent retry:
Wraps the SDK http.Client. On a 401 from a non-/auth/* endpoint:
- JWT context: read refresh token, hit /auth/refresh, persist new pair,
replay original request with new bearer.
- API-key context: pass through (no refresh semantic).
- Non-replayable body (req.GetBody == nil): pass through.
- /auth/login or /auth/refresh: pass through (no recursion).
Concurrent 401s are singleflight-coalesced via sync.Mutex — 5 parallel
calls trigger exactly 1 refresh.
SDK additions (additive, non-breaking):
- WithTransport(rt http.RoundTripper) ClientOption.
- PathAuthLogin / PathAuthRefresh constants (cli/internal/cmdutil/authretry
imports them so the CLI and SDK can't drift on path strings).
Refactor surfaced by the post-commit reviewer round:
- cmdutil.RefreshAndPersist(ctx, store, refresher, ctxName) — the
load-refresh → call-SDK → persist-pair sequence was duplicated between
the standalone `auth refresh` and the transport's refresh closure;
collapsed to one canonical implementation.
- refreshFn signature takes context.Context so Ctrl+C during a
transparent refresh cancels.
- AuthRetryTransport.CurrentToken() removed — never called.
8 + 8 + 8 unit tests cover happy path / refresh-fail / auth-endpoint
skip / api-key passthrough / singleflight under concurrency / non-
replayable-body fallback.
Roadmap: 3-2.
New v0.3 P0 entry 3-4: kubectl-style context-management subtree using
gh's `<noun> <verb>` surface convention consistent with the rest of
this CLI.
- context list: tabwriter rendering + --json envelope; reads config.yaml
only.
- context add <name> --host <url> [--user]: validates http(s) URL, first
context auto-becomes current, rejects duplicates with did-you-mean.
- context remove <name>: best-effort keyring cleanup like `auth logout`.
Removing the current context triggers exit-10 confirmation (lark-cli
skill protocol) — subsequent commands would lose their default
--context.
(`context use` predates v0.3; the subtree was previously use-only.)
Bugs caught and fixed inline by the post-commit reviewer round:
- auth login was accepting `http://` (empty host portion) because the
old validateHost only checked the scheme. New cmdutil.NormalizeHost
(shared by both login and context add) requires u.Host != "".
- context add's validateName claimed `..` was rejected but only denied
/ \\ space. Switched to positive allowlist [A-Za-z0-9._-] plus
explicit ./../path-separator rejection.
Helper consolidation:
- cli/internal/cmdutil/host.go: NormalizeHost (trim, scheme, host
non-empty) — both auth login and context add share it.
- cli/internal/format/dash.go: DashIfEmpty — promoted from copies in
cmd/auth/list.go and cmd/context/list.go.
- recordingStore test stub dropped in favor of secrets.NewMemStore;
contextKeyList test helper replaced by the existing contextKeys.
14 unit tests; 13 e2e branches verified.
Roadmap: 3-4.
The runtime chat pipeline (internal/application/service/chat_pipeline/rerank.go:56)
already skips rerank when rerank_model_id is empty, so it should not be
required at save time.
The editor incorrectly required rerank_model_id whenever a RAG KB was
selected, even though the field is optional. This fix removes the
validation so users can save agents without configuring a rerank model.
Fixes#1252
JoinByOrganizationID no longer delegates to JoinByInviteCode for direct
join, matching the product copy that searchable spaces need no invite
code. Extract joinAsViewerWithChecks for shared member-limit logic.
Co-authored-by: Cursor <cursoragent@cursor.com>
Update the bug report and question issue templates to direct users to
the Settings → System Info page and ask for the concrete "App Version"
and "UI Version" values shown there, explicitly disallowing "latest",
"main" or "master" as answers. This pairs with the new UI Version row
on the System Info page so version-mismatch bugs (e.g. SearXNG
base_url field missing on stale UI builds) can be triaged at a glance.
Inject the frontend package.json version at build time via Vite and
render it as a dedicated "UI Version" row in System Info, alongside
the existing app (backend) version returned by /system/info. When the
two versions differ a warning tag is shown so users can tell at a
glance that their weknora-ui image is out of sync with weknora-app
(e.g. missing config fields for newer providers like SearXNG).
Rename the existing version row label from "System Version" to
"App Version" to distinguish it from the new UI version row in all
four locales (zh-CN, en-US, ko-KR, ru-RU).
The docreader gRPC service has no authentication or TLS, but the
default `ports: ["50051:50051"]` mapping in docker-compose.yml binds
to 0.0.0.0, exposing an unauthenticated document parser (with URL
fetch capability) on every host interface.
The app container reaches docreader through the internal Docker
network via `docreader:50051` (the default `DOCREADER_ADDR`), so the
host port mapping is not required for normal operation.
Replace the `ports` entry with `expose: ["50051"]` so the port is
only reachable inside the WeKnora-network. Operators who need to
call docreader from the host (for debugging, etc.) can re-add a
`ports:` entry in a local override, preferably bound to 127.0.0.1.
Bump version to v0.5.2 across VERSION, Helm chart, frontend package
files, and cloud-image script comment. Update CHANGELOG and all four
language READMEs with v0.5.2 highlights (Wiki Mode at scale, MCP
human-in-the-loop approval, new LLM/vector/storage/search backends,
adaptive 3-tier chunking, global command palette, CLI preview, etc.).
Tool-calling rounds emit reasoning into the OpenAI-protocol
reasoning_content field rather than visible content, so AgentStep.Thought
is often empty in DB. The history reconstructor only read step.thought,
which made the historical step card silent for those rounds even though
the user saw the reasoning live.
Fall back to step.reasoning_content when step.thought is empty so
reload mirrors the live experience.
MiMo and DeepSeek V3.2/V4 reject multi-turn requests in thinking mode
when the prior assistant message lacks reasoning_content with HTTP 400:
"The reasoning_content in the thinking mode must be passed back to the API."
Agent's ReAct loop is the worst-case scenario — every round produces tool
calls, exactly the case DeepSeek's docs specify reasoning_content MUST
participate in subsequent context.
Plumb reasoning_content through the full assistant-message round-trip:
- chat.Message / types.ChatResponse / types.AgentStep gain a
reasoning_content field (AgentStep persists via the existing
Message.AgentSteps jsonb column, no migration needed).
- streamLLMToEventBus accumulates reasoning chunks into
result.ReasoningContent and surfaces it on the round's ChatResponse.
- engine.runReActIteration writes it onto AgentStep so cross-turn replay
preserves it.
- observe.appendToolResults attaches it to the same-turn assistant
message; agent_history.buildAssistantHistoryMessages does the same on
cross-turn replay.
- RemoteAPIChat.ConvertMessages forwards it on assistant turns to
openai.ChatCompletionMessage.ReasoningContent (already supported by
go-openai); providers that don't recognize the field ignore it.
Tests cover the three boundaries: ConvertMessages serializes it for
upstream, appendToolResults preserves it within the same turn, and
buildAssistantHistoryMessages replays it across turns.
Scope is intentionally limited to Agent mode — KnowledgeQA's chat
pipeline and Anthropic's signed thinking_blocks are separate fixes that
require schema changes (rendered_content / thinking_blocks columns).
Fixes#1302