WeKnora/internal/types
nullptr-error e39277a75e
feat(sandbox): add tenant-configurable network policy (#2995)
* feat(sandbox): add the tenant network policy with its secrets and validation

Administrators need control over sandbox network access, so a named sandbox
config gains a network policy block. It lives inside the existing config JSONB,
so there is no schema change and old rows read back as the zero value.

Both booleans are phrased so that the zero value IS the intended default --
outbound open, inbound closed. That makes "no network block" and "explicitly
saved defaults" provably the same thing, and avoids a tri-state pointer where
nil would have to mean "allowed" for egress and "closed" for ingress.

Injected header values are credentials, so they ride the encrypt-at-rest and
mask-on-read path EnvVars already uses instead of a parallel mechanism. One
CloneWithSecrets traversal serves encrypt, decrypt and mask so the three
directions cannot disagree about which fields are secret.

Validation runs at save time because the alternative is failing at
sandbox-create time, where the admin no longer has the form in front of them.
It rejects both what a provider would refuse and what a provider would accept
without doing what the admin meant -- a domain allowlist with no deny-all
fallback still leaves every unresolved destination reachable.

* feat(sandbox): apply the network policy on both providers, persist the inbound token

Resolve the stored policy into one fully specified provider-neutral policy so
that nothing downstream re-derives a default. Both booleans come out non-nil on
every path, including the nil-tenant one, because a partially specified policy
would let the adapters fall back to their providers' more permissive defaults.
The three inversions (deny-egress to allow-internet, public-inbound, rule-deny
to rule-allow) happen in exactly one place.

Every sandbox-creating path carries it, so sessions, skill installs and the
deep connectivity check all run under the same config's policy rather than
each wiring its own.

Inbound access defaults to CLOSED, deliberately stricter than either
provider's own default: a sandbox URL reachable by anyone who learns the ID is
not a reasonable default for a tenant's workspace. Closing it makes the
provider issue a per-sandbox traffic token, and only at create time -- so the
token is stored with the session binding and replayed through Connect on every
reconnect. Without that, a sandbox that was paused on TTL or rebound after a
restart would answer 403 on every exec and file read, and 403 classifies as an
authentication error rather than NotFound, so the binding would never be
replaced and the session would stay wedged.

Connect therefore takes a request struct instead of a bare ID, which gives the
token a single documented way in rather than implicit state inside an adapter.

Metadata-based recovery is the one reconnect path with no binding to read the
token from. Rather than adopt a sandbox that would be permanently 403, it is
deleted and replaced -- but only for providers that have the credential at
all, since Docker has no inbound token and its containers are still adoptable.

go-e2b keeps the token and never sends it, so for E2B the data-plane
RoundTripper attaches the header from a process-local registry. Cube's SDK
already does this itself.

* fix(sandbox): classify data-plane traffic by authority shape, not the configured domain

isDataPlane suffix-matched the configured sandbox domain, which was wrong in
both directions. Both required a configured proxy_url, so E2B Cloud's default
usage was unaffected; the affected deployments are exactly the self-hosted and
E2B-compatible ones the gateway exists for.

A configured gateway could be silently ignored. sandbox_domain is optional for
E2B -- MissingRequiredFields asks only for api_key and template_id -- and with
it empty the check returned false for everything. The general form is worse
than the empty case: go-e2b's envdBaseURL prefers the domain the control plane
reported over the client-wide one, so the authority actually dialled need not
appear in the config at all. The failure is quiet whenever public DNS happens
to resolve the sandbox domain, and the operator believes traffic is passing
through a gateway that never sees it.

Control-plane calls could be sent to the gateway. E2B's own defaults put the
API host under the sandbox domain (api.e2b.app beneath e2b.app), so the suffix
match classified every Create, Connect and List as data-plane traffic. That
is loud where the gateway does not serve the API, but where it proxies both
planes it succeeds while downgrading an X-API-Key request to plaintext for an
http:// gateway. Cube was immune to the first problem, since it requires both
fields, but not to this one.

Classify on the "<port>-<sandboxID>.<domain>" shape both SDKs generate, using
the SandboxIDFromDataPlaneHost helper that token injection on the very same
request already relies on -- so routing and injection now agree by
construction instead of applying two different rules. That helper exists
precisely because the configured domain is unreliable here; it was simply
never used for the routing half.

The control host stays as an explicit exclusion. The shape test already
rejects api.<domain>, so this only covers an API host that itself wears the
shape, such as 8080-api.example.com.

One existing assertion was replaced rather than kept: the E2B pool test proved
that the Cube fields were ignored by asserting a cube.app authority was not
data-plane traffic, which tested the domain matching rather than the field
selection. It now asserts the transport dials the E2B gateway address and not
the Cube one.

* feat(sandbox): add the network policy editor to the sandbox config drawer

The policy sits in the runtime step, directly under the settings it
constrains, rather than in a step of its own. The existing execution timeouts
were compressed into a two-column grid to make room, keeping each field's own
label and its own tip -- the reason they were one per row was that three
unlabelled numbers sharing a footnote gave no clue which one you were editing,
and two columns with per-field tips does not regress that.

Provider-specific sections are shown only for the backend that has them: the
Cube L7 rule cards, E2B's per-host header transforms, and for Docker just the
bridge/none selector, since Docker can only express the overall switch.

Injected header values are write-only. A stored one shows as configured and is
preserved when the input is left blank, which is how env-var secrets already
behave. Renaming a rule or a header is the case worth care: the merge indexes
credentials on the rule-and-header pair, so a rename means the old value can no
longer be recovered. One helper decides both the placeholder text and whether
the payload re-attaches the value, so the form cannot claim a credential is
kept while the save blanks it.

The add buttons use keys of their own instead of reusing the env-var table's
"add variable", which read wrongly under a list of network destinations and
which the i18n key check cannot catch, since the key exists in every bundle.

* docs(sandbox): document the network policy and its inbound default change

The existing network bullet described the pre-feature state -- both providers
open by default, tunable only through the neutral policy at create time --
which is no longer true of the defaults or of the admin-facing surface.

Three things needed writing down rather than restating.

The inbound default flips from publicly reachable to credential-required. A
deployment that relies on a browser or an external service reaching a sandbox
port directly, rather than going through WeKnora, has to opt back in on that
config.

The binding store now holds a bearer credential in plaintext. Leaving it
unencrypted was a deliberate trade -- per-reconnect crypto on the hot path
against Redis's own access control -- and the trade is only sound if the
requirement is actually stated, along with the code-side constraint it depends
on: a binding or a handle is never logged whole, only the sandbox ID and
whether a credential is present.

On binding loss, a closed-inbound Cube or E2B config no longer adopts its old
sandbox by metadata. The token went with the binding and the provider will not
reissue it, so adoption would produce a session that answers 403 on every
data-plane call with no way back. The sandbox is deleted and rebuilt, which
costs the /workspace scratch. Docker has no such credential and still adopts.

* fix(sandbox): fail closed when a closed-inbound create has no traffic token

A Cube/E2B sandbox created without the inbound token would be bound anyway,
then 403 on every data-plane call as an unreplaceable authentication error.
Refuse the create, destroy the sandbox, and stop Docker from persisting
inbound/egress radios it cannot honour. Canonicalize IPv4 /0 deny-all onto
0.0.0.0/0 and reject CR/LF in injected header names and values.

* fix(sandbox): satisfy pre-push lint on the network policy files

Unexport the data-plane host parser, drop file-level package docs that
revive rejected, and lowercase error-string prefixes that staticcheck
ST1005 treats as capitalized English.

* fix(sandbox): persist rotated inbound tokens and stop misreporting restricted egress

Connect can reissue the traffic token after pause/resume; write it back onto
the binding so a restart does not restore a stale credential that 403s and
cannot be replaced. Restore the volume-support i18n key, map Docker
network_mode=none onto deny-egress, inject the E2B inbound header without a
gateway pool, and fold header names when merging redacted secrets.

---------

Co-authored-by: hailongzhao <hailongzhao@tencent.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-03 10:49:46 +08:00
..
interfaces fix(models): show usage details when deletion is blocked (#2975) 2026-09-02 17:20:37 +08:00
agent.go feat(agent): keep long sandbox turns from truncating, overflowing, or rewriting files 2026-09-02 22:09:09 +08:00
agent_share_source.go fix: harden shared agent source selector validation 2026-07-30 12:28:23 +08:00
agent_share_source_test.go fix: harden shared agent source selector validation 2026-07-30 12:28:23 +08:00
agent_test.go feat(agent): keep long sandbox turns from truncating, overflowing, or rewriting files 2026-09-02 22:09:09 +08:00
agent_type_preset.go feat: Introduce agent type presets for smart-reasoning mode 2026-04-22 21:18:20 +08:00
audit_log.go feat(system-admin): add endpoint for creating users (#2722) 2026-08-20 19:51:10 +08:00
audit_log_test.go feat(system-admin): add endpoint for creating users (#2722) 2026-08-20 19:51:10 +08:00
builtin_agent_config.go fix(i18n): 修复内置智能体名称与描述不随界面语言切换的问题 (#2828) 2026-08-27 11:53:45 +08:00
builtin_agent_localization_test.go fix(i18n): 修复内置智能体名称与描述不随界面语言切换的问题 (#2828) 2026-08-27 11:53:45 +08:00
builtin_models_config.go feat(models): let system admins edit builtin model config 2026-07-14 15:56:02 +08:00
builtin_models_config_test.go feat(models): let system admins edit builtin model config 2026-07-14 15:56:02 +08:00
chat.go feat(chat): keep prompt prefixes stable and send provider cache markers 2026-09-03 00:45:52 +08:00
chat_history_config.go feat: implement chat history and retrieval settings 2026-03-09 11:46:56 +08:00
chat_manage.go feat(memory): add cross-session long-term memory 2026-08-13 18:00:02 +08:00
chunk.go feat(knowledge): support editable chunks and metadata 2026-07-28 20:23:48 +08:00
chunk_contextheader_test.go test(types): regression test that ContextHeader is hidden from JSON 2026-05-06 17:17:07 +08:00
cleanup.go init commit 2025-08-05 15:08:07 +08:00
config_redaction.go feat(sandbox): add tenant-configurable network policy (#2995) 2026-09-03 10:49:46 +08:00
config_redaction_sandbox_test.go feat(sandbox): add tenant-configurable network policy (#2995) 2026-09-03 10:49:46 +08:00
config_redaction_test.go feat(storage): support AWS S3 NoAk default credentials (#2008) 2026-08-05 11:59:02 +08:00
const.go refactor(context): replace middleware.EmbedChannelContextKey with types.EmbedChannelContextKey 2026-08-13 18:00:02 +08:00
context_clone.go refactor(context): replace middleware.EmbedChannelContextKey with types.EmbedChannelContextKey 2026-08-13 18:00:02 +08:00
context_clone_coverage_test.go refactor(context): replace middleware.EmbedChannelContextKey with types.EmbedChannelContextKey 2026-08-13 18:00:02 +08:00
context_helpers.go feat(sandbox) (#2301) 2026-08-10 21:28:20 +08:00
context_helpers_test.go fix: retry document summaries before fallback 2026-08-05 19:29:32 +08:00
custom_agent.go feat(agent): keep long sandbox turns from truncating, overflowing, or rewriting files 2026-09-02 22:09:09 +08:00
custom_agent_test.go feat(agent): keep long sandbox turns from truncating, overflowing, or rewriting files 2026-09-02 22:09:09 +08:00
dataset.go init commit 2025-08-05 15:08:07 +08:00
datasource.go fix(datasource): perform real sync deletions scoped per data source (#2690) 2026-08-13 19:52:44 +08:00
datasource_test.go feat(feishu): sync docx via blocks API with per-type downdrill (#2087) 2026-07-27 16:04:56 +08:00
docparser.go fix(chunker): trim surrounding whitespace before embedding 2026-05-06 21:16:48 +08:00
embed_channel.go refactor: remove Neo4j-based conversation memory feature 2026-07-18 23:41:48 +08:00
embed_channel_test.go feat(memory): add cross-session long-term memory 2026-08-13 18:00:02 +08:00
embedding.go feat(agent): enhance suggested questions API with tag scopes 2026-07-17 10:19:43 +08:00
errors.go init commit 2025-08-05 15:08:07 +08:00
evaluation.go fix: support custom jieba dictionary directory 2026-06-08 20:10:37 +08:00
event_bus.go refactor: Implement event-driven streaming and enhance session management 2025-11-07 22:41:41 +08:00
extract_graph.go Refactor knowledge post-processing into orchestrator task 2026-04-16 18:13:20 +08:00
faq.go fix(chunks): retain questions across content edits 2026-07-28 20:23:48 +08:00
faq_sync.go feat(knowledge): enhance FAQ export functionality 2026-07-23 10:50:32 +08:00
faq_sync_test.go feat(knowledge): enhance FAQ export functionality 2026-07-23 10:50:32 +08:00
faq_test.go feat(memory): add cross-session long-term memory 2026-08-13 18:00:02 +08:00
generated_question_revision_test.go fix(chunks): retain questions across content edits 2026-07-28 20:23:48 +08:00
generated_question_source_id_test.go fix(knowledge): bound generated question source IDs 2026-07-28 20:23:48 +08:00
graph.go feat: implement preview of user level memory graph 2026-02-25 14:29:54 +08:00
indexing_strategy.go feat: Add comprehensive documentation for WeKnora knowledge base and wiki features 2026-04-22 21:18:10 +08:00
json.go test(types): add unit test for JSON unmarshalling and validation 2026-04-22 21:17:36 +08:00
json_map.go refactor(knowledge): replace flat stage table with langfuse-style span tree 2026-05-28 15:14:45 +08:00
json_test.go test(types): add unit test for JSON unmarshalling and validation 2026-04-22 21:17:36 +08:00
knowledge.go feat: support editing document summaries (#2752) 2026-08-20 14:54:53 +08:00
knowledge_custom_metadata_test.go feat(knowledge): support editable chunks and metadata 2026-07-28 20:23:48 +08:00
knowledge_folder.go feat(knowledge): store folder path for folder uploads and expose folder tree API 2026-08-04 20:45:29 +08:00
knowledge_folder_test.go feat(knowledge): store folder path for folder uploads and expose folder tree API 2026-08-04 20:45:29 +08:00
knowledge_process.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
knowledge_process_test.go feat(docreader): support forcing PDF scanned-page OCR parsing 2026-06-25 17:30:22 +08:00
knowledge_span.go fix(spans): expand knowledge span name column and truncate long wiki names 2026-07-13 10:35:44 +08:00
knowledgebase.go fix(parser): prefer anydoc for every type it converts when linked 2026-08-31 14:58:15 +08:00
knowledgebase_auto_tag_test.go fix(knowledge): harden auto-tagging and let it respect manual tags 2026-08-07 15:35:58 +08:00
knowledgebase_test.go fix(parser): prefer anydoc for every type it converts when linked 2026-08-31 14:58:15 +08:00
mcp.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
mcp_oauth.go fix(mcp): coordinate OAuth token refresh and authorization polling 2026-07-23 15:35:24 +08:00
mcp_test.go feat(crypto): AES-GCM at-rest for MCP and DataSource credentials with graceful key-rotation handling 2026-05-17 15:27:52 +08:00
memory.go feat(memory): introduce long-term memory feature with i18n updates 2026-08-13 18:00:02 +08:00
memory_test.go feat(memory): add cross-session long-term memory 2026-08-13 18:00:02 +08:00
message.go fix(observability): nest follow-up completions under the chat trace 2026-08-31 19:42:32 +08:00
message_attachment_test.go feat(im): process file attachments in QA 2026-08-12 16:08:51 +08:00
message_suggestion.go feat(agent): add question suggestions with after-answer follow-ups 2026-07-14 11:17:25 +08:00
message_test.go feat(sandbox) (#2301) 2026-08-10 21:28:20 +08:00
model.go feat(agent): keep long sandbox turns from truncating, overflowing, or rewriting files 2026-09-02 22:09:09 +08:00
organization.go fix: harden shared agent source permissions 2026-07-30 12:28:23 +08:00
parser_engine_config_test.go fix(parser): default ppt/pptx to markitdown when no engine rules (#2882) 2026-08-31 14:58:15 +08:00
placeholder.go feat(chat): keep prompt prefixes stable and send provider cache markers 2026-09-03 00:45:52 +08:00
placeholder_time_test.go feat(chat): keep prompt prefixes stable and send provider cache markers 2026-09-03 00:45:52 +08:00
principal.go fix(session): include external users in API session group 2026-07-30 11:56:05 +08:00
principal_test.go test(api-keys): add unit tests for tenant API key validation 2026-07-07 12:00:13 +08:00
prompt_instructions.go feat(memory): add cross-session long-term memory 2026-08-13 18:00:02 +08:00
prompt_instructions_test.go Enhance knowledge base configuration by normalizing and validating custom instruction fields. Implemented length checks for prompt instructions and added tests for validation logic. Updated GraphSettings to retain custom instructions when extraction is disabled. 2026-07-13 12:16:19 +08:00
qa_request.go fix: harden shared agent source permissions 2026-07-30 12:28:23 +08:00
question_suggestion_test.go feat(agent): add question suggestions with after-answer follow-ups 2026-07-14 11:17:25 +08:00
resource.go feat(agent): preview skill artifacts and verify installs without running them (#2823) 2026-08-27 10:30:32 +08:00
resource_test.go feat(agent): preview skill artifacts and verify installs without running them (#2823) 2026-08-27 10:30:32 +08:00
retrieval_config.go Fix/hybrid search match count normalization (#2750) 2026-08-20 11:36:38 +08:00
retriever.go feat(retriever): add OpenSearch type prep ahead of Phase 3 driver 2026-05-22 20:43:50 +08:00
runtime_task.go feat(system): add cursor pagination for runtime queue tasks 2026-07-14 16:53:07 +08:00
sandbox_network_policy.go feat(sandbox): add tenant-configurable network policy (#2995) 2026-09-03 10:49:46 +08:00
sandbox_network_policy_test.go feat(sandbox): add tenant-configurable network policy (#2995) 2026-09-03 10:49:46 +08:00
sandbox_tenant_context_test.go feat(sandbox) (#2301) 2026-08-10 21:28:20 +08:00
search.go fix(chunks): 流水线重写后的检索结果不用位置匹配 (#2558) 2026-08-06 14:32:24 +08:00
search_scope_test.go refactor(agent): introduce modelcontext registry and fix handle-codec bugs 2026-07-27 11:05:43 +08:00
secret.go fix(security): harden secret redaction follow-ups for PR #1884 2026-07-02 14:41:42 +08:00
secret_test.go fix(security): harden secret redaction follow-ups for PR #1884 2026-07-02 14:41:42 +08:00
session.go feat(sandbox): expose tenant skill management over HTTP and in the console (#2789) 2026-08-25 11:28:12 +08:00
session_access_test.go feat(sandbox): expose tenant skill management over HTTP and in the console (#2789) 2026-08-25 11:28:12 +08:00
storagebackend.go feat(storage): add multi-instance storage backends per workspace 2026-07-15 14:54:39 +08:00
storagebackend_test.go feat(storage): add multi-instance storage backends per workspace 2026-07-15 14:54:39 +08:00
sync_item_error_test.go fix(feishu): resilient large-wiki sync — retry, streaming resume, failure visibility 2026-07-21 16:14:22 +08:00
system_setting.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
tag.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
task.go feat(memory): add cross-session long-term memory 2026-08-13 18:00:02 +08:00
task_dead_letter.go perf(wiki-ingest): scale to 4w-doc KBs + generic task queue / dead-letter 2026-05-10 00:01:06 +08:00
task_initiator_test.go feat(knowledge): add scoped activity audit trail 2026-07-23 10:50:32 +08:00
task_pending_op.go perf(wiki-ingest): scale to 4w-doc KBs + generic task queue / dead-letter 2026-05-10 00:01:06 +08:00
task_queue_test.go feat: 增加自动添加标签的功能 2026-08-07 15:01:01 +08:00
temporary_document.go fix: harden shared agent source permissions 2026-07-30 12:28:23 +08:00
tenant.go feat(sandbox): add tenant-configurable network policy (#2995) 2026-09-03 10:49:46 +08:00
tenant_api_key.go feat(api-keys): add platform-scoped API keys for control-plane automation 2026-07-21 17:21:35 +08:00
tenant_api_key_scope_test.go Add API key member and space capabilities 2026-07-07 12:00:13 +08:00
tenant_api_key_test.go Refine scoped API key capabilities 2026-07-07 12:00:13 +08:00
tenant_env_vars.go fix(sandbox): make skill env capture fill blanks only 2026-08-27 14:32:00 +08:00
tenant_env_vars_test.go fix(sandbox): make skill env capture fill blanks only 2026-08-27 14:32:00 +08:00
tenant_invitation.go feat(rbac): add multi-use share-link invitations for invite_only mode (#1504) 2026-05-27 19:42:01 +08:00
tenant_member.go Refine scoped API key capabilities 2026-07-07 12:00:13 +08:00
tenant_sandbox_config_entity.go feat(sandbox): 沙箱后端改为按空间配置,并补齐 E2B 模板与检测诊断 2026-08-11 15:42:02 +08:00
tenant_sandbox_config_entity_test.go feat(sandbox) (#2301) 2026-08-10 21:28:20 +08:00
tenant_sandbox_config_test.go feat(sandbox) (#2301) 2026-08-10 21:28:20 +08:00
tenant_skill.go fix(skills): store one catalog zip and cache compressed archives 2026-09-02 15:08:05 +08:00
tenant_test.go fix: preserve MinerU automatic PDF parsing 2026-08-03 15:37:07 +08:00
token_usage_test.go feat(agent): report and attribute per-turn LLM token usage 2026-08-24 10:36:19 +08:00
tracing.go feat(tracing): migrate langfuse client to OTLP/OTel wire protocol 2026-07-17 16:30:45 +08:00
user.go fix(auth): record last-active-tenant preference on SwitchTenant (#2922) 2026-09-01 15:43:40 +08:00
user_resource_favorite.go feat(favorites): implement user favorites and recents functionality 2026-05-18 17:28:58 +08:00
vectorstore.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
vectorstore_opensearch_test.go feat(retriever): activate OpenSearch k-NN driver (PR 3 of 3) 2026-05-29 16:32:27 +08:00
vectorstore_test.go feat(retriever): configure Tencent VectorDB replicas via env 2026-06-22 20:21:25 +08:00
web_search.go fix(web-search): normalize default tenant web search config at runtime 2026-04-30 15:28:35 +08:00
web_search_provider.go feat: add Exa WebSearch (#2617) 2026-08-11 17:05:52 +08:00
web_search_provider_test.go feat: add Exa WebSearch (#2617) 2026-08-11 17:05:52 +08:00
web_search_test.go fix(web-search): normalize default tenant web search config at runtime 2026-04-30 15:28:35 +08:00
weknoracloud.go refactor(weknoracloud): update credential handling and configuration 2026-04-15 11:37:24 +08:00
wiki_page.go feat(memory): add cross-session long-term memory 2026-08-13 18:00:02 +08:00
wiki_page_test.go feat(memory): add cross-session long-term memory 2026-08-13 18:00:02 +08:00