Commit Graph

60 Commits (625f9db80a8448644cd020fb0156e7aca28cd87c)

Author SHA1 Message Date
nullptr-error 18d22b014b
feat(sandbox): expose tenant skill management over HTTP and in the console (#2789)
* feat(sandbox): enhance skill management in sandbox configurations

Updated the skill management system to allow listing, installing, and removing skills within sandbox configurations. The ListSkills function now accepts a sandboxConfigID to filter skills based on the selected configuration. Introduced a new SandboxSkillsDrawer component for managing skills through the UI, along with necessary updates to the API and localization files to support these features.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix: keep skill maintenance sessions out of the console session list

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat: store and expose the installer transcript locators

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat: record the skill installer agent's conversation

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat: stream and persist the installer transcript during a skill install

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat: expose install transcript locators on the skill type

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat: add the floating skill install transcript window

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat: open the install transcript from the sandbox skills drawer

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat(sandbox): 重构技能安装体验,支持内联时间线、编辑第4步及保留镜像指针

- 替换右下角浮窗为内联安装时间线 (SkillInstallTimeline.vue)
  - 新增 GET /api/v1/sandbox-configs/:id/skills/:skillId/transcript 接口,支持回放与追尾
  - 移除 /sessions/continue-stream 依赖,避免 404 问题
  - 事件日志以 install_prompt 开头,不再将工具调用 preamble 拼入最终答案
  - 删除 SkillInstallTranscript.vue,技能条目下可展开/折叠时间线

- 编辑向导新增第 4 步「管理技能」
  - Cube/E2B 环境:连接 → 模板 → 运行配置 → 技能(第4步)
  - Docker/Local 环境:无第4步(无快照镜像)
  - 首次创建必须按 1→2→3 顺序,第3步保存成功后自动进入第4步
  - 已有配置支持步骤条数字跳转,可直接访问任意步骤
  - 卡片菜单「管理技能」打开同一抽屉并定位至第4步
  - SandboxSkillsDrawer.vue 重命名为 SandboxSkillsPanel.vue,去除独立抽屉壳

- 修复保存运行配置时冲掉 skill_image 的问题
  - 顶部展示信息(基础模板/快照ID/版本/构建时间)读取 config.skill_image
  - 编辑器保存 payload 不再包含 skill_image,合并时保留已有 SkillImage 和 VolumeMount
  - MergeSandboxConfigForUpdate 增加对 SkillImage 和 VolumeMount 的保护
  - 已冲掉的配置需重新安装技能才能恢复镜像指针

- 「查看安装过程」在安装开始时即显示
  - 技能列表中出现「正在安装」状态即展示按钮
  - 若记录尚未写入,展示等待提示并自动重试,避免误报「没有留下记录」

* 沙盒内技能目录从 UUID 改成 SKILL.md 的 name。

* fix(sandbox): heal stuck skill rows and block planted maintenance sessions

The reaper left a mid-upgrade install spinning forever when the old snapshot was still live, and restored removals to ready after the pointer had already moved. Client session create/update can no longer hide chats behind the skill_maintenance: prefix.

* style: gofumpt the skill reaper test stubs

* style: satisfy golangci on the tenant skill admin API

* feat(sandbox): rebuild skill images at turn boundaries, not mid-exec

Hold a session turn lease through AgentQA so an install cannot tear down
the live sandbox between tool calls; the next turn's first resolve picks
up the new image. Configs can instead keep existing sandboxes and boot
the new snapshot only for new sessions.

* fix(sandbox): judge stuck skill runs from the snapshot chain

InstalledSnapshotID only records the generation that installed a skill; later
installs of other skills advance the live pointer without rewriting older rows.
Comparing those IDs made the reaper fail healthy installs and delete live
skills whose removal had stalled.

---------

Co-authored-by: hailongzhao <hailongzhao@tencent.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-25 11:28:12 +08:00
wizardchen 9c5c52f0a4 feat(sandbox): Docker 后端支持会话级长驻容器与远程守护进程
将 Docker 沙箱后端从一次性 exec 容器改为会话级长驻容器,使其会话与文件
系统语义与 E2B / Cube 后端对齐:

- 新增 docker_engine / docker_remote_client / docker_rpc_timeout,直连
  Docker Engine API,支持 TCP+TLS 远程守护进程,并为拉镜像等慢操作单独
  设置超时;daemon 地址留空时跟随本机 docker context。
- 新增空闲清理器 docker_idle_sweeper,按 exec 刷新的活跃标记回收长驻
  容器;标记文件对沙箱非 root 账号可写。
- 会话容器补 PID1 以回收僵尸进程;标准镜像补入 curl。
- 镜像即模板:docker_template_catalog 只上报可识别的沙箱镜像,标准模板
  的拉取在后台进行并报告 building。
- 设置页与租户配置支持守护进程地址、TLS 与 CPU/内存限额,四种语言文案
  同步;沙箱健康检查覆盖 Docker 后端。
- 补充 docs/sandbox-docker-backend.md 与 POC,说明能力边界与快照语义;
  git 钩子的变更包检测跳过嵌套 module,避免拿 POC 的包去跑主 module 测试。
2026-08-24 12:22:58 +08:00
wizardchen 9b4f792a04 fix(docker): 默认在 app 镜像中链接 anydoc 解析引擎
Hub / compose 打包的二进制此前走 stub,设置页会显示引擎未编入。WITH_ANYDOC 改为默认开启,并补上 builder 所需的 curl。
2026-08-16 13:31:51 +08:00
wizardchen ad2f1d1450 fix(docker): copy anydoc-go go.mod before go mod download
go.mod replace-points firecrawl/anydoc to ./third_party/anydoc-go, so the vendored module must exist in the image before dependency download.
2026-08-16 12:29:42 +08:00
wizardchen e123f8247e feat(docparser): 引入 anydoc 引擎,在 Go 进程内解析 office 文档
通过 cgo 链接 anydoc Rust 静态库,把 docx/pptx/xlsx 等转为带原位图片的 Markdown;扫描版 PDF 无文字层时回落到 DocReader OCR。默认构建不链接该库。

Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
2026-08-16 11:37:27 +08:00
lyingbug 517942c846
feat(sandbox): unify remote sandbox on E2B protocol with self-hosted support (#2655)
* feat(sandbox): unify remote sandbox on E2B protocol with self-hosted support

- Add e2b.proxy_url gateway config and generic gateway transport layer
- Add envd compat layer for Basic auth and multipart /files uploads
- Support Dockerfile.sandbox runtime and e2b-backend build targets in CI
- Rename sandbox image non-root user to align with E2B template convention
- Add sandbox-protocol.md and extend cluster deployment / local repro docs

Enables connecting to any E2B-compatible control plane (E2B Cloud,
CubeSandbox, Agent-Sandbox, etc.) without per-backend client code.

* style(sandbox): gofmt tenant_resolver struct alignment
2026-08-11 20:14:19 +08:00
wizardchen ef1047bf67 feat(parser): add OpenDataLoader, PaddleOCR-VL engines, and parser improvements
Introduce opendataloader and PaddleOCR-VL parser engines with tenant-level
settings UI, replace liteparse, and harden Excel/PPT/Markdown parsing.
Optional odl-hybrid sidecar stays local-build only and is excluded from
default dev-start and full profiles.
2026-06-03 12:29:13 +08:00
wizardchen 1110615300 feat(web-search): add SearXNG provider (#1166)
支持对接自建/公共 SearXNG 实例作为网络搜索引擎,缓解免费搜索引擎在国内
网络环境下访问受限的问题。

- types: 新增 WebSearchProviderTypeSearxng 与 BaseURL 参数字段;
  类型元数据新增 RequiresBaseURL,前端可动态渲染 Instance URL 表单。
- infrastructure/web_search/searxng.go: 调用 /search?format=json,强制
  utils.ValidateURLForSSRF 校验 base_url,可选 api_key 透传给反代鉴权。
- service: isValidProviderType 与参数校验接入 searxng。
- container: 注册 NewSearxngProvider 工厂。
- frontend: WebSearchSettings 表单根据 requires_base_url 渲染 Instance
  URL 输入框;编辑回填、free 判定同步更新。
- docker: 新增可选 searxng 服务(profile=searxng/full),附带最小化
  settings.yml(启用 JSON 格式、关闭 limiter、关闭遥测),
  docker-compose 默认 SSRF_WHITELIST 包含 searxng 容器名。
- .env.example: 补充 SEARXNG_PORT / SEARXNG_SECRET 说明。

Closes #1166
2026-05-11 16:53:47 +08:00
wizardchen d65e647f95 chore: bump Go to 1.26 and slim docreader dependencies
- Bump base image in docker/Dockerfile.app from golang:1.24 to golang:1.26
  to match `go 1.26` declared in go.mod (fixes CI build failure on
  `go mod download`).
- Drop unused docreader components and their dependencies:
  - Remove `docreader/ocr/` package (paddle/vlm/dummy backends are
    unreferenced by the main flow; OCR/VLM is handled by the Go App).
  - Remove `docreader/parser/storage.py` (dead code; image persistence
    happens in the Go App via inline ImageRef bytes).
  - Remove `docreader/scripts/download_deps.py` (PaddleOCR pre-download).
  - Drop deps: paddleocr, paddlepaddle, openai, ollama, minio,
    cos-python-sdk-v5, oss2, asyncio, pypdf2, markdown, mistletoe,
    goose3, markdownify, pdfplumber, antiword, urllib3.
- Re-lock uv.lock: 145 -> 79 packages.
- Update docreader/README.md to reflect that OCR/VLM/storage are no
  longer configured at the docreader level.
2026-05-09 13:32:40 +08:00
jw_mac faaf6124b2 add output that if install package of offline protoc exitst 2026-04-24 22:42:17 +08:00
Liwx da8191cd57 fix(docker): 修复镜像构建兼容性问题 2026-04-21 23:29:41 +08:00
DaWesen bfa3341d4a Fix: Dockerfile build error (duplicate libsqlite3-0 and ffmpeg installation) 2026-04-10 01:26:07 +08:00
DaWesen b80bdb1079 feat: add video multimodal functionality 2026-04-09 19:19:42 +08:00
wizardchen 90c32b5926 feat: enhance Docker setup with entrypoint script and skill management
- Added a new entrypoint script to manage ownership of bind-mounted directories and merge built-in skills into preloaded ones.
- Updated the Dockerfile to include the `gosu` package for privilege management and to set the entrypoint to the new script.
- Ensured built-in skills are preserved and accessible after bind-mounting user directories, improving the application's flexibility and usability.

These changes streamline the container initialization process and enhance the management of skills within the application.
2026-03-05 18:14:14 +08:00
wizardchen 393ebf9f92 feat: standardize storage configuration and enhance backward compatibility
- Updated the `KnowledgeBase` struct to rename `cos_config` to `storage_config` for consistency across the application.
- Implemented a custom `UnmarshalJSON` method to maintain backward compatibility with legacy responses that still use `cos_config`.
- Modified various frontend components to reflect the new `storage_config` naming, ensuring seamless integration with the updated backend structure.
- Enhanced Dockerfiles and service configurations to support the new storage engine setup.

This update improves the clarity of storage configurations and ensures compatibility with existing data structures.
2026-03-02 21:21:49 +08:00
wizardchen 6d88619869 feat: enhance Dockerfile and build scripts for customizable APT mirror
- Added support for customizable APT mirror in the Dockerfile for the docreader service, allowing users to specify a mirror via build arguments.
- Updated docker-compose.yml to pass the APT_MIRROR argument during the build process.
- Modified build_images.sh script to include the APT_MIRROR argument when building the docreader image.
- Updated .gitignore to exclude .cursor/ directory.

This update improves flexibility in package management during the image build process.
2026-03-02 21:21:49 +08:00
wizardchen 397689d2f3 feat: introduce WeKnora Lite edition with lightweight configuration and deployment
- Added a new `.env.lite.example` file for the Lite version, providing a minimal configuration template.
- Updated `.env.example` to remove deprecated variables and include new Docreader settings.
- Enhanced Docker configurations to support the Lite version, including a new Dockerfile for the Docreader service.
- Introduced a Makefile target for building and running the Lite version, along with packaging capabilities.
- Created GitHub workflows for building and releasing Lite binaries, including Homebrew formula support.
- Implemented a new service file for managing the Lite version as a system service.

This update enables a streamlined, single-binary deployment of WeKnora, reducing external dependencies and simplifying setup.
2026-03-02 21:21:49 +08:00
begoniezhao bce968833e feat: Add async knowledge re-parse API and examples 2026-02-11 12:27:55 +08:00
nullkey 43a2c647ea feat: optimize security and deployment of agent skills 2026-02-04 20:08:49 +08:00
nullkey 3f21345546 feat: adapt preloaded skills to smart-reasoning agent 2026-02-04 20:08:49 +08:00
begoniezhao 1b0edccfc4 feat: add download_spatial target and DuckDB spatial extension handling 2026-01-05 15:01:50 +08:00
begoniezhao 7c302bea1f fix(docker): remove default value for APK_MIRROR_ARG in Dockerfile 2025-12-29 21:08:26 +08:00
begoniezhao b4f1846a84 fix(docker): streamline package installation in Dockerfile by removing upgrade step 2025-12-29 20:43:49 +08:00
begoniezhao 907e9a5522 feat: Add DataSchema tool for retrieving schema information from CSV and Excel files 2025-12-29 20:03:51 +08:00
wizardchen 7aed2415e6 feat: 更新Dockerfile,调整uvx权限,升级node版本 2025-12-15 19:13:58 +08:00
wizardchen 4cb8079dce feat: 修复Docker构建中pip安装依赖时的系统包冲突问题,添加--break-system-packages参数 2025-12-12 20:24:29 +08:00
Munish Thakur 54707b71a0 Update Dockerfile.app 2025-12-12 10:31:39 +08:00
wizardchen 0dbb853860 Merge remote-tracking branch 'github_public/main' 2025-11-20 15:49:43 +08:00
begoniezhao 2d66abedf0 feat: 新增文档模型类,调整配置与解析逻辑,优化日志及导入
移除日志设置与冗余代码,优化导入、类型提示及OCR后端管理
统一调整各文件模块导入路径为绝对导入
调整导入路径,移除部分导入,优化日志及注释
升级文档解析器为 Docx2Parser,优化超时与图片处理逻辑
2025-11-18 22:37:01 +08:00
wizardchen 9b7128d69d feat: Update docker-compose and Dockerfile for frontend build configuration
- Added build context and Dockerfile specification for the frontend service in docker-compose.yml.
- Refactored Dockerfile.app to ensure non-root user creation is idempotent and streamlined the ownership setup for application directories.
2025-11-13 20:16:15 +08:00
wizardchen ffd5cd2881 feat: Update Dockerfile to support customizable APK mirror
- Added APK_MIRROR_ARG to allow dynamic configuration of the Alpine package mirror.
- Updated the repository URL in the Dockerfile to use the specified APK mirror if provided.
- Enhanced runtime dependencies installation by including additional packages and updating pip before installing uvx.
2025-11-13 20:01:04 +08:00
wizardchen ae58801baf Merge remote-tracking branch 'github_public/main' 2025-11-13 14:28:41 +08:00
wizardchen 05154c8d75 feat: Enhance MCP service configuration with stdio transport support
- Added support for stdio transport type in MCP service configuration, allowing command execution via standard input/output.
- Updated MCPService interface to include optional stdio_config and env_vars fields for managing command arguments and environment variables.
- Enhanced frontend components to handle stdio transport, including dynamic form fields for command and arguments.
- Implemented validation logic for stdio configuration in the service update process, ensuring proper handling of required fields.
- Updated backend logic to manage stdio transport connections and ensure proper client lifecycle management.
2025-11-12 11:32:59 +08:00
begoniezhao c1f731e026 chore(docreader): 重新组织模块文件 2025-11-05 12:07:39 +08:00
orbisai-security 6d547131b6 fix: semgrep_yaml.docker-compose.security.writable-filesystem-service.writable-filesystem-service_docker/docker-compose.minio.yml_4 2025-10-31 15:09:40 +08:00
begoniezhao 0324e83b2c chore: upspeed build docker 2025-09-19 12:29:03 +08:00
begoniezhao bfd4fffbe3 chore: upspeed build docker 2025-09-19 12:29:03 +08:00
begoniezhao 53215b1f90 chore(build): Optimize Docker build configuration and process 2025-09-18 17:34:14 +08:00
begoniezhao a5c3623a02 chore(build): Optimize Docker build configuration and process 2025-09-18 17:34:14 +08:00
begoniezhao a2a9e3aed9 chore(build): Optimize Docker build configuration and process, adjust task names 2025-09-18 17:34:14 +08:00
begoniezhao 8f723b38fb chore(build): Optimize Docker build configuration and process, adjust task names 2025-09-18 17:34:14 +08:00
begoniezhao d734b80cb5 chore(build): Optimize Makefile and Dockerfile.app build configuration 2025-09-18 14:46:10 +08:00
begoniezhao 7973128f4c chore(build): Optimize Makefile and Dockerfile.app build configuration 2025-09-18 14:46:10 +08:00
wizardchen 98e47e83d2 feat(ui): Support multi knowledgebases operation 2025-09-17 16:02:08 +08:00
wizardchen 91e65d6445 feat(ui): Support multi knowledgebases operation 2025-09-17 16:02:08 +08:00
wizardchen 092b30af3e fix(docreader): Download binary by target arch in docker 2025-09-12 20:21:30 +08:00
wizardchen 74c121f7fb feat: Adjust App & Docreader log output 2025-09-11 23:14:23 +08:00
wizardchen bff0e742fa fix: try fix ocr avx not support 2025-09-11 13:21:21 +08:00
wizardchen 6f6ca84dae feat(docreader): add health check 2025-09-10 20:22:14 +08:00
wizardchen 7cfae7e0d3 fix: pre fetch ocr models in docker container 2025-09-10 17:24:26 +08:00