WeKnora/scripts
MidoriKurage 236d8c098f
fix(datasource): perform real sync deletions scoped per data source (#2690)
* test(datasource): cover scoped sync deletions

- deleted connector items remove the matching KB knowledge, scoped per data
  source via metadata datasource_id + external_id
- SyncDeletions=false neither deletes nor counts
- stream handler classifies deleted/failed items like the batch loop

* fix(datasource): perform real sync deletions scoped per data source

- applyFetchedItem now performs the KB deletion instead of only counting
- lookup is scoped to datasource_id + external_id so identical external IDs
  from two data sources cannot collide or overwrite each other
- ingestItem update-path lookup uses the same scoped query

* fix(datasource): note failed deletions retry only on full sync

- SyncResult.DeletionFailed tracks deletion failures (subset of Failed)
- streaming partial message warns that failed deletions only retry on a full
  sync, since the connector cursor is checkpointed past the deleted item

* fix(datasource): humanize deletion errors, keep raw detail in logs

- raw lookup/delete errors go to server logs only, with full context
- UI samples carry a humanized "see server logs" fallback
- add datasource.syncError.* i18n keys

Note: ko-KR / ru-RU syncError translations were generated without a
native-speaker review.

* fix(datasource): hard-delete synced rows and tighten deletion scoping

Sync-internal deletions now call HardDeleteKnowledge after the soft-delete
cascade so tombstones cannot block re-sync; subtree sweeps and URL metadata
attach failures are handled with datasource scoping and explicit errors.

* chore: add git hooks aligned with CI and PR checks

Install via scripts/install-git-hooks.sh (sets core.hooksPath). Pre-commit
runs whitespace/gofmt/golangci-lint on staged files; pre-push mirrors
app/frontend/cli workflows on the diff since origin/main.

* fix: gofmt test file; tighten pre-push to match CI app workflow

Pre-push now gofmts the full PR diff and runs go vet on all app packages
(like app.yml), not only Go files in the latest commit. Previous pushes
skipped checks via --no-verify and because hook-only commits had no .go files.

* fix(hooks): align pre-push diff with GitHub PR base via gh

Use three-dot diff against the open PR baseRefOid (same as CI) instead of
a two-dot merge-base range; run full go vet on every push with Go PR changes.

* style: gofumpt datasource_service_test.go

---------

Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-13 19:52:44 +08:00
..
cloud-image feat(compose): align env vars with code + fix helm GraphRAG bug (#2164) 2026-07-21 15:55:52 +08:00
git-hooks fix(datasource): perform real sync deletions scoped per data source (#2690) 2026-08-13 19:52:44 +08:00
build_frontend_dist.sh feat(docker): streamline frontend build process and update Docker configuration 2026-06-10 16:04:11 +08:00
build_images.sh feat(sandbox): unify remote sandbox on E2B protocol with self-hosted support (#2655) 2026-08-11 20:14:19 +08:00
check-env.sh feat(storage): support AWS S3 NoAk default credentials (#2008) 2026-08-05 11:59:02 +08:00
dev.sh fix(dev): use IPv4 loopback for local services 2026-08-13 11:21:20 +08:00
docker-entrypoint.sh feat: enhance Docker setup with entrypoint script and skill management 2026-03-05 18:14:14 +08:00
get_version.sh feat: introduce WeKnora Lite edition with lightweight configuration and deployment 2026-03-02 21:21:49 +08:00
install-git-hooks.sh fix(datasource): perform real sync deletions scoped per data source (#2690) 2026-08-13 19:52:44 +08:00
migrate.sh feat: 支持数据库迁移脏状态自动恢复和Neo4j连接重试机制 2025-12-08 19:38:16 +08:00
package-lite.sh Revert "chore: make Lite edition support and related configurations hidden" 2026-04-11 18:51:16 +08:00
package-mac-app.sh feat: implement desktop app structure with API base URL handling and bindings for Wails integration 2026-04-11 18:51:16 +08:00
quick-dev.sh refactor(tracing): remove Jaeger integration and related components 2026-06-08 21:01:04 +08:00
start_all.sh refactor(tracing): remove Jaeger integration and related components 2026-06-08 21:01:04 +08:00
test-homebrew.sh Revert "chore: make Lite edition support and related configurations hidden" 2026-04-11 18:51:16 +08:00
update-homebrew-formula.sh Revert "chore: make Lite edition support and related configurations hidden" 2026-04-11 18:51:16 +08:00
verify_frontend_pr.sh chore(cursor): run frontend CI before gh pr create 2026-07-31 11:54:20 +08:00