WeKnora/internal/types
shenk-b 5ee96019e0 feat(tracing): migrate langfuse client to OTLP/OTel wire protocol
Migrate WeKnora's Langfuse tracing from the legacy /api/public/ingestion
batch API to the OpenTelemetry Go SDK emitting OTLP/HTTP to
/api/public/otel/v1/traces — the Langfuse v3+ standard ingestion protocol.

Why: Langfuse v3 deprecated the legacy ingestion API in favor of OTLP, and
OTel-only backends (e.g. the LiteFuse Doris fork) dropped it entirely, so the
legacy client was incompatible. Migrating to the OTel SDK also adds W3C
traceparent cross-service propagation: an upstream caller's trace id is
inherited so its run and the WeKnora call land under one trace in the UI.

What:
- Manager owns an OTel TracerProvider backed by an otlptracehttp exporter
  (Basic auth + the x-langfuse-ingestion-version gate header). Spans are
  buffered by BatchSpanProcessor and exported complete on End, so a long
  trace no longer fragments across flush batches.
- *Trace/*Span/*Generation wrap OTel spans; the public API (StartTrace/
  Span/Generation, handles, InjectTracing, asynq/gin middleware, Config,
  TokenUsage) is unchanged, so callers (agent/chat/handler/scheduler/asynqdl)
  need no edits.
- W3C traceparent: GinMiddleware extracts traceparent from incoming requests;
  AsynqMiddleware + InjectTracing propagate it across the asynq payload.
  types.TracingContext gains a LangfuseTraceparent field.
- logger.CloneContext preserves the active OTel span across context rebuilds
  so child spans (agent engine, summary generation) stitch to the HTTP trace
  instead of orphaning into separate traces.
- StartSpan/StartGeneration re-establish the parent from the *Trace handle
  when the ctx lost the active OTel span, covering background/derived contexts.

Compatibility: works with Langfuse v3+ (cloud / self-hosted / the bundled
docker-compose profile) and LiteFuse. Drops Langfuse v2 (legacy-only) support.
Uses the OTel Go SDK already (indirectly) in go.mod, promoting
otlptracehttp to direct; no third-party langfuse-go library.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 16:30:45 +08:00
..
interfaces feat(agent): enhance suggested questions API with tag scopes 2026-07-17 10:19:43 +08:00
agent.go feat(agent): add per-agent citation output toggle 2026-07-17 10:19:43 +08:00
agent_test.go Fix system prompt not being used when flag is false 2026-04-15 20:21:12 +08:00
agent_type_preset.go feat: Introduce agent type presets for smart-reasoning mode 2026-04-22 21:18:20 +08:00
audit_log.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
audit_log_test.go feat(system): expand runtime queue task inspection and safe actions 2026-07-14 15:38:06 +08:00
builtin_agent_config.go feat: enhance localization and configuration support 2026-03-17 22:23:24 +08:00
builtin_models_config.go feat(models): let system admins edit builtin model config 2026-07-14 15:56:02 +08:00
builtin_models_config_test.go feat(models): let system admins edit builtin model config 2026-07-14 15:56:02 +08:00
chat.go feat(mcp): prompt and resume OAuth authorization mid-conversation 2026-06-29 23:22:28 +08:00
chat_history_config.go feat: implement chat history and retrieval settings 2026-03-09 11:46:56 +08:00
chat_manage.go feat(agent): enhance suggested questions API with tag scopes 2026-07-17 10:19:43 +08:00
chunk.go refactor(chunk): remove VideoInfo field from Chunk struct 2026-05-25 19:15:39 +08:00
chunk_contextheader_test.go test(types): regression test that ContextHeader is hidden from JSON 2026-05-06 17:17:07 +08:00
cleanup.go init commit 2025-08-05 15:08:07 +08:00
config_redaction.go fix(parser): remove workspace chat parser policy UI 2026-07-17 13:42:02 +08:00
config_redaction_test.go fix(parser): remove workspace chat parser policy UI 2026-07-17 13:42:02 +08:00
const.go feat(models): cap background LLM concurrency per model 2026-07-09 20:40:51 +08:00
context_helpers.go feat(models): cap background LLM concurrency per model 2026-07-09 20:40:51 +08:00
context_helpers_test.go test: address PR review follow-ups for OAuth and logger cleanup 2026-07-01 17:31:42 +08:00
custom_agent.go feat(agent): add per-agent citation output toggle 2026-07-17 10:19:43 +08:00
custom_agent_test.go feat(agent): add per-agent citation output toggle 2026-07-17 10:19:43 +08:00
dataset.go init commit 2025-08-05 15:08:07 +08:00
datasource.go feat(im,datasource): support Lark, Feishu's international edition 2026-07-17 11:05:54 +08:00
datasource_test.go fix(datasource): harden RSS connector and editor UX after review 2026-06-23 17:30:38 +08:00
docparser.go fix(chunker): trim surrounding whitespace before embedding 2026-05-06 21:16:48 +08:00
embed_channel.go feat(embed): add backend embed channels with auth and rate limiting (#1657) 2026-06-12 23:10:39 +08:00
embed_channel_test.go feat(embed): add backend embed channels with auth and rate limiting (#1657) 2026-06-12 23:10:39 +08:00
embedding.go feat(agent): enhance suggested questions API with tag scopes 2026-07-17 10:19:43 +08:00
errors.go init commit 2025-08-05 15:08:07 +08:00
evaluation.go fix: support custom jieba dictionary directory 2026-06-08 20:10:37 +08:00
event_bus.go refactor: Implement event-driven streaming and enhance session management 2025-11-07 22:41:41 +08:00
extract_graph.go Refactor knowledge post-processing into orchestrator task 2026-04-16 18:13:20 +08:00
faq.go feat(types): enhance FAQ metadata normalization and sanitization 2026-03-19 18:20:09 +08:00
faq_test.go feat(chunk): implement batch deletion for chunks to avoid MySQL placeholder limit 2026-03-19 18:20:09 +08:00
graph.go feat: implement preview of user level memory graph 2026-02-25 14:29:54 +08:00
indexing_strategy.go feat: Add comprehensive documentation for WeKnora knowledge base and wiki features 2026-04-22 21:18:10 +08:00
json.go test(types): add unit test for JSON unmarshalling and validation 2026-04-22 21:17:36 +08:00
json_map.go refactor(knowledge): replace flat stage table with langfuse-style span tree 2026-05-28 15:14:45 +08:00
json_test.go test(types): add unit test for JSON unmarshalling and validation 2026-04-22 21:17:36 +08:00
knowledge.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
knowledge_process.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
knowledge_process_test.go feat(docreader): support forcing PDF scanned-page OCR parsing 2026-06-25 17:30:22 +08:00
knowledge_span.go fix(spans): expand knowledge span name column and truncate long wiki names 2026-07-13 10:35:44 +08:00
knowledgebase.go feat(resource): add stable resource registry with LLM-context alias compaction 2026-07-15 17:02:43 +08:00
knowledgebase_test.go fix: deep-copy stored files and images when cloning a knowledge base 2026-06-03 14:45:59 +08:00
mcp.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
mcp_oauth.go feat: scope API MCP OAuth by principal 2026-06-30 20:01:05 +08:00
mcp_test.go feat(crypto): AES-GCM at-rest for MCP and DataSource credentials with graceful key-rotation handling 2026-05-17 15:27:52 +08:00
memory.go feat: implement preview of user level memory graph 2026-02-25 14:29:54 +08:00
message.go feat(agent): enhance suggested questions API with tag scopes 2026-07-17 10:19:43 +08:00
message_attachment_test.go feat(chat): add session-scoped temporary attachments with async parsing 2026-07-16 17:41:27 +08:00
message_suggestion.go feat(agent): add question suggestions with after-answer follow-ups 2026-07-14 11:17:25 +08:00
model.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
organization.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
parser_engine_config_test.go feat(chat): add session-scoped temporary attachments with async parsing 2026-07-16 17:41:27 +08:00
placeholder.go feat: enhance localization and configuration support 2026-03-17 22:23:24 +08:00
principal.go test(api-keys): add unit tests for tenant API key validation 2026-07-07 12:00:13 +08:00
principal_test.go test(api-keys): add unit tests for tenant API key validation 2026-07-07 12:00:13 +08:00
prompt_instructions.go Enhance knowledge base configuration by normalizing and validating custom instruction fields. Implemented length checks for prompt instructions and added tests for validation logic. Updated GraphSettings to retain custom instructions when extraction is disabled. 2026-07-13 12:16:19 +08:00
prompt_instructions_test.go Enhance knowledge base configuration by normalizing and validating custom instruction fields. Implemented length checks for prompt instructions and added tests for validation logic. Updated GraphSettings to retain custom instructions when extraction is disabled. 2026-07-13 12:16:19 +08:00
qa_request.go feat(chat): add @Skill/@MCP mentions with scoped agent runtime 2026-06-29 18:09:44 +08:00
question_suggestion_test.go feat(agent): add question suggestions with after-answer follow-ups 2026-07-14 11:17:25 +08:00
resource.go feat(resource): add stable resource registry with LLM-context alias compaction 2026-07-15 17:02:43 +08:00
retrieval_config.go feat(retrieval): configurable RRF parameters per tenant 2026-05-06 17:17:07 +08:00
retriever.go feat(retriever): add OpenSearch type prep ahead of Phase 3 driver 2026-05-22 20:43:50 +08:00
runtime_task.go feat(system): add cursor pagination for runtime queue tasks 2026-07-14 16:53:07 +08:00
search.go feat(agent): enhance suggested questions API with tag scopes 2026-07-17 10:19:43 +08:00
search_scope_test.go feat(agent): enhance suggested questions API with tag scopes 2026-07-17 10:19:43 +08:00
secret.go fix(security): harden secret redaction follow-ups for PR #1884 2026-07-02 14:41:42 +08:00
secret_test.go fix(security): harden secret redaction follow-ups for PR #1884 2026-07-02 14:41:42 +08:00
session.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
storagebackend.go feat(storage): add multi-instance storage backends per workspace 2026-07-15 14:54:39 +08:00
storagebackend_test.go feat(storage): add multi-instance storage backends per workspace 2026-07-15 14:54:39 +08:00
system_setting.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
tag.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
task.go feat(chat): add session-scoped temporary attachments with async parsing 2026-07-16 17:41:27 +08:00
task_dead_letter.go perf(wiki-ingest): scale to 4w-doc KBs + generic task queue / dead-letter 2026-05-10 00:01:06 +08:00
task_pending_op.go perf(wiki-ingest): scale to 4w-doc KBs + generic task queue / dead-letter 2026-05-10 00:01:06 +08:00
task_queue_test.go feat(chat): add session-scoped temporary attachments with async parsing 2026-07-16 17:41:27 +08:00
temporary_document.go feat(attachments): implement normalization and limits for temporary attachment IDs 2026-07-16 17:41:27 +08:00
tenant.go feat(chat): add session-scoped temporary attachments with async parsing 2026-07-16 17:41:27 +08:00
tenant_api_key.go feat(storage): add multi-instance storage backends per workspace 2026-07-15 14:54:39 +08:00
tenant_api_key_scope_test.go Add API key member and space capabilities 2026-07-07 12:00:13 +08:00
tenant_api_key_test.go Refine scoped API key capabilities 2026-07-07 12:00:13 +08:00
tenant_invitation.go feat(rbac): add multi-use share-link invitations for invite_only mode (#1504) 2026-05-27 19:42:01 +08:00
tenant_member.go Refine scoped API key capabilities 2026-07-07 12:00:13 +08:00
tenant_test.go feat: support Tencent VectorDB keyword retrieval 2026-05-10 00:02:05 +08:00
tracing.go feat(tracing): migrate langfuse client to OTLP/OTel wire protocol 2026-07-17 16:30:45 +08:00
user.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
user_resource_favorite.go feat(favorites): implement user favorites and recents functionality 2026-05-18 17:28:58 +08:00
vectorstore.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
vectorstore_opensearch_test.go feat(retriever): activate OpenSearch k-NN driver (PR 3 of 3) 2026-05-29 16:32:27 +08:00
vectorstore_test.go feat(retriever): configure Tencent VectorDB replicas via env 2026-06-22 20:21:25 +08:00
web_search.go fix(web-search): normalize default tenant web search config at runtime 2026-04-30 15:28:35 +08:00
web_search_provider.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
web_search_test.go fix(web-search): normalize default tenant web search config at runtime 2026-04-30 15:28:35 +08:00
weknoracloud.go refactor(weknoracloud): update credential handling and configuration 2026-04-15 11:37:24 +08:00
wiki_log_entry.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
wiki_page.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
wiki_page_test.go feat: Enhance wiki extraction and configuration options 2026-04-22 21:18:32 +08:00