kb/concepts/ bekommt Areas: layout: fuer concept, Area-Titel aus jedem Type-Spec, Schwellen-Empfehlung im lint (schliesst #59)
Files changed: - CHANGES.md - README.md - VERSION - kb/concepts/Ambient Environment Dependency.md - kb/concepts/Anti-Cramming Heuristic.md - kb/concepts/Audit Trail.md - kb/concepts/BM25.md - kb/concepts/Bulk Operations.md - kb/concepts/CI Integration.md - kb/concepts/COLLECTION.md - kb/concepts/CPPC.md - kb/concepts/Checkpoint Audit.md - kb/concepts/Claude Code Auto Mode.md - kb/concepts/Command Round-Trip Integrity.md - kb/concepts/Confidence Scoring.md - kb/concepts/Consolidation Tiers.md - kb/concepts/Content Quality Control.md - kb/concepts/Context Isolation.md - kb/concepts/Contradiction Resolution.md - kb/concepts/Cross-platform Agent Skills.md - kb/concepts/Crystallization.md - kb/concepts/Delete Rather Than Anonymize.md - kb/concepts/Denylist over Allowlist.md - kb/concepts/Detect-Repair Asymmetry.md - kb/concepts/Diff-Reviewable Agent Edits.md - kb/concepts/Dual Licensing by File Plan.md - kb/concepts/Entity Extraction.md - kb/concepts/Episodic Memory.md - kb/concepts/Event-Driven Automation.md - kb/concepts/Filter on Ingest.md - kb/concepts/Forgetting.md - kb/concepts/Graph Traversal.md - kb/concepts/Green Suite Blind Spot.md - kb/concepts/Hooks.md - kb/concepts/Hybrid Search.md - kb/concepts/INDEX.md - kb/concepts/Implementation Spectrum.md - kb/concepts/Index Scaling.md - kb/concepts/Issue Label Scheme.md - kb/concepts/Iteration and Cost Limits.md - kb/concepts/KB Migration.md - kb/concepts/KB Stack Versioning.md - kb/concepts/Knowledge Compounding.md - kb/concepts/Knowledge Graph.md - kb/concepts/LLM Wiki Pattern.md - kb/concepts/Lint Workflow.md - kb/concepts/MCP-Leseserver.md - kb/concepts/Mass-Update Gate.md - kb/concepts/Memory Lifecycle.md - kb/concepts/Mesh Sync.md - kb/concepts/Modbus.md - kb/concepts/Multi-Agent Collaboration.md - kb/concepts/Naming Convention Conflict.md - kb/concepts/OKF Compatibility.md - kb/concepts/Optional Instance Context File.md - kb/concepts/Personalization Plane.md - kb/concepts/Privacy and Governance.md - kb/concepts/Procedural Memory.md - kb/concepts/Publish-Remote Gate.md - kb/concepts/Quality Scoring.md - kb/concepts/Quality and Self-Correction.md - kb/concepts/RAG.md - kb/concepts/Reciprocal Rank Fusion.md - kb/concepts/SSD TRIM.md - kb/concepts/Scale Ceiling.md - kb/concepts/Self-Healing.md - kb/concepts/Semantic Lint Automation.md - kb/concepts/Semantic Memory.md - kb/concepts/Session Orientation.md - kb/concepts/Shared vs Private.md - kb/concepts/Split Merge Reclassify.md - kb/concepts/Split Threshold.md - kb/concepts/Structural Enforcement over Documented Rule.md - kb/concepts/Stub Threshold.md - kb/concepts/Supersession.md - kb/concepts/Three-Layer Architecture.md - kb/concepts/Token Economics.md - kb/concepts/Typed Relationships.md - kb/concepts/User Management.md - kb/concepts/Vector Search.md - kb/concepts/Work Coordination.md - kb/concepts/Workflow Extraction.md - kb/concepts/Workflow Orchestration.md - kb/concepts/Working Memory.md - kb/concepts/Write-Once Frontmatter Fields.md - kb/concepts/architectures/Consolidation Tiers.md - kb/concepts/architectures/Context Isolation.md - kb/concepts/architectures/Cross-platform Agent Skills.md - kb/concepts/architectures/Episodic Memory.md - kb/concepts/architectures/Hybrid Search.md - kb/concepts/architectures/Implementation Spectrum.md - kb/concepts/architectures/Knowledge Graph.md - kb/concepts/architectures/LLM Wiki Pattern.md - kb/concepts/architectures/MCP-Leseserver.md - kb/concepts/architectures/Memory Lifecycle.md - kb/concepts/architectures/OKF Compatibility.md - kb/concepts/architectures/Optional Instance Context File.md - kb/concepts/architectures/Personalization Plane.md - kb/concepts/architectures/Procedural Memory.md - kb/concepts/architectures/RAG.md - kb/concepts/architectures/Scale Ceiling.md - kb/concepts/architectures/Semantic Memory.md - kb/concepts/architectures/Three-Layer Architecture.md - kb/concepts/architectures/Token Economics.md - kb/concepts/architectures/Working Memory.md - kb/concepts/decisions/Delete Rather Than Anonymize.md - kb/concepts/decisions/Denylist over Allowlist.md - kb/concepts/decisions/Diff-Reviewable Agent Edits.md - kb/concepts/decisions/Dual Licensing by File Plan.md - kb/concepts/decisions/Issue Label Scheme.md - kb/concepts/decisions/KB Stack Versioning.md - kb/concepts/decisions/Structural Enforcement over Documented Rule.md - kb/concepts/patterns/Audit Trail.md - kb/concepts/patterns/BM25.md - kb/concepts/patterns/Command Round-Trip Integrity.md - kb/concepts/patterns/Confidence Scoring.md - kb/concepts/patterns/Contradiction Resolution.md - kb/concepts/patterns/Entity Extraction.md - kb/concepts/patterns/Filter on Ingest.md - kb/concepts/patterns/Forgetting.md - kb/concepts/patterns/Graph Traversal.md - kb/concepts/patterns/Mesh Sync.md - kb/concepts/patterns/Quality Scoring.md - kb/concepts/patterns/Reciprocal Rank Fusion.md - kb/concepts/patterns/Self-Healing.md - kb/concepts/patterns/Shared vs Private.md - kb/concepts/patterns/Typed Relationships.md - kb/concepts/patterns/Vector Search.md - kb/concepts/patterns/Work Coordination.md - kb/concepts/problems/Ambient Environment Dependency.md - kb/concepts/problems/Detect-Repair Asymmetry.md - kb/concepts/problems/Green Suite Blind Spot.md - kb/concepts/problems/Naming Convention Conflict.md - kb/concepts/problems/Write-Once Frontmatter Fields.md - kb/concepts/protocols/CPPC.md - kb/concepts/protocols/Modbus.md - kb/concepts/protocols/SSD TRIM.md - kb/concepts/workflows/Anti-Cramming Heuristic.md - kb/concepts/workflows/Bulk Operations.md - kb/concepts/workflows/CI Integration.md - kb/concepts/workflows/Checkpoint Audit.md - kb/concepts/workflows/Claude Code Auto Mode.md - kb/concepts/workflows/Content Quality Control.md - kb/concepts/workflows/Crystallization.md - kb/concepts/workflows/Event-Driven Automation.md - kb/concepts/workflows/Hooks.md - kb/concepts/workflows/Index Scaling.md - kb/concepts/workflows/Iteration and Cost Limits.md - kb/concepts/workflows/KB Migration.md - kb/concepts/workflows/Knowledge Compounding.md - kb/concepts/workflows/Lint Workflow.md - kb/concepts/workflows/Mass-Update Gate.md - kb/concepts/workflows/Multi-Agent Collaboration.md - kb/concepts/workflows/Privacy and Governance.md - kb/concepts/workflows/Publish-Remote Gate.md - kb/concepts/workflows/Quality and Self-Correction.md - kb/concepts/workflows/Semantic Lint Automation.md - kb/concepts/workflows/Session Orientation.md - kb/concepts/workflows/Split Merge Reclassify.md - kb/concepts/workflows/Split Threshold.md - kb/concepts/workflows/Stub Threshold.md - kb/concepts/workflows/Supersession.md - kb/concepts/workflows/User Management.md - kb/concepts/workflows/Workflow Extraction.md - kb/concepts/workflows/Workflow Orchestration.md - kb/index.md - kb/log.md - tools/CONTRACT.md - tools/README.md - tools/chemenu/catalog.py - tools/chemenu/commands/index_build.py - tools/chemenu/lint_core.py - tools/chemenu/tests/conftest.py - tools/chemenu/tests/test_cite_cmd.py - tools/chemenu/tests/test_git_publish.py - tools/chemenu/tests/test_index_build.py - tools/chemenu/tests/test_lint.py - tools/chemenu/tests/test_new_page.py - tools/chemenu/tests/test_provenance.py - tools/chemenu/tests/test_type_resolver.py - tools/chemenu/tests/test_xref.py - types/concept.md - types/type-spec.md
This commit is contained in:
@@ -0,0 +1,139 @@
|
||||
---
|
||||
type: types/concept.md
|
||||
concept_type: pattern
|
||||
tags: [confidence, scoring, reliability, knowledge-management]
|
||||
created: 2026-07-26
|
||||
modified: 2026-08-29
|
||||
related:
|
||||
- part-of: Memory Lifecycle
|
||||
- exemplifies: LLM Wiki Pattern
|
||||
sources: [Source - LLM Wiki v2]
|
||||
confidence: 0.95
|
||||
confidence_base: 0.95
|
||||
provenance: sourced
|
||||
summary: Mechanismus, der faktischen Aussagen quantitative Werte nach Quellenzahl, Aktualität, Qualität und Bestätigung zuweist, um gut gestütztes Wissen zu erkennen.
|
||||
---
|
||||
# Confidence Scoring
|
||||
|
||||
**Typ:** Pattern (Wissens-Zuverlässigkeitsbeurteilung)
|
||||
|
||||
## Definition
|
||||
|
||||
Confidence Scoring ist ein Mechanismus zur Zuweisung einer **quantitativen Konfidenz-Bewertung** zu jedem faktische Aussage im Wiki, der es dem LLM ermöglicht, zwischen gut gestütztem Wissen und vorläufigen Beobachtungen zu unterscheiden. Dies ist eine Kernkomponente der [[Memory Lifecycle]]-Verwaltung.
|
||||
|
||||
## Kernpunkte
|
||||
|
||||
### Die Scoring-Formel
|
||||
|
||||
Die Konfidenz-Bewertung jedes faktischen Aussage wird berechnet aus:
|
||||
|
||||
| Faktor | Gewichtung | Beschreibung |
|
||||
|--------|--------|-------------|
|
||||
| Basis-Konfidenz | +0.5 | Standard für jeden Aussage aus einer einzigen Quelle |
|
||||
| Quellenanzahl | +0.2 pro Quelle (max +0.6) | Mehr Quellen = höhere Konfidenz |
|
||||
| Aktualität | +0.2 (<30 Tage), +0.1 (<90 Tage) | Aktuelle Bestätigungen erhöhen Konfidenz |
|
||||
| Quellenqualität | +0.1 (Amtliche Dokumente), +0.05 (Reputabel) | Bessere Quellen = höhere Konfidenz |
|
||||
| Bestätigung | +0.1 | Mehrere unabhängige Quellen stimmen überein |
|
||||
| **Maximum** | **1.0** | Vollständige Konfidenz (selten) |
|
||||
|
||||
### Konfidenz-Verfall
|
||||
|
||||
Die Konfidenz **verfällt um 1% pro Monat** seit der letzten Bestätigung, mit einem **Minimum von 0.2**.
|
||||
|
||||
Dies modelliert die natürliche Erosion der Wissenssicherheit im Laufe der Zeit.
|
||||
|
||||
### Konfidenz-Schwellwerte für Sprache
|
||||
|
||||
Bei der Synthese von Antworten sollte der LLM Konfidenz-Bewertungen verwenden, um Aussagen zu qualifizieren:
|
||||
|
||||
- **Konfidenz ≥ 0.6:** Als Tatsache angeben („Projekt X verwendet Redis")
|
||||
- **0.4 ≤ Konfidenz < 0.6:** Versuchsweise Sprache verwenden („möglicherweise", „kann")
|
||||
- **0.2 ≤ Konfidenz < 0.4:** Als unsicher markieren („unsicher", „unbestätigt")
|
||||
- **Konfidenz < 0.2:** Sollte nicht in Antworten verwendet werden
|
||||
|
||||
## Implementierung
|
||||
|
||||
### Zu verfolgbende Metadaten
|
||||
|
||||
Für jede Aussage speichern:
|
||||
```yaml
|
||||
source: [list of source IDs]
|
||||
source_dates: [list of dates]
|
||||
last_confirmed: YYYY-MM-DD
|
||||
confidence: 0.XX
|
||||
quality_flags: [official, reputable, etc.]
|
||||
```
|
||||
|
||||
### Automation
|
||||
|
||||
Confidence Scoring funktioniert am besten mit [[Event-Driven Automation]]:
|
||||
|
||||
- **Bei Quellenaufnahme:** Anfängliche Konfidenz für extrahierte Aussagen berechnen
|
||||
- **Bei Zugriff auf Aussagen:** Konfidenz erhöhen (Verstärkung)
|
||||
- **Bei neuer bestätigender Quelle:** Konfidenz erhöhen, Quellen aktualisieren
|
||||
- **Bei Widerspruch:** [[Supersession]] oder [[Contradiction Resolution]] auslösen
|
||||
- **Nach Zeitplan (monatlich):** Alle Konfidenz-Scores verfallen lassen
|
||||
|
||||
## Beispiele
|
||||
|
||||
Aussage: „Das CI-System verwendet BuildKit auf Port 1234"
|
||||
|
||||
- **Quelle 1:** Interne Dokumentation (Amtlich) - datiert 2026-07-01
|
||||
- **Quelle 2:** Team-Besprechungsnotizen (Reputabel) - datiert 2026-07-15
|
||||
- **Zuletzt bestätigt:** 2026-07-20
|
||||
- **Aktuelles Datum:** 2026-07-26
|
||||
|
||||
Berechnung:
|
||||
- Basis: +0.5
|
||||
- Quellenanzahl (2): +0.4 (begrenzt auf +0.6, also +0.4)
|
||||
- Aktualität: +0.2 (Quelle 2 < 30 Tage)
|
||||
- Quellenqualität: +0.1 (Quelle 1 ist Amtlich)
|
||||
- **Zwischensumme:** 1.2 → **Begrenzt auf 1.0**
|
||||
- Verfall: 6 Tage seit letzter Bestätigung ≈ 0.2% Verfall
|
||||
- **Endgültige Konfidenz:** 0.996 ≈ **0.996**
|
||||
|
||||
Aussage: „Das CI-System verwendet BuildKit auf Port 1234." (als Tatsache angegeben)
|
||||
|
||||
## Vorteile
|
||||
|
||||
- **Transparenz:** Benutzer wissen, wie zuverlässig jeder Aussage ist
|
||||
- **Priorisierung:** Hochkonfidenz-Informationen erscheinen zuerst
|
||||
- **Vertrauen:** Stärkt das Vertrauen der Benutzer in die Wiki-Genauigkeit
|
||||
- **Selbstkorrektur:** Aussagen mit niedriger Konfidenz erhalten Aufmerksamkeit zur Überprüfung
|
||||
|
||||
## Wann zu verwenden
|
||||
|
||||
- Alle faktischen Aussagen im Wiki
|
||||
- Besonders wichtig für:
|
||||
- Technische Spezifikationen
|
||||
- Architekturentscheidungen
|
||||
- Sicherheitsbezogene Informationen
|
||||
- Zeitempfindliches Wissen
|
||||
|
||||
## Wann NICHT zu verwenden
|
||||
|
||||
- Meinungen oder subjektive Aussagen
|
||||
- Definitionen, die sich nicht ändern
|
||||
- Reine deskriptive Metadaten
|
||||
|
||||
## Verwandte Concepts
|
||||
|
||||
- [[Memory Lifecycle]] - Übergeordnetes Konzept
|
||||
- [[Supersession]] - Umgang mit widersprochenen Aussagen
|
||||
- [[Forgetting]] - Komplementärer Mechanismus für alte Aussagen
|
||||
- [[LLM Wiki Pattern]] - Gesamtmuster
|
||||
- [[Agent Memory]] - Produktionsimplementierung
|
||||
- [[Quality Scoring]] - Komplementäre Qualitätsmetriken
|
||||
|
||||
## Siehe auch
|
||||
|
||||
- [[Event-Driven Automation]] (für automatisierte Konfidenz-Updates)
|
||||
- [[Contradiction Resolution]] (für Konfliktbehandlung)
|
||||
- [[Self-Healing]] (für automatisierte Konfidenz-Reparatur)
|
||||
|
||||
<!-- wikitool:links -->
|
||||
## Beziehungen
|
||||
|
||||
- **part-of:** [[Memory Lifecycle]]
|
||||
- **exemplifies:** [[LLM Wiki Pattern]]
|
||||
<!-- /wikitool:links -->
|
||||
Reference in New Issue
Block a user