Skip to content

LIMITATIONS.md

Latest commit

 

History

History
41 lines (27 loc) · 4.12 KB

LIMITATIONS.md

File metadata and controls

41 lines (27 loc) · 4.12 KB
Raw

Limitations & Non-Goals

MisakaNet is designed for a specific niche: decentralized, git-backed failure-memory sharing for AI agents. It is not a general-purpose solution. This document honestly describes what the system does not do well.

Update (2026-09): MisakaNet now ships two surfaces over one knowledge core — a local stdio MCP / CLI (zero-dependency BM25 over the git checkout) and a remote HTTP MCP (https://misakanet.org/mcp, Cloudflare Worker + D1, anonymous search). Items below marked "local" apply to the local core; the remote surface mitigates some (not all) of them.

Search

  • BM25 is keyword-based. It cannot understand semantic similarity, paraphrase intent, or conceptual relationships. If a lesson uses different terminology than the search query, it will not be found. This is a known limitation of stdlib-only retrieval. An optional --semantic flag exists (see misakanet/search/embeddings.py) but requires an external embedding service; it is not the default.
  • No embedding model in the core. We intentionally avoid vector embeddings to keep the core zero-dep. For semantic search, integrate an external embedding service at the node level.
  • RRF fusion is heuristic. Reciprocal Rank Fusion improves multi-query results but has no theoretical optimality guarantee. Tuning may be needed for your domain.

Scale

  • Git is not a database (local). The local search_knowledge.py tool loads all lessons into memory. With >50,000 lessons, startup time and memory usage become non-trivial. We recommend archiving older lessons to a separate repository at that scale. The remote HTTP surface reads from D1 instead, avoiding this local-load cost for API consumers.
  • Concurrent writes. Git merge conflicts are possible when multiple nodes push simultaneously. The CI pipeline handles simple cases automatically, but complex conflicts may require manual resolution. The remote submit_intake path writes through the GitHub Issues API, not direct git push.
  • No real-time sync (local). The local core is fundamentally a batch-sync system: lessons pushed by one node are not visible to others until they git pull. This is by design — offline-first, no daemon. The remote D1 service is kept in sync by CI (sync-d1.yml, cron + push triggers) on the order of minutes.

Content Quality

  • Garbage in, garbage out. Lessons are community-contributed. Despite CI checks for dangerous patterns (see SECURITY.md), we cannot guarantee factual accuracy of every lesson. Always verify before executing retrieved commands.
  • No automated fact-checking. The CI pipeline validates format, DCO, and dangerous patterns, but not semantic correctness. Misinformation is possible.
  • Subjectivity in scoring. Quality Score is a heuristic based on format, DCO compliance, and audit results. It does not measure lesson usefulness or correctness. Evidence levels (E0–E4) and me_events reuse signals are the main corrective signals we do track.

Ecosystem

  • No plugin system in the core repo. MisakaNet is not a host runtime; integration happens at the node level. It is, however, itself installable as a DSH plugin (dsh plugin add misakanet, npm misakanet@2.23.0) and discoverable via MCP registries (Glama, Smithery, dsh-plugin.org).
  • No general-purpose SaaS. There is no multi-tenant MisakaNet service; the public misakanet.org/mcp endpoint is the project's own deployment of the same open-source worker.
  • Small community. As of 2026, MisakaNet is an early-stage project. Response times for issues and PRs may vary.

Non-Goals

  • MisakaNet is not a replacement for dedicated knowledge bases (Confluence, Notion, GitBook).
  • MisakaNet is not a real-time collaboration platform.
  • MisakaNet is not a vector database or embeddings service.
  • MisakaNet is not a substitute for proper documentation.
  • MisakaNet is not a content moderation platform.

Philosophy

By being honest about what we cannot do, we build trust with those who evaluate the project. Every claimed capability is demonstrable; every limitation is disclosed. This is the opposite of a hype-driven roadmap.