Under the hood: RAG orchestration, LLM inference, vector databases, and institutional deployment patterns.
Retrieval-Augmented Generation (RAG) is the core of Nursing Copilot. When a clinician asks a question, the system retrieves relevant institutional knowledge and grounds the LLM response in documented policy.
Convert user query to dense vector using BGE-M3 multilingual encoder. Supports clinical terminology, acronyms, and institutional jargon.
Search Weaviate vector database with approximate nearest neighbor (ANN) algorithms. Retrieves top-k candidates from institutional knowledge base.
BGE-Reranker-v2-M3 scores and orders results by relevance. Filters out marginal matches. Ensures highest-confidence sources appear first.
Haystack orchestration engine builds final prompt with system instructions, retrieved context, citation metadata, and user query.
Quantized local LLM (Qwen3.6-35B MoE) generates response grounded in retrieved context. Temperature-controlled for consistency.
Response includes direct citations: document title, version, SOP code, page number. Every claim is traceable back to policy.
| Layer | Component | Purpose | Configuration |
|---|---|---|---|
| Orchestration | Haystack | RAG pipeline orchestration, workflow management, retrieval optimization | Multi-stage retrieval with dynamic component routing, hybrid search |
| Vector DB | Weaviate | Semantic search over institutional knowledge base | HNSW ANN indexing, multi-collection design for specialties |
| Embeddings | BGE-M3 | Convert text to dense vectors for semantic search | Multilingual, dense+sparse hybrid mode for accuracy |
| Reranker | BGE-Reranker-v2-M3 | Score and rank retrieved candidates by relevance | Fine-tuned on medical/clinical text pairs |
| LLM Inference | Xinference / vLLM / llama.cpp | Model serving with quantization support | Q3_K_XL / AWQ / GGUF formats, batch inference |
| LLM | Qwen3.6-35B-A3B (MoE) | Main generative model for clinical decision support | Temperature=0.2 (consistent), seed control for reproducibility |
| Backend | Express.js + Node.js | REST API layer, session management, middleware | Async handlers, error recovery, request logging |
| ORM | Prisma | Type-safe database access, schema migrations | Auto-generated query builders, relationship handling |
| Database | PostgreSQL | Structured data (users, sessions, audit logs, KPIs) | ACID compliance, row-level security, backups |
| Frontend | React + TypeScript | User-facing interface, real-time chat, dashboards | Dark mode, responsive design, accessibility (WCAG 2.1) |
Select your available GPU hardware below to see recommended models and configurations:
Air-Gapped Infrastructure: Nursing Copilot runs entirely on institutional hardware. No cloud dependencies. No patient data leaves the institution. HIPAA-compliant by design.
Institutional email login, MFA support, Microsoft Entra ID integration (SSO ready).
Automated daily backups, systemd timer-based scheduling, manual restore capabilities, technical admin controls.
Every query logged with timestamp, user, query text, model used, citations generated. Full compliance trail.
PowerShell deployment to institutional Windows nursing stations. Group Policy-managed auto-updates.
Switch between Qwen2.5-3B, Qwen3-8B, Qwen3-14B, Qwen3.6-27B without downtime. Per-quantization testing.
Architecture designed for multi-node Kubernetes (future). Currently optimized for single-node institutional servers.
Haystack pipeline orchestration enables multi-stage retrieval architecture:
Note: Citation sparsity remediation in progress. Testing indicates ~10 of thousands of indexed policies currently being cited. Planned improvements: raising global top_k to ~8, parallel retrieval nodes per specialty, and fine-tuning BGE-M3 on institutional terminology.
Percentage of LLM responses with relevant, traceable citations from institutional knowledge base.
Active usage tracking per department, specialty, shift. Adoption metrics by clinical unit.
How many institutional SOPs/protocols are actively being retrieved and used in real queries.
Query-to-answer latency (target: <3 seconds). Includes embedding, retrieval, reranking, generation.
Thumbs-up/down ratings on responses. Feedback collection for continuous model improvement.
Estimated minutes saved per query vs manual policy lookup. Aggregate institutional time savings.
Executive dashboards provide real-time visibility into system health, AI quality, safety metrics, usage patterns, and compliance status. Role-based access controls ensure appropriate visibility by user tier.
Generate 1,000–10,000 specialized oncology-nursing QA pairs from reference books and institutional SOPs. Fine-tune Qwen via LoRA.
Test Q4 quantization of Qwen3.6-35B MoE for improved performance on modest hardware. Evaluate inference speedup vs quality trade-off.
Decouple from GitHub Actions. Full self-hosted CI infrastructure independent of GitHub cloud (GitHub = storage/backup only).
Click-through from citations to open source PDFs at the exact page/section. Integration with Odoo for institutional document URLs.
Destructive operation with MFA, typed confirmation, dry-run preview, integrity checks, pre-restore snapshot, full audit trail.
Evaluate i18n effort and retrieval/generation quality in Arabic. Knowledge base currently English-only; multilingual generation only.
See how these components work together in a live institution. Schedule a technical deep-dive with our team.