Field notes · ai
Building private AI search for a Windows file share
My on-prem file-share search pilot raised recall at 5 from 0.70 to 0.97 by combining dense and sparse retrieval, ACL trimming, and citations in tests.
I wanted staff to ask ordinary questions of a Windows file share without sending municipal documents to a hosted model or exposing files they could not already open. The result is an on-premises retrieval system that extracts documents, indexes chunks, filters every search by the caller’s directory groups, and cites the source files used for each answer.
The hardest part was not running a language model locally. It was making the retrieval path accurate, permission-aware, current, and honest about what it could not find.
Key takeaways
- A local model does not make a search system private by itself.
- Apply file permissions before retrieved text reaches the model.
- Combine semantic and keyword retrieval for mixed office documents.
- Require citations and a refusal when the evidence is missing.
- Measure retrieval separately from answer quality.
- Keep a visible report of files the index could not read.
What does private AI search mean in this project?
Private means document text, embeddings, retrieval, and generation stay on systems I control, while each query is trimmed to content the caller is allowed to read. It also means queries and cited sources are auditable, failures do not fall open, and unsupported answers are refused rather than improvised.
That definition has four boundaries:
- Data location: source documents and derived chunks remain on premises.
- Identity: the server resolves the caller rather than trusting a username or group list supplied by the browser.
- Authorization: retrieval filters chunks by the caller’s effective group security identifiers before prompt construction.
- Accountability: answers cite documents, and the service records which sources were presented to the model.
The word private is often reduced to “the model runs locally.” That is only the first boundary. A local model can still disclose another department’s document if the retrieval service gives it the wrong context. It can still produce an uncited answer. It can still retain sensitive paths in an error log.
I approached this with the same principle I use for read-only access to a vendor database: the useful capability should be narrower than the underlying data access, and the restriction should be enforced technically rather than remembered by the operator.
How does the file-share search pipeline work?
The pipeline crawls files, extracts structured text, divides it into bounded chunks, creates dense and sparse representations, stores metadata with each chunk, retrieves an authorized set, and asks the local model to answer only from that context. Citations map the answer back to copyable source paths.
Windows shares
-> crawl and ACL capture
-> format-aware extraction and triage
-> chunking with source metadata
-> dense embeddings plus sparse terms
-> vector store with permission payloads
-> caller identity and group expansion
-> permission-filtered hybrid retrieval
-> grounded answer with citations or refusal
Each chunk carries more than text. It includes a canonical source path, chunk number, file modification time, content hash, source share, and the allow-list of directory security identifiers captured from the file’s access control list. That metadata is what makes freshness, deduplication, authorization, and citations possible later.
The embedding service converts passages and questions into vectors. Ollama’s official embeddings documentation describes the basic query and document embedding pattern. I also create a sparse representation so exact terms, codes, and names do not have to win through semantic similarity alone.
Why did semantic search alone underperform?
Semantic search was good at paraphrased questions, but it was weaker on exact identifiers and repeated administrative language. Adding a sparse retrieval arm, removing duplicate content, and fusing the ranked results improved recall at 5 from 0.70 to 0.97 on my controlled pilot question set.
That number is not a universal benchmark. It reflects one real collection and a fixed set of questions whose relevant source documents I reviewed by hand. It does show why I would not replace filename or keyword search with embeddings.
The search modes solve different problems:
| Search method | Strong at | Weak at |
|---|---|---|
| Filename search | Known document names and paths | Questions expressed differently from filenames |
| Full-text or sparse search | Exact names, codes, and phrases | Paraphrases and conceptual similarity |
| Dense vector search | Natural-language questions and related meaning | Rare identifiers and repeated boilerplate |
| Hybrid retrieval | Balancing meaning with exact terms | More tuning, metadata, and evaluation work |
I retrieve candidates from both dense and sparse indexes, apply the same permission filter to both arms, fuse their rankings, and optionally rerank the small combined set. The language model sees only the final authorized chunks.
Qdrant’s filtering documentation
explains how payload conditions can be combined with retrieval. Its Match Any
condition is useful here because a chunk is eligible when at least one stored
allow SID appears in the caller’s transitive SID set.
How are Windows file permissions enforced?
The crawler captures an allow-list of security identifiers from NTFS access control data and stores it with every indexed chunk. At query time, the server resolves the authenticated user’s own SID and transitive group SIDs, then asks the vector store to return only chunks whose allow-list intersects that set.
Microsoft documents the Active Directory tokenGroups attribute as a computed list of SIDs produced by transitive group-membership expansion. That is the shape of identity information the filter needs. The browser never gets to assert its own group membership.
The authorization rule is simple:
allow chunk if intersection(chunk.allow_sids, caller.sids) is not empty
The failure behavior is more important than the expression. If an ACL cannot be read, a deny rule appears, or the crawler cannot produce a safe allow-list, the document is not indexed. If a returned chunk ever fails a second application-side intersection check, the query stops rather than generating an answer.
I tested the boundary with separate simulated identities and documents assigned to different groups. Authorized questions retrieved the intended material; cross-department questions returned no protected chunks. Those tests gave me a security claim about the cases exercised, not a guarantee that every possible NTFS arrangement is understood.
Nested groups, inheritance, explicit denies, share permissions, and stale tokens all complicate the phrase “what the user can read.” A production design needs a documented policy for each one and test fixtures that fail closed.
How do citations and refusals keep answers grounded?
Every retrieved chunk receives a source number in the prompt, and the model is
instructed to answer only from that context using inline references such as
[1]. If the supplied text does not answer the question, the expected behavior
is an explicit refusal, not a plausible continuation.
The response layer validates citation numbers against the actual retrieved source list. It then adds a deduplicated source appendix for only the documents the answer cited. This gives the reader a path back to the record and gives me a way to inspect whether a fluent answer rested on the right evidence.
I measure retrieval and generation separately:
- Recall at 5: Did the top five chunks include the known relevant source?
- Answer correctness: Did a reviewer judge the response accurate against the source?
- Citation validity: Did references point to chunks actually retrieved?
- Refusal behavior: Did unanswerable questions stop cleanly?
- Authorization: Did each identity retrieve only its allowed corpus?
In the same controlled pilot, answer correctness improved from 2.87 to 4.30 on a five-point rubric after deduplication and hybrid retrieval. The tested responses had no invented citation numbers, and the refusal tests passed. Those results were good enough to justify the next stage, not good enough to eliminate review.
What went wrong in the first full crawl?
The first crawl taught me that a document collection is not a folder full of clean PDFs. It contained legacy Office files, scanned pages, empty templates, duplicate copies, temporary lock files, table-heavy documents, oversized files, and files whose permissions could not be interpreted safely.
The crawler therefore produces a verdict for every candidate:
| Verdict | Meaning | Action |
|---|---|---|
| Clean | Useful text extracted normally | Index it |
| Messy | Text extracted with limitations | Index or review by policy |
| Unreadable | No useful text was recovered | Report it, do not index |
| Unsupported | No extractor is configured | Report it |
| Oversize | File exceeded the configured extraction cap | Report or route separately |
| Error | Extraction failed unexpectedly | Retry or investigate |
That report is part of the product. Without it, “the index contains the file share” is an uncheckable claim. A later article in this cluster covers the Office-format and table problems in detail. The lesson is similar to decompiling a vendor application to verify its real behavior: inspect what the system actually stores and returns before designing around an assumption.
Would I deploy this as a replacement for file search?
No. I would deploy it as another route to documents, with clear citations and a visible coverage report. Filename search remains faster when the user knows what the file is called, and ordinary full-text search remains excellent for exact phrases.
The AI layer earns its place when the question and the document use different language, when the answer spans several passages, or when staff do not know which department created the record. Even then, the cited document remains the authority.
I also would not describe this as a complete enterprise search or governance platform. The pilot has a specific corpus, identity path, extraction policy, evaluation set, and operational owner. Records retention, legal holds, source system permissions, and broader identity governance remain separate concerns.
Frequently asked questions
Private file-share search is an authorization and data-quality project wrapped around an AI model, not an AI model with a shared folder attached.
Does the model need internet access?
No. Embedding, retrieval, and generation can run on local infrastructure. The larger privacy question is whether document text, telemetry, updates, or user queries leave that boundary through any other component.
Why store permissions on every chunk?
Retrieval operates on chunks, so the authorization metadata must travel with the retrievable unit. Filtering only after generation would expose unauthorized text to the model before the application could hide it.
Can a user trust the generated answer instead of opening the file?
The answer is a navigation and synthesis aid. For policy, legal, financial, or records decisions, the cited source remains the authoritative record.