If you are running local LLMs, your hard drive is likely bleeding gigabytes without you realizing it. Between default model weights, duplicate quantization formats, and forgotten vector embeddings, local AI setups are silent storage hogs.

Here is how you can systematically track down and clean up the clutter directly from your terminal:

• Locate hidden Hugging Face and Ollama model weights: By default, Hugging Face caches everything in ~/.cache/huggingface/hub and Ollama stores models under ~/.ollama/models. Run du -sh ~/.cache/huggingface/ to see how much space is currently locked up.
• Prune redundant quantization formats and unused embedding databases: Review your downloaded models and delete redundant variations (like keeping both Q4_K_M and Q8_0 when you only use one). Clear out stale Chroma, FAISS, or Pinecone local vector database caches residing in your project directories.
• Automate routine garbage collection: Set up a lightweight shell script to periodically check cache growth and alert you before your drive hits capacity.

Fore More Information

I put together the complete, production-ready automated cleanup script along with an interactive storage calculator to help map out your directories.

Direct links to the complete article.

drop a comment below

I have finished a RAG system with a demo account and wanted to ask how to find work, and the rules reference a link to a Discord group for job requests, and the link does not seem to work. Can anyone point me to resources finding clients, or any helpful insight in how to do so? Thank you.

This is the context:

Hi, so I've made a corrective rag pipeline that goes from ingesting documents to retrieving files, reranking, using a LLM model to decide if the documents returned are okay and selecting the ones to use to generate an answer. It is a pipeline to help service desk employees to better answer tickets. It's around 1000 articles of documentation, most of them with 1000 tokens. I've chunk to max of 512 tokens per chunk. There are also some rest full api docs, that show the parameters.

Overall, even though the documentation is lacking and a bit outdated, it manages to retrieve and answer.

For this, I'm using some small models (8b) for answers, due to current constraints; I may be given access to a better model with API if I manage to show why and how this can be an asset.

Some details about why this was done:

The quality of the answer our clients get, depends a lot on who the agent answering is. We have a lot of knowledge gaps and many things that people don't know how to answer. This was a movement to try to address and allow them to give better answers

Some minor details:

I've been given this task even though it's outside my area because I was already looking into it, but I'll have to make a presentation soon. Unfortunately the base was done with IA before I got it and it made some weird choices, some that I couldn't simply take it back without redoing everything

My doubts are (if you guys can give me a direction, may it be scientific articles, docs, wiki... I would appreciate it)

1 How could I ingest the service desk tickets and issues from azure DevOps? Most of them lack a clear answer, so I would like an idea of a standard to propose for the answers, since ingesting now seems like it would make more noise for the retrieval

2 One thing that they want is to use an agent to connect onto the clients DB to analyze problems and situations passed by the client, but I'm unsure of the best way to do it.i would appreciate a direction

3 overall, what is a good strategy to deal with more articles? I feel like the documentation is very similar from one article to another and not in depth enough

4 any other tips to give me, would be appreciated

Thanks and sorry for the long text

Following up other recent posts also about the same issue.

About 1 year ago we established RAG quality monitoring where in particular we tracked content precision and recall using Ragas. But recently we discovered a significant degradation in this, and the root cause was removal of a bunch of docs belonging to another track (don’t ask me why, just corporate work moments).

It was the moment we realised that we need somehow to track data quality itself, and not something we see as in/out. Some metrics that we consider we colud track:

- Staleness - how many of vector embeddings contain outdated information
- Orphaned embeddings - how many of vector embeddings point to data sources that no longer exist
- Deleted-but-retrievable - how many times RAG returns vector embeddings that we actually (we think) deleted from RAG

I see it as kinda scheduled job that does this assessment although it needs quite some time to write and test it. Any advice on what we can use?

Yesterday I nearly declared a production retrieval emergency that didn't exist, and the mechanism is general enough that anyone running hybrid search should check for it.

**Setup:** hybrid retrieval over personal memory — vector similarity + BM25, fused with Reciprocal Rank Fusion, optional cross-encoder rerank on top for some tiers. Every search logs `top_score` for quality monitoring.

**The scare:** analyzing 10,706 logged searches, I applied the obvious threshold — top_score < 0.3 = weak retrieval. Result: 62% "failures," a dozen users at "100% failure with avg score 0.017," and a terrifying month-over-month "degradation" trend. One of the "100% failed" users was a paying customer with a thousand searches. I was halfway into incident mode.

**The tell:** a search for an exact entity name — a guaranteed hit — logged top_score 0.0426. And those "failing" users all averaged 0.016–0.021. Then it clicked: RRF scores are 1/(k + rank) with the standard k=60. Top rank = 1/60 ≈ 0.0167. My "catastrophic" users weren't failing — **their top result was rank-1 almost every time.** avg 0.017 is what perfect RRF retrieval looks like.

What actually happened: requests that go through the reranker log cosine-style scores (0–1 scale, 0.3+ = good). Requests on the raw RRF path log fusion scores (0.016–0.05 scale, where 0.017 = excellent). Both landed in the same `top_score` column with no scale tag. Every aggregate over that column — means, z-scores, my failure thresholds, even the health monitoring cron — was averaging apples with orbital velocities. The "month-over-month degradation" was just the RRF-path share growing as more traffic moved to hybrid.

**What survived scale-correction:** true failure (zero results) was 9–13%, driven mostly by two accounts whose agents were querying literally empty stores — a real integration problem, but a completely different one than "retrieval is broken."

**Lessons, generalized:**

1. **A fused ranking score is not a similarity.** RRF outputs rank information, not confidence. The moment you fuse, your score's absolute value stops meaning what your dashboards think it means.
2. **Never store scores from different scoring regimes in one unlabeled column.** Log a `score_kind` (or a scale-aware quality label computed at write time, which is what we shipped: strong/weak/no_match with per-scale bands). Analysis-time guessing is how you get 3am false incidents.
3. **The only scale-free failure signal is emptiness.** Zero results means the same thing on every path. When in doubt, count zeros, not thresholds.
4. **Validate your alarm against a known-good query before believing it.** One exact-match search that "scored 0.04" saved me from paging myself.

Sources for the RRF math: Cormack, Clarke & Buettcher (2009), "Reciprocal Rank Fusion outperforms Condorcet and individual rank learning methods" — the k=60 default everyone inherits comes from there.

Disclosure per rule 3: the production system is Mengram (mengram.io), a memory layer for AI agents — but the trap applies to any RAG stack mixing rerankers with fusion scoring. Nothing here requires my product to check: grep your score column and look for a bimodal cluster around 1/60.