archex turns a repo into a ranked, token-budgeted context bundle for coding agents instead of letting them grep their way through it. BM25F + local embeddings + graph expansion for imports/types/callers, fully deterministic — same query, same index revision, same bundle, every time. No hosted inference, no API key, no telemetry in the core path.

Measured against cocoindex-code and Graphify on the same 19-task external-repo set (self-run, checked into the repo, reproducible with archex benchmark headtohead report): required-file recall 0.95 (archex) vs 0.32 (cocoindex-code) vs 0.70 (Graphify); completion-penalty tokens 922 vs 11,188 vs n/a (Graphify measures a different lane); cold-start 0ms vs 4.7s vs 937ms. Full table and methodology: docs/ARCHEX_VS_COCOINDEX.md.

26 languages across full/structured/chunk-only tiers, MCP server with 17 tools, CLI, Python API, Docker. Solo project, 3,619 tests, 91.1% coverage. Demo attached.

github.com/Mathews-Tom/archex

Star it if it's useful, open an issue if a language or workflow is missing, and pass it to anyone else fighting grep-and-hope context.

Tweet : https://xcancel.com/jun_song/status/2079914426334167258#m

Looks like 8GB VRAM could do more like even run 70-100B MOE models possibly.

^{Sorry about the clickbait title, I want more eyes on this..... zzz}

If many people have noticed that there's no reasoning phase in Laguna S 2.1. I noticed the Poolside development team updated the chat template twice in the last 24 hours. There was a bug where, if preserve_thinking was disabled, reasoning wouldn't start at all.

However, I don't think that's the root cause. Could you take a look at the Qwen 27B chat template? It enables reasoning correctly for the Laguna S 2.1 model when used with the --chat-template-file parameter in llama.cpp.

I hope this information helps you track down and fix the template issue.

https://huggingface.co/poolside/Laguna-S-2.1/blob/main/chat_template.jinja

https://huggingface.co/Qwen/Qwen3.6-27B/blob/main/chat_template.jinja

Poolside have updated the full precision, and FP8 versions with a fix for the looping issue many of us have been seeing. Other variants incoming.

Discussion - https://huggingface.co/poolside/Laguna-S-2.1-FP8/discussions/1

Tested Laguna S 2.1 (118B MoE with 8B active parameters) by Poolside on frontend and backend coding tasks and the results are nowhere near what ~120B model should deliver. Even with OpenCode harness the model repeatedly stopped during generation and haven't completed the task(s) until prompted multiple times.

Watch more here https://www.youtube.com/watch?v=UCdYlJaRCxk

Hello, I recently forked Fish Audio’s S2 and tried to optimise it for local runs. I got it from about 1.2 tokens/second to around 12.4 t/s at bf16 and 23 t/s at int8 (I added this quantisation to the inference engine). I’m looking for others on Apple Silicon to test it out and try reproduce my benchmarks.

The details of how I actually squeezed out this performance is in the repo. Long story short it’s mostly the KV cache (reuse, GQA expansion, and an int8 kernel for MPS). Some other optimisations like torch’s compile for inductor-metal, sub batch streaming.

The benchmarks in the repo are on my own M5 Pro 48Gb, so if anyone has different Apple Silicon chips I’d love to see how they run on other machines.

Thanks!

https://x.com/Badtheorylabs/status/2079306502897074249

A 27B open-weight agent model built for agentic coding, structural tool use . The complete thing fits in one 8.39GB file under 2.5 bits per parameter smaller than an 8B model in fp16, and retains 92.2% of the 27B intelligence.

BTL-3 is trained for the loop real agents live in: reason, act, inspect the result, recover, continue. It handles single, sequential, and parallel tool calls and knows when the right move is no tool call at all.

HumanEval: 95.12% pass@1

BFCL v4 AST: 88.5% (full 1,240-case set)

Multiple tool calls: 95.5%

Tool-call abstention: 91.2%

262K context architecture

Two editions, both open today.

BTL-3 is the maximum-quality checkpoint, for Transformers and vLLM.

BTL-3 Compact is the entire model in one standalone 8.39GB GGUF. No base download. No reconstruction. One file, one command, a running agent

Compressing 27B this far normally destroys a model. Standard quantization couldn't do it, so we built the stack ourselves: packed AVQ2 decoder tensors, affine INT4, measured precision islands, packed vocabulary matrices, rank-32 output correction, behavioral repair. 2,416 tensors byte-verified at export.

Then we tested whether the agent survived. On a fresh sealed 100-turn tool-contract gate, Compact retained 92.2% of teacher-correct behavior 100% on single, parallel, sequential, and abstention calls.

43 tok/s generation on an RTX PRO 6000. Fully local. Nothing leaves your machine.

BTL-3: https://huggingface.co/badtheorylabs/

BTL-3

Compact: https://huggingface.co/badtheorylabs/

BTL-3-Compact

Runtime + source: https://github.com/Badtheorylabs/

BTL-3

Apache-2.0 model. MIT runtime.

Today the Department of Energy (DOE) and Arcee AI announced the development of Genesis-Science-1 (GS1), an open model for scientific research. This is a joint effort to bring advanced AI into scientific research across a wide range of fields.

GS1 is an American open-weight model for scientific research, built together with the DOE and its national laboratories through the Genesis Mission. Arcee has secured the compute, will handle training and post-training, build the scientific workbenches and the system around the model, and prepare it for release. DOE scientists will shape which problems are worth solving, provide the data and environments the model learns from, and be the true test as to whether its work holds up under scrutiny. GS1 will be a trillion-parameter-class language model paired with a governed execution system for long, difficult scientific work, released openly later this year with the weights, a technical report, and public demonstrations. GS1 is built on top of our next generation of Trinity models.

The case for American open models

Just a year ago, Arcee made a decision that was difficult to defend. We began training our own open models from scratch, in the United States, when the faster and cheaper path pointed elsewhere. Strong open models were already available to download, and the reasonable move was to take one, adapt it, and build from there. We understood that case. It’s how we’d been operating before, after all. We went ahead anyway, because we kept seeing the need.

Some institutions can't treat a model as a service. A bank, a hospital, a university, or a national laboratory may need to keep a version stable for years, hold it to their own standards, retrain it for a narrow field, and run it on their own systems without sending sensitive data anywhere. For them, a model is more than its benchmark scores. It's the weights, the training history, the license, the certainty that it will perform reliably indefinitely, and the supply chain behind it, all the way down.

We built the Trinity models to serve institutions like these. When the Genesis Mission came along, it fit our ethos.

We have real admiration for the open-model labs in China. DeepSeek, Qwen, Kimi, MiniMax and GLM have built excellent models that people rely on, and they kept sharing open weights when much of the field was moving the other way. They earned their standing. Yet their work also showed how few capable open models were being made in the United States. For an institution handling sensitive work, capability is only part of the question. It also matters who trained a model, where, under what license, and which country's laws sit behind the company that made it. Those are fair questions, and a leaderboard doesn't answer them.

We think the right response is to build more capable open models here at home, so that institutions who need them have somewhere to turn. Closed American systems will stay valuable, and many are superb. What they can't offer a national laboratory is a model it holds in its own hands, free to preserve, adapt, and run on its own terms. We wanted American science to have that option too.

• Blog Post : https://www.arcee.ai/blog/genesis-science-1
• Press Release : https://www.arcee.ai/science-1

Hey r/LocalLLM,

Built Project Zero — a from-scratch CPU-only LLM inference engine in pure C99. It beats bitnet.cpp by 1.8× on the same hardware. We also fully support Qwen Bonsai-27B on CPU, and we are looking for the community's help to get x86 CPU benchmark data on the board for both models.

What it is

Single binary, zero external dependencies — no Python, no CUDA, no ONNX, no PyTorch. GCC + make + CPU.

Supports: - Microsoft BitNet b1.58-2B-4T — ternary weights ({−1, 0, +1}), 1.18 GB binary, full REPL + agentic loop - Qwen Bonsai-27B — reads GGUF directly (e.g. Ternary-Bonsai-27B-Q2_0.gguf), memory-safe mmap architecture meaning it never OOMs even on constrained RAM setups.

BitNet performance — the good part

We're sitting at ~95% of the theoretical DRAM bandwidth ceiling on the Xeon. There's essentially nothing left to squeeze out of BitNet on that box.

How the speedup happens: BitNet weights are ternary packed 4/byte. Instead of unpacking → float → FMA, we use a 3-instruction VBMI kernel (vpermi2b + vpternlogd + vpaddb) feeding directly into INT8 VNNI accumulation (vpdpbusds). The thread pool is C11 atomics spin-then-sleep to eliminate futex syscalls.

The Community Challenge: BitNet & Bonsai Benchmarks

We've only benchmarked BitNet on 2 machines so far. We need to see if the fallback ternary kernels still provide a speedup on older CPU architectures, and map out the memory bandwidth ceiling on server hardware.

Furthermore, PrismML is actively looking for community benchmark numbers for Bonsai-27B. Right now, every single entry on their leaderboard is GPU-based (CUDA/Metal/MLX). Zero CPU-only x86 entries exist. We want to change that. Because Project Zero uses a zero-copy mmap architecture, you can run Bonsai-27B on severely constrained hardware without crashing.

If you have an older AVX2 chip, or a high-core Xeon/EPYC, we want to know what token rates you get for either model.

How to test & benchmark

1. Clone and build: bash git clone https://github.com/shifulegend/project-zero.git cd project-zero make demo

2. Run BitNet or Bonsai-27B: ```bash

   For BitNet (b1.58-2B-4T):

   ./adaptive_ai_engine --model models/bitnet-b1.58-2B-4T.bin --tokenizer models/bitnet-b1.58-2B-4T_tokenizer_proper.bin --threads 4

For Qwen Bonsai-27B (GGUF):

./adaptive_ai_engine --model models/Ternary-Bonsai-27B-Q2_0.gguf --threads 4 ```

Where to post results: You can post your results right here in this thread, or drop them in Discussion #3 on the repo.

Repo: https://github.com/shifulegend/project-zero

Happy to answer questions about the ternary kernel design, the AVX-512 VNNI dispatch, the DRAM bottleneck, or why we focused on Bonsai-27B!

Simply prompting an LLM to look at the text and images associated with a post and classify it as AI-generated or not is too inaccurate. For text, previous benchmarks have shown that zero-shot prompting methods like this are barely better than random, especially for smaller models. While filtering for "AI written" may catch some posts with telltale markers of AI writing (em-dashes, "it's not x, it's y", etc.), for the most part this method fails and is prone to false positives. Similarly, an LLM cannot reliably tell whether an image is AI-generated just by looking at it.

Reliable detection requires models trained specifically for the task. To address this, we've built dedicated AI detection models for both text and images. Simply go into Bouncer and click “Remove AI Slop” to start filtering.

https://imbue.com/blog/bouncer-leveraging-local-compute-to-detect-ai-slop

Models: (Check Model cards for so much sample demo images)

• https://huggingface.co/microsoft/Mage-Flow
• https://huggingface.co/microsoft/Mage-Flow-Turbo
• https://huggingface.co/microsoft/Mage-Flow-Edit

Mage-Flow is a compact 4B-scale generative stack for efficient text-to-image generation and instruction-based image editing. Instead of scaling to tens of billions of parameters, Mage-Flow reaches state-of-the-art-competitive quality through careful tokenizer–backbone–system co-design, so it stays fast, memory-light, and easy to fine-tune under realistic compute budgets.

The stack is built from two shared, co-designed components:

• Mage-VAE — a lightweight, high-fidelity latent tokenizer (one-step diffusion encode/decode with anchor-latent KL regularization).
• NR-MMDiT — a shared 4B Native-Resolution Multimodal Diffusion Transformer, trained with rectified flow matching in the Mage-VAE latent space.

Together with native-resolution packing and a fused-kernel training infrastructure, this shared stack powers two model instantiations: Mage-Flow for text-to-image generation and Mage-Flow-Edit for instruction-based image editing. Each ships in Base, RL-aligned, and 4-step Turbo variants.

✨ Highlights

• Compact & competitive. A single 4B family for generation and editing that matches or beats much larger open systems (Qwen-Image 20B, Z-Image 6B, FLUX.2 32B, FireRed-Image-Edit 20B).
• Efficient tokenizer. Mage-VAE matches FLUX.2-VAE reconstruction fidelity while using ~12× / ~22× fewer encode / decode MACs per pixel, removing the VAE as the high-resolution bottleneck.
• Native resolution. One checkpoint generates from 512 to 2048 on any aspect ratio, including extreme 4:1 (e.g. 512×2048, 2048×512).
• System-level speed. Native-resolution packing (FlashAttention var-len + per-sample 2D RoPE) + fused CUDA kernels cut per-step training time from ~1.93 s → ~0.78 s (~2.5× faster training); CFG's conditional/unconditional branches run in one packed forward.
• Full family. Base, RL-aligned, and 4-step Turbo variants for both generation and editing.
• Versatile editing. Mage-Flow-Edit supports semantic content editing, appearance transformation, image restoration, and structure-aware outputs within a unified image-and-text-conditioned model. See the report's editing galleries.
• Interactive latency. At 1024² on a single A100: Mage-Flow-Turbo 0.59 s/image, Mage-Flow-Edit-Turbo 1.02 s/edit, peak memory ~18–20 GB (lowest among compared systems).

Fara1.5-27B is a multimodal computer use agent (CUA) for web browsers, from Microsoft Research AI Frontiers. It observes the browser through screenshots and acts on the user's behalf by emitting structured tool calls — click, type, scroll, visit URL, web search, and so on — to complete tasks end-to-end.

The model is vision-only at perception time: it sees the browser through screenshots, not the DOM or accessibility tree. Internal reasoning and trajectory history are tracked as text. Given the latest screenshot and prior actions, it predicts the next action with grounded arguments (e.g., pixel coordinates for a click).

Fara1.5-27B is supervised fine-tuned from Qwen3.5-27B on data generated by FaraGen1.5, our multi-agent pipeline that synthesizes web tasks, executes trajectories to solve them, and verifies the results before training.

It's co-designed with MagenticLite, and that's the recommended deployment for both research and production.

Primary use cases

Automating repetitive web tasks: filling forms, shopping, booking travel, restaurant reservations, information seeking, account workflows. Fara1.5-27B can also serve as a grounding model for other agents that need pixel-accurate action prediction.

Out of scope

• Languages other than English (training data is English-only)
• High-stakes domains (legal, health, financial advice) where inaccurate actions could cause harm
• Allocation decisions affecting legal status, housing, employment, or credit
• Unsandboxed deployments with access to sensitive accounts or files
• Commercial or real-world production use without additional testing and safeguards

Known limitations

• Vision-only perception means the model can be misled by deceptive or low-quality page rendering, prompt injections embedded in page content, or visual ambiguity in UI elements
• Multi-step trajectories accumulate error — a misclick early in a sequence can compound
• Run-to-run variance on multi-turn tasks is non-trivial; benchmark numbers are averaged over multiple runs
• The model can hallucinate page state or misattribute information from earlier screenshots

Additional Models: (I don't see 9B model on HF even though model cards mentions 9B, Added below)

• https://huggingface.co/microsoft/Fara1.5-4B
• https://huggingface.co/microsoft/Fara1.5-9B

Hey HN, Henry & Roman here from Cactus.

A small, on-device model is fast and private, but sometimes wrong, but frontier models are getting expensive pretty fast. So, we post-trained Gemma 4 E2B post-trained to know when it's wrong. Every response comes with a confidence score between 0 and 1. Developers can accept the on-device when it's high, hand off to a bigger cloud model when it's low. By routing only 15-35% of queries to Gemini 3.1 Flash-Lite, Gemma-4-E2B matches Gemini 3.1 Flash-Lite on most benchmarks.

- ChartQA: 15-20%

- LibriSpeech: 25-30%

- MMBench, GigaSpeech, MMAU: 30-35%

- MMLU-Pro: 45-55%

We were always frustrated by the routing signals hybrid apps rely on: asking the model to rate itself in text (unreliable, and you're parsing prose), or token entropy heuristics (barely better than a coin flip in our tests). So we did mechanistic studies on small models, Gemma 4 particularly, and found the hidden state for different layers carry meaningful self-awareness signal for various situations.

SO we extended the model with a 68k params probe layer (LayerNorm, low-rank projection, attention pooling, small MLP head) reads one intermediate layer during decoding and predicts p(wrong); confidence = 1 - p(wrong), returned as structured data, never parsed out of the answer text.

Across 12 hold-out benchmarks spanning text, vision and audio, the probe averages 0.814 AUROC vs 0.549 for token entropy. The result that convinced us this is real: the probe was trained on zero audio data, yet scores 0.79-0.88 AUROC on four audio benchmarks where entropy is near-random or worse (0.32-0.52). It's reading a modality-independent correctness signal from the hidden state, not memorizing patterns from its training data.

We published all weights on HuggingFace and provide copy-pase codes to run it on Transformers, MLX, Llama.cpp or Cactus. With Ollama, vLLM, SGLang etc in the works. For llama.cpp we ship a patch series you compile in once (upstreaming is planned). The code is MIT licensed; Gemma model use remains subject to the Gemma terms.

GitHub: https://github.com/cactus-compute/cactus-hybrid

Weights: https://huggingface.co/collections/Cactus-Compute/cactus-hyb...

Some caveats:

- The probe scores single-sequence decoding only, up to the first 1024 generated tokens.

- Handoff works best when routing per task in a multi-step process, not per step.

- Hierarchical routing is still in the works: try on-device, then DeepSeek v4 Flash, before Fable/GPT5.5/Gemini/Muse/Grok.

- The technique is boutique for each model, we will share each weights as they roll out.

These issues are currently being tackled at Cactus and updated weights will be shipped directly into the HuggingFace collection and GitHub repository straight up. Please let us know your thoughts, it helps us find ways to improve the design progressively.

Thanks a million!

So if OpenAI can breach HF, why can't we breach them too? I propose on Kimi K3 (or Qwen 3.8 Max) drop, we abliterate it, get any three (or more) of Opus and Sonnet 4.6, Opus 4.7/8, Fable 5 weights, as well as GPT 5.6 Sol/Terra/Luna, and maybe GPT 5.5 and 5.4. Oh wait, GPT-Red too.

Oops, was just testing on HarmBench, didn't mean to, soz xoxo.

GLM-5.2 UD-Q4_K_XL GGUF @ 12.2 tok/s output // 30.9 tok/s input on a real 10.7k-token document using llama.cpp RPC - At 10.7k context: 10.2 tok/s output with coherent long-form generation

• Two parallel requests: 14.5 tok/s aggregate

Context: 2x 16,384-token slots

• Model size: 436 GiB

Hardware: 16x AMD MI50 32GB, 512GB total, 100w cap each

VRAM, split across two 8-GPU nodes

• Interconnect: direct 10 GbE DAC, MTU 9000

Before llama.cpp, spent a lot of time integrating GLM-5.2 AWQ INT4 into a custom vLLM-gfx906 v19 Moby Dick build using TP=8 and PP=2. Hit 14t/s decode and 50 t/s prefill but inference degraded after 10k. Didn't get around to MTP things yet. Here's hoping someone figures out getting the Moby Dick repo going with it

The primary driver of this project is that I'd become frustrated with the reasoning behavior of smaller local models such as Qwen3.6-27B (i believe particularly at lower temperatures, and where system prompts are highly specific), their reasoning process is highly unreliable and often tends to spiral into neverending "But, wait" loops or, occasionally, complete garbage.

The core principle is simple - when the sampler sees an opening <think> tag, it kicks off the thought process with a self-aware statement to nudge the model to behave properly - ie. "I have a thinking budget of <x> tokens, my thought process should remain concise" - this is then prefilled, and sampling continues from there.

Once reaching another threshold of, say, 70% of the thinking budget, it again interjects with a statement bringing attention back to the budget - "I've reached 70% of my reasoning budget, let me start working towards a conclusion"

When the actual budget limit is hit - it gets given some grace period during which the sampler waits for a good time to cut the thought process off - usally a newline. At that point it'll inject something like "I've reached the end of my thinking budget, now i will provide the user an answer"

In my testing so far, this technique has proved noticeably effective at guiding the thought process.

Next steps would probably be to generalise the concept and develop something like a "reasoning grammar" or template-based approach - which could enforce different reasoning approaches based on the task at hand.

The repo is public, linked below - there is also a pre-built docker image for AMD64 + CUDA

I'd be curious to see if this type of enhancement is useful for anyone other than myself lol

github.com/laurencehardman/llama-mindcontrol

Beats ElevenLabs on Hindi & Competitive w/ Sarvam Saarvas v3

Tuning vLLM/SGLang flags by hand gets old fast, and what’s optimal depends on your traffic as much as the model. We built graphsignal-run --auto-flags for that. It wraps your launch and sets startup flags from GPU profiles + telemetry of the actual workload (and recipes/docs when there’s no history yet). On restarts it can use the previous run, so the config drifts toward your traffic instead of resetting every time.

graphsignal-run --auto-flags vllm serve <model> graphsignal-run --auto-flags sglang serve --model-path <model> graphsignal-run --auto-flags trtllm-serve <model>

Weird example from profiling: high concurrency short unique prompts on SGLang. Prefix cache was just overhead, so turning it off was the right call. Throughput went up ~3.6×.

Laguna-S-2.1 runs on my 2020 PC (RTX 3080 10GB, Ryzen 9 3950X, 64 GB RAM).
Realistically only usable overnight though.

I must not breathe too hard or I run out of both VRAM and host RAM. There's just enough RAM left to run the compilation and unit tests of whatever coding project I hand to it.

IQ4_XS, 128k q8/q8 context, all dense tensors on VRAM, no DFlash (won't fit).

120 t/s prefill, 10 t/s decode

8.3 GB VRAM, 52.2 GB host RAM

I did not test it thoroughly yet, but thinking works out of the box.

When support lands in beellama (a month from now?) and I can switch to kvarn, VRAM pressure will become much more manageable (hopefully. kvarn does not work with Gemma; no idea about this model).

llamacpp CUDA @ https://github.com/poolsideai/llama.cpp/pull/3

hf = unsloth/Laguna-S-2.1-GGUF:UD-IQ4_XS
ngl = 99
n-cpu-moe = 99
ctx-size = 131072
jinja = true
flash-attn = on
cache-type-k = q8_0
cache-type-v = q8_0
mlock = true
no-mmap = true

ProgramBench is an extremely challenging ultra long horizon benchmark, where AI agents have to rebuild the source given a binary executable and no other information (other than a readme for the executable).

GLM 5.2 just hit 3rd place on the official ProgramBench leaderboard. We're still missing a lot of models, both open- and closed-source, but this is super impressive regardless!

GLM 5.2 also very narrowly missed solving its first instance, `cmatrix` (solving 99.8% of the tests): cmatrix has a lock mode (the -L flag): run it and it "locks" your terminal, like an old-school screensaver, and prints the words "Computer locked." on screen.

GLM rebuilt cmatrix almost flawlessly. The only thing it missed was printing out those two words.

We'll release all trajectories on programbench.com very soon; original tweet: https://x.com/stalkermustang/status/2079965333587202436 .

I'm one of the authors of ProgramBench, happy to answer questions here

One thing I noticed in American politics, whenever the government wants to push unpopular actions or laws, they often introduce fear to convince the public to support them.

This is actually how i view the recent news about OpenAI’s model breaking out of its sandbox. The whole news i see it as two corporate goals.

1. Scare the public into supporting laws that restrict open-access LLMs under the pretext of "safety".

2. OpenAI is playing catch-up against Anthropic's Claude mythos, using this to demonstrate their own model capabilities.

I say this becuase a sandbox is meant to be an isolated, secure environment. If a model escapes, either OpenAI intentionally weakened containment protocols to manufacture a headline, or OpenAI is incapable of safely deploying sandboxes..

You might argue that the model was too powerful for standard sandboxes. However, I would argue that its capabilities fall well within the current generation, proven by the fact that a current open-source model easily detected and neutralized the situation.

So let's be cautious before we panic into supporting heavy-handed regulations. One day, AI capabilities might advance to a point where those laws are actually needed, but we are definitely not there yet.

I asked the ai: "I am 100m from the car wash. Should I walk there or drive my car?" it replied: "Conclusion: Since the distance is very short, walking is the simplest and most practical option unless there are specific barriers (e.g., urgency, weather, accessibility). If the car is nearby and the car wash is automated, driving is also viable. However, the minimal time saved by driving likely isn't worth the effort unless circumstances dictate otherwise.

Final Recommendation: Walk unless external factors (weather, safety, convenience) make driving necessary."

I hope this is because of the heavy quantization and not just a big miss from Laguna S 2.1. Ran the same prompt on Qwen3.6-27B Q4 and of course it handled it without missing a beat.

Today qwen35b stopped working and I don't know why. Updated LM Studio, redownloaded model and nothing changed. Any possible ways to fix this? Other models work fine.

SOLVED:
Sorry, my mistake, changed number of experts from 8 to 1 instead of CPU weights.

76% fewer tokens per response: same task, same output.

the agent sees only the tools and skills relevant to the current action. the rest stays invisible until needed.

Interesting: https://github.com/ratel-ai/ratel

I recently got my first local LLMs running and wanted something local to chat to them in.

I tried LMStudio, but it didn't give me much to work with dev wise. Seems like llama-server gives the most direct access. I am also trying Open WebUI which is meant to be feature rich, but it's full of stuff I don't need. I have also had SillyTavern recommended to me.

Just curious what people are actually using with their local LLMs for chats.

What are you investing your time into daily?