Hi everyone,

I've been working on a new language model architecture called BCMT (Blockwise Causal Memory Transformer) and I'd really appreciate feedback from people interested in efficient transformer scaling.

The main idea is to replace full causal attention with:

• Dense causal attention within fixed-size blocks
• A compact memory representation summarizing each block
• Causal memory propagation between successive blocks

The goal is to reduce the computational and memory cost of long-context training while preserving language modeling performance.

Current results (WikiText-103, context length 256):

• Validation loss: 4.5931 (Dense Transformer: 4.5752)
• +57.4% training throughput
• −23.1% peak GPU memory

I've also included an ablation model (BCMT-HOnly) to isolate the contribution of the inter-block memory mechanism.

This is an early version of the project, and I'm especially interested in feedback on:

• the architectural design,
• additional scaling experiments,
• evaluation protocols,
• and comparisons with other efficient attention or memory-based approaches.

Paper, code, and training details are available here:

https://github.com/rachidlabs/BCMT

I'd be grateful for any comments or suggestions. Thanks!

GPU cloud operators: how do you decided which customers get capacity when you’re supply constrained? Is this manual or automated?

Posting here specifically because most "AI chat app" releases treat local models as an afterthought bolted onto a cloud-first UI. Eaon flips that — Ollama, llama.cpp, and MLX are full citizens: same tool-calling, same agent loop, same everything a hosted model gets, plus a live hardware-fit check (comfortable/tight/too-big) before you download a model that won't run. The part I think this sub will actually care about: Agent mode isn't gated to models that are great at function-calling. There's a text-fence fallback baked into the system prompt for models that ignore or mishandle native tool calls — I've been testing this against small stuff like Nemotron 3 Nano and it holds up. It'll write files, run them, read the output, and fix its own mistakes, same loop regardless of which model's driving. Also shipped a terminal CLI (eaon-cli) if you live in a terminal instead of a GUI — same agent, same local-model routing, npm-installableux, Windows should work too but. You can download at eaon.dev

[https://arxiv.org/pdf/2607.13511](https://arxiv.org/pdf/2607.13511))

the core idea is, we cannot have ternary PTQ with fixed matrix size, trying to do that is dead end. so i tried decomposing the matrix to 2 ternary matrices and inner diagonal scaling matrix. now that the inner rank can be arbitrarily large the accuracy can be arbiratily small. and its not that it has to be very large too i also showed that it does take only slightly more vram then current quantisation methods. the slight more vram is worth it if we abuse the ternary math.

Hello guys, I saw today a great cost/task matrix today that showed all the SOA LLM models broken down by different reasoning efforts and side-by-side. For example, it showed that GPT 5.6 performs very well, while Fable is very expensive. (And that SOA models with low reasoning are sometimes even quite dumb.)

The whole thing was presented in a table, with the cells colored green, yellow, or red according to the result.

Unfortunately, I can't find the link anymore. Can anyone help me find it?

Welcome to A Tale of Two Nations, a contamination-proof, cross-domain adversarial stress-testing suite designed to push frontier large language models (LLMs) to their absolute logical limits.

Unlike traditional low-context benchmarks that suffer from data contamination, this ecosystem uses a highly intricate, multi-layered systemic scenario to evaluate an AI's long-horizon reasoning, context-gating integrity, and synthetic logic capabilities under zero-shot conditions.

Will your local model pass? Open-sourcing with scoring framework~✨

Links to the project:

• GitHub Repository: https://github.com/SMahjuba/A-Tale-of-Two-Nations-AI-Benchmark
• Hugging Face Dataset: https://huggingface.co/datasets/SMahjuba/A-Tale-of-Two-Nations-AI-Benchmark

A gigantic new model from Moonshot - the biggest open-source LLM by a large margin (but note that it's more sparse than before).

We have also scaled up Mixture of Experts (MoE) sparsity, effectively activating 16 out of 896 experts when paired with a Stable LatentMoE framework. Together with refined training and data recipes, these structural changes yield an approximate 2.5× improvement in overall scaling efficiency compared to Kimi K2, allowing the model to convert compute into intelligence more effectively.

It looks a bit better than Opus 4.8/GPT 5.5 but a bit worse than Fable/GPT 5.6.

I am struck by the fact that they made so little progress on Humanity's Last Exam (58.7%, vs 54% for Kimi K-2.6, which was released in February). LLMs appear to be stalling out at around 60% on Humanity's Last Exam (note that Grok 4 Heavy scored 50.7% over a year ago) despite making rapid progress in other benchmarks.

To be honest, I am now pretty suspicious of that benchmark - particularly after FrontierMath and SWE-Bench Pro were found to have lots of unanswerable/unscorable questions.