全部文章0
Hacker Newsxenova··访问 1

Bonsai 27B: A 27B-Class model that runs on a phone

原网页

Article URL: https://prismml.com/news/bonsai-27b

Comments URL: https://news.ycombinator.com/item?id=48910545

Points: 522

# Comments: 187

Hacker News 讨论

522 points · 187 comments · 查看原帖

  1. SwellJoe

    What I most want to see it compared to is Gemma 4 12B in the 4-bit QAT version. It's barely bigger than this at just under 7GB, so it also runs on just about any modern device and is remarkably smart for its size. It's an excellent tool user, crazy good vision for its size. I'm still trying to wrap my head around how much is lost with each step down in resolution, but the QAT versions from Google seem to prove the answer is "very little" at four bits.

  2. motbus3

    I need help understanding this. I understood that the magic here is the quantization that allows it to use from 50G to 4G and their process retain most of the intelligence within Pareto limits of gain. And then they proceed to compare with other quantized models as in the level of intelligence per size. It gets to my attention though that the performance in tool calling is mostly affected which is a problem for other small models. How does this model compare to a recent 4G model? How do we know it retained intelligence from the parent rather then being fine tuned for the benchmarks? I am not shtng on them or anything. I'd rather find it amazing, BUT given my limited knowledge, I feel the results miss fair comparison plots and the ones might be misleading. Buy I also reckon it might be me the problem. Anyone care to explain this poor silly fellow some of those points?

  3. kristianp

    Apparently Apple is "in talks" with the PrismML: https://www.cnbc.com/2026/07/14/apple-prismml-ai-compression...

  4. simonw

    The models themselves are showing up on Hugging Face here: https://huggingface.co/prism-ml/models I've tried a couple in LM Studio - the GGUF one and the MLX one - but neither worked there. Anyone else get them to work? Might be that LM Studio needs to upgrade their llama.cpp or MLX engines first.

  5. RugnirViking

    maybe its nitpicking here but the demo shows them asking the model what to cook and its recipie sounds like it wouldn't be very good and also it totally gets the macronutrients wrong. 25g protein for "spaghetti, carrots, peppers, garlic and herbs"?

  6. hham

    This is accelerant #3 and #4 from our article converging in one release: a 27B-class model, built on Qwen (already one of our examples of local models "good enough to matter"), now running on an iPhone. The hardware layer and the local-model layer aren't just going to converge in the future, they're doing it right now! https://news.ycombinator.com/item?id=48892559

  7. Arcuru

    Awesome! I've been waiting for them to start scaling ternary models for over a year[1]. Excited to try it out, typical Qwen 27B is too heavy for me to run on my local hardware at reasonable speeds. [1] https://jackson.dev/post/dont-sleep-on-bitnet/

  8. athrowaway3z

    So first off, phenomenal stuff to see a 1bit model at 90% capability. However, this is the 5th product post in 2 weeks that proclaims that AI use is shifting, and why [insert tradeoffs] are the perfect fit. Paradigms shift don't happen in the release announcements. I suspect this is an AI-ism, making all the release posts sound so paradigmshiftery.