I Just Tried The Brand New Ternary Model And It's Great!
www.youtube.com
PrismML just dropped the first Ternary Models (not the same as 1Bit!) and what we have on our hands is 1Bit models but with *a bit more intelligence*. In thi...

Pretty good foot in door video. I’m looking forward to a 27B dense or 35B MoE that can fit entirely within 5-8GB vram 🤞🤞🤞

Speccing out the smallest reasonable rig that might presently run such a thing (used Optiplex with a 3090) is roughly a $1500 ask with a $250/yr running cost. If they can shrink a 27B-35B down to something that can run on low end hardware that sips power…well…that will be an interesting thing indeed.