• 3 Posts
  • 14 Comments
Joined 1 month ago
cake
Cake day: July 29th, 2026

help-circle
  • Well the hashes are always much smaller than the actual data, otherwise downloading the hashes would take just as long as downloading the data lol.

    So 16KB blocks means 131072 bits per block, subtract the 256 bit hash size and that means 2^130816 hash collisions

    That’s a lot lol, you’ll never find which one is the correct one. Even if you instantly had a list of all values that match the desired hash, you’d have to check them all and somehow determine which one was correct.







  • Number of Parameters: 125B with 6B activated, plus 51B n-gram embedding and 4B MTP

    I bet you could run this in Q4 on 96GB RAM and 16GB VRAM, maybe even less. The benchmark scores seem good, beating 27b and DeepSeek Flash.

    The n-gram embeddings sound very similar to Gemma 4 e4b embeddings. Need llama.cpp to support streaming n-grams from SSD, mmap would be less efficient than having explicit support.

    GGUFs are starting to be available now