Answering @RiversHaveWings's question of what happens when you take the kv cache of an instruct model and feed it to the base model

Nov 27, 2023 · 9:40 AM UTC

2
1
65
gist: gist.github.com/sekstini/024…

Answering what happens when you give the KV-Cache of an instruct model to its base model. (update:...

Answering what happens when you give the KV-Cache of an instruct model to its base model. (update: better version in the comments) - KVCacheFun.ipynb

gist.github.com
10
Replying to @fluffykittnmeow @RiversHaveWings
Interesting, though Mistral has probably seen Instruct data in its training corp if its used any web data post 2022
1
1
It may have. Actually I was a bit dissatisfied with the methodology in my original post, as I should have used the proper instruct template and given a command instead of a question. The results when doing that are way more interesting: gist.github.com/sekstini/91b…

KV Cache Fun, but using the proper instruct template, and comparing outputs

KV Cache Fun, but using the proper instruct template, and comparing outputs - KVCacheFunv2.ipynb

gist.github.com
1
3
more replies