lots of mysteries to unravel here

Mar 13, 2023 · 6:41 PM UTC

22
52
19
606
1
26
Colin Fraser | @colin-fraser.net on bsky @colin_fraser
Mar 14
My original tweet makes the performance look a bit better than it is, if you're looking for exact answers. Here's the accuracy in terms of exact solutions.
7
Replying to @colin_fraser
I wonder if it varies changing number representation (base, language, digits vs words, etc). Would be very interested in reading the source.
1
7
It's my own experimentation. I used the API to get 40000 solutions to randomly generated arithmetic problems (500 pairs of numbers * 4 operators * 20 solutions per pair). I pull out whatever the last number in the ChatGPT response is as the solution.
1
1
19
more replies
Replying to @colin_fraser
I assume these are operations on integer values only?
1
Yes
1
more replies
Replying to @colin_fraser
subtraction and division are cursed, and always have been.
1
1
I would say that this does make subtraction worse than maybe it is, and multiplication look better than it is, if you're looking for exact solutions. See my followup post
Colin Fraser | @colin-fraser.net on bsky @colin_fraser
Mar 14
My original tweet makes the performance look a bit better than it is, if you're looking for exact answers. Here's the accuracy in terms of exact solutions.
2
Replying to @colin_fraser
where's this from?
1
1
My computer
1
32
more replies
Load more