ππ΄π’π―π«@gwernJan 16Some people say GPT-2 doesn't scale. They don't understand: the only thing that matters isβBRUTE STRENGTH! They want Reformer (they're probably beta orbiters).
But TPUs haveβBRUTE STRENGTH!
TPUs run GPT-2-117M with 30,000 context windows for MIDI generationβwith BRUTE STRENGTH! pic.twitter.com/lMTsqvJLUT
ππ΄π’π―π«@gwernJan 16slatestarcodex.com/2015/06/02/andβ¦ is a fun story.
Anyway, we have 537k MIDI files (~9GB) which we're converting to ABC (text is just easier to work with and it's only somewhat lossy). Our TPU swarms hunger. Should be interesting to compare with @OpenAI's MuseNet...
14,017
434
3.1%
View Tweet activity
ππ΄π’π―π«@gwernJan 24help I finally got my cat to pay attention to videos on my monitor and now he's addicted to videos-for-cats on YouTube and won't stop sitting in front of my monitor waiting for the birds to come back
ππ΄π’π―π«@gwernJan 7...
- GPT-2-1.5b-IRC
- scaling StyleGAN 2 up to our TPU swarm for training on 1024px Danbooru2018 images (codename: "This Booru Does Not Exist")
- Danbooru2019 dataset release (+~580k images)
ππ΄π’π―π«@gwernJan 4Delayed due to Typhoid Mary visiting for the holidays...
("Breath is the least appreciated gift of the gods.
None sing hymns to it, praising the good air, breathed by king and beggar, master and dog alike.
But, oh to be without it!")
10,576
21
0.2%
View Tweet activity
ππ΄π’π―π«@gwernJan 11Exactly! So many things in DL are like 'we tried a dozen variants of X and it didn't work, but variant #13 did'. The papers then hide it and pretend they had a theory of #13 all along. Infuriating. Like resnets. 'Grad student descent' but does paper mention that? ofc not.
10,344
140
1.4%
View Tweet activity
ππ΄π’π―π«@gwernJan 8The alpha release of Danbooru2019 (+358k/425GB; n=3,692,577 total) is now up on rsync for testing:
`rsync --verbose --recursive rsync://78.46.86.149:873/danbooru2019/ ./danbooru2019/`
If you have Danbooru2018 downloaded already, simply rename to './danbooru2019/' & rsync.
10,297
38
0.4%
View Tweet activity
ππ΄π’π―π«@gwernJan 19In ππ’π΅π΄, cats unzip their fursuits and another cat steps out. This has raised questions about how the ππ’π΅π΄verse works ('are these cats nude if they have no clothes?') but this overlooks the main import: this proves reproduction in the ππ’π΅π΄verse is by purformationism!
9,900
38
0.4%
View Tweet activity
ππ΄π’π―π«@gwernJan 14Shower thought: terminal velocity of a car is 100-200mph; sports car tires can handle <200mph easily. So, with enough heat shielding and a tall curved ramp and good aim, you could drive a car from orbit to ground.
ππ΄π’π―π«@gwernJan 1315-kun keeps his secrets; nevertheless: docs.google.com/file/d/1nyHvaaβ¦
"...the one secret concerning the atomic bomb which might have been kept and which was given to the public & to all potential enemies without the least inhibition, was that of the possibility on its construction..."
9,747
152
1.6%
View Tweet activity
Get your Tweets in front of more people.
Use Tweet Activity to track how your Tweets are doing.
![](data:image/svg+xml,<svg xmlns="http://www.w3.org/2000/svg" width="128" height="128"><rect fill-opacity="0"/></svg>)
