ππ΄π’π―π«@gwernFeb 1Nobody was born caring deeply about better modeling of ImageNet, or ALE, or Wikitext. You want to do something cool, useful, & interesting in DL? Do literally anything you are interested in and you will almost immediately find yourself on virgin ground with an endless TODO list.
ππ΄π’π―π«@gwernFeb 1(Surely these are not 'artificial artificial neural networks' but *artisanal* artificial neural networks?)
6,172
127
2.1%
View Tweet activity
ππ΄π’π―π«@gwernFeb 2Seems rather flag-plant-y. I wonder if these results are worse than what's already been posted to Twitter by hobbyists - blackbox search, not even exploiting the ability to use gradient descent through CLIP and the GANs?
ππ΄π’π―π«@gwernFeb 1β¬ Am I awake or do I dream? / The strangest pictures I have seen...
TwilightβI only meant to stay a while
TwilightβI gave you time / to steal my mind / away from me...
You brought me here, but can you take me back again? π pic.twitter.com/zZIHyLXyOl
ππ΄π’π―π«@gwernFeb 1@Nearcyan makes a good point: while there may be countless downstream users, at the mine face of DL, in any niche, there is ~no one. For all ML/DL furry stuff, there's... @arfafax. For anime, you can count on one hand the people (updating reddit.com/r/AnimeResearcβ¦ is very easy).
2,925
62
2.1%
View Tweet activity
ππ΄π’π―π«@gwernFeb 2It's funny you complain about that, I was just staring at a font proposal (unicode.org/L2/L2020/20275β¦) with ever-escalating sense of dread about how subtle the differences between various 'C' and 'F' variables all looking nearly-identical could become...
ππ΄π’π―π«@gwernFeb 2This amounts to the claim that the bias-variance tradeoff does not exist and scaling is irrelevant.
1,376
50
3.6%
View Tweet activity
ππ΄π’π―π«@gwernFeb 3A good analogy. How do our current computers compare to resources available 20 or 30 years from now? There's no reason to think Transformers are truly NN's final form. (This is why I'm interested in what you can do with FCs only.)
1,242
74
6.0%
View Tweet activity
ππ΄π’π―π«@gwernJan 31Perhaps it's hindsight talking, but I recall a fair bit of skepticism about SELU, once you got past the clowning about its appendix, same as any new activation, and that within a month or so, enough people had reported on Reddit no benefit that it was pretty clearly a dud.
1,056
37
3.5%
View Tweet activity
ππ΄π’π―π«@gwernFeb 3The bias-variance tradeoff is one of the most fundamental and widely employed concepts in ML, it's not fancy in the slightest bit.
Many models perform poorly at the small-scale and win in the large-scale. We just witnessed Transformers eating everything when large enough!
![](data:image/svg+xml,<svg xmlns="http://www.w3.org/2000/svg" width="128" height="128"><rect fill-opacity="0"/></svg>)
