Does it perform better than pigeons[1] though? ;)<p>[1]: <a href="https://twitter.com/emollick/status/1388594078837878788/photo/4" rel="nofollow">https://twitter.com/emollick/status/1388594078837878788/phot...</a>
Well, that blog post was rather uninspiring. The winning solution basically amounted to a whole bunch of hyperparameter tweaking using tons of GPU compute on a UNet architecture that has existed for many years now. We've definitely reached a plateau of sorts when it comes to vision-based deep learning. There is definitely progress still happening, but a lot of it is very incremental and there hasn't really been any big fundamental overall performance improvements in a while. That being said, there has been a bunch of work on improving efficiency. Which is nice, but it only buys you so much when the only way to "improve" overall performance is to throw in several million more parameters with larger models to eke out another few percent in accuracy or whatever metric you are tracking.
"Training was done on eight NVIDIA A100 GPUs for 1000 epochs"
"trained on four NVIDIA V100 GPUs for 300 epochs"
"model was trained on a NVIDIA DGX-1 cluster using eight GPUs"<p>No doubt these are model innovations. But can't help wondering how much ready access to arrays of GPU might give them advantage... You know how many different models need to be tested before getting to the winning ones.
i wonder if the average professional coder, still has. chance to beat the big teams ?<p>Reminded me of years ago(It might have been on HN) where some company/group trained a state of the art nlp model to classify if a financial statment or press release was possitive or negative. They had some good results until someone did:<p>Lets look how far down the press-release-statement the 'numbers' are :)<p>If it was positive-sentiment it usually was the case that the numbers were high up the page. If it was bad thr bulk of the numbers were much lower in the page.<p>Almost make sense,if you had goof numbers you want to screM it out. If you had bad numbers you want to preface/explain why first ??
Perhaps they should impose rules that constrain the number of GPU resources you can use to achieve your result, that levels the playing field and keeps competitions like these open to everyone instead of just institutions with access to tons of capital. They do that for Formula 1 as well.