You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Jan 5, 2024. It is now read-only.
Repository navigation
This repository was archived by the owner on Jan 5, 2024. It is now read-only.
Hi, I see here that you list the layer that you used to compare to V1 but I cant find the layers you used for V2, V4, IT, or Behavior. I was wondering where I can find this information for all the candidate models you all benchmarked
Hi aaprasad, we started storing this information only recently again, so older models unfortunately don't have it stored in a way accessible by the website. Please see here for meta information in json format for the models scored up until 2019, on older versions of the benchmarks.
Thank you so much for this, its super helpful. Just to clarify, how come the layer listed on the website for V1 is different from the one in the JSON? Were they not scored on the same benchmarks?
The file I sent you is meta information from 2019. We have since introduced new versions of the benchmarks. For V1, the most notable difference is that the V1 benchmarks now properly show images at 4 degrees visual angle -- for instance, if the model declares its standard input as 8 degrees (all models on the site right now), then the image will be zommed-in/gray-padded to simulate how the image would appear to a primate subject on the screen. If you scroll down all the way in http://www.brain-score.org/model/639, there's an illustration of this.
Activity
mschrimpf commentedon Dec 17, 2021
Hi aaprasad, we started storing this information only recently again, so older models unfortunately don't have it stored in a way accessible by the website. Please see here for meta information in json format for the models scored up until 2019, on older versions of the benchmarks.
aaprasad commentedon Dec 17, 2021
Dear Martin,
Thank you so much for this, its super helpful. Just to clarify, how come the layer listed on the website for V1 is different from the one in the JSON? Were they not scored on the same benchmarks?
mschrimpf commentedon Dec 17, 2021
The file I sent you is meta information from 2019. We have since introduced new versions of the benchmarks. For V1, the most notable difference is that the V1 benchmarks now properly show images at 4 degrees visual angle -- for instance, if the model declares its standard input as 8 degrees (all models on the site right now), then the image will be zommed-in/gray-padded to simulate how the image would appear to a primate subject on the screen. If you scroll down all the way in http://www.brain-score.org/model/639, there's an illustration of this.