It would be good to develop a set of criteria a benchmark system should typically meet in terms of data quality, structure availability, etc.
Originally, I thought that we would be able to develop a universal set of such criteria (i.e. high quality structures of such-and-such a resolution, ITC or SPR binding affinities, etc.), but then as the paper developed we realized that different types of data are needed depending on the purpose of a test, as in Section II.A ("hard" and "soft" benchmarks). So, it may not be that we can provide a universal set of criteria -- but it would be good to discuss criteria that might apply in the different categories.
It would be good to develop a set of criteria a benchmark system should typically meet in terms of data quality, structure availability, etc.
Originally, I thought that we would be able to develop a universal set of such criteria (i.e. high quality structures of such-and-such a resolution, ITC or SPR binding affinities, etc.), but then as the paper developed we realized that different types of data are needed depending on the purpose of a test, as in Section II.A ("hard" and "soft" benchmarks). So, it may not be that we can provide a universal set of criteria -- but it would be good to discuss criteria that might apply in the different categories.