SNIP Doesn't Work (at least not for area studies)

It has been our practice at the Crawford School to highlight on the website etc. all journal articles that ranked as A or A* (the top 20% of journals) in the ARC's ERA journal ranking list. Unfortunately, the ARC abolished this ranking. We seem to be one of the few groups that were unhappy with that. Most Australian researchers seemed overjoyed.

So we had to look for a replacement listing. One option was Elsevier's SNIP which
I blogged about in May. SNIP is designed to take into account the varying citation practices in different fields. This is very important in an interdisciplinary school like Crawford. If we adopt a common metric like impact factor or article influence score then our colleagues in smaller and book-oriented fields will be disadvantaged relative to those in large journal-oriented fields like economics.

But we found that at least in the case of area studies, rather than ameliorating the problem, SNIP exacerbates it. Using an article influence score of 1 and above, I identified 78 economics journals out of a total of 305 in the Journal Citations Report and only 3 in area studies out of a total of 60 in the JCR. Using a SNIP of 2 or more, I identified 86 economics journals but only one * of the top 15 area studies journals from the JCR had a SNIP of 2 or more. So SNIP was 3 times as discriminatory against area studies than Article Influence Score.

So, reluctantly, we have decided to continue using the ERA journal ranking list for the next year using judgement to decide whether new journals qualify as leading journals. We could use the top 25% of journals in each field in the JCR as an alternative but decided that that would be less transparent. Energy Economics would not be included in the top 25% because it is outside the top 25% in economics but Ecological Economics would be included because it is in the top 25% in environmental studies, though outside the top 25% in economics. This would end up confusing people. Maybe we'll try it in 2013, unless SNIP improves in the meantime.

* This journal - African Affairs - was ranked C by the ARC!