In some social science disciplines, talk of impact factors will be routine. In others, they will be mentioned sometimes but some of your colleagues (even highly respected researchers) won’t know anything about them. In most humanities disciplines, they are unheard of.
Even if you can state the impact factor for a particular journal, do you really know what one is?
And why the disciplinary differences?
Measuring the impact on the advancement of knowledge
Impact factors are a statistical measure of the impact on the advancement of knowledge. They are a measure of the impact of a journal, not a specific article.
More accurately, they are a measure of the probability of an article in a particular journal being cited in other journal articles.
The data set used to come up with this number is the relevant citation index.
Variable validity
Because the data set is the citation index, it only includes citations in journals. The validity of the measure thus varies considerably by discipline.
In disciplines, like the hard sciences, where close to 100% of all scholarly publishing is in peer reviewed journals, the impact factor is an excellent measure of the probability of your article being cited if you published in a particular journal.
In these disciplines, the methodology captures all (or close enough to all not to make a difference) of the citations.
In disciplines where significant scholarly publishing is in books (monographs and/or edited collections), the measure is not valid. The data set does not include citations of journal articles that are in books, thus missing significant numbers of citations. The humanities fall in this group.
Mixed disciplines
In many social science disciplines, the validity varies between sub-fields and is reflected in journal rankings based on impact factors.
The reason journals that publish quantitative work in sociology and political science are ranked highly in journal rankings based on impact factors is because researchers who do quantitative sociology and political science tend to only or mainly publish in journals. The measure captures all of the citations in their subfields. The extent that other sociologists/political scientists might cite them in books is likely to be small enough not to make a significant difference.
However, other sub-fields of those disciplines, particularly more theoretical or qualitative researchers, will publish a mix of journal articles and books. Their work will be cited in both journal articles and books. And a significant proportion of their citations will not be captured in the methodology used to construct impact factors and rate journals.
This may affect how you are evaluated if the evaluators (T&P committee, grant selection committee, etc) rely heavily on impact factors in comparing candidates. The composition of peer-review committees is crucial. Your peers are those within your discipline with similar publishing patterns, who should be able to evaluate the likely impact of your work even if impact factors are not valid in your sub-field.
Are they useful even in mixed fields?
Impact factors can be helpful when you are making decisions about publishing.
If you are comparing a few journals of the same type, where the error margin is likely to be similar, you could assume that the ranking retains some validity.
Thus, if you give primacy to reaching the right audience, the audience that is engaged in the debate you are engaged in, and compare a few journals that are reaching that audience, impact factors can help you decide which journal is likely to reach more of that audience or be more highly valued by that audience (and thus likely to be taken seriously and cited in their own work).
It is not a good idea to look at the journal rankings for your (mixed) discipline and then target the top journals if those journals do not reach the people you want to reach.
The future of impact measures
I don’t know what is happening behind the scenes on this but it seems to me that we now have (or are close to having) the technology to improve the validity of impact measures in fields where book publishing is significant.
For example, Google Scholar captures citations in books. And I assume other digitization initiatives will also do this.
The questions this raises include:
- Is anyone updating the impact factor methodology to capture these citations?
- Would it be useful to do so?
On the second question, I think there is probably a case in the social sciences, where impact factors already have considerable purchase in decisions due to their validity in some sub-fields. I’m not sure the humanities value quantitative measures enough to care. Though in wider university or political venues where they are being compared to scientists and social scientists (collectively, if not individually), they could make a difference.
Want to know more?
If you want to know more about impact factors, including how to find them for your discipline, ask your librarian. This is the kind of thing that librarians know a lot about. And they will know how to access the relevant databases in your institution.
Your librarian is also the person you might get to come and explain citation indexes, impact factors, and related issues to your graduate students. A lot has changed since you were a graduate student, and enlisting the help of “information professionals” makes your life lots easier as both a teacher and a researcher.
Want help with your own publishing plans?
Is this making you think differently about your own writing and publishing plans?
Would it help to talk to someone about what you have on your desk and where you might publish it?
A 30-minute “Push out of the snowbank” ($50) might be all you need.
Me, on the phone, brainstorming with you on how to move forward.
Or, for $150, you could send me your publications list and a list of works in progress. I would look that over and then we’d talk for up to an hour on the phone to work out where you are, where you want to be and how to get from one to the other.
I’d record the call for you so you don’t have to worry too much about taking notes and I’d send you a longish e-mail summarizing the key points. You would also be able to e-mail me for clarification as you put the plan in place.
