Be Known by the Company You Keep: Citations - Quality or Chance?

I previously blogged about how highly downloaded working papers lead to higher downloads of other papers in the same series. So I was interested to read that having your paper appear in the same issue of a journal as a highly cited article increases your number of citations. The implication is that weak papers can benefit from being in journals with high impact factors. Another interesting point in this paper is that self-citations help increase citations to a paper. I've suspected that this is true. Advertising your paper by citing it gets more people to be aware of it. But we can't really establish causality here. Maybe better authors just tend to cite themselves more and/or get more opportunities to do so.

Interview with Editor of the Quarterly Journal of Economics

The QJE is currently widely regarded as the top economics journal. It is also well-known for leading the trend towards "desk-rejections", where papers are rejected by the editors without sending them to external referees. This greatly speeds up the process for papers that are obviously not likely to pass the refereeing process. One of my roles at Ecological Economics as associate editor is giving advice about whether to desk reject papers and I think we are desk rejecting more over time.

I found this interview with the editor of the QJE that I thought had some useful tips about submitting papers to top journals. At Ecological Economics, sloppiness won't be an automatic desk reject if the paper looks interesting. We are more likely to throw out papers because the basic thesis makes no sense or the authors do not justify at all what they do. But really bad English that hinders understanding will result in a desk reject too. Unless your English is close to native speaker standard it is worth getting a native English speaker to look over your paper before submission whether for free or for a fee.

Open Access: What's the Big Deal?

There is currently a lot of discussion about open-access to academic literature. A lot of people seem to be really passionate about the issue. I agree with Gans that the fact that much research is publicly funded does not mean that the public should be able to access it free of charge. Though the major commercial journal publishers do seem to have high rates of profit, the services they provide still do have costs. One could argue that these oligopolists need to be regulated, though I think that their profits will not be maintained forever.

But is access to scientific literature so limited that government should be funding or mandating free access? The fact is that it should usually be possible to get a free copy of any article. In economics and in disciplines such as physics, most papers also exist in preprint form available for free (though the NBER still charges $5 a paper for some reason...). If you check out the information on the SHERPA/RoMEO website you'll find that many publishers including Elsevier, allow authors to put a final version of the paper on their institution's website.* Of course, a lot of authors don't do this. But if you don't and could, it seems hypocritical to complain about the lack of open access. And, of course, an increasing number of open-access journals are being published. Finally, most authors will be happy to e-mail you a copy of their paper.

The main issue, apart from the lack of convenience where direct open-access isn't available, would seem to be the archival literature. A lot is included in JSTOR. Their per article fees are lower than commercial publishers but still $10 a paper.

* A lot of these, plus "illegal" copies are available through Google Scholar. The ARC for example mandates publishing through open access or archiving publications in institutional repositories.

SHERPA/RoMEO

In economics there is a very strong tradition of working papers and journals do not have a problem with a working paper version of the paper (a "preprint") already having been "published". But this can be a real problem in some other disciplines. Putting up a working paper on the web could preclude publication in a refereed journal. Usually, in biology you can assume that a preprint is not OK. For interdisciplinary researchers working out what they are allowed to do in this regard could be a real problem. Fortunately, there is a really useful website called SHERPA/RoMEO that lists journal's policies on preprints and posting of final versions of papers. Information is missing on some journals, but it does provide a huge amount of information on most.

Becoming a Researcher

A great post on becoming a researcher. Well, it's actually titled: "How to find problems to work on", which is something I have found sometimes puzzles non-academics. I don't have 55 things I am working on but I do have a list with about twenty things on it which are papers from very sketchy ideas to papers under review. Probably, a smaller number of things makes sense in economics than in paleontology. Most economists publish less papers. How many things are in your list?

Credibility Intervals for Medical Journals

A paper by Darren Greenwood in BMC Medical Research Methodology estimates confidence intervals for impact factors based on observed fluctuations in the impact factor over time. He comes up with these intervals for journals in research and experimental medicine in 2005:



As you might expect, the credibility intervals * are wider for lower ranked journals. These estimates do not use actual citation data to individual articles, as I did, and Greenwood notes in the article that using such data would further widen the credibility intervals.

It is pretty clear that confidence intervals for impact factors for low ranked journals are high. The providers of these statistics should really provide a measure of precision. I'm surprised that this doesn't seem to be an extensively researched area in bibliometrics.

* Bayesian equivalent to confidence intervals.

Trends in RePEc Downloads and Abstract Views per Paper

A couple of months ago, I reported on trends in RePEc downloads and abstract views per person. As I was commenting on this issue on the RePEc Blog, I thought I'd do a follow on post here. Here are the same trends per paper:



Both are also on a downward trend over time. In my post in November I suggested a number of reasons that might explain the trends in per person terms. The number of papers per registered member is unchanged over recent years. So that rules out any explanations about there being fewer papers per person to download as RePEc grows. Here are possible explanations for the per paper trends here:

1. The average quality of papers is declining as more working paper series are added from lower quality institutions.

2. Too much supply relative to demand. Economists can only read so many papers.

3. The average paper is getting older and older papers will be downloaded less.

4. People are downloading more papers through services like Google Scholar, bypassing RePEc servers. This might be supported by the recent decline in total downloads and abstract views on RePEc that I mentioned back in November and Christian hinted at today.

I suspect it is a bit of each of these. If you have any ideas about how to test these hypotheses let me know!

5 Year Impact Factor for the Journal of Economic Growth

Following yesterday's post on the 2 year impact factor for the Journal of Economic Growth I tried to compute the 5 year factor. According to Journal Citation Reports it's 3.467 but I came up with 3.283 with a standard deviation of 0.52. It's not unusual apparently to not be able to reproduce the exact impact factors published in the JCR. Anyway, the 95% confidence interval is from 2.25 to 4.32.

Wayback in 1997

My homepage from the relatively early days of the World Wide Web. A lot of the links do work. My first web page would have been in 1995 but that is no longer accessible. The current version is here. You'll notice that this page has something in common with the 1997 homepage.

Standard Errors of Impact Factors

A few times recently I've seen it claimed on the web that there is no correlation between journals ISI impact factors and the number of citations that papers published in them get. On the face of it, this sounds a priori silly. The 2 year impact factor for 2010 is the average number of citations that articles published in a journal in 2008 and 2009 got in 2010 in the Web of Science. The relationship could be weak but it's unlikely to be zero. I've checked this for my own journal articles and the correlation between Article Influence Score and citations per year is 0.20.

A different way to look at this is to ask what is the standard error of the estimated impact factor? I decided to compute this for the Journal of Economic Growth. This journal has a reasonably high impact factor - 2.458 in 2010 - and only publishes twelve articles a year, which makes it easy to compute. Here are the number of citations each paper published in the previous two years got in 2010:



As you can see, half the papers got 1 or 0 citations. This doesn't necessarily mean they are duds, maybe they were published near the end of 2009 or are slow to gain popularity. The top three papers got 13, 10, and 8 citations respectively and the fourth only 4. The standard deviation of the mean is 0.69. That means that a 95% confidence interval stretches from 1.07 to 3.84. Possibly one can be more confident about impact factors for journals with larger numbers of papers or for 5 year impact factors, we'll have to see.

Crawford School Working Papers in December 2011

Following on from yesterday's post, here are some highlights of our working papers series in December 2011. Across all six series we got 2445 abstract views and 968 downloads. The most downloaded papers in each series were:

Crawford School Research Papers:

David Stern - From Correlation to Granger Causality

Arndt-Corden Department of Economics, Departmental Working Papers:

Pierre van der Eng - Total Factor Productivity and Economic Growth in Indonesia

ASARC Working Papers:

Khondoker Abdul Mottaleb and Kaliappa Kalirajan - Determinants of Foreign Direct Investment in Developing Countries: A Comparative Analysis

Asia Pacific Economics Papers:

Kazuki Onji - TAX CONSOLIDATION AND THE STRUCTURE OF CORPORATE GROUPS: EVIDENCE FROM THE JAPANESE TAX REFORM 2002

CCEP Working Papers:

Mutsuyoshi Nishimura and Akinobu Yasumoto - In Search of a New Effective International Climate Framework for Post-2020: A Proposal for an Upstream Global Carbon Market

Development Policy Centre Discussion Papers:

Matthew Morris - Measuring Poverty in the Pacific

Congratulations to all the authors!

Crawford School Working Papers

I am taking over the role of research director in the Crawford School. Apart from being responsible for the PhD programs I also deal with a set of research-related issues. Instead of just posting about CCEP working papers I plan to highlight developments across all our publications in future blogposts.

I've put together a page with some links to our publications as they appear in various online services. We have six currently active working paper series that are indexed on RePEc (let me know if I missed something out):

Crawford School Research Papers : This is our new school-wide series, which is cataloged on both RePEc and SSRN and is intended to cover the range of disciplines represented at the school. However, different disciplines have different traditions of working papers. In some disciplines putting out a working paper precludes publication in a journal (in much of biology, for example). By contrast, economics has a very strong working paper tradition. So not all disciplines are likely to be equally represented here.

Arndt-Corden Department of Economics, Departmental Working Papers: This is the long-running working paper series of the Arndt-Corden Department of Economics which is now a department within the Crawford School and was formerly the economics department of the Research School of Pacific and Asian Studies. The series focuses on economic development and trade particularly in Asia and the Pacific.

ASARC Working Papers: This series is put out by the Australia South Asia Research Centre.

Asia Pacific Economics Papers: This series is published by the Australia-Japan Research Centre.

CCEP Working Papers: Readers of this blog will be familar with this series produced by CCEP.

Development Policy Centre Discussion Papers: Another new series from the Development Policy Centre at Crawford.

Most Popular Posts, 2011

What were the most popular posts on Stochastic Trend in 2011? It was pretty random :)

1. 2009 Journal Citation Report Released (1525 hits). People are very interested in learning what are the top journals. Even if the report is a year out of date.

2. IAmScientist (939 hits). Also they are interested in scientific social networking.

3. Marginal Cost Curve for Crude Oil (606 hits). As oil prices rose again, the cost of oil production remains a popular topic.

4. How Does CSIRO Compare to Australian Universities? (544 hits). This is the most popular post actually written in 2011. Discussion of a paper comparing CSIRO's publication performance with the top Australian universities.

5. ERA Ranked Journal List is Out(368 hits). Even though the ranked journal list was abolished a lot of people are still using it.

6. Energy Mix and Energy Intensity (343 hits). Down from 3rd place last year, this is the most popular post on my own research content.

7. How Does PLoS ONE Have Such a High Impact Factor? (308 hits). This post is only a couple of months old but is already one of the more popular ones.

8. The Ecological Economics Critique (237 hits). Down from 4th place last year. Another part of my serialization of this paper.

8. Synthesizing Diesel Fuel Using Cyanobacteria (220 hits).

9. SD Memory Cards (214 hits).

So, only three posts actually written in 2011. I'll have to take some lessons from this when writing in 2012!

My Year in Review 2011

I started the year as a "Visiting Fellow" (i.e. unemployed but with an office etc. at the university) and ended as a full professor (appointed 15th August). In between I was an associate professor (from 10th January to 15th August). Of course, until mid 2007 I was a tenured associate professor in the US so this wasn't quite as meteoric a rise as it might seem :) (though it's debatable whether a US associate prof is equivalent to Australian one or to an Australian senior lecturer). I gave a "foundation seminar" for the new position on 1st November. The other major career highlight for the year was getting my first ARC grant.



I went on four international trips in the year. In April I went to Austria to be interviewed for a full professor position at Graz, which I didn't blog about at the time. It was a fun trip though pretty tiring. In the end they hired Martin Wagner, who I think is much more suitable for the position than I would have been. In July, I went to Korea for the IPCC, Working Group III first lead author meeting. I also gave a presentation at KEEI and did some sightseeing. In August, another new country - India - for a workshop on climate change policy. The final trip of the year was in late September and October to Ann Arbor, Michigan for another workshop.

On the publication front there were three papers which were officially published in 2011 - my review of energy and growth in Ecological Economics Reviews, publication finally of my paper on elasticities of substitution, and a paper on malaria and climate change with Simon Hay. The latter two papers were each submitted to four journals I think till we got them accepted... A couple of papers were also accepted, which hopefully will be forthcoming soon: Where in the world is it cheapest to cut carbon emissions? and The role of energy in the industrial revolution and modern economic growth. Both of these were published at the first place we sent them, but after a lot of revision. I also wrote a chapter for the Encyclopedia of Environmetrics on ecological economics.

Between all this I did manage to do some teaching :) I taught an introductory microeconomics course - Economic Way of Thinking I and gave a guest lecture at the Treasury as well as a series of three lectures in our flagship CRWF 8000 course.

That's probably enough of me talking about myself! Tomorrow there'll be a post on the most popular blogposts of 2011.

Job Opportunities at University of Maryland

University of Maryland, College Park.

The College of Behavioral and Social Sciences is seeking (3) computational social scientists to expand Maryland's strengths in the computational aspects of global environmental change through interdisciplinary joint appointments. Rank will start at associate professor and tenure will be in the department closest to the applicant's background. Applicants should have disciplinary backgrounds in the social sciences and most importantly, have advanced computational skills which include experience integrating social science data into computational models.

One appointment will be in economics/geography. Experience is preferred in sustainability science in combination with one or more of the following: computational economics, economic geography, spatial modeling or data visualization. Individuals selected will lead their own research, including sustaining external funding through grants, but will participate in ongoing and future interdisciplinary, environmental projects within their departments, and across the college. Faculty hired under this search will be expected to develop a college-wide course on computational social science that would fit into a sequence taken by students to cultivate research and computational skills. Student mentoring and service to the university community will be expected.

Each applicant should submit a cover letter specifying the joint position for which they are applying, along with their relevant qualifications, a curriculum vita, names of three references, and one journal article. Material should be submitted electronically (reference position 117853). Review of applications will continue until the positions are filled, however applications received by February 3, 2012 will receive best consideration. Applicants will be notified prior to reference checks. For questions, please contact the search coordinator, Sarah Goff-Tlemsani. The University of Maryland is an affirmative action/equal opportunity employer and is proud of its diverse faculty, staff, and student body. Women, minorities, veterans, disabled veterans, and individuals with disabilities are encouraged to apply.

Shuang's Blog Has a New Name

It's still at the same URL but has a new title: "Collective decision-making under uncertainty" instead of "Bioinvasion and ecoservices", which reflects the evolution of her research interests over the last 4 years since coming to CSIRO.

Paper Accepted by The Energy Journal

My paper with Astrid Kander The Role of Energy in the Industrial Revolution and Modern Economic Growth has been accepted for publication in The Energy Journal. I discussed the paper when the working paper was added to RePEc. This is the first building block in the research program which is a large part of our ARC grant - the "past" energy transition. A couple more related papers are already in preparation and much more is planned.

Comparing Author Citation Profile Services

There are now three leading services that provide author citation profiles online. I recently discussed the most recent entrant, Google Scholar Citations. The other two are Researcher ID (from the Web of Science) and Scopus Author Details. There are considerable differences between these services:



Google Scholar and Researcher ID require you to set up your profile yourself while Scopus does this for everyone automatically. To do this on Researcher ID you need to search for your publications on the Web of Science and so you will only add those that really belong to you. The profile does not update automatically and so there will be no noise from publications that don't belong to you but it will become out of date if you don't update it. Scopus profiles are usually very noisy unless your name is unique and there may be multiple profiles for the same author. You can request changes online but as the profile updates automatically noise is likely to be added again if your name is common. Google Scholar will suggest groups of articles that might be yours and you can search for missing articles and you can then self edit the profile. But because the profile updates automatically noise is likely to be added again. Google allows editing of the details of entries - you can correct titles, dates of publication etc. Scopus and Researcher ID only allow addition or deletion of articles as a whole. Automatic updating is really nice but it is noisy is the bottom line, especially if your name is something like S Liu.

Scopus profiles are not available publicly - only to other Scopus users which is a big downside for you as a researcher but because they are created automatically they are often my first place to look at a profile of someone. Researcher ID and Google Scholar profiles are still both rare in most fields. CSIRO requires staff to have a Researcher ID profile. I've also noted that in dinosaur paleontology several people have been able to list up to ten coauthors on their Google Scholar profile but this won't be common yet in other fields.

All three services allow you to sort the results by date or times cited. Google Scholar and Researcher ID also allow sorting by title.

The other features in the table are functions of the databases themselves. Google Scholar has the widest coverage of different types of publications. Scopus does include citations to books in the journals it covers but the author details does not. Google Scholar is limited by only looking for citations in materials that are online which does tend to favor recent citations. Do you really think that Einstein's citations increased by so much in recent years?* Scopus only has citations from post 1996 documents. The Web of Science has citations from documents going back to 1945 (if your institution subscribes that far back).

Bottom line is that Google is the most user friendly service but quite "noisy" and so far limited in who is covered. Researcher ID has the lowest noise level and is fairly user friendly. Scopus is noisy, non-public, and not very user friendly, but has the broadest coverage of researchers.

* If you think that Einstein has a lot of citations check out Pierre Bourdieu!

Transmission of Infectious Parasites Slows with Rising Temperatures, Researchers Find

So says a commentary in Nature on an article in Biology Letters*. This result is from a study with rodent malaria rather than human malaria. If this extends to human malaria it could be another factor explaining why we don't find a positive relationship between temperature and malaria.** Our assumption has been that interventions have been successful in overcoming the tendency to more malaria in warmer climates. But maybe that tendency is not so strong in the first place.

* Link to letter in Biology Letters from Nature is broken and I can't find it on their website.
** My coauthor, Simon Hay, is quoted in the Nature piece.

SNIP Doesn't Work (at least not for area studies)

It has been our practice at the Crawford School to highlight on the website etc. all journal articles that ranked as A or A* (the top 20% of journals) in the ARC's ERA journal ranking list. Unfortunately, the ARC abolished this ranking. We seem to be one of the few groups that were unhappy with that. Most Australian researchers seemed overjoyed.

So we had to look for a replacement listing. One option was Elsevier's SNIP which
I blogged about in May. SNIP is designed to take into account the varying citation practices in different fields. This is very important in an interdisciplinary school like Crawford. If we adopt a common metric like impact factor or article influence score then our colleagues in smaller and book-oriented fields will be disadvantaged relative to those in large journal-oriented fields like economics.

But we found that at least in the case of area studies, rather than ameliorating the problem, SNIP exacerbates it. Using an article influence score of 1 and above, I identified 78 economics journals out of a total of 305 in the Journal Citations Report and only 3 in area studies out of a total of 60 in the JCR. Using a SNIP of 2 or more, I identified 86 economics journals but only one * of the top 15 area studies journals from the JCR had a SNIP of 2 or more. So SNIP was 3 times as discriminatory against area studies than Article Influence Score.

So, reluctantly, we have decided to continue using the ERA journal ranking list for the next year using judgement to decide whether new journals qualify as leading journals. We could use the top 25% of journals in each field in the JCR as an alternative but decided that that would be less transparent. Energy Economics would not be included in the top 25% because it is outside the top 25% in economics but Ecological Economics would be included because it is in the top 25% in environmental studies, though outside the top 25% in economics. This would end up confusing people. Maybe we'll try it in 2013, unless SNIP improves in the meantime.

* This journal - African Affairs - was ranked C by the ARC!

More Musings about Schmith et al.

I got an e-mail from Søren Johansen about my post about the Schmith et al. paper that he coauthored. In response I thought a bit more about whether there could be a long run relationship between atmospheric temperature, sea level, and radiative forcing. I had said that that didn't make much physical sense to me and proposed a statistical explanation for what they found. But the fact is that atmospheric temperature is in equilibrium with the temperature of the ocean surface or the mixed layer (generally the top few tens of metres of the ocean). But heat is being slowly transmitted to the deep ocean and ocean height reflects the temperature of the entire ocean. So the relation between ocean height and atmospheric temperature might need the forcing variable to complete it over the length of the annual time series available. Also of course the oceans are not simply rising from thermal expansion but due to meltwater from glaciers and ice sheets.

Some Kind of Deal is Agreed in Durban

So Kyoto will be extended but be restricted to Europe. In the meantime most other countries have their Copenhagen pledges to fulfill. Negotiations will proceed to agree a global treaty with "legal force" (whatever that means) by 2015 to come into force in 2020.

This is good, assuming things stay on track because we know that China needs to peak emissions by at least 2020.

I'm not too concerned about whether countries' commitments would limit climate change to 3.5C or 2C. The main thing is to continue the momentum that will foster the innovation that will solve this problem at a reasonable cost. We are seeing this technological change happening in a significant way.

Economics of the Singularity

I recently read part of this book: "The Lights in the Tunnel". I don't think it makes a lot of sense in terms of the outcome - the economy collapses from lack of demand as unemployment rises. But the basic idea is intriguing. Up till now machines and computers have taken over many tasks but they have acted as "q-complements" to labor in general and particularly skilled labor.* But could the future be one where all simple tasks are automated and the remaining jobs are too hard for people of ordinary ability to do?** And what happens when/if the "Singularity" occurs - when computers become more intelligent than humans? Does inequality continue to grow beyond anything we see today? Will redistribution increase? Is there a middle way? Is this whole idea totally mistaken? Or will humans be enhanced by genetic manipulation or implanting of non-biological computers?

There are economists and entrepreneurs/technologists who have thought about this. But I don't think anyone has any real answers yet.

* If an increase in an input increases the marginal product of another input then those two inputs are q-complements.
** More precisely the most that anyone is willing to pay for a human to do these automatable tasks is less than the minimum amount needed for subsistence.

Microsoft Academic Search

It turns out that Microsoft already had my idea of computing H-Indices for Institutions. I've got no idea how they calculate it, though. Probably based on the number of publications from that institution cited h times rather than the number of researchers cited h times as I suggested.

Microsoft Academic Search is another citation service that is apparently little known outside of library science circles. I only just discovered it despite being a citations junkie. According to them I have only 691 citations, which is just a bit above my count on RePEc and less than half my count on the Web of Science. They also think I'm still at RPI. But that is just my main entry. A whole bunch of my publications seem to be listed under separate identities at the University of Michigan and ANU as well as some smaller entries. So, I'm a bit skeptical about this service.

Contrary to some blogs I read, this service still seems to be under development.

CCEP Working Papers in November 2011

Downloads were still good this month after relaxing from last month's craziness:



The two top papers from last month continued to get a lot of hits. We added one new paper by Paul Burke. There is a second paper dated November that didn't quite squeeze into the sample. More details of the new papers below:

The National-Level Energy Ladder and its Carbon Implications

Paul J. Burke

Abstract: This paper documents an energy ladder that nations ascend as their per capita incomes increase. On average, economic development results in an overall substitution from the use of biomass to fulfill energy needs to energy sourced from fossil fuels, and then toward nuclear power and certain low-carbon modern renewables such as wind power. The results imply an inverse-U shaped relationship between per capita income and the carbon intensity of energy, which is borne out in the data. Fossil fuel-poor countries are more likely to climb to the upper rungs of the national-level energy ladder and experience reductions in the carbon intensity of energy as they develop than fossil fuel-rich countries. Leapfrogging to low-carbon energy sources on the upper rungs of the national-level energy ladder is one route via which developing countries can reduce the magnitudes of their expected upswings in carbon dioxide emissions.

In Search of a New Effective International Climate Framework for Post-2020: A Proposal for an Upstream Global Carbon Market

Mutsuyoshi Nishimura and Akinobu Yasumoto

Abstract: Given the urgency and the magnitude of emission cuts required to arrest the global temperature rise at an acceptable level (like 2 degrees Celsius), it is imperative that action to mitigate climate change is taken at the lowest cost. This can be done if a cost effective set of policy tools with a focus on carbon pricing is applied as broadly as possible across all emission sources. In view of the emerging consensus on the temperature target like 2 degrees Celsius, it is imperative that climate scheme caps global emissions rather than allowing governments to arbitrarily pledge their intended cuts. Global emissions must be contained within the limit of carbon budget that achieves temperature objectives. Emission allowances must be issued in accordance with such limit and be sold to the global demand of emitters. Such sales of carbon budget give rise to both the most accurate carbon pricing as well as new revenue that can be used for much needed climate financing for developing countries. A new climate regime along those lines would stop global warming at an acceptable level, provide a new large climate funding that would integrate developing countries to a global low-carbon growth and transformation and keep all economies thriving, whether they are developing, emerging or developed. The post-2020 climate regime must be nimble and effective, not unwieldy and least burdensome. It must also be durable and fully congruent to the economic realities of the coming decades