The system is human

Recently a client got turned down for a small amount of money she’d applied for from her university’s internal research grants competition.

Not surprisingly, she was devastated.

The gremlins were telling her that if she couldn’t even get this small amount of money from this not terribly prestigious institution that she clearly isn’t good enough to have an academic career.

It’s hard to cheer someone up in that situation.

I want to recognize how awful it feels and how legitimate those feelings are.

And at the same time I want to tell my client that the gremlins are wrong.

Other academics suggest that it’s okay to feel crappy, and maybe to swear at the idiots who made the decision (in the privacy of your own home, obviously), and then pick yourself up and keep going.

Which is fine, but as someone said “I keep hearing lots of reinforcement about how stupid the system is. But how does that help me feel better about it?”

Good point.

Maybe they aren’t idiots

Here’s a story from back when I was an academic and sat on one of those internal committees. Not sure if it’ll help but it might illumanate the process.

We had some money in the School of Social Sciences to provide research funding for doctoral students.

Decisions about who would receive the money were made by a multi-disciplinary committee with members from each department in the school. I was our department’s representative.

When an application from one of Cultural Studies students came up for discussion in the meeting, someone from Economics asked if her work was “real research”.

How I reacted then

I was shocked and angry. But I did my best to respond rationally, reassuring him that yes, indeed, this was real research.

In my head, I was calling him an idiot. And a rude idiot for saying that out loud.

I am not that keen on economics research. I don’t think mathematical modelling is appropriate for all questions. And too often, the assumptions that ground an economics project are given less prominence than they should be, especially when considering the relationship between economics research findings and any possible applications in the “real world”.

But I would never say that out loud in a meeting like that one. In that meeting, I took the position that economics is a valid discipline, albeit not my discipline, and that economics applicants should be judged by the standards of their discipline.

The fact that the economist on the committee asked whether a cultural studies student’s work was “real research” seemed to me at the time to be an indication that he was not extending my discipline the same courtesy.

Hindsight is 20/20

Thinking about the same situation now, I realize that my interpretation was probably wrong.

What the economist on the committee was doing was articulating the question that all of us must ask in a multi-disciplinary adjudication committee. His wording was not eloquent but I shouldn’t fault him for that.

In order to judge applicants by the standards of their discipline, we sometimes need to ask the representative of that discipline to clarify those standards.

Multi-disciplinary peer review is fiendishly difficult

Although we all use similar words to describe the research process, we mean different things by them.

The word “discipline” has meaning. Different disciplines are precisely different ways of asking questions, seeking answers, and supporting conclusions.

Even within most disciplines, “excellence” is routinely debated and revised to account for new sources, new methods, and new ways of disseminating information.

Many disciplines are also concerned with other equally contested terms like “originality” or “innovation”.

Committee members are human

They bring their own understandings of the standards to the table.

They bring their own imperfect understandings of the bigger picture to the table.

I agree with Michele Lamont that the vast majority of academics who sit on review committees, whether for small internal grants or for prestigious national fellowships, are genuinely committed to making fair decisions based on standards of excellence.

They are doing their best to compare applications that are almost fundamentally incomparable, to reconcile possibly incommensurable definitions of excellence, and to judge the potential impact of an unpredictable process (research itself).

And sometimes they aren’t doing their best. Sometimes they are overcommitted, tired, and giving less than the optimum level of attention and care to the task. Sometimes they have agendas they are trying to advance and other members of the committee are either oblivious or are not effectively preventing them from doing so.

All human behaviour.

Their judgments are not global

It is perhaps better to keep in mind that you are being judged in a particular context.

The fact that you did not receive this funding, or get this article published in this particular journal, or get this particular job or scholarship is not a judgment on the overall quality of your work.

Nor is it an indication that you won’t ever receive funding, get published, get a job or a scholarship.

It is only a judgment of how this particular panel of peers judged this particular application/manuscript/whatever in relation to the others in this competition given the particular constraints they were under.

You might be doing excellent work in an area they don’t understand (or don’t value).

They might have genuinely wanted to fund more research than they did but were operating under budget constraints that they do not control.

They might have been operating under guidelines that privilege some applicants over others on criteria additional to the quality of the proposal.

Small comfort

It’s not much. I know.

But I also know that while you have to compete and you have to do your best to put forward your best performance in each competition, you will not always come home with the medal.

It feels crappy. And that’s okay.

But it doesn’t necessarily mean you are not cut out for this.

You don’t have to do this alone

If you would find it helpful to have someone to discuss the bumps in the road with. Someone to help you figure out what you really want to be doing and how to get there. I do that.

Athletes have coaches. There’s no reason you shouldn’t, too.

I can help you evaluate the little setbacks, improve your performance, and keep moving forward.

I can also help you figure out what your goals are. Because sometimes we’re so lost in what we were told we should be doing we lose sight of what really got us into this stuff in the first place. Or the other paths open to us from here.