Great post. Fortunately, some journals do explicitly seek these goods. From the Journal of the APA's instructions to referees:
"The Journal aims to publish papers only of the highest quality. This means that we will reject many very good papers. You should bear this in mind when you review a submission. Recommend papers only when they stand out. In general, we hope to avoid papers that
• embed an interesting point in a lengthy discussion of the literature or argue preemptively against every possible objection;
• simply add an epicycle to a well-known thesis;
• are needlessly technical or introduce gratuitous formalism;
• are accessible only to specialists steeped in the topic.
The editors have a preference for papers that go out on a limb, that exhibit daring, that challenge the status quo, papers that defend surprising conclusions, even when the author’s arguments are not watertight or otherwise impervious to criticism. We rely heavily on reviewers’ judgment and good sense in this regard."
Yes, I think lots of philosophers broadly agree that these are the right goals. One thing I'm wondering is what *specific questions/directions* are most likely to orient referees to write their reviews with these values in mind.
JAPA's instructions look like a good start! But I do wonder if it might still be too easy to dismiss them as "boilerplate" and subconsciously return to our automatic "objection-finding" mode that we all learn in grad school. So I wonder if more specific questions (of the sort I suggest in the post) might help more...?
Honestly, "the pursuit of the unobjectionable" is the main reason I've tired of sending work to journals. And since I'm not interested in writing books, it basically means that I'm not publishing much -- which, given that I'm a full professor, is fine. But still: I wish it were otherwise.
I once received a referee report from Phil Studies that said the main problem with the view presented in my paper was that it was obviously false, but that it was "a testament to the undoubted philosophical skills of the author" that I was "able to make such an interesting case for an obviously false view." The editor rejected my paper on this basis. It was the first paper I submitted for publication as a young tenure-track prof, and I never sent it out again. (A version of the paper later appeared in an edited volume.)
I think this is a great essay, and I agree with basically all of it.
I would also add as a category to ask referees - is it well-written? I don't mean this to be strictly distinguished from content - I think that thought-provoking examples, illuminating characterisations, and memorable taxonomies are matters of both form and content, and play a great role in shaping a debate. By contrast, the safe and unobjectionable work you object to is often written in a leaden and formulaic style. I think we gain more, as readers, from lucid and lively defences of novel ideas, than from ponderous marches to obvious conclusions.
This is excellent advice! In the last couple of years, I have tried to move my editorial/referee work in this direction! For the sake of the profession, I hope others do as well.
It's almost as if blogging, with its measures of subscriber count, inlink count, and quality of commenters, does a better job than many journals, isn't it? It certainly filters out the boring and the verbose pretty well.
Sadly it doesn't seem to count for much professionally. But yeah, I think there are lots of philosophy blogs (from Bentham's Newsletter to Fake Nous) that are more reliably interesting to read than any philosophy journal.
It's interesting to think about ways to try to institutionalize this to make academic philosophy more like blogging, in a post-journal, "publish then filter" world. I'll probably write a separate post on this in the not-too-distant future.
I agree with the general point in this post. I think publication is too cautious, and I think if we implemented your suggestions it would have a net positive effect. It also reminds me of Adam Mastroianni’s excellent blog post, “Science is a strong-link problem,” which makes a similar point about how the peer review process in science serves largely to filter out bad science rather than identify and amplify good science.
However, I worry about some of the concrete proposals you suggest for what reviewers should look for. Ambitious, creative, clever, and novel arguments are probably some of the things we want, but an emphasis on these characteristics could still run afoul of Goodhart’s law (if we use a proxy X to measure target Y, X becomes the target, and stops being a good measure of Y).
I don’t want philosophy to become (if it isn’t often already) a game of signaling how clever one is by inventing some bizarre position or ingenious argument as ends in themselves. I want the field to converge on what’s true or useful, not whatever will appeal to our intellectual sensibilities.
Many famous papers are famous because he sparked conversation, and that might seem like a good thing, but I’m not sure it (always or entirely) is. Famous papers can become lightning rods for discussion because they mask ambiguities or foster confusions that lead the field down dead ends it can take decades to recover from, if ever. The controversy ends up serving as a kind of recreational center of gravity around which storms of papers are published, not because the field is making progress, but because it isn’t. In short: I worry that a “yes” to whether a paper would reshape a debate, get cited a lot, or be remembered later will often be a bad thing.
If we want academic philosophy to improve, we should pay greater attention to reevaluating the methods and aims of the field itself, and that means more metaphilosophy. We need to get a better handle on what it is we’re doing, what our methods are, and how (or even whether) those methods work.
One reason for this state of affairs is the imitation of the sciences in analytic philosophy. I make this point, amongst others, in this recent paper in Metaphilosophy:
Great post. Fortunately, some journals do explicitly seek these goods. From the Journal of the APA's instructions to referees:
"The Journal aims to publish papers only of the highest quality. This means that we will reject many very good papers. You should bear this in mind when you review a submission. Recommend papers only when they stand out. In general, we hope to avoid papers that
• embed an interesting point in a lengthy discussion of the literature or argue preemptively against every possible objection;
• simply add an epicycle to a well-known thesis;
• are needlessly technical or introduce gratuitous formalism;
• are accessible only to specialists steeped in the topic.
The editors have a preference for papers that go out on a limb, that exhibit daring, that challenge the status quo, papers that defend surprising conclusions, even when the author’s arguments are not watertight or otherwise impervious to criticism. We rely heavily on reviewers’ judgment and good sense in this regard."
Yes, I think lots of philosophers broadly agree that these are the right goals. One thing I'm wondering is what *specific questions/directions* are most likely to orient referees to write their reviews with these values in mind.
JAPA's instructions look like a good start! But I do wonder if it might still be too easy to dismiss them as "boilerplate" and subconsciously return to our automatic "objection-finding" mode that we all learn in grad school. So I wonder if more specific questions (of the sort I suggest in the post) might help more...?
Honestly, "the pursuit of the unobjectionable" is the main reason I've tired of sending work to journals. And since I'm not interested in writing books, it basically means that I'm not publishing much -- which, given that I'm a full professor, is fine. But still: I wish it were otherwise.
I once received a referee report from Phil Studies that said the main problem with the view presented in my paper was that it was obviously false, but that it was "a testament to the undoubted philosophical skills of the author" that I was "able to make such an interesting case for an obviously false view." The editor rejected my paper on this basis. It was the first paper I submitted for publication as a young tenure-track prof, and I never sent it out again. (A version of the paper later appeared in an edited volume.)
I think this is a great essay, and I agree with basically all of it.
I would also add as a category to ask referees - is it well-written? I don't mean this to be strictly distinguished from content - I think that thought-provoking examples, illuminating characterisations, and memorable taxonomies are matters of both form and content, and play a great role in shaping a debate. By contrast, the safe and unobjectionable work you object to is often written in a leaden and formulaic style. I think we gain more, as readers, from lucid and lively defences of novel ideas, than from ponderous marches to obvious conclusions.
This is excellent advice! In the last couple of years, I have tried to move my editorial/referee work in this direction! For the sake of the profession, I hope others do as well.
It's almost as if blogging, with its measures of subscriber count, inlink count, and quality of commenters, does a better job than many journals, isn't it? It certainly filters out the boring and the verbose pretty well.
Sadly it doesn't seem to count for much professionally. But yeah, I think there are lots of philosophy blogs (from Bentham's Newsletter to Fake Nous) that are more reliably interesting to read than any philosophy journal.
It's interesting to think about ways to try to institutionalize this to make academic philosophy more like blogging, in a post-journal, "publish then filter" world. I'll probably write a separate post on this in the not-too-distant future.
I agree with the general point in this post. I think publication is too cautious, and I think if we implemented your suggestions it would have a net positive effect. It also reminds me of Adam Mastroianni’s excellent blog post, “Science is a strong-link problem,” which makes a similar point about how the peer review process in science serves largely to filter out bad science rather than identify and amplify good science.
However, I worry about some of the concrete proposals you suggest for what reviewers should look for. Ambitious, creative, clever, and novel arguments are probably some of the things we want, but an emphasis on these characteristics could still run afoul of Goodhart’s law (if we use a proxy X to measure target Y, X becomes the target, and stops being a good measure of Y).
I don’t want philosophy to become (if it isn’t often already) a game of signaling how clever one is by inventing some bizarre position or ingenious argument as ends in themselves. I want the field to converge on what’s true or useful, not whatever will appeal to our intellectual sensibilities.
Many famous papers are famous because he sparked conversation, and that might seem like a good thing, but I’m not sure it (always or entirely) is. Famous papers can become lightning rods for discussion because they mask ambiguities or foster confusions that lead the field down dead ends it can take decades to recover from, if ever. The controversy ends up serving as a kind of recreational center of gravity around which storms of papers are published, not because the field is making progress, but because it isn’t. In short: I worry that a “yes” to whether a paper would reshape a debate, get cited a lot, or be remembered later will often be a bad thing.
If we want academic philosophy to improve, we should pay greater attention to reevaluating the methods and aims of the field itself, and that means more metaphilosophy. We need to get a better handle on what it is we’re doing, what our methods are, and how (or even whether) those methods work.
One reason for this state of affairs is the imitation of the sciences in analytic philosophy. I make this point, amongst others, in this recent paper in Metaphilosophy:
https://philarchive.org/rec/CONSAA-8