It’s quite common for me to have students tell me “the analysis didn’t work out” or “the figure looks bad” or “I don’t have any useful results.” And it’s also quite common for the students to be wrong. Sometimes, students have amazing results but are all disappointed because the results aren’t what they had expected. These students fail to see the data for what they are. More commonly, the students may be right in that the data aren’t that great, but usually I can see something in the data that the student didn’t. In either case, it is important that we look at the data together, because jointly we will see more than either of us individually would have seen.
However, while some students are happy to show me their “failed” analyses and complain about how nothing works, others are more reserved, sometimes to the point of reluctance. The latter students don’t feel comfortable showing me their preliminary results, or sometimes any results at all, unless they think the work is completed.1 These students are probably under the mistaken impression that I will judge them for “failed” analyses, or that I expect a complete analysis, with proper interpretation of all data points, at all times, or that having remaining open questions means I’ll think the student did a poor job. On the contrary, I want to see all these preliminary, incomplete, and confusing results. The data are the data, and I want to be able to draw my own conclusions about them.
Furthermore, I submit that unless you know for sure you made a mistake (say, your code has a bug and you know it), you will always be better off discussing your work with other people than silently deciding it’s not good enough. And if your results seem too trivial or wrong to talk about them with your advisor, then at least talk them through with a fellow student or postdoc.2 Think about it this way: every time you generate a result or make a figure, and then you delete it before you show it to somebody else, you’re preempting the possibility that somebody else might see something useful in your work. And while a lot of what you’re doing may indeed be worthless, I’d argue that unless you’re a complete disaster, you’ll probably do at least one thing every day that is actually worthwhile. So every day, you should produce some sort of result that you then discuss with somebody else.3
Importantly, the necessity of sharing preliminary work with other people doesn’t stop once you graduate. Just because you have a PhD doesn’t mean that you’re suddenly able to always see the data exactly for what they are. Sometimes even the most experienced scientists are overly attached to a particular hypothesis or miss some critical detail in their data. That’s why many experienced scientists routinely talk to their colleagues about work in progress, why they present preliminary results at conferences or seminar talks, and why they request comments on manuscript drafts and preprints. In my mind, even traditional peer review serves primarily the purpose of having another set of eyes look over the data and check whether the authors are over- or underhyping their results.4
The only thing that really changes once you graduate and/or complete your postdoc is that suddenly there is no adviser anymore who makes you show him/her your latest work. Now it’s entirely on you to seek out the advice you need and to receive the feedback that will make your work better. And, if you are advising students yourself, it’s now your job to make them talk to you and show you what they’re doing, warts and all.
If you’re my student and you think this blog post is about you, let me tell you: you’re not the only one.↩︎
But realize that sometimes students talk each other down. Just because your fellow grad students say what you did was stupid doesn’t mean it actually was; they may just lack perspective.↩︎
You might argue that you’re working in a field where results accumulate slowly, hence you can’t possibly discuss results from individual days. E.g., you’re collecting beetles in the field, and you need three field seasons before you have enough data to test your hypothesis. My response is that nevertheless there are tons of preliminary data that you should discuss with somebody. E.g., if you’re collecting beetles every day, you know how many you found each day, where you found them, what physical characteristics they had, etc., and how these quantities change over time. All of these are worthwhile preliminary results that you should look at, graph, and discuss with a third party.↩︎
The second case, where reviewers see more in the data than the authors, is more common than you may think. Many papers get really good only after a solid round of peer review, and frequently the reviews in those cases are not “X is wrong” but rather “You have done X but you should also do Y, and in combination you could conclude Z which would be really cool.”↩︎
