Doing good science is hard. There are so many ways how science can go wrong. Our studies are too small. We confuse correlation with causation. We are biased. Our hypotheses are not fairly tested. We overfit. We fail to understand the literature. Our data is not shared. And, arguably, we could avoid much of this by better planning our studies.
Talking with friends and mentees, it seems that everyone feels overwhelmed about all this. We got to find questions, decide on hypotheses, find past papers and an audience etc. So it is too much. Instead of planning, a lot of folks just start out by collecting data. And then hope that in the end everything comes together into a nice paper. But, arguably, this both makes for ugly papers and for non rigorous science.
So in any case, I just finished building an app. I made a long list of shortcomings that projects often have. I asked LLMs and colleagues. I ended up with about 25 pages worth of potential problems of scientific projects. Then I wrote a guide. Then I wrote a system that automatically checks guides against plans. Then I wrote an app that exposes this nicely to users (and therefore I had no time to blog in three weeks or so). And iterated to make it non overwhelming. I then did a lot of user testing. And now many users like it. There is no doubt that I am still missing a lot.
So, this is an announcement post. Try my scientific project planning app. Spend a few hours planning your next project. It will probably help. And then, please provide feedback (DM anywhere @kordinglab or email koerding@gmail.com).
Here is what I am hoping to learn:
Are there problems I flag that should not be flagged?
Are there problems I should flag that are not flagged?
Are there aspects of the workflow that should be more intuitive?
Did you get lost somewhere?
I would be truly grateful for any feedback.
Test the Scientific Project Planner here:
https://scientific-paper-planner-test.vercel.app/
