Once you have a detailed explanation of your research question, you are ready to fully engage with evaluation of the idea it expresses. However, ideas don’t exist in a vacuum. 

‍

• Evaluation requires examining a question in context. 
• Evaluating an idea can be intimidating! It’s easy to get stuck on unknowns. 

‍

Help is on the way! Here we have a new framework to break the process into manageable and memorable pieces. As we work our way through, we will first rate our research question in each category, then strategize to achieve the quality and clarity we need to move ahead.

As we work through the categories of this framework, we want to perform our evaluation in an actionable way. Rather than trying to use a quantitative scale, we therefore suggest a qualitative approach, summarizing whether each category overall reflects a strength or a challenge, or whether it’s not possible to fully evaluate a category without further information. 

• Strengths
  • What does your question in your context have going for it? 
  • These are your bragging points!
• Challenges
  • Where do you have enough information to know that the category is not ideal?
  • These are good places to pause and consider solutions or alternatives to improve your question and smooth the path ahead.
• Uncertainties
  • Where is unknown context blocking your ability to fully evaluate your research question? 
  • It isn’t always possible to know everything, but these areas help us focus our efforts on clearing away the doubts.

To evaluate our idea, we will adapt a mnemonic framework, coined in Hulley et al. (2007). 

Feasible

“Possible” isn’t always practical! How hard will your project be? Feasibility considerations include possible obstacles between your research question and a rigorous answer. 

‍

• A few dimensions to consider:some text
  • Time and Money.
  • Access and Limitations.
  • Skills.
  • Technical practicality and scope.

‍

Interesting

Shiny objects can appear dull from a different angle! An interesting research question should capture and retain attention from beginning to end.

• Every audience has different needs:some text
  • You!some text
    • Passion project or practical choice?
  • Peerssome text
    • Blank stares are a red flag. Ignore them at your peril.
  • Funderssome text
    • Will reviewers of your next grant be wide-eyed or skeptically squinting?

‍

Novel

Be intentional about replication! A novel question meets a clear need in the quest for understanding,

• A great study may confirm, refute, or extend knowledge.
• Uncertainty here is common. Look for help!

‍

Ethical

You can do it, but should you? An ethical question emerges from careful considerations of risk.

• A few dimensions to consider:some text
  • Minimizing harm.
  • Anticipating institutional review. some text
    • E.g. for human subjects, animal care and use, hazardous materials, etc.
  • Considering equity.

‍

Relevant

Let’s have a talk with your “why.” A relevant question generates a meaningful outcome with value to the broader community.

• Is your outcome truly relevant to your purpose?some text
  • Is it worth the time and effort?
  • Are your Who, What, and How, ideal for your Why?
• How will your purpose serve:some text
  • The broader research community?
  • Patients or other target groups?
  • Policymakers?

‍

Wrap-up Discussion

• How useful was this process? What possible rigor challenges may have been avoided?
• How would you like to apply what you have learned to early project development in your lab?

1. A carefully designed research question boosts rigor, prevents waste, and avoids many common pitfalls.

2. Supportive scientific communities are key to resolving challenges and uncertainties.