New to the lab? Start with Getting Started. Supporting materials are now in Resources, Journals, and AI. Exercise answers are stored in this browser when storage is available; use Save notes to keep a copy.
Research Guide · AIT Brain Lab
·
Generation
Refinement
Validation
Testing
Storytelling
Five stages of the research journey0% complete
How to Develop a Research Contribution
Develop one question through generation, refinement, validation, testing, and storytelling. Use the exercises to produce work for discussion and revision.
Use ← →, the buttons, or the session menu. Press F for fullscreen.
Research Journey
Stage
Main question
Idea Generation
Where could a good idea come from?
Idea Refinement
How to sharpen the idea?
Idea Validation
Is the idea new and important?
Idea Testing
How can we test the idea convincingly?
Idea Storytelling
How to storytell the idea?
01
Idea generation
Where could a good idea come from?
Idea generation
Four ways to find a starting idea
Notice a problem. What problem do people still face?
Notice a surprise. What happened differently from what you expected?
Borrow a theory. Can a theory help explain a problem you care about?
Extend a recent paper. What useful question could come next?
1. Notice a problem
Look at the people and situations around you. What problem do you notice repeatedly?
When does it happen, and why does it matter?
Problem:
People may find it difficult to understand and reflect on their emotions.
When:
This can happen when people cannot connect their feelings with what happened during the day.
Why it matters:
Understanding emotions can help people manage their feelings and communicate with others.
Initial idea:
Can we use bodily sensors to help people understand their emotions?
What the paper did:
The researchers designed a diary that combines body data, personal notes, and pictures to
support emotional reflection.
Affective Diary system · Interaction Design Foundation
Exercise
Think about a problem you repeatedly notice in study, work, or daily life.
Problem
When
Why it matters
Initial idea
How you could study it
Check your answer
Name a concrete person or situation and a repeated difficulty. Explain the consequence. “People need a better app” does not yet identify what is difficult or why it matters.
Bring to discussion: your answer, one uncertainty, and a proposed next step.
2. Notice a surprise
Start from a result that did not match what you expected. The difference can become a research question.
Situation:
Language models can role-play characters with different personalities.
What researchers expected:
A more agreeable AI character should be friendly and helpful.
What actually happened:
More agreeable characters were often more likely to agree with the user, even when they should challenge the user's view.
Initial idea:
Does making an AI character nicer also make it more sycophantic?
What the paper did:
The researchers tested personas with different levels of agreeableness and measured how often the models agreed with users.
AI sycophancy · India Today
Exercise
Use an observation from your study, work, experiment, or daily life.
Situation
What you expected
What actually happened
Initial idea
How you could study it
Check your answer
State the expectation and observation separately. Consider a measurement or implementation error before treating a surprise as a new phenomenon. Propose an observation that could distinguish the explanations.
Bring to discussion: your answer, one uncertainty, and a proposed next step.
3. Borrow a theory
Start with a problem you care about. Use a theory from another field to explain it and guide what to test.
Explain what the theory predicts or helps you observe. Naming a theory is insufficient. Your question and proposed evidence should change because of the theory.
Bring to discussion: your answer, one uncertainty, and a proposed next step.
4. Extend a recent paper
Start with a recent paper. Find what it did not answer, then ask one focused next question.
What the paper studied:
A Bayesian visualization that helps people manage many issues while negotiating with AI.
Main finding:
The visualization helped people reach agreements more efficiently without reducing their control.
What is still difficult or unknown:
Showing the most promising options may focus people too early and discourage them from exploring unusual but better solutions.
Next idea:
Does Bayesian guidance improve negotiation speed while reducing creative exploration?
How it could be studied:
Compare negotiation with and without the visualization. Measure speed, variety of offers, and whether people discover the best shared outcome.
What comes next? · Maya Say
Exercise
Choose one recent paper related to your interests. Read enough to complete the table.
What the paper studied
Main finding
What is still difficult or unknown
Next idea
How you could study it
Check your answer
Identify what the paper actually establishes, then a consequential unresolved question. Check newer citing papers before claiming that the question remains open.
Bring to discussion: your answer, one uncertainty, and a proposed next step.
Summary: Idea generation
Starting point
Question to explore
A repeated problem
Who faces it, and what remains difficult?
A surprise
Which expectation failed, and why?
A useful theory
What does it explain or help us observe?
A recent paper
Which important question remains unanswered?
Choose one idea. Explain why the answer matters, how you could begin, and the biggest uncertainty.
How to choose between two promising ideas
An ambitious chatbot may be interesting but require data or expertise you cannot access. A narrower question about when users misunderstand its advice may be feasible and reveal something useful. Compare importance, your curiosity, available support, and the first investigation. Difficulty alone is not a reason to choose a project.
02
Idea refinement
How can we turn a broad idea into a strong research question?
Idea refinement
Five ways to make an idea stronger
Challenge an assumption. What are you taking for granted?
Consider another explanation. What else could cause the result?
Use first principles. What must be true at the most basic level?
Think about the whole system. How do the parts affect one another?
Consider second-order effects. What may happen after the first benefit?
1. Challenge an assumption
Find what your question assumes is already true. Rewrite it so the assumption can be tested.
Starting question: Can Bayesian reasoning help people make better decisions in complex negotiations?
Hidden assumption: If probabilities are mathematically correct, people will understand and use them correctly.
Challenge: People may misunderstand probabilities, trust them too much, or ignore information that is not shown.
Better question: When does Bayesian support improve negotiation decisions, and when does it create false confidence?
Why it is better: The question identifies a condition and a possible failure, rather than assuming the method is helpful.
Visible claims and hidden assumptions · Lead Beyond Code
Exercise
Starting question
Hidden assumption
Reason it may fail
Better question
Useful comparison
Check your answer
Name an assumption your original question depends on. Explain when it might fail and how the revised question could reveal that failure. Adding an extra variable without a reason is insufficient.
Bring to discussion: your answer, one uncertainty, and a proposed next step.
2. Consider another explanation
Ask what else could produce the same result. Compare your explanation with the strongest alternative.
Starting question: Does Bayesian visualization improve negotiation performance?
Expected explanation: Performance improves because Bayesian reasoning helps people understand uncertainty.
Alternative explanation: Any clear visual summary may reduce effort; the improvement may not come from Bayesian reasoning.
Better question: Does Bayesian information help beyond the benefit of a simpler non-Bayesian visualization?
Why it is better: It separates the proposed explanation from a simpler competing explanation.
More than one possible path · Unsplash
Exercise
Expected result
Your explanation
Alternative explanation
Better question
Evidence needed
Check your answer
Your alternative should plausibly produce the same observation. Propose a comparison that separates the explanations; repeating the original comparison with more participants may not do so.
Bring to discussion: your answer, one uncertainty, and a proposed next step.
3. Use first principles
Break the problem into its most basic needs, decisions, and constraints before choosing a method.
Starting question: How can we use Bayesian reasoning to improve negotiation?
Break it down: Negotiation requires people to express preferences, compare trade-offs, handle uncertainty, and choose an offer.
Basic need: The user does not need “Bayesian reasoning” itself; the user needs help seeing uncertain trade-offs without losing control.
Better question: Which uncertain trade-offs should the system estimate, and how should it show them so people can still make the final choice?
Why it is better: It begins with the human decision problem, then selects mathematics only where it is useful.
First-principles thinking · Laws of Software Engineering
Exercise
Starting question
Basic human goal
Smallest parts
Necessary role of the method
Better question
Check your answer
Describe the human goal without mentioning your favourite method. Explain which part requires the method and which part a simpler approach could handle.
Bring to discussion: your answer, one uncertainty, and a proposed next step.
4. Think about the whole system
Identify the people, technology, and feedback loops that influence one another over time.
Starting question: Can Bayesian guidance help a person choose better negotiation offers?
Parts of the system: The person, AI agent, probability model, visualization, offers, and responses all influence one another.
Feedback loop: A probability display changes the next offer; that offer changes the AI response; the response changes the next probability.
Better question: How does Bayesian guidance change the negotiation over several rounds, including offers, trust, and the AI response?
Why it is better: It studies the changing interaction, not only one user choice or the final agreement.
Systems-thinking workshop · Emile Silvis
Exercise
People and components
Important connections
Feedback loop
Possible system behavior
Better question
Check your answer
Trace one complete feedback loop. Explain how one change affects another and eventually returns to influence the starting point. A list of components alone is not a systems explanation.
Bring to discussion: your answer, one uncertainty, and a proposed next step.
5. Consider second-order effects
Look beyond the immediate benefit. Ask how people may adapt and what new effect may appear next.
Starting question: Can Bayesian guidance help people reach an agreement faster?
First-order effect: People may identify promising offers and converge more quickly.
Second-order effect: They may depend on the guidance, explore fewer unusual offers, or overlook options outside the model.
Better question: Does Bayesian guidance improve convergence while reducing exploration or independent judgment?
Why it is better: It tests both the immediate benefit and the new problem that the benefit may create.
First- and second-order consequences · Lead Beyond Code
Exercise
Expected benefit
Behavior change
Second-order effect
Better question
Measures
Check your answer
Link the immediate benefit to a later behaviour change. Identify evidence for both. Reduced effort does not automatically imply reduced independent judgment.
Bring to discussion: your answer, one uncertainty, and a proposed next step.
Summary: Idea refinement
Use the technique that reveals a weakness in your current question. You do not need to apply all five every time.
Question
What changes?
Can we use Bayesian reasoning in negotiation?
A method and a topic; the knowledge we seek is unclear.
Does a Bayesian display improve negotiation outcomes?
A testable effect; a simpler visual summary may explain it.
Does uncertainty information improve outcomes beyond a matched visual summary?
A comparison that helps distinguish two explanations.
Illustrative follow-up question, not an established gap or finding.
Check the refinement
Explain which assumption changed, what evidence the new question requires, and why the difference matters. A longer title or a more advanced method does not automatically create a stronger question. If a simpler explanation is sufficient, revise the mechanism you claim.
03
Idea validation
Is the idea truly novel and worth studying?
1. Validate the research gap
Your knowledge gap is not yet a research gap.
Is there really a gap? Past work must leave an important question unanswered, inconsistent, or poorly supported.
Is the gap in the field—not only in your reading? Start with strong CHI, ACL, and NeurIPS papers where relevant. Trace references and newer citing papers; search neighbouring fields and specialist venues. State what you searched and what uncertainty remains. A search cannot prove that no answer exists.
Is it a clear next step? Aim for a focused advance that changes understanding or capability. A model swap, new domain, or combination of components needs evidence of a meaningful difference. You do not need a breakthrough.
Is there a research contribution? Building an application or apparatus is not enough. It must answer a research question or produce evidence, design knowledge, or a tested technical capability.
Past work APast work BPast work C
Important question still unanswered
A research gap exists in the literature—not only in your reading.
Build the gap from the closest work
Separate what a paper demonstrates from the next question you infer.
A whole-interface comparison does not by itself isolate the value of uncertainty information from other interface differences.
Candidate next question
Does uncertainty information help beyond a matched visual summary?
Still to verify
Search component comparisons and neighbouring decision-support research. A question left open by one paper may already be answered elsewhere.
Compare weak and stronger gap statements
Weak: “Nobody has built our combination of widgets.”
Stronger candidate: “We need to distinguish the benefit of uncertainty information from the benefit of a clear summary.” This states a meaningful difference, but it becomes a defensible field-level gap only after comparing the closest verified work.
Exercise
Past work A
Past work B
Past work C
However
We investigate / propose / develop
Check your answer
Use specific verified papers for the literature rows. State what each establishes and the meaningful distinction your work would test. “No paper uses our combination” is insufficient. Include your search boundary and the uncertainty that remains.
Bring to discussion: your answer, one uncertainty, and a proposed next step.
04
Idea testing
What kind of contribution will the paper make?
Choose the contribution first
Different contributions need different methodologies
Experimental: evidence about an effect or causal relationship.
Design research: design knowledge, principles, or rationale.
Artifact or system: a working technical capability.
Exploratory qualitative: understanding of a new situation.
Model or dataset: measurable computational performance and limitations.
Check three kinds of validity
Internal validity
Did X really cause the change in Y?
Check: confounds, controls, randomization, counterbalancing, and alternative explanations.
Construct validity
Did you measure the concept you claim to study?
Check: operational definitions, validated measures, manipulation checks, and whether the task represents the construct.
External validity
Where should the result generalize?
Check: participants, tasks, settings, realism, cultural context, and replication.
Experimental contribution
Typical question: Does X affect Y? Use an experiment when the contribution is evidence that an independent variable changes a dependent variable.
Specify: IV and levels; DV and operational measure; between-, within-, or mixed design; task; procedure; sample; randomization or counterbalancing; controls and likely confounds.
Typical evidence and analysis: Descriptive plots; t-test; ANOVA; regression or mixed-effects models; effect sizes, confidence intervals, manipulation checks, and assumption checks.
Typical question: What design knowledge can we produce? The contribution is theory-informed rationale, principles, tensions, or implications—not merely proof that one interface is better.
Specify: human need; grounding theory; design goals; alternatives; rationale; iterations; context; and what transfers beyond the artifact.
Typical evidence and analysis: Formative interviews or observations; co-design; workshops; diaries; field deployment; think-aloud; thematic analysis; reflection across iterations and triangulation.
Attention regulation and feedback · Niksirat et al.
Artifact or system contribution
Typical question: What new capability does the artifact provide? Its evaluation should show that it works, can be used, and meets the technical requirements connected to its claim.
Specify: capability; architecture; implementation; intended users and tasks; baseline; success criteria; limitations and deployment constraints.
Typical evidence and analysis: Accuracy, latency, reliability, scalability, or resource use; usability and task performance; interviews or observations; benchmark comparison and ablation when components matter.
Typical question: What is happening in this new situation? This approach fits a topic, community, behavior, or technology that is too new for precise variables or hypotheses.
Specify: phenomenon and context; why exploration is needed; purposeful participant selection; researcher position; protocol; how interpretations will be checked.
Typical evidence and analysis: Semi-structured interviews, observations, fieldwork, diaries, focus groups, or documents; thematic analysis, grounded theory, or qualitative content analysis; negative cases, reflexivity and triangulation.
Typical question: How well does it work—and where does it fail? Average scores are insufficient: evaluation must expose meaningful strengths and weaknesses.
Specify: data source and representativeness; train/validation/test separation; metrics; strong baselines; reproducibility; leakage, bias, and ethical risks.
Typical evidence and analysis: Benchmark comparison; statistical uncertainty; ablations; error analysis; robustness and stress tests; subgroup analysis; human evaluation when automated metrics miss quality.
Choose one topic: mental health, simulated humans, ontology + LLM, medical VQA, or Raman / EEG. Design only the evidence needed for one clear contribution.
Research question
Contribution type
Closest paper
Method and procedure
Analysis and quality check
Check your answer
Connect each intended claim to a measure, observation, or analysis. Explain why the comparison is fair and what result would change your interpretation. Match the method to the contribution; an interview is not inherently weaker than an experiment.
Bring to discussion: your answer, one uncertainty, and a proposed next step.
Summary: Idea testing
Worked planning example: does uncertainty information add value beyond a clear summary?
Design decision
Reasoning
Competing explanations
Users benefit from uncertainty information; or any clear summary reduces effort.
Comparison
Compare an uncertainty display with a credible summary. Match task, underlying evidence, layout, training, and time as far as the question permits.
Evidence
Measure decision quality and understanding of uncertainty. Faster completion or satisfaction alone cannot establish better judgment.
What could weaken the claim?
If a sufficiently sensitive comparison shows no meaningful advantage, revise the claim that uncertainty information adds value in this setting.
Before running the main study
Check that the summary is genuinely useful, the two interfaces differ as intended, and the measures reflect your concepts. Justify the sample and analysis for the question. If the comparison is too noisy or underpowered, a null result is inconclusive.
Agree a pilot budget and review point. A pilot checks feasibility and measurement; do not repeatedly alter the test until a preferred result appears. For exploratory work, specify the next observation and how interpretations will be challenged instead of inventing a causal hypothesis.
05
Idea storytelling
How can readers follow one clear research story?
People remember stories
A research paper is easier to understand and remember when five parts form one connected story. The problem is the turning point: it moves readers from what is known to what your study must do.
Background: Establish the important topic and synthesize what is known.
Problem: Turn the story with “However.”
Methodology: Explain how the paper answers the problem.
Findings: Reveal what the evidence shows.
Contribution / significance: End with what changes because of the work.
Tip—choose the verb carefully: investigates a relationship; explores a new situation; proposes an idea; develops an artifact; evaluates performance or usefulness.
Photo: Laura Kapfer · Unsplash
One reusable template
Write the story in five parts
Background: Emotion regulation is important because … Prior work shows … (use many relevant citations)
Problem:However, existing digital meditation tools mainly use spoken instructions or fixed content, giving users limited ways to regulate attention through their own action.
Methodology: This paper proposes an attention-regulation framework and develops Pause, an interaction based on slow, continuous movement.
Findings: The key findings are …
Contribution / significance: This paper contributes … This matters because …
Your title is the first line of the story
Mediocre title
“A Study of a Meditation Application”
Too broad. Readers cannot see the mechanism, contribution, or research area.
Stronger title
“Pause: Using Slow, Continuous Movement to Support Attention Regulation in Interactive Mindfulness Meditation”
Use clear research keywords so the right readers can find the paper. A short, meaningful hook can help them remember it.
Example: Revise the reasoning, then the wording
Illustrative follow-up proposal. The study has not been run.
Initial paragraph
“AI negotiation is important. Our new Bayesian tool will make people better negotiators and preserve their agency.”
Supervisor comments
What does “better” mean? What is the closest comparison? How will you measure agency? You are predicting success before collecting evidence.
Revised paragraph
“We will test whether uncertainty information improves decision quality beyond a matched visual summary. Both conditions will use the same negotiation task and underlying evidence. We will measure decision quality and uncertainty understanding. Any claim about agency will require separate evidence of meaningful user control.”
The revision makes the comparison, evidence, and claim boundaries explicit. Findings remain pending.
What improved?
The paragraph now states an answerable question, names the strongest comparison, and connects measures to claims. It does not promise a positive result. After the study, replace the plan with what was done and what the evidence supports, including uncertainty and failures.
Exercise
Background
Problem
Methodology
Findings
Contribution / significance
Check your answer
Each sentence should prepare the next: the gap motivates the question, the method can answer it, and the finding supports the contribution. If evidence is not available, write “findings pending”. Do not fill the results row with a prediction presented as fact.
Bring to discussion: your answer, one uncertainty, and a proposed next step.
