What is your favorite research method? Under what circumstances do you feel it's best applied? What are the pros and cons of using this methodology?

Question

Anonymous · Accepted Answer

"My favorite research method is A/B testing, especially when evaluating specific
features or design elements in a product to understand their impact on user
behavior. A/B testing is ideal when we want to determine the effectiveness of a
change by directly comparing two or more versions of an experience. It’s best
applied when we have a clear hypothesis, measurable outcomes, and a large enough
sample to achieve statistical significance.
Pros: One major advantage of A/B testing is that it provides clear, actionable
insights by isolating variables. This helps determine causation rather than just
correlation, which is valuable for making confident, data-backed decisions.
Additionally, A/B testing is highly iterative, allowing teams to continuously
optimize features based on real user feedback in real-time environments.
Cons: However, A/B testing does have limitations. It requires a significant user
base to achieve statistical significance, which may not be feasible for niche
products or low-traffic features. Also, it can be narrow in scope—testing one
variable at a time often doesn’t capture the complexity of user experiences
where multiple factors interact. Finally, if not designed carefully, A/B tests
can introduce biases or fail to account for confounding variables, leading to
misleading results.

Anonymous · Answer

My favourite research method is user interviews. The circumstance it would be
best applied in is the discovery phase of a product. When you are trying to gain
as much insight into the users and the context of where and how they would use
the product. The pros of using this methodology, is since it is generative in
nature, you gain a lot of rich data on the users and the context of the product
use. This research method can be unstructured giving the interviewer complete
freedom on how the interview is shaped or semi structured, giving the
interviewer more control in the questions asked and the insights discovered. A
con of using this methodology is that since it is generative in nature, unless
the interviewer is focused, the interview can get away from them, leading to an
information dump from the user. This could lead to waste of time and a lot of
unrelevant data.

What is your favorite research method? Under what circumstances do you feel it's best applied? What are the pros and cons of using this methodology?

Did you come across this question in an interview?

Answers

Also asked as

Try Our AI Interviewer