I am trying to make better user experience for my poll application and one of ideas I have for this is breaking multi choices poll to binary polls.

Is statistical result of a question with these 4 options (a or b or c or d) equals with sum of statistical results of these 6 binary questions?:

(a or b), (a or c), (a or d), (b or c), (b or d), (c or d)

assume I ask these questions from separate people and I ask these questions equally

for example: assume we have a society with 1000 members if we ask all of them the first question, people will answer it with this distribution: a:60%, d:20%, b:15%, c:5%

now assume if we ask those six binary questions from all members of that society and then we sum each vote of winners of each of those questions.

we ask 1000 times a or b and answer is a:600,b:400

we ask 1000 times a or c and answer is a:500,c:500

we ask 1000 times a or d and answer is a:100,d:900 ....)

we sum number of votes on a and it is 1300 =600+500+100

we do similar thing with b,c and d, is this result similar with result of first question?(a:60%, d:20%, b:15%, c:5% ) and is meaning of this result similar with first one? ===update=== @Bruno mention to inconsistency of result of this approach and I think it can be solved if we do not show all combination to all voters. If voter1 chooses "a" between "a" and "b" we do not show questions with any combination of "b".

  • 3
    $\begingroup$ I'm voting to close this question as off-topic because it is a better fit for CrossValidated.SE $\endgroup$ – AliceD Mar 1 '16 at 14:30

That's a good question, but... No way, man, they're not equal!! When you ask someone to choose between 4 options, their answer just means about their pick for the 4 options. And this, and only this, is the way to know the preference of people for those 4 options!

The way you're purposing could even generate an inconsistency... Suppose that one person prefer A over B and C, but not over D. It doesn't means that they would necessarily prefer D over B and C, nor means that A or D would be the choice among all. Our choices are not rational, so, if just the order you offer the 4 (or 2) options could induce people to prefer one of the options, asking all vs all could generate a lot of noise!

I'm not saying it's a bad idea, depends on the purpose of your question, of course, and on what you want to know!

I don't know what's your objective, exactly, but maybe you want to ask people to put them in order of importance! e.g., you might be wanting to ask people: "Classify these options in order of importance". This is what does it looks like, by your question, but it's just a guess at all.

It all will depend on what you want to know and the evolved specificity. In some cases, it would be a good idea, in others not.

  • $\begingroup$ what if I ask these questions by a little intelligence: if I ask a or b and voter picks a i will not ask questions that has "b" as a choice? $\endgroup$ – Mohammad Reza Esmaeilzadeh Feb 29 '16 at 7:31
  • $\begingroup$ I am trying to make user experience of my poll application more comfortable and one of ideas I have for this is breaking multi choices poll to binary polls $\endgroup$ – Mohammad Reza Esmaeilzadeh Feb 29 '16 at 7:33
  • $\begingroup$ @MohammadRezaEsmaeilzadeh, I understand your idea for making it more comfortable and easy to do. This is why I think asking the user to rank could be a good idea! Anyway, I don't believe the results could be comparable in any way. $\endgroup$ – Bruno Mar 14 '16 at 14:44
  • $\begingroup$ Hey @MohammadRezaEsmaeilzadeh, I was reading an interesting text on this matter a couple of days ago, while going deeper into my literature investigation on origins of Behavioral Finance. It's Marshack's chapter "Utilities, Psychological Values and the Training of Decision Makers", in Allais&Hagen (1979)'s "EXPECTED UTILITY HYPOTHESES AND THE ALLAIS PARADOX". I think it's worth to read!! I'm editing the answer to include this example. $\endgroup$ – Bruno Dec 17 '17 at 12:40

It slightly depends on what you are asking and you might want to post on one of the mathematical stack exchanges instead.

If you are asking "Do I get the same information by asking binary comparisons as by asking the four-option choice?" Then my answer is that you actually get more information, both about the pattern in the group and about a particular individual. Imagine if person X chooses A in the four option case, all you know is that they prefer A the best. If they then complete all 6 binary options, you will still find this out, but you will also know how they feel about B vs. C etc, so you will be able to find the complete ranking for each person. For the whole group, you would certainly be able to find the overall ranking, but I don't think you'd be able to sum the choices in the way you state.

If you are asking "will people respond in the same way?", then the answer is probably "No" because, as Bruno notes, people are not always rational and will be affected by the number of choices. Even if you imagine there is some random "noise" in their decision, then making six independent choices will produce some conflicts and potentially amplify this "error".

At any rate, it doesn't seem like this is going to make the user experience more comfortable! This is not my area, but you could read something about "rational choice theory" to get some background about the math/assumptions. https://en.wikipedia.org/wiki/Rational_choice_theory

  • $\begingroup$ I do not ask all of 6 questions from one person. I can ask multiple a or b and if he answered "a" then I can ignore all "b or c" and "b or d" questions. $\endgroup$ – Mohammad Reza Esmaeilzadeh Feb 29 '16 at 13:11
  • $\begingroup$ @MohammadRezaEsmaeilzadeh yes, you could. But that still requires 3 separate questions to get essentially the same information as the 4 option case (assuming they keep choosing A over everything else, it will get more complicated if they do not). You might want to randomly simulate some data and see what happens. $\endgroup$ – splint Mar 1 '16 at 7:37

Your Answer

