Further Interruption Survey Analysis

Note from Ben: After just one reader commented that there may be some flaws is Koopmann’s analysis (these flaws don’t change the overall conclusion about the data), Koopmann took upon himself to invest considerable time in re-working the data. Below is the re-work of that data directly from him, with the original article (here) preserved as it was originally published.

Methodology

The poll was presented as post with an embedded iframe on The Brooks Review website. The post, titled Interruption Survey, contained the following introduction and note:

I’d appreciate you taking a moment to fill this out (note this is an iframe, you need to scroll the frame to get to the “finish” button): Note: I’m assuming you want to interrupt someone and they are using the device.

Instrument

The poll consisted to two questions:

  1. I am more likely to interrupt someone using:
    • a pad of paper to write.
    • an iPad.
    • a Desktop.
    • a Laptop.
    • an iPhone.
  2. I am least likely to interrupt someone using:
    • a pad of paper to write.
    • an iPhone.
    • a Desktop.
    • an iPad.
    • a Laptop.

Each of these items was required and only one choice was allowed per item.

Results

Respondents

There were 499 submissions received between 2014-06-11 18:44:51 and 2014-06-14 14:34:49[^GMT?].

Note that some percentages may not sum to 100% due to rounding.

Geography

The majority of respondents (58%) were from the United States; the next-largest group of respondents (9%) was from the United Kingdom. The remaining 33% of respondents came from 49 other countries with each country contributing less than 5% of respondents.

See Respondents by Country for full breakdown.

Operating System

Of the 441 (88%) respondents where the platform was detected, the overwhelming majority (93%) were running an iOS (56%) or OS X (38%) device. Windows (5%) and Linux (2%) were the remaining identified operating systems; the tool was unable to identify platform for the remaining 58 (12%) respondents.

iOS 7.1 (52%) and Mac OS X 10.9 Mavericks (33%) were the largest group of respondents; 9 respondents (2%) were bleeding-edge dev-types running iOS 8; 14 respondents (3%) were old-school OS X 10.1 Puma (released Sep 2001).

See [Respondents by Operating System & Platform][respondentsByPlatform] and [Respondents by Browser Agent][respondentsByAgent] for full breakdowns.

Q1. I am more likely to interrupt someone using

The frequency plot below shows the portion of responses per option.

Q1 Frequency Plot

We can see significant deviation1 in the deviation plot below if we assume an equal 20% of responses per response option (100% across 5 options). Specifically the an iPhone2 option was selected at a higher rate and the a Laptop3 , a pad of paper to write4, a Desktop5, and an iPad6 response options were all selected at lower rates than expected.

Q1 Deviation Plot (assumed equal distribution)

OS X Respondents

Q1 Frequency Plot for OS X users

If we assume an even distribution of responses per response option we can see that OS X users (33% of respondents) responded with significant deviation7 as well. The an iPhone8 option was selected at a higher rate and the a Laptop9 and a pad of paper to write10 options were selected at lower rates than expected. The a Desktop11 and an iPad12 response options were selected at a rate consistent with expectations.

Q1 Deviation Plot for OS X users (assumed equal distribution)

If instead we assume OS X respondents would respond similar to the entire respondent pool (identical to the pattern shown in Q1 Frequency Plot above), we can see that these users responded as expected13. Since these respondents did not deviate on the whole, there is no need to inspect individual response options.

Q1 Deviation Plot for OS X users (relative to respondent pool)

See [Q1 Responses by Operating System][q1ByOs] for details.

iOS Respondents

Q1 Frequency Plot for iOS users

If we assume an even distribution of responses per response option, we can see that iOS users (49% of respondents) responded with significant deviation14. The an iPhone15 option was selected at a higher rate and the a Laptop16, a pad of paper to write17, a Desktop18, and an iPad19 response options were selected at lower rates than expected.

Q1 Deviation Plot for iOS users (assumed equal distribution)

If instead we assume iOS respondents would respond similar to the entire respondent pool, we can see that these users responded as expected20. Since these respondents did not deviate on the whole, there is no need to inspect individual response options.

Q1 Deviation Plot for iOS users (relative to respondent pool)

See [Q1 Responses by Operating System][q1ByOs] for details.

iPhone Respondents

Q1 Frequency Plot for iPhone users

If we assume an even distribution of responses per response option, we can see that iPhone users (34% of respondents) responded with significant deviation21. The an iPhone22 option was selected at a higher rate and the a Laptop23 and a pad of paper to write24 options were selected at lower rates than expected. The a Desktop25 and an iPad26 response options were selected at a rate consistent with expectations.

Q1 Deviation Plot for iPhone users (assumed equal distribution)

If instead we assume iPhone respondents would respond similar to the entire respondent pool, we can see that these users responded as expected27. Since these respondents did not deviate on the whole, there is no need to inspect individual response options.

Q1 Deviation Plot for iPhone users (relative to respondent pool)

See [Q1 Responses by iOS Agent][q1ByIosAgent] for details.

iPad Respondents

Q1 Frequency Plot for iPad users

If we assume an even distribution of responses per response option, we can see thatiPad users (15% of respondents) responded with significant deviation28. The an iPhone29 option was selected at a higher rate and the a Laptop30 option was selected at a lower rate than expected. The a Desktop31, an iPad32, and a pad of paper to write33 response options were selected at a rate consistent with expectations.

Q1 Deviation Plot for iPad users (assumed equal distribution)

If instead we assume iPad respondents would respond similar to the entire respondent pool, we can see that these users responded as expected34. Since these respondents did not deviate on the whole, there is no need to inspect individual response options.

Q1 Deviation Plot for iPad users (relative to respondent pool)

See [Q1 Responses by iOS Agent][q1ByIosAgent] for details.

Q2. I am least likely to interrupt someone using

The frequency plot below shows the portion of responses per response option.

Q2 Frequency Plot

We can see significant deviation35 in the deviation plot below if we assume an equal 20% of responses per response option. Specifically, the a pad of paper to write36 option was selected at a higher rate while the a Laptop37 and an iPad38 response options were selected at lower rates than expected. The an iPhone39 and a Desktop40 response options were selected at rates consistent with expectations.

Q2 Deviation Plot (assumed equal distribution)

OS X Respondents

Q2 Frequency Plot for OS X users

If we assume an even distribution of responses per response option we can see that OS X users (33% of respondents) responded with significant deviation41 as well. The a pad of paper to write42 option was selected at a higher rate while the an iPad43, a Laptop44, and an iPhone45 options were selected at lower rates than expected. The a Desktop46 response option was selected at a rate consistent with expectations.

Q2 Deviation Plot for OS X users (assumed equal distribution)

If instead we assume OS X respondents would respond similar to the entire respondent pool (identical to the pattern shown in Q2 Frequency Plot above), we can see that these users responded as expected[^q2-osx-chi]. Since these respondents did not significantly deviate on the whole, there is no need to inspect individual response options.

Q2 Deviation Plot for OS X users (relative to respondent pool)

iOS Respondents

Q2 Frequency Plot for iOS users

If we assume an even distribution of responses per response options, we can see that iOS users (49% of respondents) responded with significant deviation47. The a pad of paper to write48 option was selected at a higher rate while the a Laptop49 and an iPad50 response options were selected at lower rates than expected. The a Desktop51 and an iPhone52 response options were selected at a rate consistent with expectations.

Q2 Deviation Plot for iOS users (assumed equal distribution)

If instead we assume iOS respondents would respond similar to the entire respondent pool, we can see that these users responded as expected[^q2-osx-chi]. Since these respondents did not deviate on the whole, there is no need to inspect individual response options.

Q2 Deviation Plot for iOS users (relative to respondent pool)

iPhone Respondents

Q2 Frequency Plot for iPhone users

If we assume an even distribution of responses per response options, we can see that iPhone users (34% of respondents) responded with significant deviation53. The a pad of paper to write54 option was selected at a higher rate while the an iPad and a Laptop55 options were selected at lower rates than expected. The a Desktop56 and an iPhone57 response options were selected at a rate consistent with expectations.

Q2 Deviation Plot for iPhone users (assumed equal distribution)

If instead we assume iPhone respondents would respond similar to the entire respondent pool, we can see that these users responded as expected58. Since these respondents did not deviate on the whole, there is no need to inspect individual response options.

Q2 Deviation Plot for iPhone users (relative to respondent pool)

iPad Respondents

Q2 Frequency Plot for iPad users

If we assume an even distribution of responses per response option, we can see that iPad users (15% of respondents) responded with significant deviation59. The a pad of paper to write60 option was selected at a higher rate while the an iPad61 and a Laptop62 options were selected at lower rates than expected. The a Desktop63 and an iPhone64 response options were selected at a rate consistent with expectations.

Q2 Deviation Plot for iPad users (assumed equal distribution)

If instead we assume iPad respondents would respond similar to the entire respondent pool, we can see that these users responded as expected65. Since these respondents did not deviate on the whole, there is no need to inspect individual response options.

Q2 Deviation Plot for iPad users (relative to respondent pool)

Interaction

The matrix plot below shows responses to Q1 and Q2 (data points jittered to reduce overlap of plotted data points). We can see that the most-frequent combination was more likely to interrupt someone using an iPhone and least likely to interrupt someone using a pad of paper to write (29% of respondents). The next most-frequent combination was more likely to interrupt someone using an iPhone and least likely to interrupt someone using a desktop (14% of respondents).

Q1-Q2 Response Matrix (jittered)

There were only 3 respondents (0.6% of respondents) who selected the same option for both questions.

Appendix

Respondents by Country

Respondents by Country
Country Frequency Percent Cume Freq Cume Pct
United States 289 58% 289 58%
United Kingdom 44 9% 333 67%
Australia 22 4% 355 71%
Germany 17 3% 372 75%
Canada 16 3% 388 78%
Italy 11 2% 399 80%
Netherlands 10 2% 409 82%
Unknown 10 2% 419 84%
Sweden 8 2% 427 86%
India 6 1% 433 87%
Belgium 5 1% 438 88%
Austria 4 0% 442 89%
France 3 0% 445 89%
New Zealand 3 0% 448 90%
Poland 3 0% 451 90%
Portugal 3 0% 454 91%
Singapore 3 0% 457 92%
China 2 0% 459 92%
Costa Rica 2 0% 461 92%
Denmark 2 0% 463 93%
Hungary 2 0% 465 93%
Mexico 2 0% 467 94%
Norway 2 0% 469 94%
Saudi Arabia 2 0% 471 94%
Switzerland 2 0% 473 95%
Taiwan 2 0% 475 95%
Argentina 1 0% 476 95%
Armenia 1 0% 477 96%
Bermuda 1 0% 478 96%
Brazil 1 0% 479 96%
Finland 1 0% 480 96%
Greece 1 0% 481 96%
Hong Kong 1 0% 482 97%
Indonesia 1 0% 483 97%
Ireland 1 0% 484 97%
Japan 1 0% 485 97%
Korea, Republic of 1 0% 486 97%
Latvia 1 0% 487 98%
Lithuania 1 0% 488 98%
Malaysia 1 0% 489 98%
Malta 1 0% 490 98%
Peru 1 0% 491 98%
Philippines 1 0% 492 99%
Puerto Rico 1 0% 493 99%
Romania 1 0% 494 99%
Serbia 1 0% 495 99%
South Africa 1 0% 496 99%
Spain 1 0% 497 100%
Thailand 1 0% 498 100%
Ukraine 1 0% 499 100%

Respondents by Operating System & Platform

Respondents by Platform
OS Platform Frequency Percent Cume Freq Cume Pct
unknown 58
iOS iOS 7.1 229 52% 229 52%
iOS 8 9 2% 238 54%
iOS 7 5 1% 243 55%
iOS 5.1 1 0% 244 55%
iOS 6.1 1 0% 245 56%
OS X Mac OS X 10.9 145 33% 390 88%
Mac OS X 10.1 14 3% 404 92%
Mac OS X 10.6 4 0% 408 93%
Mac OS X 10.8 4 0% 412 93%
Windows Windows 7 16 4% 428 97%
Win7 2 0% 430 98%
WinVista 1 0% 431 98%
WinXP 1 0% 432 98%
Windows 8 1 0% 433 98%
Linux Linux 8 2% 441 100%

Respondents by Operating System & Browser Agent

Respondents by Browser Agent
OS Agent Frequency Percent Cume Freq Cume Pct
unknown 58
iOS iPhone 170 39% 170 39%
iPad 73 17% 243 55%
Safari 5.1 1 0% 244 55%
iPod Touch 1 0% 245 56%
OS X Safari 0.0 129 29% 374 85%
Chrome 0.0 34 8% 408 93%
Safari 5.1 3 0% 411 93%
Safari 6.0 1 0% 412 93%
Windows Chrome 0.0 16 4% 428 97%
IE 10.0 2 0% 430 98%
IE 9.0 2 0% 432 98%
Opera 12.00 1 0% 433 98%
Linux Safari 0.0 4 0% 437 99%
Safari 4.0 4 0% 441 100%

Q1 Responses by Operating System

Q1 Responses by Operating System
Response unknown iOS OS X Windows Linux Total
iphone 30 132 79 9 5 225
54% 47% 43% 63%
ipad 9 35 30 3 1 69
14% 18% 14% 13%
desktop 9 36 26 4 1 67
15% 16% 19% 13%
paper 6 32 19 4 1 56
13% 11% 19% 13%
laptop 4 10 13 1 0 24
4% 8% 5% 0%
Total 245 167 21 8 441

Q1 Responses by iOS Agent

Q1 Responses by iOS Browser Agent
Col Pct iPhone iPad Safari 5.1 iPod Touch Total
iphone 97 34 0 1 132
57% 47% 0% 100%
desktop 24 12 0 0 36
14% 16% 0% 0%
ipad 24 11 0 0 35
14% 15% 0% 0%
paper 18 13 1 0 32
11% 18% 100% 0%
laptop 7 3 0 0 10
4% 4% 0% 0%
Total 170 73 1 1 245

Q2 Responses by Operating System

Q2 Responses by Operating System
Response unknown iOS OS X Windows Linux Total
paper 25 116 70 8 5 199
47% 42% 38% 63%
desktop 13 59 32 6 2 99
24% 19% 29% 25%
iphone 9 46 45 5 1 97
19% 27% 24% 13%
laptop 10 19 15 2 0 36
8% 9% 0% 0%
ipad 1 5 5 0 0 10
2% 3% 0% 0%
Total 245 167 21 8 441

Q2 Responses by iOS Agent

Q2 Responses by iOS Browser Agent
Col Pct iPhone iPad Safari 5.1 iPod Touch Total
paper 82 33 0 1 116
48% 45% 0% 100%
desktop 38 21 0 0 59
22% 29% 0% 0%
iphone 31 14 1 0 46
18% 19% 100% 0%
laptop 15 4 0 0 19
9% 5% 0% 0%
ipad 4 1 0 0 5
2% 1% 0% 0%
Total 170 73 1 1 245

  1. χ² = 317.7635; df = 4; p < 0.0001; 

  2. Z = -17.3693; p < 0.0001; 

  3. Z = -8.0355; p < 0.0001; 

  4. Z = -4.2304; p < 0.0001; 

  5. Z = -2.6636; p = 0.0077; 

  6. Z = -2.4398; p = 0.0147; 

  7. χ² = 82.9102; df = 4; p < 0.0001; 

  8. Z = 8.8216; p < 0.0001; 

  9. Z = -3.9465; p < 0.0001; 

  10. Z = -2.7858; p = 0.0053; 

  11. Z = -1.4316; p = 0.1523; 

  12. Z = -0.6577; p = 0.5107; 

  13. χ² = 2.6182; df = 4; p = 0.6236; 

  14. χ² = 184.9796; df = 4; p < 0.001; 

  15. Z = 13.2567; p < 0.0001; 

  16. Z = -6.2290; p < 0.0001; 

  17. Z = -2.7152; p = 0.0066; 

  18. Z = -2.0763; p < 0.0001; 

  19. Z = -2.2361; p = 0.0253; 

  20. χ² = 1.8008.; df = 4; p = 0.7723; 

  21. χ² = 151.5882; df = 4; p < 0.0001; 

  22. Z = 12.0797; p < 0.0001; 

  23. Z = -5.1770; p < 0.0001; 

  24. Z = -3.0679; p = 0.0022; 

  25. Z = -1.9174; p = 0.0552; 

  26. Z = -1.9174; p < 0.0552; 

  27. χ² = 2.7051.; df = 4; p = 0.6083; 

  28. χ² = 36.5205; df = 4; p < 0.0001; 

  29. Z = 5.6765; p < 0.0001; 

  30. Z = -3.3942; p = 0.0007; 

  31. Z = -0.7608; p = 0.4468; 

  32. Z = -1.0534; p = 0.2922; 

  33. Z = -0.4682; p = 0.6397; 

  34. χ² = 2.3735.; df = 4; p = 0.6674; 

  35. χ² = 264.4569; df = 4; p < 0.0001; 

  36. Z = 13.8999, p < 0.0001; 

  37. Z = -6.0210, p < 0.0001; 

  38. Z = -9.9381, p < 0.0001; 

  39. Z = 0.6939, p = 0.4878; 

  40. Z = 1.3654, p = 0.1721; 

  41. χ² = 1.9227.; df = 4; p = 0.7500; 

  42. Z = 7.0805; p < 0.0001; 

  43. Z = -5.4941; p < 0.0001; 

  44. Z = -3.5596; p = 0.0004; 

  45. Z = 2.2441; p = 0.0248; 

  46. Z = -0.2708; p = 0.7865; 

  47. χ² = 151.7143; df = 4; p < 0.0001; 

  48. Z = 10.7012; p < 0.0001; 

  49. Z = -4.7916; p < 0.0001; 

  50. Z = -7.0276; p < 0.0001; 

  51. Z = 1.5972; p = 0.1102; 

  52. Z = -0.4792; p = 0.6318; 

  53. χ² = 105.5882; df = 4; p < 0.0001; 

  54. Z = 9.2036; p < 0.0001; 

  55. Z =-3.6431; p = 0.0003; 

  56. Z = 0.7670; p = 0.4431; 

  57. Z =-0.5752; p = 0.5651; 

  58. χ² = 1.1940.; df = 4; p = 0.8791; 

  59. χ² = 46.3836; df = 4; p < 0.0001; 

  60. Z = 5.3839; p < 0.0001; 

  61. Z =-3.9794; p < 0.0001; 

  62. Z =-3.1016; p = 0.0019; 

  63. Z = 1.8727; p = 0.0611; 

  64. Z =-0.1756; p = 0.8606; 

  65. χ² = 2.7858.; df = 4; p = 0.5943;