Further Interruption Survey Analysis

Note from Ben: After just one reader commented that there may be some flaws is Koopmann’s analysis (these flaws don’t change the overall conclusion about the data), Koopmann took upon himself to invest considerable time in re-working the data. Below is the re-work of that data directly from him, with the original article (here) preserved as it was originally published.

Methodology

The poll was presented as post with an embedded iframe on The Brooks Review website. The post, titled Interruption Survey, contained the following introduction and note:

I’d appreciate you taking a moment to fill this out (note this is an iframe, you need to scroll the frame to get to the “finish” button):

Note: I’m assuming you want to interrupt someone and they are using the device.

Instrument

The poll consisted to two questions:

  1. I am more likely to interrupt someone using:
    • a pad of paper to write.
    • an iPad.
    • a Desktop.
    • a Laptop.
    • an iPhone.
  2. I am least likely to interrupt someone using:
    • a pad of paper to write.
    • an iPhone.
    • a Desktop.
    • an iPad.
    • a Laptop.

Each of these items was required and only one choice was allowed per item.

Results

Respondents

There were 499 submissions received between 2014-06-11 18:44:51 and 2014-06-14 14:34:49[^GMT?].

Note that some percentages may not sum to 100% due to rounding.

Geography

The majority of respondents (58%) were from the United States; the next-largest group of respondents (9%) was from the United Kingdom. The remaining 33% of respondents came from 49 other countries with each country contributing less than 5% of respondents.

See Respondents by Country for full breakdown.

Operating System

Of the 441 (88%) respondents where the platform was detected, the overwhelming majority (93%) were running an iOS (56%) or OS X (38%) device. Windows (5%) and Linux (2%) were the remaining identified operating systems; the tool was unable to identify platform for the remaining 58 (12%) respondents.

iOS 7.1 (52%) and Mac OS X 10.9 Mavericks (33%) were the largest group of respondents; 9 respondents (2%) were bleeding-edge dev-types running iOS 8; 14 respondents (3%) were old-school OS X 10.1 Puma (released Sep 2001).

See [Respondents by Operating System & Platform][respondentsByPlatform] and [Respondents by Browser Agent][respondentsByAgent] for full breakdowns.

Q1. I am more likely to interrupt someone using

The frequency plot below shows the portion of responses per option.

Q1 Frequency Plot

We can see significant deviation[^q1equal-chi] in the deviation plot below if we assume an equal 20% of responses per response option (100% across 5 options). Specifically the an iPhone[^q1equal-iphone] option was selected at a higher rate and the a Laptop[^q1equal-laptop] , a pad of paper to write[^q1equal-paper], a Desktop[^q1equal-desktop], and an iPad[^q1equal-ipad] response options were all selected at lower rates than expected.

Q1 Deviation Plot (assumed equal distribution)

[^q1equal-chi]: χ² = 317.7635; df = 4; p < 0.0001;
[^q1equal-iphone]: Z = -17.3693; p < 0.0001;
[^q1equal-ipad]: Z = -2.4398; p = 0.0147;
[^q1equal-paper]: Z = -4.2304; p < 0.0001;
[^q1equal-laptop]: Z = -8.0355; p < 0.0001;
[^q1equal-desktop]: Z = -2.6636; p = 0.0077;

OS X Respondents

Q1 Frequency Plot for OS X users

If we assume an even distribution of responses per response option we can see that OS X users (33% of respondents) responded with significant deviation[^q1equal-osx-chi] as well. The an iPhone[^q1equal-osx-iphone] option was selected at a higher rate and the a Laptop[^q1equal-osx-laptop] and a pad of paper to write[^q1equal-osx-paper] options were selected at lower rates than expected. The a Desktop[^q1equal-osx-desktop] and an iPad[^q1equal-osx-ipad] response options were selected at a rate consistent with expectations.

[^q1equal-osx-chi]: χ² = 82.9102; df = 4; p < 0.0001;
[^q1equal-osx-iphone]: Z = 8.8216; p < 0.0001;
[^q1equal-osx-ipad]: Z = -0.6577; p = 0.5107;
[^q1equal-osx-paper]: Z = -2.7858; p = 0.0053;
[^q1equal-osx-laptop]: Z = -3.9465; p < 0.0001;
[^q1equal-osx-desktop]: Z = -1.4316; p = 0.1523;

Q1 Deviation Plot for OS X users (assumed equal distribution)

If instead we assume OS X respondents would respond similar to the entire respondent pool (identical to the pattern shown in Q1 Frequency Plot above), we can see that these users responded as expected[^q1-osx-chi]. Since these respondents did not deviate on the whole, there is no need to inspect individual response options.

[^q1-osx-chi]: χ² = 2.6182; df = 4; p = 0.6236;

Q1 Deviation Plot for OS X users (relative to respondent pool)

See [Q1 Responses by Operating System][q1ByOs] for details.

iOS Respondents

Q1 Frequency Plot for iOS users

If we assume an even distribution of responses per response option, we can see that iOS users (49% of respondents) responded with significant deviation[^q1equal-ios-chi]. The an iPhone[^q1equal-ios-iphone] option was selected at a higher rate and the a Laptop[^q1equal-ios-laptop], a pad of paper to write[^q1equal-ios-paper], a Desktop[^q1equal-ios-desktop], and an iPad[^q1equal-ios-ipad] response options were selected at lower rates than expected.

[^q1equal-ios-chi]: χ² = 184.9796; df = 4; p < 0.001;
[^q1equal-ios-iphone]: Z = 13.2567; p < 0.0001;
[^q1equal-ios-ipad]: Z = -2.2361; p = 0.0253;
[^q1equal-ios-paper]: Z = -2.7152; p = 0.0066;
[^q1equal-ios-laptop]: Z = -6.2290; p < 0.0001;
[^q1equal-ios-desktop]: Z = -2.0763; p < 0.0001;

Q1 Deviation Plot for iOS users (assumed equal distribution)

If instead we assume iOS respondents would respond similar to the entire respondent pool, we can see that these users responded as expected[^q1-ios-chi]. Since these respondents did not deviate on the whole, there is no need to inspect individual response options.

[^q1-ios-chi]: χ² = 1.8008.; df = 4; p = 0.7723;

Q1 Deviation Plot for iOS users (relative to respondent pool)

See [Q1 Responses by Operating System][q1ByOs] for details.

iPhone Respondents

Q1 Frequency Plot for iPhone users

If we assume an even distribution of responses per response option, we can see that iPhone users (34% of respondents) responded with significant deviation[^q1equal-ios-iphone-chi]. The an iPhone[^q1equal-ios-iphone-iphone] option was selected at a higher rate and the a Laptop[^q1equal-ios-iphone-laptop] and a pad of paper to write[^q1equal-ios-iphone-paper] options were selected at lower rates than expected. The a Desktop[^q1equal-ios-iphone-desktop] and an iPad[^q1equal-ios-iphone-ipad] response options were selected at a rate consistent with expectations.

[^q1equal-ios-iphone-chi]: χ² = 151.5882; df = 4; p < 0.0001;
[^q1equal-ios-iphone-iphone]: Z = 12.0797; p < 0.0001;
[^q1equal-ios-iphone-ipad]: Z = -1.9174; p < 0.0552;
[^q1equal-ios-iphone-paper]: Z = -3.0679; p = 0.0022;
[^q1equal-ios-iphone-laptop]: Z = -5.1770; p < 0.0001;
[^q1equal-ios-iphone-desktop]: Z = -1.9174; p = 0.0552;

Q1 Deviation Plot for iPhone users (assumed equal distribution)

If instead we assume iPhone respondents would respond similar to the entire respondent pool, we can see that these users responded as expected[^q1-iphone-chi]. Since these respondents did not deviate on the whole, there is no need to inspect individual response options.

[^q1-iphone-chi]: χ² = 2.7051.; df = 4; p = 0.6083;

Q1 Deviation Plot for iPhone users (relative to respondent pool)

See [Q1 Responses by iOS Agent][q1ByIosAgent] for details.

iPad Respondents

Q1 Frequency Plot for iPad users

If we assume an even distribution of responses per response option, we can see thatiPad users (15% of respondents) responded with significant deviation[^q1equal-ios-ipad-chi]. The an iPhone[^q1equal-ios-ipad-iphone] option was selected at a higher rate and the a Laptop[^q1equal-ios-ipad-laptop] option was selected at a lower rate than expected. The a Desktop[^q1equal-ios-ipad-desktop], an iPad[^q1equal-ios-ipad-ipad], and a pad of paper to write[^q1equal-ios-ipad-paper] response options were selected at a rate consistent with expectations.

[^q1equal-ios-ipad-chi]: χ² = 36.5205; df = 4; p < 0.0001;
[^q1equal-ios-ipad-iphone]: Z = 5.6765; p < 0.0001;
[^q1equal-ios-ipad-ipad]: Z = -1.0534; p = 0.2922;
[^q1equal-ios-ipad-paper]: Z = -0.4682; p = 0.6397;
[^q1equal-ios-ipad-laptop]: Z = -3.3942; p = 0.0007;
[^q1equal-ios-ipad-desktop]: Z = -0.7608; p = 0.4468;

Q1 Deviation Plot for iPad users (assumed equal distribution)

If instead we assume iPad respondents would respond similar to the entire respondent pool, we can see that these users responded as expected[^q1-ipad-chi]. Since these respondents did not deviate on the whole, there is no need to inspect individual response options.

[^q1-ipad-chi]: χ² = 2.3735.; df = 4; p = 0.6674;

Q1 Deviation Plot for iPad users (relative to respondent pool)

See [Q1 Responses by iOS Agent][q1ByIosAgent] for details.

Q2. I am least likely to interrupt someone using

The frequency plot below shows the portion of responses per response option.

Q2 Frequency Plot

We can see significant deviation[^q2-chi] in the deviation plot below if we assume an equal 20% of responses per response option. Specifically, the a pad of paper to write[^q2-paper] option was selected at a higher rate while the a Laptop[^q2-laptop] and an iPad[^q2-ipad] response options were selected at lower rates than expected. The an iPhone[^q2-iphone] and a Desktop[^q2-desktop] response options were selected at rates consistent with expectations.

[^q2-chi]: χ² = 264.4569; df = 4; p < 0.0001;
[^q2-iphone]: Z = 0.6939, p = 0.4878;
[^q2-ipad]: Z = -9.9381, p < 0.0001;
[^q2-paper]: Z = 13.8999, p < 0.0001;
[^q2-laptop]: Z = -6.0210, p < 0.0001;
[^q2-desktop]: Z = 1.3654, p = 0.1721;

Q2 Deviation Plot (assumed equal distribution)

OS X Respondents

Q2 Frequency Plot for OS X users

If we assume an even distribution of responses per response option we can see that OS X users (33% of respondents) responded with significant deviation[^q2-osx-chi] as well. The a pad of paper to write[^q2-osx-paper] option was selected at a higher rate while the an iPad[^q2-osx-ipad], a Laptop[^q2-osx-laptop], and an iPhone[^q2-osx-iphone] options were selected at lower rates than expected. The a Desktop[^q2-osx-desktop] response option was selected at a rate consistent with expectations.

[^q2-osx-chi]: χ² = 78.4790; df = 4; p < 0.001
[^q2-osx-desktop]: Z = -0.2708; p = 0.7865;
[^q2-osx-laptop]: Z = -3.5596; p = 0.0004;
[^q2-osx-paper]: Z = 7.0805; p < 0.0001;
[^q2-osx-ipad]: Z = -5.4941; p < 0.0001;
[^q2-osx-iphone]: Z = 2.2441; p = 0.0248;

Q2 Deviation Plot for OS X users (assumed equal distribution)

If instead we assume OS X respondents would respond similar to the entire respondent pool (identical to the pattern shown in Q2 Frequency Plot above), we can see that these users responded as expected[^q2-osx-chi]. Since these respondents did not significantly deviate on the whole, there is no need to inspect individual response options.

[^q2-osx-chi]: χ² = 4.1709.; df = 4; p = 0.3834;

Q2 Deviation Plot for OS X users (relative to respondent pool)

iOS Respondents

Q2 Frequency Plot for iOS users

If we assume an even distribution of responses per response options, we can see that iOS users (49% of respondents) responded with significant deviation[^q2-ios-chi]. The a pad of paper to write[^q2-ios-paper] option was selected at a higher rate while the a Laptop[^q2-ios-laptop] and an iPad[^q2-ios-ipad] response options were selected at lower rates than expected. The a Desktop[^q2-ios-desktop] and an iPhone[^q2-ios-iphone] response options were selected at a rate consistent with expectations.

[^q2-ios-chi]: χ² = 151.7143; df = 4; p < 0.0001;
[^q2-ios-desktop]: Z = 1.5972; p = 0.1102;
[^q2-ios-laptop]: Z = -4.7916; p < 0.0001;
[^q2-ios-paper]: Z = 10.7012; p < 0.0001;
[^q2-ios-ipad]: Z = -7.0276; p < 0.0001;
[^q2-ios-iphone]: Z = -0.4792; p = 0.6318;

Q2 Deviation Plot for iOS users (assumed equal distribution)

If instead we assume iOS respondents would respond similar to the entire respondent pool, we can see that these users responded as expected[^q2-osx-chi]. Since these respondents did not deviate on the whole, there is no need to inspect individual response options.

[^q2-osx-chi]: χ² = 1.9227.; df = 4; p = 0.7500;

Q2 Deviation Plot for iOS users (relative to respondent pool)

iPhone Respondents

Q2 Frequency Plot for iPhone users

If we assume an even distribution of responses per response options, we can see that iPhone users (34% of respondents) responded with significant deviation[^q2-ios-iphone-chi]. The a pad of paper to write[^q2-ios-iphone-paper] option was selected at a higher rate while the an iPad and a Laptop[^q2-ios-iphone-laptop] options were selected at lower rates than expected. The a Desktop[^q2-ios-iphone-desktop] and an iPhone[^q2-ios-iphone-iphone] response options were selected at a rate consistent with expectations.

[^q2-ios-iphone-chi]: χ² = 105.5882; df = 4; p < 0.0001;
[^q2-ios-iphone-desktop]: Z = 0.7670; p = 0.4431;
[^q2-ios-iphone-laptop]: Z =-3.6431; p = 0.0003;
[^q2-ios-iphone-paper]: Z = 9.2036; p < 0.0001;
[^q2-ios-iphone-ipad]: Z =-5.7522; p < 0.0001;
[^q2-ios-iphone-iphone]: Z =-0.5752; p = 0.5651;

Q2 Deviation Plot for iPhone users (assumed equal distribution)

If instead we assume iPhone respondents would respond similar to the entire respondent pool, we can see that these users responded as expected[^q2-iphone-chi]. Since these respondents did not deviate on the whole, there is no need to inspect individual response options.

[^q2-iphone-chi]: χ² = 1.1940.; df = 4; p = 0.8791;

Q2 Deviation Plot for iPhone users (relative to respondent pool)

iPad Respondents

Q2 Frequency Plot for iPad users

If we assume an even distribution of responses per response option, we can see that iPad users (15% of respondents) responded with significant deviation[^q2-ios-ipad-chi]. The a pad of paper to write[^q2-ios-ipad-paper] option was selected at a higher rate while the an iPad[^q2-ios-ipad-ipad] and a Laptop[^q2-ios-ipad-laptop] options were selected at lower rates than expected. The a Desktop[^q2-ios-ipad-desktop] and an iPhone[^q2-ios-ipad-iphone] response options were selected at a rate consistent with expectations.

[^q2-ios-ipad-chi]: χ² = 46.3836; df = 4; p < 0.0001;
[^q2-ios-ipad-desktop]: Z = 1.8727; p = 0.0611;
[^q2-ios-ipad-laptop]: Z =-3.1016; p = 0.0019;
[^q2-ios-ipad-paper]: Z = 5.3839; p < 0.0001;
[^q2-ios-ipad-ipad]: Z =-3.9794; p < 0.0001;
[^q2-ios-ipad-iphone]: Z =-0.1756; p = 0.8606;

Q2 Deviation Plot for iPad users (assumed equal distribution)

If instead we assume iPad respondents would respond similar to the entire respondent pool, we can see that these users responded as expected[^q2-ipad-chi]. Since these respondents did not deviate on the whole, there is no need to inspect individual response options.

[^q2-ipad-chi]: χ² = 2.7858.; df = 4; p = 0.5943;

Q2 Deviation Plot for iPad users (relative to respondent pool)

Interaction

The matrix plot below shows responses to Q1 and Q2 (data points jittered to reduce overlap of plotted data points). We can see that the most-frequent combination was more likely to interrupt someone using an iPhone and least likely to interrupt someone using a pad of paper to write (29% of respondents). The next most-frequent combination was more likely to interrupt someone using an iPhone and least likely to interrupt someone using a desktop (14% of respondents).

Q1-Q2 Response Matrix (jittered)

There were only 3 respondents (0.6% of respondents) who selected the same option for both questions.

Appendix

Respondents by Country

Respondents by Country
Country Frequency Percent Cume Freq Cume Pct
United States 289 58% 289 58%
United Kingdom 44 9% 333 67%
Australia 22 4% 355 71%
Germany 17 3% 372 75%
Canada 16 3% 388 78%
Italy 11 2% 399 80%
Netherlands 10 2% 409 82%
Unknown 10 2% 419 84%
Sweden 8 2% 427 86%
India 6 1% 433 87%
Belgium 5 1% 438 88%
Austria 4 0% 442 89%
France 3 0% 445 89%
New Zealand 3 0% 448 90%
Poland 3 0% 451 90%
Portugal 3 0% 454 91%
Singapore 3 0% 457 92%
China 2 0% 459 92%
Costa Rica 2 0% 461 92%
Denmark 2 0% 463 93%
Hungary 2 0% 465 93%
Mexico 2 0% 467 94%
Norway 2 0% 469 94%
Saudi Arabia 2 0% 471 94%
Switzerland 2 0% 473 95%
Taiwan 2 0% 475 95%
Argentina 1 0% 476 95%
Armenia 1 0% 477 96%
Bermuda 1 0% 478 96%
Brazil 1 0% 479 96%
Finland 1 0% 480 96%
Greece 1 0% 481 96%
Hong Kong 1 0% 482 97%
Indonesia 1 0% 483 97%
Ireland 1 0% 484 97%
Japan 1 0% 485 97%
Korea, Republic of 1 0% 486 97%
Latvia 1 0% 487 98%
Lithuania 1 0% 488 98%
Malaysia 1 0% 489 98%
Malta 1 0% 490 98%
Peru 1 0% 491 98%
Philippines 1 0% 492 99%
Puerto Rico 1 0% 493 99%
Romania 1 0% 494 99%
Serbia 1 0% 495 99%
South Africa 1 0% 496 99%
Spain 1 0% 497 100%
Thailand 1 0% 498 100%
Ukraine 1 0% 499 100%

Respondents by Operating System & Platform

Respondents by Platform
OS Platform Frequency Percent Cume Freq Cume Pct
unknown 58
iOS iOS 7.1 229 52% 229 52%
iOS 8 9 2% 238 54%
iOS 7 5 1% 243 55%
iOS 5.1 1 0% 244 55%
iOS 6.1 1 0% 245 56%
OS X Mac OS X 10.9 145 33% 390 88%
Mac OS X 10.1 14 3% 404 92%
Mac OS X 10.6 4 0% 408 93%
Mac OS X 10.8 4 0% 412 93%
Windows Windows 7 16 4% 428 97%
Win7 2 0% 430 98%
WinVista 1 0% 431 98%
WinXP 1 0% 432 98%
Windows 8 1 0% 433 98%
Linux Linux 8 2% 441 100%

Respondents by Operating System & Browser Agent

Respondents by Browser Agent
OS Agent Frequency Percent Cume Freq Cume Pct
unknown 58
iOS iPhone 170 39% 170 39%
iPad 73 17% 243 55%
Safari 5.1 1 0% 244 55%
iPod Touch 1 0% 245 56%
OS X Safari 0.0 129 29% 374 85%
Chrome 0.0 34 8% 408 93%
Safari 5.1 3 0% 411 93%
Safari 6.0 1 0% 412 93%
Windows Chrome 0.0 16 4% 428 97%
IE 10.0 2 0% 430 98%
IE 9.0 2 0% 432 98%
Opera 12.00 1 0% 433 98%
Linux Safari 0.0 4 0% 437 99%
Safari 4.0 4 0% 441 100%

Q1 Responses by Operating System

Q1 Responses by Operating System
Response unknown iOS OS X Windows Linux Total
iphone 30 132 79 9 5 225
54% 47% 43% 63%
ipad 9 35 30 3 1 69
14% 18% 14% 13%
desktop 9 36 26 4 1 67
15% 16% 19% 13%
paper 6 32 19 4 1 56
13% 11% 19% 13%
laptop 4 10 13 1 0 24
4% 8% 5% 0%
Total 245 167 21 8 441

Q1 Responses by iOS Agent

Q1 Responses by iOS Browser Agent
Col Pct iPhone iPad Safari 5.1 iPod Touch Total
iphone 97 34 0 1 132
57% 47% 0% 100%
desktop 24 12 0 0 36
14% 16% 0% 0%
ipad 24 11 0 0 35
14% 15% 0% 0%
paper 18 13 1 0 32
11% 18% 100% 0%
laptop 7 3 0 0 10
4% 4% 0% 0%
Total 170 73 1 1 245

Q2 Responses by Operating System

Q2 Responses by Operating System
Response unknown iOS OS X Windows Linux Total
paper 25 116 70 8 5 199
47% 42% 38% 63%
desktop 13 59 32 6 2 99
24% 19% 29% 25%
iphone 9 46 45 5 1 97
19% 27% 24% 13%
laptop 10 19 15 2 0 36
8% 9% 0% 0%
ipad 1 5 5 0 0 10
2% 3% 0% 0%
Total 245 167 21 8 441

Q2 Responses by iOS Agent

Q2 Responses by iOS Browser Agent
Col Pct iPhone iPad Safari 5.1 iPod Touch Total
paper 82 33 0 1 116
48% 45% 0% 100%
desktop 38 21 0 0 59
22% 29% 0% 0%
iphone 31 14 1 0 46
18% 19% 100% 0%
laptop 15 4 0 0 19
9% 5% 0% 0%
ipad 4 1 0 0 5
2% 1% 0% 0%
Total 170 73 1 1 245