358 lines
14 KiB
Markdown
358 lines
14 KiB
Markdown
---
|
||
created_at: '2014-06-05T13:50:13.000Z'
|
||
title: Selection bias and bombers (2008)
|
||
url: http://www.johndcook.com/blog/2008/01/21/selection-bias-and-bombers/
|
||
author: awjr
|
||
points: 91
|
||
story_text: ''
|
||
comment_text:
|
||
num_comments: 18
|
||
story_id:
|
||
story_title:
|
||
story_url:
|
||
parent_id:
|
||
created_at_i: 1401976213
|
||
_tags:
|
||
- story
|
||
- author_awjr
|
||
- story_7851927
|
||
objectID: '7851927'
|
||
year: 2008
|
||
|
||
---
|
||
[Source](https://www.johndcook.com/blog/2008/01/21/selection-bias-and-bombers/ "Permalink to Selection bias example with WWII bombers")
|
||
|
||
# Selection bias example with WWII bombers
|
||
|
||
[ ![John D. Cook][1] ][2]
|
||
|
||
[Skip to content][3]
|
||
|
||
* [ABOUT][4]
|
||
* [SERVICES][5]
|
||
* [APPLIED MATH][6]
|
||
* [STATISTICS][7]
|
||
* [COMPUTATION][8]
|
||
* [WRITING][9]
|
||
* [BLOG][2]
|
||
* [TECHNICAL NOTES][10]
|
||
* [JOURNAL ARTICLES][11]
|
||
* [TWITTER][12]
|
||
* [PRESENTATIONS][13]
|
||
* [NEWSLETTER][14]
|
||
* [CLIENTS][15]
|
||
* [ENDORSEMENTS][16]
|
||
|
||
[(832) 422-8646][17]
|
||
|
||
[Contact][18]
|
||
|
||
# Selection bias and bombers
|
||
|
||
Posted on [21 January 2008][19] by [John][20]
|
||
|
||
During WWII, statistician Abraham Wald was asked to help the British decide where to add armor to their bombers. After analyzing the records, he recommended adding more armor to the places where there was no damage!
|
||
|
||
This seems backward at first, but Wald realized his data came from bombers that survived. That is, the British were only able to analyze the bombers that returned to England; those that were shot down over enemy territory were not part of their sample. These bombers’ wounds showed where they could afford to be hit. Said another way, the undamaged areas on the survivors showed where the lost planes must have been hit because the planes hit in those areas did not return from their missions.
|
||
|
||
Wald assumed that the bullets were fired randomly, that no one could accurately aim for a particular part of the bomber. Instead they aimed in the general direction of the plane and sometimes got lucky. So, for example, if Wald saw that more bombers in his sample had bullet holes in the middle of the wings, he did _not_ conclude that Nazis liked to aim for the middle of wings. He assumed that there must have been about as many bombers with bullet holes in every other part of the plane but that those with holes elsewhere were not part of his sample because they had been shot down.
|
||
|
||
![Click to learn more about Bayesian statistics consulting][21]
|
||
|
||
|
||
|
||
Categories : [Statistics][22]
|
||
|
||
Tags : [Probability and Statistics][23]
|
||
|
||
Bookmark the [permalink][24]
|
||
|
||
# Post navigation
|
||
|
||
Previous Post[Repairing tumors][25]
|
||
|
||
Next Post[C# verbatim strings vs. PowerShell here-strings][26]
|
||
|
||
## 42 thoughts on “Selection bias and bombers”
|
||
|
||
1. Pingback: [Best early blog posts — The Endeavour][27]
|
||
|
||
2. Ling
|
||
|
||
[ 26 November 2009 at 15:09 ][28]
|
||
|
||
So is the assumption of random firing justified?
|
||
|
||
3. John
|
||
|
||
[ 27 November 2009 at 23:48 ][29]
|
||
|
||
Ling, I think the random firing assumption was justified at the time, given the state of technology. The standard error may have been larger than the size of an airplane.
|
||
|
||
4. Speedmaster
|
||
|
||
[ 6 December 2010 at 09:53 ][30]
|
||
|
||
Fantastic example.
|
||
|
||
5. Eugene
|
||
|
||
[ 25 February 2011 at 13:24 ][31]
|
||
|
||
Thanks, interesting story!
|
||
|
||
6. Alfredo
|
||
|
||
[ 25 February 2011 at 15:08 ][32]
|
||
|
||
Can you cite the source of this excellent example?
|
||
|
||
7. Pingback: The Importance of Selection Effects « Daniel J. Smith
|
||
|
||
8. Pingback: [The Importance of Selection Effects — Marginal Revolution][33]
|
||
|
||
9. Rafael
|
||
|
||
[ 13 September 2011 at 09:31 ][34]
|
||
|
||
Wald, Abraham. (1943). A Method of Estimating Plane Vulnerability Based on Damage of Survivors.
|
||
|
||
<http://cna.org/sites/default/files/research/0204320000.pdf>
|
||
|
||
10. John Thacker
|
||
|
||
[ 13 September 2011 at 09:40 ][35]
|
||
|
||
Alfredo:
|
||
|
||
[Here’s a reprint of one of his papers on the subject][36], and [here’s another source.][37]
|
||
|
||
However, while I’ve heard this story many time before, I had not heard it claimed that he did this for the British, but rather for the US Navy.
|
||
|
||
11. Pingback: [Thinking Statistically | Delightfully Distinctive COLRS][38]
|
||
|
||
12. Pingback: [The Spamlist! » The Importance of Selection Effects][39]
|
||
|
||
13. Alfredo
|
||
|
||
[ 14 September 2011 at 04:12 ][40]
|
||
|
||
Thanks. This is good stuff as teaching material.
|
||
|
||
14. Pingback: Selection effects » Manlyisms
|
||
|
||
15. Pingback: [Statistics in a time of war | Blog of Leonid Mamchenkov][41]
|
||
|
||
16. Pingback: [Selection Effects « Marc Gawley][42]
|
||
|
||
17. Pingback: [Can selection bias shoot down an argument? « The Melbourne Urbanist][43]
|
||
|
||
18. [Visa][44]
|
||
|
||
[ 18 September 2011 at 03:12 ][45]
|
||
|
||
I thought that was beautiful, and something that should be a part of basic education for everybody. How many decisions are poorly made because of the problem of silent evidence? This plagues everything from our social lives to government policy, and as a species it’s about time we got over this.
|
||
|
||
19. Pingback: [British World War II Bombers – An Example of Why I love Economics « Portrait of the Economist as a Young Man][46]
|
||
|
||
20. Suzie
|
||
|
||
[ 20 September 2011 at 07:40 ][47]
|
||
|
||
Wait, so what were the final results? Did a higher %age of bombers come back after the fleet was fitted with the armor in Wald’s recommended areas?
|
||
|
||
21. Pingback: [More on Democratic Consolidation and Time « Dart-Throwing Chimp][48]
|
||
|
||
22. Pingback: [DiSH lab blog » Blog Archive » A neat science riddle – can you solve it?][49]
|
||
|
||
23. Pingback: Selection Bias and WWII Bombers | ≈42
|
||
|
||
24. Pingback: [Brian’s Links 20 October 2011: Carnivory, Science, and Art « TheMoralMindfield][50]
|
||
|
||
25. Pingback: On Bias and Statistics « Tropewell
|
||
|
||
26. Pingback: [On Bias and Statistics « Tropewell][51]
|
||
|
||
27. Pingback: A Few Random Morning Links … | The Pretense of Knowledge
|
||
|
||
28. [Glenn][52]
|
||
|
||
[ 28 September 2012 at 12:41 ][53]
|
||
|
||
Dan Meyer did a nice version of this for the high school crowd.
|
||
<http://blog.mrmeyer.com/?p=507>
|
||
|
||
Really nice graphic he created to go with the material.
|
||
|
||
29. Pingback: [Success » Diigo Links (weekly)][54]
|
||
|
||
30. Pingback: 7 Quick Takes (9/23/11)
|
||
|
||
31. Pingback: [Today’s Unemployment Number | Points and Figures][55]
|
||
|
||
32. Pingback: [Military intelligence from serial numbers | John D. Cook][56]
|
||
|
||
33. Pingback: [Selection bias and bombers « Another Word For It][57]
|
||
|
||
34. Ronald Chappell
|
||
|
||
[ 16 April 2015 at 14:27 ][58]
|
||
|
||
Cute statistical study but also suffers from randomness bias. Armor the most vulnerable points; fuel, controls and cockpit, engines etc. Then apply statistics to the rest if you have any weight budget left.
|
||
|
||
35. John
|
||
|
||
[ 16 April 2015 at 15:06 ][59]
|
||
|
||
I believe the point of the study was to determine what the most vulnerable parts were.
|
||
|
||
36. Pingback: [TWiV 340: No shift, measles][60]
|
||
|
||
37. Pingback: [A Thought On Gun Violence Points and Figures][61]
|
||
|
||
38. Pingback: [Second languages and selection bias][62]
|
||
|
||
39. Pingback: [TWiV 340: No shift, measles | This Week in Virology][63]
|
||
|
||
40. Pingback: [The birthday problem | Datant][64]
|
||
|
||
41. Toph Tucker
|
||
|
||
[ 7 October 2016 at 14:04 ][65]
|
||
|
||
I am commenting years later because this is so good. I only came across it more recently and now I cite it to people all the time. I am not yet at the point where I can intuitively spot the pattern in action, but I hope to get there. I reckon the world is full of people solving the most survivable problems and ignoring the problems that remove their victims from the sample.
|
||
|
||
42. Pingback: [Sunday Morning Insight: D'avions et d'Intelligence Artificielle en France [in French] | A bunch of data][66]
|
||
|
||
### Leave a Reply [Cancel reply][67]
|
||
|
||
Your email address will not be published. Required fields are marked *
|
||
|
||
Comment
|
||
|
||
Notify me of followup comments via e-mail
|
||
|
||
Name *
|
||
|
||
Email *
|
||
|
||
Website
|
||
|
||
Search for:
|
||
|
||
![John D. Cook][68]
|
||
|
||
**John D. Cook, PhD**
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
# Latest Posts
|
||
|
||
* [Fibonacci / Binomial coefficient identity][69]
|
||
* [Painless project management][70]
|
||
* [New animation feature for exponential sums][71]
|
||
* [Quantifying normal approximation accuracy][72]
|
||
* [Ordinary Potential Polynomials][73]
|
||
|
||
# Categories
|
||
|
||
CategoriesSelect CategoryBusinessClinical trialsComputingCreativityGraphicsMachine learningMathMusicPowerShellPythonScienceSoftware developmentStatisticsTypographyUncategorized
|
||
|
||
| ----- |
|
||
| ![Subscribe to blog by email][74] | [Subscribe via email][75] |
|
||
| ![RSS icon][76] | [Subscribe via RSS][77] |
|
||
| ![Newsletter icon][78] | [Monthly newsletter][79] |
|
||
|
||
### [John D. Cook][2]
|
||
|
||
© All rights reserved.
|
||
|
||
Search for:
|
||
|
||
[1]: https://www.johndcook.com/blog/wp-content/themes/ThemeAlley.Business.Pro/images/Logo.svg
|
||
[2]: https://www.johndcook.com/blog/
|
||
[3]: https://www.johndcook.com#content "Skip to content"
|
||
[4]: https://www.johndcook.com/blog/top/
|
||
[5]: https://www.johndcook.com/blog/services-2/
|
||
[6]: https://www.johndcook.com/blog/applied-math/
|
||
[7]: https://www.johndcook.com/blog/applied-statistics/
|
||
[8]: https://www.johndcook.com/blog/applied-computation/
|
||
[9]: https://www.johndcook.com/blog/writing/
|
||
[10]: https://www.johndcook.com/blog/notes/
|
||
[11]: https://www.johndcook.com/blog/articles/
|
||
[12]: https://www.johndcook.com/blog/twitter_page/
|
||
[13]: https://www.johndcook.com/blog/presentations/
|
||
[14]: https://www.johndcook.com/blog/newsletter/
|
||
[15]: https://www.johndcook.com/blog/clients-new/
|
||
[16]: https://www.johndcook.com/blog/endorsements/
|
||
[17]: tel:8324228646
|
||
[18]: https://www.johndcook.com/blog/contact/
|
||
[19]: https://www.johndcook.com/blog/2008/01/21/selection-bias-and-bombers/ "01:13"
|
||
[20]: https://www.johndcook.com/blog/author/john/ "View all posts by John"
|
||
[21]: https://www.johndcook.com/Bayes_consulting2.png
|
||
[22]: https://www.johndcook.com/blog/category/statistics/
|
||
[23]: https://www.johndcook.com/blog/tag/probability-and-statistics/
|
||
[24]: https://www.johndcook.com/blog/2008/01/21/selection-bias-and-bombers/ "Permalink to Selection bias and bombers"
|
||
[25]: https://www.johndcook.com/blog/2008/01/19/repairing-tumors/
|
||
[26]: https://www.johndcook.com/blog/2008/01/22/c-verbatim-strings-vs-powershell-here-strings/
|
||
[27]: http://www.johndcook.com/blog/2009/02/10/best-early-blog-posts/
|
||
[28]: https://www.johndcook.com/blog/2008/01/21/selection-bias-and-bombers/#comment-16061
|
||
[29]: https://www.johndcook.com/blog/2008/01/21/selection-bias-and-bombers/#comment-16062
|
||
[30]: https://www.johndcook.com/blog/2008/01/21/selection-bias-and-bombers/#comment-16063
|
||
[31]: https://www.johndcook.com/blog/2008/01/21/selection-bias-and-bombers/#comment-16064
|
||
[32]: https://www.johndcook.com/blog/2008/01/21/selection-bias-and-bombers/#comment-16065
|
||
[33]: http://marginalrevolution.com/marginalrevolution/2011/09/the-importance-of-selection-effects.html
|
||
[34]: https://www.johndcook.com/blog/2008/01/21/selection-bias-and-bombers/#comment-16068
|
||
[35]: https://www.johndcook.com/blog/2008/01/21/selection-bias-and-bombers/#comment-16069
|
||
[36]: http://cna.org/sites/default/files/research/0204320000.pdf
|
||
[37]: http://www.jstor.org/pss/2288257
|
||
[38]: http://ddcolrs.wordpress.com/2011/07/14/thinking-statistically/
|
||
[39]: http://www.thespamlist.net/2011/09/13/the-importance-of-selection-effects/
|
||
[40]: https://www.johndcook.com/blog/2008/01/21/selection-bias-and-bombers/#comment-16072
|
||
[41]: http://mamchenkov.net/wordpress/2011/09/15/statistics-in-a-time-of-war/
|
||
[42]: http://marcgawley.com/2011/09/15/selection-effects/
|
||
[43]: http://melbourneurbanist.wordpress.com/2011/09/17/can-selection-bias-shoot-down-an-argument/
|
||
[44]: http://www.visakanv.com
|
||
[45]: https://www.johndcook.com/blog/2008/01/21/selection-bias-and-bombers/#comment-16077
|
||
[46]: http://robekulick.wordpress.com/2011/09/20/british-world-war-ii-bombers-an-example-of-why-i-love-economics/
|
||
[47]: https://www.johndcook.com/blog/2008/01/21/selection-bias-and-bombers/#comment-16079
|
||
[48]: http://dartthrowingchimp.wordpress.com/2011/09/25/more-on-democratic-consolidation-and-time/
|
||
[49]: http://www.dishlab.org/blog/?p=75
|
||
[50]: http://moralmindfield.wordpress.com/2011/10/20/brians-links-20-october-2011-carnivory-science-and-art/
|
||
[51]: http://tropewell.wordpress.com/2011/10/12/selection-bias-and-wwii-bombers/
|
||
[52]: http://mrwaddell.net
|
||
[53]: https://www.johndcook.com/blog/2008/01/21/selection-bias-and-bombers/#comment-16087
|
||
[54]: http://blog.mrwaddell.net/archives/594
|
||
[55]: http://pointsandfigures.com/2013/06/07/todays-unemployment-number/
|
||
[56]: http://www.johndcook.com/blog/2010/12/06/military-intelligence-from-serial-numbers/
|
||
[57]: http://tm.durusau.net/?p=61723
|
||
[58]: https://www.johndcook.com/blog/2008/01/21/selection-bias-and-bombers/#comment-529805
|
||
[59]: https://www.johndcook.com/blog/2008/01/21/selection-bias-and-bombers/#comment-529818
|
||
[60]: http://www.twiv.tv/2015/06/07/twiv-340/
|
||
[61]: http://pointsandfigures.com/2015/10/04/a-thought-on-gun-violence/
|
||
[62]: http://www.johndcook.com/blog/2015/10/07/second-languages-and-selection-bias/
|
||
[63]: http://www.microbe.tv/twiv/twiv-340/
|
||
[64]: https://datant.wordpress.com/2016/08/12/the-birthday-problem/
|
||
[65]: https://www.johndcook.com/blog/2008/01/21/selection-bias-and-bombers/#comment-903352
|
||
[66]: http://abunchofdata.com/sunday-morning-insight-davions-et-dintelligence-artificielle-en-france-in-french/
|
||
[67]: https://www.johndcook.com/blog/2008/01/21/selection-bias-and-bombers/#respond
|
||
[68]: https://www.johndcook.com/jdc_20170630.jpg
|
||
[69]: https://www.johndcook.com/blog/2018/02/22/fibonacci-binomial-coefficient-identity/ "Permanent link to Fibonacci / Binomial coefficient identity"
|
||
[70]: https://www.johndcook.com/blog/2018/02/20/painless-project-management/ "Permanent link to Painless project management"
|
||
[71]: https://www.johndcook.com/blog/2018/02/19/new-animation-feature-for-exponential-sums/ "Permanent link to New animation feature for exponential sums"
|
||
[72]: https://www.johndcook.com/blog/2018/02/19/quantifying-normal-approximation-accuracy/ "Permanent link to Quantifying normal approximation accuracy"
|
||
[73]: https://www.johndcook.com/blog/2018/02/17/ordinary-potential-polynomials/ "Permanent link to Ordinary Potential Polynomials"
|
||
[74]: https://www.johndcook.com/contact_email.svg
|
||
[75]: https://feedburner.google.com/fb/a/mailverify?uri=TheEndeavour&loc=en_US
|
||
[76]: https://www.johndcook.com/contact_rss.svg
|
||
[77]: https://www.johndcook.com/blog/feed
|
||
[78]: https://www.johndcook.com/newsletter.svg
|
||
[79]: https://www.johndcook.com/blog/newsletter
|
||
|