---
created_at: '2017-12-30T08:07:21.000Z'
title: How to test a random number generator (2010)
url: https://www.johndcook.com/blog/2010/12/06/how-to-test-a-random-number-generator-2/
author: Cieplak
points: 68
story_text: 
comment_text: 
num_comments: 42
story_id: 
story_title: 
story_url: 
parent_id: 
created_at_i: 1514621241
_tags:
- story
- author_Cieplak
- story_16034353
objectID: '16034353'

---
[Source](https://www.johndcook.com/blog/2010/12/06/how-to-test-a-random-number-generator-2/ "Permalink to How to test a random number generator")

# How to test a random number generator

[ ![John D. Cook][1] ][2]

[Skip to content][3]

* [ABOUT][4]
* [SERVICES][5]
    * [APPLIED MATH][6]
    * [STATISTICS][7]
    * [COMPUTATION][8]
* [WRITING][9]
    * [BLOG][2]
    * [TECHNICAL NOTES][10]
    * [JOURNAL ARTICLES][11]
    * [TWITTER][12]
    * [PRESENTATIONS][13]
    * [NEWSLETTER][14]
* [CLIENTS][15]
    * [ENDORSEMENTS][16]

[(832) 422-8646][17]

[Contact][18]

# How to test a random number generator

Posted on [6 December 2010][19] by [John][20]

Last year I wrote a chapter for O’Reilly’s book [Beautiful Testing][21] (ISBN 0596159811). The publisher gave each of us permission to post our chapters online, and so here is Chapter 10: [How to test a random number generator][22].

![Beautiful Testing: Leading Professionals Reveal How They Test][23]

Categories : [Software development][24] [Statistics][25]

Tags : [Books][26] [Probability and Statistics][27] [Programming][28] [Quality][29] [RNG][30]

Bookmark the [permalink][31]

# Post navigation

Previous Post[Military intelligence from serial numbers][32]

Next Post[Maybe you only need it because you have it][33]

##  24 thoughts on “How to test a random number generator”

1. Wen Chen

[ 6 December 2010 at 14:59  ][34]

hi john,

thank you for the chapter.

i’ve been following your blog this year. just wondering if you were planning to write a numerical computing book. or if there are books you would recommend.

2. John

[ 6 December 2010 at 15:08  ][35]

Wen, Thanks. I don’t have any plans to write a numerical computing book.

One of my favorite numerical books is an old one, [Numerical Methods that Work][36]. It’s not a cookbook. It’s a book on how to think about numerical computing.

3. [Hansi][37]

[ 6 December 2010 at 15:45  ][38]

Thanks for that, great read. Any suggestions for a programming recipe book that covers the same or similar tests?

4. John

[ 6 December 2010 at 15:49  ][39]

Hansi: You can read more about some of the tests in Knuth’s [Seminumerical Algorithms][40], TAOCP volume 2. You might also check [Numerical Recipes][41].

5. [Hansi][37]

[ 6 December 2010 at 15:54  ][42]

Great, I’m pretty sure there is a copy of Knuth’s TAOCP at work. I’ll have a look tomorrow.

Thanks again.

6. SteveBrooklineMA

[ 7 December 2010 at 13:07  ][43]

Acton’s book is an entertaining read, but how well has it stood up? Computing power and software have advanced so much in the 40 years since he wrote it, it’s not clear that the cautionary tales he tells are still as relevant. He might tell you a story of how 20 minutes of thought could have saved an investigator 20 hours of computing time. Now however, that same computer program, or a more robust off-the-shelf one, might run in a couple of seconds. So why think? 😉

You could argue that the complexity of problems has grown as computing power has grown, so that thought is as necessary as ever. However, I might counter that more complex problems are difficult to tackle in the way the problems in Acton’s book can be tackled. Don’t the problems in Acton’s book seem rather… quaint?

I think if I were going to recommend a book for learning practical, hands-on numerical computing, I would recommend Numerical Recipes.

7. John

[ 7 December 2010 at 14:52  ][44]

I think the principles in Acton’s book are still valuable, though the specifics are outdated. For example, most computing is now done in double precision rather than single precision. Loss of accuracy still happens when you’re not careful; you see the same problems but for a smaller range of inputs.

Acton doesn’t deal with sorting, but bubble sort is still slower than heap sort, even though both are instantaneous for small data sets. The idea of what is “small” has changed over time, but the principle remains.

However, there is one important area that Acton doesn’t address at all if I remember correctly, and that is random methods: random number generation, Monte Carlo integration, etc. These methods have become far more common in practice (at least in my practice :)).

8. SteveBrooklineMA

[ 7 December 2010 at 15:11  ][45]

I think the most out-of-date aspect of Acton’s book is that it has an entire chapter on Fourier Series, but doesn’t mention the FFT.

9. [mat roberts][46]

[ 8 December 2010 at 06:26  ][47]

I liked the chapter, in particular the line “The point is not only to test the quality of the library itself, but also to test the user's understanding of the library.”

I’m surprised you didn’t recommend plotting the data and taking a look at it – always seems valuable.

10. Pingback: igorbrejc.net » Fresh Catch For December 12th

11. Jason B

[ 11 January 2011 at 11:57  ][48]

I like this one for testing RNG’s.

<http://portal.acm.org/citation.cfm?doid=1268776.1268777>

There is a nice battery of tests ready to go in C++. If the RNG makes it through that in good shape, great.

One kind of disturbing aspect of the chapter is that its existence implies software developers should be going off and writing their own RNGs. Why bother when there are good ones out there that have been through very rigorous scrutiny?

The problem with plotting random data, especially uniform data, is that we can’t visually detect any meaningful flaws in the data.

12. John

[ 11 January 2011 at 12:52  ][49]

Jason: I don’t want to encourage anyone to develop their own uniform random number generation algorithm. But some people will need to implement existing algorithms in new languages. And even more people will need to transform a uniform random sequence into a sequence with another distribution, maybe a distribution unique to their problem.

13. Jason B

[ 11 January 2011 at 13:18  ][50]

John,

Thanks for the reply. I hope the readership of the book takes your perspective as well!

Jason

14. Khameel

[ 28 September 2011 at 04:41  ][51]

Hi John,

Even though, this post is no longer a current subject of discussion. I write to appreciate your efforts in putting together a collection of numerical gems in this blog. I have benefitted quite a lot from this blog. Remain blessed.

From Singapore.

15. Pingback: [Pseudo-Random vs. Random Numbers in R at johnramey][52]

16. Jim Dukelow

[ 26 March 2012 at 11:20  ][53]

Hi John,

I share your love of Acton’s book and Steve’s recognition that it is, in some respects, outdated. I like Acton’s mindset of thinking about what you are trying to calculate and being willing to find different algorithms when you are up against the potential for or reality of catastrophic loss of precision. The original book had the title embossed into the red cover, with the letters highlighted in silver. Also embossed, but not highlighted, was the word “usually”. Lovely.

17. Pingback: [Random number sequence overlap — The Endeavour][54]

18. Pingback: [Statistical Toolbox: The Kolmogorov-Smirnov Test – AK Tech Blog][55]

19. Pingback: [Random or not? That is the question! « Holy Hash!][56]

20. Matthew Wyatt

[ 4 December 2012 at 12:01  ][57]

Just came across this link from the CompSciFact twitter account. Are you aware of the typo in the first footnote on page 131? The word “lest” should be “least”, I believe. I know this is a few years old, so maybe it’s not helpful now.

21. Ben Bradley

[ 4 December 2012 at 12:18  ][58]

I’m reminded that sometimes, even though you may THINK you want random, it may not be what you want. From this tweet from earlier today, what the tweeter really wants is “random, or even a cheap approximation, unless the current entry has come up TOO recently:” “iPad, I appreciate your shuffle. I do. And you know I like Enya “The Longships.” But not as every third song. #shuffleparalysis”

And I hope I’m not too much of an insufferable pedant in pointing out that this is about testing pseudo-random number generators (PRNG) – a “true” random number generator will (almost?!) always pass such tests.

22. Pingback: [Comment tester un générateur de nombres aléatoires | L'Endormitoire][59]

23. Pingback: [Random is as random does][60]

24. Pingback: [New top story on Hacker News: How to test a random number generator (2010) – ÇlusterAssets Inc.,][61]

### Leave a Reply [Cancel reply][62]

Your email address will not be published. Required fields are marked *

Comment

Notify me of followup comments via e-mail

Name *

Email *

Website

Search for:

![John D. Cook][63]

**John D. Cook, PhD**

  
  
  
  

  
  

# Latest Posts

* [Fibonacci / Binomial coefficient identity][64]
* [Painless project management][65]
* [New animation feature for exponential sums][66]
* [Quantifying normal approximation accuracy][67]
* [Ordinary Potential Polynomials][68]

# Categories

CategoriesSelect CategoryBusinessClinical trialsComputingCreativityGraphicsMachine learningMathMusicPowerShellPythonScienceSoftware developmentStatisticsTypographyUncategorized

| ----- |
| ![Subscribe to blog by email][69] | [Subscribe via email][70] | 
| ![RSS icon][71] | [Subscribe via RSS][72] | 
| ![Newsletter icon][73] | [Monthly newsletter][74] | 

### [John D. Cook][2]

© All rights reserved.

Search for:

[1]: https://www.johndcook.com/blog/wp-content/themes/ThemeAlley.Business.Pro/images/Logo.svg
[2]: https://www.johndcook.com/blog/
[3]: https://www.johndcook.com#content "Skip to content"
[4]: https://www.johndcook.com/blog/top/
[5]: https://www.johndcook.com/blog/services-2/
[6]: https://www.johndcook.com/blog/applied-math/
[7]: https://www.johndcook.com/blog/applied-statistics/
[8]: https://www.johndcook.com/blog/applied-computation/
[9]: https://www.johndcook.com/blog/writing/
[10]: https://www.johndcook.com/blog/notes/
[11]: https://www.johndcook.com/blog/articles/
[12]: https://www.johndcook.com/blog/twitter_page/
[13]: https://www.johndcook.com/blog/presentations/
[14]: https://www.johndcook.com/blog/newsletter/
[15]: https://www.johndcook.com/blog/clients-new/
[16]: https://www.johndcook.com/blog/endorsements/
[17]: tel:8324228646
[18]: https://www.johndcook.com/blog/contact/
[19]: https://www.johndcook.com/blog/2010/12/06/how-to-test-a-random-number-generator-2/ "12:35"
[20]: https://www.johndcook.com/blog/author/john/ "View all posts by John"
[21]: http://www.amazon.com/gp/product/0596159811?ie=UTF8&amp;tag=theende-20&amp;linkCode=xm2&amp;camp=1789&amp;creativeASIN=0596159811
[22]: https://www.johndcook.com/Beautiful_Testing_ch10.pdf
[23]: //www.johndcook.com/51B+qoLmASL._SL160_.jpg
[24]: https://www.johndcook.com/blog/category/software-development/
[25]: https://www.johndcook.com/blog/category/statistics/
[26]: https://www.johndcook.com/blog/tag/books/
[27]: https://www.johndcook.com/blog/tag/probability-and-statistics/
[28]: https://www.johndcook.com/blog/tag/programming/
[29]: https://www.johndcook.com/blog/tag/quality/
[30]: https://www.johndcook.com/blog/tag/rng/
[31]: https://www.johndcook.com/blog/2010/12/06/how-to-test-a-random-number-generator-2/ "Permalink to How to test a random number generator"
[32]: https://www.johndcook.com/blog/2010/12/06/military-intelligence-from-serial-numbers/
[33]: https://www.johndcook.com/blog/2010/12/07/cascading-needs/
[34]: https://www.johndcook.com/blog/2010/12/06/how-to-test-a-random-number-generator-2/#comment-18821
[35]: https://www.johndcook.com/blog/2010/12/06/how-to-test-a-random-number-generator-2/#comment-18822
[36]: http://www.amazon.com/gp/product/0883854503?ie=UTF8&tag=theende-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=0883854503
[37]: http://www.hlynsson.com
[38]: https://www.johndcook.com/blog/2010/12/06/how-to-test-a-random-number-generator-2/#comment-18823
[39]: https://www.johndcook.com/blog/2010/12/06/how-to-test-a-random-number-generator-2/#comment-18824
[40]: http://www.amazon.com/gp/product/0201896842?ie=UTF8&tag=theende-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=0201896842
[41]: http://www.amazon.com/gp/product/0521880688?ie=UTF8&tag=theende-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=0521880688
[42]: https://www.johndcook.com/blog/2010/12/06/how-to-test-a-random-number-generator-2/#comment-18825
[43]: https://www.johndcook.com/blog/2010/12/06/how-to-test-a-random-number-generator-2/#comment-18826
[44]: https://www.johndcook.com/blog/2010/12/06/how-to-test-a-random-number-generator-2/#comment-18827
[45]: https://www.johndcook.com/blog/2010/12/06/how-to-test-a-random-number-generator-2/#comment-18828
[46]: http://moleseyhill.com
[47]: https://www.johndcook.com/blog/2010/12/06/how-to-test-a-random-number-generator-2/#comment-18829
[48]: https://www.johndcook.com/blog/2010/12/06/how-to-test-a-random-number-generator-2/#comment-18831
[49]: https://www.johndcook.com/blog/2010/12/06/how-to-test-a-random-number-generator-2/#comment-18832
[50]: https://www.johndcook.com/blog/2010/12/06/how-to-test-a-random-number-generator-2/#comment-18833
[51]: https://www.johndcook.com/blog/2010/12/06/how-to-test-a-random-number-generator-2/#comment-18834
[52]: http://johnramey.net/blog/2011/11/25/pseudo-random-vs-random-numbers-in-r/
[53]: https://www.johndcook.com/blog/2010/12/06/how-to-test-a-random-number-generator-2/#comment-18836
[54]: http://www.johndcook.com/blog/2012/04/16/random-number-sequence-overlap/
[55]: http://blog.aggregateknowledge.com/2012/04/18/statistical-toolbox-the-kolmogorov-smirnov-test/
[56]: http://holyhash.wordpress.com/2012/09/13/random-or-not-that-is-the-question/
[57]: https://www.johndcook.com/blog/2010/12/06/how-to-test-a-random-number-generator-2/#comment-18840
[58]: https://www.johndcook.com/blog/2010/12/06/how-to-test-a-random-number-generator-2/#comment-18841
[59]: https://endormitoire.wordpress.com/2016/01/09/comment-tester-un-generateur-de-nombres-aleatoires/
[60]: http://www.johndcook.com/blog/2012/04/19/random-is-as-random-does/
[61]: https://clusterassets.wordpress.com/2017/12/31/new-top-story-on-hacker-news-how-to-test-a-random-number-generator-2010/
[62]: https://www.johndcook.com/blog/2010/12/06/how-to-test-a-random-number-generator-2/#respond
[63]: https://www.johndcook.com/jdc_20170630.jpg
[64]: https://www.johndcook.com/blog/2018/02/22/fibonacci-binomial-coefficient-identity/ "Permanent link to Fibonacci / Binomial coefficient identity"
[65]: https://www.johndcook.com/blog/2018/02/20/painless-project-management/ "Permanent link to Painless project management"
[66]: https://www.johndcook.com/blog/2018/02/19/new-animation-feature-for-exponential-sums/ "Permanent link to New animation feature for exponential sums"
[67]: https://www.johndcook.com/blog/2018/02/19/quantifying-normal-approximation-accuracy/ "Permanent link to Quantifying normal approximation accuracy"
[68]: https://www.johndcook.com/blog/2018/02/17/ordinary-potential-polynomials/ "Permanent link to Ordinary Potential Polynomials"
[69]: https://www.johndcook.com/contact_email.svg
[70]: https://feedburner.google.com/fb/a/mailverify?uri=TheEndeavour&loc=en_US
[71]: https://www.johndcook.com/contact_rss.svg
[72]: https://www.johndcook.com/blog/feed
[73]: https://www.johndcook.com/newsletter.svg
[74]: https://www.johndcook.com/blog/newsletter

