--- created_at: '2017-12-11T22:32:40.000Z' title: Approximating a solution that doesn't exist (2009) url: https://www.johndcook.com/blog/2009/08/11/approximating-a-solution-that-doesnt-exist/ author: kawera points: 52 story_text: comment_text: num_comments: 13 story_id: story_title: story_url: parent_id: created_at_i: 1513031560 _tags: - story - author_kawera - story_15901251 objectID: '15901251' year: 2009 --- [Source](https://www.johndcook.com/blog/2009/08/11/approximating-a-solution-that-doesnt-exist/ "Permalink to Approximating a solution that doesn't exist | diff eqn") # Approximating a solution that doesn't exist | diff eqn [ ![John D. Cook][1] ][2] [Skip to content][3] * [ABOUT][4] * [SERVICES][5] * [APPLIED MATH][6] * [STATISTICS][7] * [COMPUTATION][8] * [WRITING][9] * [BLOG][2] * [TECHNICAL NOTES][10] * [JOURNAL ARTICLES][11] * [TWITTER][12] * [PRESENTATIONS][13] * [NEWSLETTER][14] * [CLIENTS][15] * [ENDORSEMENTS][16] [(832) 422-8646][17] [Contact][18] # Approximating a solution that doesn’t exist Posted on [11 August 2009][19] by [John][20] The following example made an impression on me when I first saw it years ago. I still think it’s an important example, though I’d draw a different conclusion from it today. **Problem**: Let _y_(_t_) be the solution to the differential equation _y_‘ = _t_2 \+ _y_2 with _y_(0) = 1. Calculate _y_(1). If we use Euler’s numerical method with a step size _h_ = 0.1, we get _y_(1) = 7.19. If we reduce the step size to 0.05 we get _y_(1) = 12.32. If we reduce the step size further to 0.01, we get _y_(1) = 90.69. That’s strange. Let’s switch over to a more accurate method, Runge-Kutta. With a step size 0.1 the Runge-Kutta method gives 735.00, and if we use a step size of 0.01 we get a result larger than 1015. What’s going on? The problem presupposes that a solution exists at _t_ = 1 when in fact no solution exists. General theory (Picard’s theorem) tells that a unique solution exists for _some_ interval containing 0, but it does not tell us how far that interval extends. With a little work we can show that a solution exists for t at least as large as π/4. However, the solution becomes unbounded somewhere between π/4 and 1. When I first saw this example, my conclusion was that it showed how important theory is. If you just go about numerically computing solutions without knowing that a solution exists, you can think you have succeeded when you’re actually computing something that doesn’t exist. Prove existence and uniqueness before computing. Theory comes first. Now I think the example shows the importance of the interplay between theory and numerical computation. It would be nice to know how big the solution interval is before computing anything, but that’s not always possible. Also, it’s not obvious from looking at the equation that there should be a problem at _t_ = 1. The difficulties we had with numerical computation suggested there might be a theoretical problem. I first saw this problem in an earlier edition of [Boyce and DiPrima][21]. The book goes on to approximate the interval over which the solution does exist using a combination of analytical and numerical methods. It looks like the solution becomes unbounded somewhere near _t_ = 0.97. I wouldn’t say that theory or computation necessarily come first. I’d say you iterate between them, starting with the approach that is more tractable. Theoretical results are  more satisfying when they’re available, but theory often doesn’t tell us as much as we’d like to know. Also, people make mistakes in theoretical computation just as they do in numerical computation. It’s best when theory and numerical work [validate each other][22]. The problem does show the importance of being concerned with existence and uniqueness, but theoretical methods are not the only methods for exploring existence. Good numerical practice, i.e. trying more than one step size or more than one numerical method, is also valuable. In any case, the problem shows that without some diligence — either theoretical or numerical — you could naively compute an approximate “solution” where no solution exists. **Related**: [Consulting in differential equations][23] Categories : [Math][24] Tags : [Differential equations][25] Bookmark the [permalink][26] # Post navigation Previous Post[Reasoning about code][27] Next Post[Physical explanation of median][28] ## 10 thoughts on “Approximating a solution that doesn’t exist” 1. Keith [ 11 August 2009 at 11:10 ][29] I thought the article below had an interesting comment about theory vs. practice from an engineering standpoint. Q: You were developing transonic theory after the sound barrier had already been broken. Hasn't much of your historical study also involved engineering problems that were "solved" in a practical sense before they were understood theoretically? A: Yes, and I think that's a typical situation in technology. You have to look hard to find cases in which the theory is well worked out before the practice. Look at the steam engine and thermodynamics; that whole vast science got started because people were trying to explain and calculate the performance of the reciprocating steam engines that had been built. 2. [Daniel Lemire][30] [ 11 August 2009 at 20:25 ][31] While the term thermodynamics came about after the first engines, we already had a lot of theory worked out before. Pressure was a well known concept. So, I wouldn’t go so far as to say that “practical application” predates theory. I suspect it is more of an entanglement… theory without practice is barren… practice without theory is crude… 3. ekzept [ 11 August 2009 at 23:41 ][32] Very nice illustration! Just to finish off the scholarship, the Boyce and DiPrima (first) edition I have (INTRODUCTION TO DIFFERENTIAL EQUATIONS, 1970, SBN 471-09338-6) puts the illustration of this problem in section 7.7, out at pages 277-278. It is interesting it is dumped in a section introducing numerical methods rather than being placed far ahead, in the Introduction. That Introduction uses Newton’s Second Law as a start, and follows with the general form of ODEs. I think this shows, in part, how heavily computation has influenced our collective view of this material. To balance, however, I’d say there are some algorithms and processes which give answers with phenomena and data theory is quite hopeless at dicing. (Think Navier-Stokes.) Sure, they have and need parts to be robustly built, and existence solutions are critical. And, sure, people — including _me_ — could do with more careful attention to theoretical underpinnings in nearly everything I do. I try. We try. But, for example, there’s a lot of good engineering that can be done with Laplace transforms that doesn’t really need an understanding of the proof of Lerch’s Theorem. I’d say that’s good! 4. Pingback: [Carnival of Mathematics #56 « Reasonable Deviations][33] 5. Jan Van lent [ 15 December 2011 at 14:05 ][34] Note that the location of the singularity is given by `t(1)`, where `t(w)` is the solution of the initial value problem `t'(w) = 1/((1-w)^2*t(w)^2+1), t(0) = 0`. The value of `t(1) = 0.96981072` can easily be found numerically. The solution to the original problem can be written using Bessel functions. 6. Pingback: [Sensitive dependence on initial conditions | John D. Cook][35] 7. Pingback: [Three views of differential equations | John D. Cook][36] 8. Pingback: [Boundary conditions are the hard part][37] 9. Pingback: [Life lessons from differential equations][38] 10. Paul Abbott [ 30 January 2017 at 00:45 ][39] There is no problem at t=1. You can continue the exact solution involving Bessel functions (easily obtained using, say, Mathematica), past the (first) singularity (which is at 0.930564508526…), and find that y(1)=-14.379106277… ### Leave a Reply [Cancel reply][40] Your email address will not be published. Required fields are marked * Comment Notify me of followup comments via e-mail Name * Email * Website Search for: ![John D. Cook][41] **John D. Cook, PhD** # Latest Posts * [Fibonacci / Binomial coefficient identity][42] * [Painless project management][43] * [New animation feature for exponential sums][44] * [Quantifying normal approximation accuracy][45] * [Ordinary Potential Polynomials][46] # Categories CategoriesSelect CategoryBusinessClinical trialsComputingCreativityGraphicsMachine learningMathMusicPowerShellPythonScienceSoftware developmentStatisticsTypographyUncategorized | ----- | | ![Subscribe to blog by email][47] | [Subscribe via email][48] | | ![RSS icon][49] | [Subscribe via RSS][50] | | ![Newsletter icon][51] | [Monthly newsletter][52] | ### [John D. Cook][2] © All rights reserved. Search for: [1]: https://www.johndcook.com/blog/wp-content/themes/ThemeAlley.Business.Pro/images/Logo.svg [2]: https://www.johndcook.com/blog/ [3]: https://www.johndcook.com#content "Skip to content" [4]: https://www.johndcook.com/blog/top/ [5]: https://www.johndcook.com/blog/services-2/ [6]: https://www.johndcook.com/blog/applied-math/ [7]: https://www.johndcook.com/blog/applied-statistics/ [8]: https://www.johndcook.com/blog/applied-computation/ [9]: https://www.johndcook.com/blog/writing/ [10]: https://www.johndcook.com/blog/notes/ [11]: https://www.johndcook.com/blog/articles/ [12]: https://www.johndcook.com/blog/twitter_page/ [13]: https://www.johndcook.com/blog/presentations/ [14]: https://www.johndcook.com/blog/newsletter/ [15]: https://www.johndcook.com/blog/clients-new/ [16]: https://www.johndcook.com/blog/endorsements/ [17]: tel:8324228646 [18]: https://www.johndcook.com/blog/contact/ [19]: https://www.johndcook.com/blog/2009/08/11/approximating-a-solution-that-doesnt-exist/ "10:38" [20]: https://www.johndcook.com/blog/author/john/ "View all posts by John" [21]: http://www.amazon.com/gp/product/0471433381?ie=UTF8&tag=theende-20&linkCode=xm2&camp=1789&creativeASIN=0471433381 [22]: https://www.johndcook.com/blog/2008/01/10/complementary-validation/ [23]: https://www.johndcook.com/blog/partial-differential-equations/ [24]: https://www.johndcook.com/blog/category/math/ [25]: https://www.johndcook.com/blog/tag/differential-equations/ [26]: https://www.johndcook.com/blog/2009/08/11/approximating-a-solution-that-doesnt-exist/ "Permalink to Approximating a solution that doesn’t exist" [27]: https://www.johndcook.com/blog/2009/08/11/reasoning-about-code/ [28]: https://www.johndcook.com/blog/2009/08/12/physical-median/ [29]: https://www.johndcook.com/blog/2009/08/11/approximating-a-solution-that-doesnt-exist/#comment-12092 [30]: http://www.daniel-lemire.com/ [31]: https://www.johndcook.com/blog/2009/08/11/approximating-a-solution-that-doesnt-exist/#comment-12093 [32]: https://www.johndcook.com/blog/2009/08/11/approximating-a-solution-that-doesnt-exist/#comment-12094 [33]: http://stochastix.wordpress.com/2009/08/27/carnival-of-mathematics-56/ [34]: https://www.johndcook.com/blog/2009/08/11/approximating-a-solution-that-doesnt-exist/#comment-12096 [35]: http://www.johndcook.com/blog/2013/11/12/sensitive-dependence-on-initial-conditions/ [36]: http://www.johndcook.com/blog/2011/01/07/three-views-of-differential-equations/ [37]: http://www.johndcook.com/blog/2012/01/24/boundary-conditions/ [38]: http://www.johndcook.com/blog/2015/07/23/life-lessons-from-differential-equations/ [39]: https://www.johndcook.com/blog/2009/08/11/approximating-a-solution-that-doesnt-exist/#comment-924962 [40]: https://www.johndcook.com/blog/2009/08/11/approximating-a-solution-that-doesnt-exist/#respond [41]: https://www.johndcook.com/jdc_20170630.jpg [42]: https://www.johndcook.com/blog/2018/02/22/fibonacci-binomial-coefficient-identity/ "Permanent link to Fibonacci / Binomial coefficient identity" [43]: https://www.johndcook.com/blog/2018/02/20/painless-project-management/ "Permanent link to Painless project management" [44]: https://www.johndcook.com/blog/2018/02/19/new-animation-feature-for-exponential-sums/ "Permanent link to New animation feature for exponential sums" [45]: https://www.johndcook.com/blog/2018/02/19/quantifying-normal-approximation-accuracy/ "Permanent link to Quantifying normal approximation accuracy" [46]: https://www.johndcook.com/blog/2018/02/17/ordinary-potential-polynomials/ "Permanent link to Ordinary Potential Polynomials" [47]: https://www.johndcook.com/contact_email.svg [48]: https://feedburner.google.com/fb/a/mailverify?uri=TheEndeavour&loc=en_US [49]: https://www.johndcook.com/contact_rss.svg [50]: https://www.johndcook.com/blog/feed [51]: https://www.johndcook.com/newsletter.svg [52]: https://www.johndcook.com/blog/newsletter