856 lines
36 KiB
Markdown
856 lines
36 KiB
Markdown
---
|
|
created_at: '2016-08-18T06:09:52.000Z'
|
|
title: Git bisect (2006)
|
|
url: http://yarchive.net/comp/linux/git_bisect.html
|
|
author: '99_00'
|
|
points: 45
|
|
story_text:
|
|
comment_text:
|
|
num_comments: 11
|
|
story_id:
|
|
story_title:
|
|
story_url:
|
|
parent_id:
|
|
created_at_i: 1471500592
|
|
_tags:
|
|
- story
|
|
- author_99_00
|
|
- story_12310683
|
|
objectID: '12310683'
|
|
|
|
---
|
|
[Source](http://yarchive.net/comp/linux/git_bisect.html "Permalink to git bisect (Linus Torvalds) ")
|
|
|
|
# git bisect (Linus Torvalds)
|
|
|
|
[Index][1] [Home][2] [About][3] [Blog][4]
|
|
|
|
* * *
|
|
|
|
|
|
From: Linus Torvalds <torvalds@osdl.org>
|
|
Newsgroups: fa.linux.kernel
|
|
Subject: Re: git pull on Linux/ACPI release tree
|
|
Date: Tue, 10 Jan 2006 19:30:42 UTC
|
|
Message-ID: <[fa.fv9n3b6.j067qs@ifi.uio.no][5]>
|
|
Original-Message-ID: <[Pine.LNX.4.64.0601101111110.4939@g5.osdl.org][6]>
|
|
|
|
On Tue, 10 Jan 2006, Linus Torvalds wrote:
|
|
>
|
|
> Now, the git history is _not_ really a two-dimensional surface, so it's
|
|
> just an analogy, not an exact identity. But from a visualization
|
|
> standpoint, it's a good way to think of each "git bisect" as adding a
|
|
> _line_ on the surface rather than a point on a linear line.
|
|
|
|
Actually, the way I think of it is akin to the "light cones" in physics. A
|
|
point in space-time doesn't define a fully ordered "before and after": but
|
|
it _does_ describe a "light cone" which tells you what is reachable from
|
|
that point, and what that point reaches. Within those cones, that
|
|
particular point ("commit") has a strict ordering.
|
|
|
|
And exactly as in physics, in git there's a lot of space that is _not_
|
|
ordered by that commit. And the way to bisect is basically to find the
|
|
right points in "git space" to create the right "light cone" that you
|
|
find the point where the git space that is reachable from that commit has
|
|
the same volume as the git space that isn't reachable.
|
|
|
|
And maybe that makes more sense to you (if you're into physics), or maybe
|
|
it makes less sense to you.
|
|
|
|
Now, since we always search the "git space" in the cone that is defined by
|
|
"reachable from the bad commit, but not reachable from any good commit",
|
|
the way we handle "bad" and "good" is actually not a mirror-image. If we
|
|
fine a new _bad_ commit, we know that it was reachable from the old bad
|
|
commit, and thus the old bad commit is now uninteresting: the new bad
|
|
commit forms a "past light cone" that is a strict subset of the old one,
|
|
so we can totally discard the old bad commit from any future
|
|
consideration. It doesn't tell us anything new.
|
|
|
|
In contrast, if we find a new _good_ commit, the "past light cone" (aka
|
|
"set of commits reachable from it") is -not- necessarily a proper superset
|
|
of the previous set of good commits, so when we find a good commit, we
|
|
still need to carry the _other_ good commits around, and the "known good"
|
|
universe is the _union_ of all the "good commit past lightcones".
|
|
|
|
Then the "unknown space" is the set difference of the "past lightcone of
|
|
the bad commit" and of this "union of past lightcones of good commits".
|
|
It's the space that is reachable from the known-bad commit, but not
|
|
reachable from any known-good commit.
|
|
|
|
So this means that when doing bisection, what we want to do is find the
|
|
point in git space that has _new_ "reachability" within that unknown space
|
|
that is as close to half that volume as space as possible. And that's
|
|
exactly what "git-rev-list --bisect" calculates.
|
|
|
|
So every time, we try to either move the "known bad" light-cone down in
|
|
time in the unknown space, _or_ we add a new "known good" light-cone. In
|
|
either case, the "unknown git space" keeps shrinking by half each time.
|
|
|
|
("by half" is not exact, because git space is not only quanticized, it
|
|
also has a rather strange "distance function". In other words, we're
|
|
talking about a rather strange space. The good news is that the space is
|
|
small enough that we can just enumerate every quantum and simply
|
|
calculate the volume it defines in that space. IOW, we do a very
|
|
brute-force thing, and it works fine).
|
|
|
|
Linus
|
|
|
|
|
|
|
|
|
|
* * *
|
|
|
|
|
|
From: Linus Torvalds <torvalds@osdl.org>
|
|
Newsgroups: fa.linux.kernel
|
|
Subject: Re: rc6 keeps hanging and blanking displays where rc4-mm1 works
|
|
Date: Fri, 12 Aug 2005 16:54:50 UTC
|
|
Message-ID: <[fa.g0ad5b8.m0o5qk@ifi.uio.no][7]>
|
|
Original-Message-ID: <[Pine.LNX.4.58.0508120937140.3295@g5.osdl.org][8]>
|
|
|
|
On Fri, 12 Aug 2005, Helge Hafting wrote:
|
|
>
|
|
> > at the moment. The setup is fine with 2.6.13-rc4-mm1 x86-64, no
|
|
> > problems there.
|
|
>
|
|
> The problem still exists in 2.6.13-rc6. Usually, all I get is a
|
|
> suddenly black display, solveable by resizing.
|
|
|
|
Is there any chance you could try bisecting the problem? Either just
|
|
binary-searching the patches or by using the git bisect helper scripts?
|
|
|
|
Obviously the git approach needs a "good" kernel in git, but if
|
|
2.6.13-rc4-mm1 is ok, then I assume that 2.6.13-rc4 is ok too? That's a
|
|
fair number of changes:
|
|
|
|
git-rev-list v2.6.13-rc4..v2.6.13-rc6 | wc
|
|
340 340 13940
|
|
|
|
but if you can tighten it up a bit (you already had trouble at rc5, I
|
|
think), it shouldn't require testing more than a few kernels.
|
|
|
|
Git has had bisection support for a while, but the helper scripts to use
|
|
it sanely are fairly new, so I think you'd need the git-0.99.4 release for
|
|
those. But then you'd just do
|
|
|
|
git bisect start
|
|
git bisect bad v2.6.13-rc5
|
|
git bisect good v2.6.13-rc4
|
|
|
|
and start bisecting (that will check out a mid-way point automatically,
|
|
you build it, and then do "git bisect bad" or "git bisect good" depending
|
|
on whether the result is bad or good - it will continue to try to find
|
|
half-way points until it has found the point that turns from good to
|
|
bad..)
|
|
|
|
Linus
|
|
|
|
|
|
|
|
* * *
|
|
|
|
|
|
From: Linus Torvalds <torvalds@osdl.org>
|
|
Newsgroups: fa.linux.kernel
|
|
Subject: Re: Simple script that locks up my box with recent kernels
|
|
Date: Sat, 07 Oct 2006 21:27:08 UTC
|
|
Message-ID: <[fa./oMGTFml4fnJMGJZgMEDjT9Ip0Q@ifi.uio.no][9]>
|
|
|
|
On Sat, 7 Oct 2006, Jesper Juhl wrote:
|
|
>
|
|
> > Can I bother you to just bisect it?
|
|
>
|
|
> Sure, but it will take a little while since building + booting +
|
|
> starting the test + waiting for the lockup takes a fair bit of time
|
|
> for each kernel
|
|
|
|
Sure. That said, we've tried to narrow down things that took hours or days
|
|
(under real loads, not some nice test-script) to reproduce, and while it
|
|
doesn't always work, the real problem tends to be if the problem case
|
|
isn't really reproducible. It sounds like yours is pretty clear-cut, and
|
|
that will make things much easier.
|
|
|
|
> and also due to the fact that my git skills are pretty
|
|
> limited, but I'll figure it out (need to improve those git skills
|
|
> anyway) :-)
|
|
|
|
"git bisect" in particular isn't that hard to use, and it will really do
|
|
a lot of heavy lifting for you.
|
|
|
|
Although since it will just select a random commit (well, it's not
|
|
"random": it's strictly as half-way as it can possibly be, but it's
|
|
automated without any regard for anything else), you can sometimes hit a
|
|
situation where git will ask you to test a kernel that simply doesn't work
|
|
at all, and you can't even test whether it reproduces your particular bug
|
|
or not.
|
|
|
|
For example, "git bisect" might pick a kernel that just doesn't compile,
|
|
because of some stupid bug that was fixed almost immediately afterwards.
|
|
In those cases, the total automation of "git bisect" ends up being
|
|
something that has to be helped along by hand, and then it definitely
|
|
helps to know more about how git works.
|
|
|
|
Anyway, the quick tutorial about "git bisect" is that once you've given it
|
|
the required first "good" and "bad" points, it will create a new branch in
|
|
the repository (called "bisect", in case you care), and after that point
|
|
it will do a search in the commit DAG (aka "history tree" - it's not a
|
|
tree, it's a DAG, since merges will join branches together) for the next
|
|
commit that will neatly "split" the DAG into two equal pieces. It will
|
|
keep splitting the commit history until you get fed up, or until it has
|
|
pinpointed the single commit that caused the problem.
|
|
|
|
The nicest tool to use during bisection is to just do a
|
|
|
|
git bisect visualize
|
|
|
|
that simply starts up "gitk" (the default git history visualizer) to show
|
|
what the current state of bisection is. Now, if there are thousands and
|
|
thousands of commits, you'll have a really hard time getting a visual clue
|
|
about what is going on, but especially once you get to a smaller set of
|
|
commits, it's very useful indeed.
|
|
|
|
And it's _especially_ useful if you hit one of the problem spots where you
|
|
can't test the resulting tree for some unrelated reason. When that
|
|
happens, you should _not_ mark the problematic commit as being "bad",
|
|
because you really don't know - the "badness" of that commit is probably
|
|
not related to the "badness" that you're actually searching for.
|
|
|
|
Instead, you should say "ok, I refuse to test this commit at all, because
|
|
it's got other problems, and I will select another commit instead". The
|
|
bisection algorithm doesn't care which commit you pick, as long as it's
|
|
within the set of "unknown" commits that you'll see with the visualization
|
|
tool.
|
|
|
|
Of course, for efficiency reasons, the _closer_ you get to the half-way
|
|
mark, the better. So it's useful to try to pick a commit that is close to
|
|
the one that "git bisect" originally chose for you, but that's not a
|
|
correctness issue, that's just an issue of "if we have a thousand
|
|
potential commits, we're better off bisecting it 400/600 rather than
|
|
1/999, even if the exact half-way point isn't testable".
|
|
|
|
So if you need to decide to pick another point than the one "git bisect"
|
|
chose for you automatically, just select that commit in the visualizer
|
|
(which will cut the SHA1 name of it), and then do
|
|
|
|
git reset --hard <paste-sha1-here>
|
|
|
|
to reset the "bisect" branch to that point instead. And then compile and
|
|
test that kernel instead (and then if that's good or bad, you can do the
|
|
"git bisect good" or "git bisect bad" thing to mark it so, and git will
|
|
continue to bisect the set of commits).
|
|
|
|
It can be a bit boring, but damn, it's effective. I've used "git bisect"
|
|
several times when I've been too lazy to try to really think about what is
|
|
going on - I'll happily brute-force bug-finding even if it might take a
|
|
little longer, if it's guaranteed to find it (and if the bug is
|
|
reproducible, git bisect definitely guarantees to find what made it
|
|
appear, even if that may not necessarily be the deeper _cause_ of the bug)
|
|
|
|
Linus
|
|
|
|
|
|
|
|
* * *
|
|
|
|
|
|
From: Linus Torvalds <torvalds@linux-foundation.org>
|
|
Newsgroups: fa.linux.kernel
|
|
Subject: Re: 2.6.21-rc1: known regressions (part 2)
|
|
Date: Thu, 01 Mar 2007 23:40:46 UTC
|
|
Message-ID: <[fa.63Ct4ClRD8hjd7Z3taISujCscb8@ifi.uio.no][10]>
|
|
|
|
On Thu, 1 Mar 2007, Ingo Molnar wrote:
|
|
>
|
|
> * Ingo Molnar <mingo@elte.hu> wrote:
|
|
>
|
|
> > update: f3ccb06f3b8e0cf42b579db21f3ca7f17fcc3f38 works for me too, and
|
|
> > 01363220f5d23ef68276db8974e46a502e43d01d is broken. I too will attempt
|
|
> > to bisect this.
|
|
>
|
|
> hm. There's some weird bisection artifact here. Here are the commits i
|
|
> tested, in git-log order:
|
|
>
|
|
> #1 commit 01363220f5d23ef68276db8974e46a502e43d01d bad
|
|
> #2 commit ee404566f97f9254433399fbbcfa05390c7c55f7 bad
|
|
> #3 commit f3ccb06f3b8e0cf42b579db21f3ca7f17fcc3f38 good
|
|
> #4 commit c827ba4cb49a30ce581201fd0ba2be77cde412c7 bad
|
|
|
|
Use "git bisect visualize" to see what bisect ends up doing.
|
|
|
|
> if i tell git-bisect that #1 is bad and #3 is good, then it offers me #2
|
|
> - that's OK. But when i tell it that #2 is bad, it offers #4 - which is
|
|
> out of order!
|
|
|
|
No it's not. "git bisect" does exactly the right thing. There is no simple
|
|
ordering in a complex branch-merge scenario, you can't just put the
|
|
commits in some "ordering" and test things in time order. That would be
|
|
totally broken, and idiotic. It doesn't give the right results.
|
|
|
|
What git bisect does is to find the commit that most closely *bisects* the
|
|
history of commits, so that if it is marked good/bad, it will leave you
|
|
with about 50% of the commits left. But if you are looking at date order,
|
|
you're entirely confused.
|
|
|
|
For example, let's take a really simple case
|
|
|
|
a <- bad
|
|
/
|
|
b c
|
|
| |
|
|
d e
|
|
| |
|
|
f g
|
|
/
|
|
h
|
|
|
|
|
* <-good
|
|
|
|
and if you are looking to find something "in the middle", you might thing
|
|
that "d" or "e" are the best choices, since time-wise, they are in the
|
|
middle.
|
|
|
|
But that's not true AT ALL.
|
|
|
|
If you actually want to bisect that kind of history, you need to choose
|
|
"b" or "c", even though they may both be *much* more "recent" than the
|
|
others. Why? Because if you pick "d", you're really only testing three
|
|
commits ('d' 'f' and 'h') out of the 8 commits you have to test.
|
|
|
|
In contrast, if you pick 'b', you are testing the effects of *four*
|
|
commits ('b', 'd', 'f' and 'h') and you have thus neatly bisected the
|
|
commits into two equal groups for testing (one group _with_ those four
|
|
commits, and one group _without_) instead of having partitioned them as 3
|
|
commits vs 5 commits.
|
|
|
|
So please realize that non-linear history very much means that you MUST
|
|
NOT think that you just pick a commit "in the middle". No, git bisect is a
|
|
LOT smarter than that - it picks a commit that *reaches* about half the
|
|
commits you have left to test.
|
|
|
|
> The bisection goes off into la-la land after that and
|
|
> never gets back to a commit that is /after/ the good commit. How is this
|
|
> possible? (I upgraded from git-1.4.4 to 1.5.0 to make sure this isnt
|
|
> some git bug that's already fixed.)
|
|
|
|
It's possible because git knows what it is doing, and you didn't think
|
|
things through.
|
|
|
|
The commits that "git bisect" picked out are the right ones. Quite often,
|
|
there may be two or more "equally good" commits (in my example above, you
|
|
can choose either "b" or "c", and it will bisect the set of untested
|
|
commits equally well - in two groups of four, but two *different* groups
|
|
of four commits), and yes, it's possible that git has a bug that makes it
|
|
pick the wrong ones, but quite frankly, I seriously doubt it. "git bisect"
|
|
has been very successful indeed, and is generally a *lot* better at
|
|
picking a commit "in the middle" than people are, exactly because it's
|
|
quite hard to see which commit "reaches" half the commits if you have lots
|
|
of merges and branches.
|
|
|
|
Try out
|
|
|
|
git bisect visualize
|
|
|
|
and it will literally show you what it is doing.
|
|
|
|
What can be confusing is that if the "good" and "bad" markers are ON
|
|
DIFFERENT BRANCHES OF DEVELOPMENT, you may not even *see* the "good"
|
|
marker, because you may well have something like this:
|
|
|
|
|
|
a <- bad
|
|
|
|
|
b * <- good
|
|
| |
|
|
c d
|
|
/
|
|
e
|
|
|
|
|
f
|
|
|
|
|
...
|
|
|
|
and what do you think "git bisect visualize" will actually show you?
|
|
|
|
Since 'd', 'e' and 'f' are all in the "good" set (they both exist as
|
|
commits in something leading up to a commit that has already been deemed
|
|
fine), they aren't *interesting* - they can't be introducing the bug,
|
|
since if that was the case, the good commit wouldn't have been good. So as
|
|
far as bisection is concerned, the tree actually looks like
|
|
|
|
a <- bad
|
|
|
|
|
b
|
|
|
|
|
c
|
|
|
|
|
...
|
|
|
|
and you have just three commits that are potentially interesting: 'a', 'b'
|
|
and 'c'.
|
|
|
|
Now, with three commits, you cannot test them half-and-half, so you have
|
|
to test it in groups of 1 vs 2 commits, so it's arbitrary whether you
|
|
choose 'b' or 'c' to test, but you'd test one of them. Say that you choose
|
|
'b', and it turns out to be good. If so, you're done: 'a' is bad and 'b'
|
|
is good, so the bug was introduced in 'a'. But if it turns out to be bad,
|
|
you'll still have to test 'c' too, since you don't know if the bug was
|
|
*introduced* in 'b' or not.
|
|
|
|
See?
|
|
|
|
> i'll try to straighten this out manually
|
|
|
|
Don't. You're just going to make your bisection much less effective. The
|
|
whole point of bisection is that you can usually cut the number of commits
|
|
to test pretty exactly in half. If you start mucking with the commits to
|
|
test, and you don't understand about the reachability graph, you'll just
|
|
choose a much worse set of commits to test than "git bisect" will do.
|
|
|
|
So learn to trust "git bisect". It really does know what it is doing.
|
|
|
|
Linus
|
|
|
|
|
|
|
|
* * *
|
|
|
|
|
|
From: Linus Torvalds <torvalds@linux-foundation.org>
|
|
Newsgroups: fa.linux.kernel
|
|
Subject: Re: 2.6.21-rc1: known regressions (part 2)
|
|
Date: Fri, 02 Mar 2007 00:31:00 UTC
|
|
Message-ID: <[fa.NTa0foc6yshUr2H9+kfHCLAmL60@ifi.uio.no][11]>
|
|
|
|
On Thu, 1 Mar 2007, Ingo Molnar wrote:
|
|
>
|
|
> git-bisect gets royally confused on those ACPI merge branches around
|
|
> commit c0cd79d11412969b6b8fa1624cdc1277db82e2fe. Here are my test
|
|
> results so far:
|
|
|
|
Looks like git bisect worked for you, and wasn't confused at all. You
|
|
started out with 2931 commits between your first known-bad and known-good
|
|
commits, which means that you usually end up having to check "log2(n)+1"
|
|
kernels, ie I'd have expected you to have to do 12-13 bisection attempts
|
|
to cut it down to one.
|
|
|
|
You seem to have done 14 (you list 16 commits, two of which are the
|
|
starting points), which is right in that range. The reason you sometimes
|
|
get more is:
|
|
|
|
- you "help" git bisect by choosing other commits than the optimal ones.
|
|
|
|
- with bad luck, it can be hard to get really close to "half the commits"
|
|
in the reachability analysis, especially if you have lots of merges
|
|
(and *especially* if you have octopus merges that merge more than two
|
|
branches of development). For example, say that you have something like
|
|
|
|
a
|
|
|
|
|
+---+---+---+---+
|
|
| | | | |
|
|
b c d e f
|
|
|
|
where you have six commits - you can't test any "combinations" at all,
|
|
since they are all independent, so "git bisect" cannot test them three
|
|
and three to cut down the time, so if you don't know which one is bad,
|
|
you'll basically end up testing them all.
|
|
|
|
The bad luck case never really happens to that extreme in practice, and
|
|
even when it does you can sometimes be lucky and just hit on the bug early
|
|
(so "bad luck" may end up being "good luck" after all), but it explains
|
|
why you can get more - or less - than log2(n)+1 attempts. More commonly
|
|
one more.
|
|
|
|
A much *bigger* problem is if you mark something good or bad that isn't
|
|
really. Ie if the bug comes and goes (it might be timing-dependent, for
|
|
example), the problem will be that you'll always narrow things down
|
|
(that's what bisection does), but you may not narrow it down to the right
|
|
thing!
|
|
|
|
We've had that happen several times. If the bug (for example) means that
|
|
suspend *often* breaks, but sometimes works just by luck, you might mark a
|
|
kernel "good" when it really wasn't and then "git bisect" will *really* go
|
|
out in the weeds, and won't even try to test the commits that may have
|
|
introduced the bug, because you told it that those commits resulted in a
|
|
good kernel..
|
|
|
|
> commit 01363220f5d23ef68276db8974e46a502e43d01d: bad
|
|
> commit 255f0385c8e0d6b9005c0e09fffb5bd852f3b506: bad
|
|
> commit c0cd79d11412969b6b8fa1624cdc1277db82e2fe: bad
|
|
> commit c24e912b61b1ab2301c59777134194066b06465c: good
|
|
> commit e9e2cdb412412326c4827fc78ba27f410d837e6e: bad
|
|
> commit 79bf2bb335b85db25d27421c798595a2fa2a0e82: bad
|
|
> commit fc955f670c0a66aca965605dae797e747b2bef7d: good
|
|
> commit 70c0846e430881967776582e13aefb81407919f1: good
|
|
> commit 414f827c46973ba39320cfb43feb55a0eeb9b4e8: bad
|
|
> commit f3ccb06f3b8e0cf42b579db21f3ca7f17fcc3f38: good
|
|
> commit 5f0b1437e0708772b6fecae5900c01c3b5f9b512: bad
|
|
> commit b878ca5d37953ad1c4578b225a13a3c3e7e743b7: bad
|
|
> commit c2902c8ae06762d941fab64198467f78cab6f8cd: bad
|
|
> commit 12e74f7d430655f541b85018ea62bcd669094bd7: bad
|
|
> commit 3388c37e04ec0e35ebc1b4c732fdefc9ea938f3b: bad
|
|
> commit 9f4bd5dde81b5cb94e4f52f2f05825aa0422f1ff: bad
|
|
|
|
Looks like it's claiming that 9f4bd5dde81b5cb94e4f52f2f05825aa0422f1ff is
|
|
the bad commit. Which is extremely unlikely, since it only seems to affect
|
|
the emu10k sound driver, which I don't think even exists on any ThinkPad
|
|
laptops (correct me if I'm wrong).
|
|
|
|
Btw, you seem to have re-ordered the commits - the above is not the order
|
|
you did the bisection in. The known-good commit (f3ccb06..) is in the
|
|
middle. That's totally bogus. Please use the git bisection log (see
|
|
git/BISECT_LOG), and don't think that you know some "better" order. You
|
|
really don't.
|
|
|
|
> the results are totally reproducible (i re-tried a few of both the good
|
|
> and the bad commits), i.e. it's not a sporadic condition. Also, a number
|
|
> of the 'bad' commits have no dynticks stuff in them at all, so i'd
|
|
> exclude dynticks.
|
|
>
|
|
> could someone suggest a sane way to go with this? Perhaps suggest
|
|
> specific commit IDs to test?
|
|
|
|
You claim that 9f4bd5dd is bad, but you indirectly claim that its direct
|
|
parent (5986a2ec) is good by saying that f3ccb06f is good. This is why
|
|
"git bisect" will claim that 9f4bd5dd must be the bad commit.
|
|
|
|
I would suggest testing commit 5986a2ec explicitly. If that one is good,
|
|
then, since you claim that 9f4bd5dd is bad, then yes, 9f4bd5dd *is* the
|
|
bad commit (because 5986a2ec is its direct parent).
|
|
|
|
But most likely, 9f4bd5dd is actually already bad, and what you are seeing
|
|
is two *different* bugs that just have the same symptoms ("suspend doesn't
|
|
work").
|
|
|
|
What happens is that you've chased them *both*, and you cannot bisect that
|
|
kind of behaviour totally automatically and mindlessly, simply because
|
|
when you say "git bisect bad", that means that *one* of the bugs is
|
|
active, but not necessarily both of them. So you may well be marking
|
|
kernels that are "good" (as far as the other bug is concerned) as bad -
|
|
and that just means that bisection won't even test them.
|
|
|
|
When that happens, you need to basically
|
|
|
|
- be able to separate the bugs out some way (so that you can still mark a
|
|
non-working kernel "good" if it's good *with*respect*to* the particular
|
|
bug you're chasing)
|
|
|
|
This is often practically impossible, _especially_ with suspend, where
|
|
the behaviour is so unhelpful that it's usually not possible to
|
|
separate out "ACPI is broken" from "one particular device driver is
|
|
broken", because they both have exactly the same symptoms: the machine
|
|
doesn't resume.
|
|
|
|
HOWEVER. Even if you can't actually separate the bugs out, you can usually
|
|
find where *one* of the bugs starts, and that point you can generally find
|
|
the fix for it too. In this case, we already know one of the bugs: it's
|
|
the ACPI bug that was apparently fixed by f3ccb06f3 (or maybe another one
|
|
in that series).
|
|
|
|
Once you have that, you now actually have a way to "correct" for that
|
|
known bug, and by correcting for the known bug, you now *can* separate the
|
|
behaviour of the two bugs:
|
|
|
|
- You can now re-do a totally mindless git bisection for the *other* bug,
|
|
but what you now need to do is that at each bisection step, you look at
|
|
whether the bisection point has the known bug, and if so, you apply the
|
|
known fix for that known bug, and thus you can test the kernel
|
|
*without* the interaction of the bug you already found.
|
|
|
|
This makes bisection a lot less automated (you have to apply the "fix" for
|
|
the other bug at each step), but it still allows "total automation" in the
|
|
sense that you don't actually need to know at all what you're looking for:
|
|
you're just following a known pattern, and you're basically just
|
|
correcting for the effects of another bug that you're no longer interested
|
|
in, since you already know what the fix for that bug was.
|
|
|
|
The other alternative is to actually have a clue what you're searching
|
|
for, and/or look deeply at where the fix was merged, and trying to narrow
|
|
things down by actually understanding the problem. But at that point,
|
|
bisection won't much help you, except perhaps as a way to find a mid-way
|
|
point to test out theories with ("which drivers that I actually use have
|
|
changed in between" kinds of experiments where you simply undo part of
|
|
the changes entirely, and bisection ends up being just a way to pick
|
|
points that are hopefully "interestingly far apart").
|
|
|
|
Linus
|
|
|
|
|
|
|
|
* * *
|
|
|
|
|
|
From: Linus Torvalds <torvalds@linux-foundation.org>
|
|
Newsgroups: fa.linux.kernel
|
|
Subject: Re: 2.6.21-rc1: known regressions (part 2)
|
|
Date: Fri, 02 Mar 2007 00:45:25 UTC
|
|
Message-ID: <[fa.8mGUFyTdx8CqCcb1RxAuXU3++W0@ifi.uio.no][12]>
|
|
|
|
On Thu, 1 Mar 2007, Linus Torvalds wrote:
|
|
>
|
|
> Once you have that, you now actually have a way to "correct" for that
|
|
> known bug, and by correcting for the known bug, you now *can* separate the
|
|
> behaviour of the two bugs:
|
|
>
|
|
> - You can now re-do a totally mindless git bisection for the *other* bug,
|
|
> but what you now need to do is that at each bisection step, you look at
|
|
> whether the bisection point has the known bug, and if so, you apply the
|
|
> known fix for that known bug, and thus you can test the kernel
|
|
> *without* the interaction of the bug you already found.
|
|
>
|
|
> This makes bisection a lot less automated (you have to apply the "fix" for
|
|
> the other bug at each step)
|
|
|
|
Side note: it's still usually fairly easy. Especially if you have a known
|
|
fix for the other bug, you can usually just do the equivalent of
|
|
|
|
git cherry-pick <fixcommit>
|
|
|
|
at each point during this bisect (or just have a known patch that you keep
|
|
applying and un-applying), and you're largely done.
|
|
|
|
Of course, if the area with the fix keeps changing, or if the fix is
|
|
really intrusive and nasty, this gets hairy, but at least in this case the
|
|
patch is fairly trivial and it shouldn't cause any trouble at all to do
|
|
this.
|
|
|
|
The only real down-side is just the mindless extra work, and the possible
|
|
added confusion you get from modifying the history at the points you're
|
|
testing. "git bisect" is not necessarily happy about auto-picking a new
|
|
bisection point with a dirty tree, for example, so before you mark
|
|
something "good" or "bad", you should generally try to do so with a clean
|
|
git tree (ie if you apply a patch at each stage, do "git reset --hard" to
|
|
remove the patch before you do the "git bisect bad/good" stage).
|
|
|
|
Similarly, especially at the end of the bisection run, if you actually use
|
|
"git cherry-pick" to *add* a commit, the bisection will now take that
|
|
added commit into account when trying to pick the next commit to check,
|
|
which is not what you really want. It probably doesn't matter that much
|
|
during the early stages (when bisection is really jumping around wildly
|
|
anyway, and one commit more or less doesn't really matter), but again, it
|
|
might be a good idea to make a habit of undoing the cherry-pick, the same
|
|
way you'd undo a patch (eg "git reset --hard HEAD^" would do it, if you
|
|
have exactly one cherry-pick that you tested).
|
|
|
|
Linus
|
|
|
|
|
|
|
|
* * *
|
|
|
|
|
|
From: Linus Torvalds <torvalds@linux-foundation.org>
|
|
Newsgroups: fa.linux.kernel
|
|
Subject: Re: Regression: USB is nfg after suspend/resume(RAM) cycle on Intel
|
|
Date: Tue, 29 May 2007 17:42:48 UTC
|
|
Message-ID: <[fa.zWzTluJWEfNpa88JpCctkkanoBA@ifi.uio.no][13]>
|
|
|
|
On Tue, 29 May 2007, Mark Lord wrote:
|
|
>
|
|
> Ugh. Is there a way to tell bisect to only work around the USB updates?
|
|
|
|
Well, you _can_ actually give "git bisect" a pathspec (the same way you
|
|
git "git log" and friends), and tell it to only care about stuff that
|
|
changed that pathspec.
|
|
|
|
However, that was broken in some older git versions, and in general hasn't
|
|
had a huge amount of testing even in new ones, so I'm a tad nervous about
|
|
recommending people do it. But yes, you should be able to say
|
|
|
|
git bisect start drivers/usb/
|
|
git bisect good v2.6.21
|
|
git bisect bad v2.6.22-rc3
|
|
|
|
and off you go. However, if you do this, you need to make sure that you
|
|
have at LEAST git-1.5.1.
|
|
|
|
The other downside of path limiting is that if it turns out that the bug
|
|
was really introduced by something else, and just happened to _look_ like
|
|
it's USB-related, the path limiting will then cause "git bisect" to blame
|
|
a commit that just happens to be the next commit after the _real_ bug was
|
|
introduced.
|
|
|
|
In this case, I don't think it's likely to be an issue, but it *could*
|
|
obviously be something else.
|
|
|
|
(In contrast, the non-path-limited "git bisect" should work in pretty much
|
|
any situation and with any git version)
|
|
|
|
> Still, that'll take a few hours, and frankly I'm getting sick of having
|
|
> to re-debug the USB layer with each new kernel rev.
|
|
|
|
Yeah, I'm not surprised. USB is probably the worst possible case for
|
|
suspend/resume (at least if you ignore ACPI-related problems). It's nasty.
|
|
|
|
> Got a pointer to the "bisect how-to" ?
|
|
|
|
It's so disgustingly simple that I don't think we've ever done any
|
|
specific bisection tutorial, but the "git-bisect" man-page does exist, and
|
|
it talks about the only half-way interesting case, namely the case where
|
|
the automatic selection of a half-way point causes git to pick a point
|
|
that doesn't work for some other reason (ie stupid compile problem or
|
|
whatever). In which case you have to pick another one manually.
|
|
|
|
So that kind of gotcha is at least _mentioned_ in the git-bisect man-page,
|
|
even if it doesn't get much further than that.
|
|
|
|
There's also the git users manual, but I think the man-page is more
|
|
detailed. But for future reference, just do
|
|
|
|
git user manual
|
|
|
|
in google, and press "I'm feeling lucky". It finds the right thing at
|
|
least for me (and at least right now).
|
|
|
|
Linus
|
|
|
|
|
|
|
|
* * *
|
|
|
|
|
|
From: Linus Torvalds <torvalds@linux-foundation.org>
|
|
Newsgroups: fa.linux.kernel
|
|
Subject: Re: Linux 2.6.22 released
|
|
Date: Tue, 10 Jul 2007 15:39:56 UTC
|
|
Message-ID: <[fa.DEwDXyqrl1AOg838sXIQpMmK16U@ifi.uio.no][14]>
|
|
|
|
On Tue, 10 Jul 2007, Stefano Rivoir wrote:
|
|
>
|
|
> 2.6.22 hangs at boot on my box. Here attached a original dmesg from 2.6.21,
|
|
> and a copy of it where it stops on 2.6.22 (I can't attach the original 2.6.22
|
|
> dmesg because it's not logged to disk yet); it actually stops right after
|
|
> 'init' launches.
|
|
|
|
Ok, without any oops or hang in any really obvious place, that doesn't
|
|
really tell anybody anything specific enough to even start trying to debug
|
|
this, so you'd need to do one of two things:
|
|
|
|
- poke a lot at the machine to try to get more specific information. In
|
|
particular, get things like SysRQ-T output. That, in turn, probably
|
|
would mean trying to get a serial console hooked up or something.
|
|
|
|
The next thing that you got on 2.6.21 after the point it hangs was
|
|
apparently
|
|
|
|
..
|
|
input: PC Speaker as /class/input/input1
|
|
ieee1394: Initialized config rom entry `ip1394'
|
|
ACPI: PCI Interrupt Link [APC4] enabled at IRQ 19
|
|
ACPI: PCI Interrupt 0000:04:05.0[A] -> Link [APC4] -> GSI 19 (level, low) -> IRQ 19
|
|
usbcore: registered new interface driver hiddev
|
|
..
|
|
|
|
so you could _try_ to disable the PC speaker or firewire, and see
|
|
what's up. Did you switch from the old firewire drivers to the new one,
|
|
for example? Or if you didn't, try it.
|
|
|
|
IOW, we'd need a lot more debug information.
|
|
|
|
The second alternative will take some time, but is really a lot easier:
|
|
|
|
- Get a kernel git tree, and do a "git bisect".
|
|
|
|
There's almost 7000 commits in between 2.6.21 and 22, but that still
|
|
means that in about fourteen recompiles/reboots, "git bisect" should
|
|
tell us where your problem starts, which will hopefully make it obvious
|
|
what the problem is (or at least pinpoint it a *lot*).
|
|
|
|
Doing a git bisect isn't really that hard, but fourteen compiles/reboots
|
|
will take some time (well, the compiles will, the reboots aren't that
|
|
bad). But even if you're not a git user, it really is very simple:
|
|
|
|
- get started with 'git': on most distros it's now as simple as doing
|
|
something like
|
|
|
|
yum install git
|
|
|
|
and while you might not get the latest version (Debian stable is at
|
|
some ancient 1.4.4.4 version that isn't as nice as the 1.5.x series),
|
|
for something like this you won't care that deeply.
|
|
|
|
- get the kernel git tree (this will take a while to download about
|
|
180MB)
|
|
|
|
git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6
|
|
|
|
- start the "git bisect" with
|
|
|
|
git bisect good v2.6.21
|
|
git bisect bad v2.6.22
|
|
|
|
and it will pick a kernel version about half-way between the two
|
|
points, and you can now start testing. For each kernel you try, if it
|
|
boots fine, do "git bisect good", otherwise boot into a working kernel,
|
|
and then do "git bisect bad". Git will then pick the next "halfway"
|
|
kernel for that case.
|
|
|
|
Thanks,
|
|
|
|
Linus
|
|
|
|
|
|
|
|
* * *
|
|
|
|
|
|
From: Linus Torvalds <torvalds@linux-foundation.org>
|
|
Newsgroups: fa.linux.kernel
|
|
Subject: Re: GIT bisection range errors
|
|
Date: Thu, 08 May 2008 23:00:39 UTC
|
|
Message-ID: <[fa.lMeHf8njqvCZ0Y4BjVIQtc3VesI@ifi.uio.no][15]>
|
|
|
|
On Fri, 9 May 2008, Rene Herman wrote:
|
|
>
|
|
> I'm in a git bisect and am experiencing strangeness. I did a
|
|
>
|
|
> $ git checkout -b rc v2.6.26-rc1
|
|
> $ git bisect start
|
|
> $ git bisect bad
|
|
> $ git bisect good v2.6.25
|
|
>
|
|
> Yet, during this I'm finding myself at 2.6.25-rc6 and 2.6.25-rc8
|
|
> as the last two results (both good...).
|
|
|
|
This is very normal.
|
|
|
|
Why?
|
|
|
|
Because a lot (in fact, *most*) of the code that was merged after v2.6.25
|
|
was released was actually *written* and committed long before v2.6.25.
|
|
|
|
It just got merged into my tree much later.
|
|
|
|
So what happens? The bisection run starts walking into all that history,
|
|
and that history is *not* based on the released v2.6.25 at all, it's based
|
|
on much earlier kernels (eg the -rc kernels).
|
|
|
|
So what you see is perfectly normal and expected. It's only unexpected if
|
|
you think of history as a linear thing, but it isn't - it's full of
|
|
merging of code that was branched off from (much) earlier code points.
|
|
|
|
Linus
|
|
|
|
|
|
|
|
* * *
|
|
|
|
[Index][1] [Home][2] [About][3] [Blog][4]
|
|
|
|
[1]: http://yarchive.net/index.html
|
|
[2]: http://yarchive.net/home.html
|
|
[3]: http://yarchive.net/about.html
|
|
[4]: http://yarchive.net/blog
|
|
[5]: http://groups.google.com/groups/search?as_umsgid=fa.fv9n3b6.j067qs%40ifi.uio.no
|
|
[6]: http://mid.gmane.org/Pine.LNX.4.64.0601101111110.4939%40g5.osdl.org
|
|
[7]: http://groups.google.com/groups/search?as_umsgid=fa.g0ad5b8.m0o5qk%40ifi.uio.no
|
|
[8]: http://mid.gmane.org/Pine.LNX.4.58.0508120937140.3295%40g5.osdl.org
|
|
[9]: http://groups.google.com/groups/search?as_umsgid=fa.%2foMGTFml4fnJMGJZgMEDjT9Ip0Q%40ifi.uio.no
|
|
[10]: http://groups.google.com/groups/search?as_umsgid=fa.63Ct4ClRD8hjd7Z3taISujCscb8%40ifi.uio.no
|
|
[11]: http://groups.google.com/groups/search?as_umsgid=fa.NTa0foc6yshUr2H9+kfHCLAmL60%40ifi.uio.no
|
|
[12]: http://groups.google.com/groups/search?as_umsgid=fa.8mGUFyTdx8CqCcb1RxAuXU3++W0%40ifi.uio.no
|
|
[13]: http://groups.google.com/groups/search?as_umsgid=fa.zWzTluJWEfNpa88JpCctkkanoBA%40ifi.uio.no
|
|
[14]: http://groups.google.com/groups/search?as_umsgid=fa.DEwDXyqrl1AOg838sXIQpMmK16U%40ifi.uio.no
|
|
[15]: http://groups.google.com/groups/search?as_umsgid=fa.lMeHf8njqvCZ0Y4BjVIQtc3VesI%40ifi.uio.no
|
|
|