From mingo@elte.hu Wed Mar 11 12:39:29 2009
Return-Path: mingo@elte.hu
Received: from zmta01.collab.prod.int.phx2.redhat.com (LHLO
 zmta01.collab.prod.int.phx2.redhat.com) (10.5.5.31) by
 mail03.corp.redhat.com with LMTP; Wed, 11 Mar 2009 12:39:29 -0400 (EDT)
Received: from localhost (localhost.localdomain [127.0.0.1])
	by zmta01.collab.prod.int.phx2.redhat.com (Postfix) with ESMTP id 5485291974;
	Wed, 11 Mar 2009 12:39:28 -0400 (EDT)
Received: from zmta01.collab.prod.int.phx2.redhat.com ([127.0.0.1])
	by localhost (zmta01.collab.prod.int.phx2.redhat.com [127.0.0.1]) (amavisd-new, port 10024)
	with ESMTP id bnJ+lnWnSnaL; Wed, 11 Mar 2009 12:39:28 -0400 (EDT)
Received: from int-mx1.corp.redhat.com (int-mx1.corp.redhat.com [172.16.52.254])
	by zmta01.collab.prod.int.phx2.redhat.com (Postfix) with ESMTP id 1F6BC91976;
	Wed, 11 Mar 2009 12:39:28 -0400 (EDT)
Received: from mx3.redhat.com (mx3.redhat.com [172.16.48.32])
	by int-mx1.corp.redhat.com (8.13.1/8.13.1) with ESMTP id n2BGdUHR005355;
	Wed, 11 Mar 2009 12:39:30 -0400
Received: from mx2.mail.elte.hu (mx2.mail.elte.hu [157.181.151.9])
	by mx3.redhat.com (8.13.8/8.13.8) with ESMTP id n2BGdAPa019954;
	Wed, 11 Mar 2009 12:39:10 -0400
Received: from elvis.elte.hu ([157.181.1.14])
	by mx2.mail.elte.hu with esmtp (Exim)
	id 1LhRRg-0000Wa-CQ
	from <mingo@elte.hu>; Wed, 11 Mar 2009 17:38:23 +0100
Received: by elvis.elte.hu (Postfix, from userid 1004)
	id 6B5883E2136; Wed, 11 Mar 2009 17:38:09 +0100 (CET)
Date: Wed, 11 Mar 2009 17:38:12 +0100
From: Ingo Molnar <mingo@elte.hu>
To: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Kyle McMartin <kyle@redhat.com>,
        "Pallipadi, Venkatesh" <venkatesh.pallipadi@intel.com>,
        Suresh Siddha <suresh.b.siddha@intel.com>,
        Arjan van de Ven <arjan@infradead.org>, Dave Jones <davej@redhat.com>,
        cebbert@redhat.com
Subject: Re: Fedora-11 x86-32 kernel failures?
Message-ID: <20090311163812.GA25947@elte.hu>
References: <20090310200543.GB31017@kibblesnbits.boston.devel.redhat.com> <20090310202107.GA10949@elte.hu> <20090310203141.GE31017@kibblesnbits.boston.devel.redhat.com> <20090310205858.GA4204@elte.hu> <20090310211452.GG31017@kibblesnbits.boston.devel.redhat.com> <20090310213103.GC14867@elte.hu> <20090310214510.GI31017@kibblesnbits.boston.devel.redhat.com> <20090311034251.GK31017@kibblesnbits.boston.devel.redhat.com> <20090311104717.GA2282@elte.hu> <alpine.LFD.2.00.0903110913550.32478@localhost.localdomain>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <alpine.LFD.2.00.0903110913550.32478@localhost.localdomain>
User-Agent: Mutt/1.5.18 (2008-05-17)
Received-SPF: neutral (mx2: 157.181.1.14 is neither permitted nor denied by domain of elte.hu) client-ip=157.181.1.14; envelope-from=mingo@elte.hu; helo=elvis.elte.hu;
X-ELTE-VirusStatus: clean
X-ELTE-SpamScore: -1.5
X-ELTE-SpamLevel: 
X-ELTE-SpamCheck: no
X-ELTE-SpamVersion: ELTE 2.0 
X-ELTE-SpamCheck-Details: score=-1.5 required=5.9 tests=BAYES_00 autolearn=no SpamAssassin version=3.2.3
	-1.5 BAYES_00               BODY: Bayesian spam probability is 0 to 1%
	[score: 0.0000]
X-RedHat-Spam-Score: -1.429 
X-Scanned-By: MIMEDefang 2.58 on 172.16.52.254
X-Scanned-By: MIMEDefang 2.63 on 172.16.48.32


* Linus Torvalds <torvalds@linux-foundation.org> wrote:

> > @@ -521,6 +521,7 @@ static int split_large_page(pte_t *kpte,
> >  	 * pagetable protections, the actual ptes set above control the
> >  	 * primary protection behavior:
> >  	 */
> > +	__flush_tlb_all();
> >  	__set_pmd_pte(kpte, address, mk_pte(base, __pgprot(_KERNPG_TABLE)));
> >  	base = NULL;
> 
> I think this one is wrong. Your previous one was better.
> 
> I suspect the __flush_tlb_all() should be _after_ the 
> __set_pmd_pte() (your previous patch had it both before and 
> after).
> 
> And see my previous email - I think it's an Atom errata. And 
> putting the __flush_tlb_all() after the __set_pmd_pte will 
> likely hide it, but until we know exactly what triggers it, I 
> don't know how to _really_ fix it if both CPU's are actually 
> active.

hm, i see:

AAH41        Code Fetch May Occur to Incorrect Address After a Large Page is Split
             Into 4-KByte Pages
Problem:     If software clears the PS (page size) bit in a present PDE (page directory entry), that
             will cause linear addresses mapped through this PDE to use 4-KByte pages instead of
             using a large page after old TLB entries are invalidated. Due to this erratum, if a code
             fetch uses this PDE before the TLB entry for the large page is invalidated then it may
             fetch from a different physical address than specified by either the old large page
             translation or the new 4-KByte page translation. This erratum may also cause
             speculative code fetches from incorrect addresses.
Implication: The processor may fetch code from an incorrect address after a large page is
             converted into 4-Kbyte pages.
Workaround:       None
Status       For the steppings affected, see the Summary Tables of Changes.

How about the one below? I think it's better to have this within 
__set_pmd_pte().

	Ingo

-------------->
Subject: x86: work around Fedora-11 x86-32 kernel failures on Intel Atom CPUs
From: Ingo Molnar <mingo@elte.hu>
Date: Tue, 10 Mar 2009 22:31:03 +0100

Work around Intel Atom erratum AAH41 (probabilistically).

Reported-by: Linus Torvalds <torvalds@linux-foundation.org>
Tested-by: Kyle McMartin <kyle@redhat.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
---
 arch/x86/mm/pageattr.c |    9 +++++++++
 1 file changed, 9 insertions(+)

Index: tip/arch/x86/mm/pageattr.c
===================================================================
--- tip.orig/arch/x86/mm/pageattr.c
+++ tip/arch/x86/mm/pageattr.c
@@ -339,6 +339,15 @@ static void __set_pmd_pte(pte_t *kpte, u
 		}
 	}
 #endif
+	/*
+	 * Intel Atom errata AAH41 workaround.
+	 *
+	 * The real fix is in a microcode update, but we
+	 * also probabilistically try to reduce the window
+	 * of having a large TLB mixed with 4K TLBs while
+	 * instruction fetches are going on.
+	 */
+	__flush_tlb_all();
 }
 
 static int

