WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

[Xen-devel] RE: [PATCH] Fix cache flush bug of cpu offline

To: Keir Fraser <keir@xxxxxxx>, "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>
Subject: [Xen-devel] RE: [PATCH] Fix cache flush bug of cpu offline
From: "Liu, Jinsong" <jinsong.liu@xxxxxxxxx>
Date: Sat, 12 Mar 2011 00:50:25 +0800
Accept-language: en-US
Acceptlanguage: en-US
Cc: "Jiang, Yunhong" <yunhong.jiang@xxxxxxxxx>, "Shan, Haitao" <haitao.shan@xxxxxxxxx>, "Wei, Gang" <gang.wei@xxxxxxxxx>, "Yu, Ke" <ke.yu@xxxxxxxxx>, "Li, Xin" <xin.li@xxxxxxxxx>
Delivery-date: Fri, 11 Mar 2011 08:51:16 -0800
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <C99FFD99.2B708%keir@xxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <BC00F5384FCFC9499AF06F92E8B78A9E1FCCF29C97@xxxxxxxxxxxxxxxxxxxxxxxxxxxx> <C99FFD99.2B708%keir@xxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Thread-index: Acvf+Wq8eUV5g7VvTPu6ARIzvUA0UgAD5esZAAA/DUA=
Thread-topic: [PATCH] Fix cache flush bug of cpu offline
Keir Fraser wrote:
> On 11/03/2011 14:34, "Liu, Jinsong" <jinsong.liu@xxxxxxxxx> wrote:
> 
>> Fix cache flush bug of cpu offline
>> 
>> Current xen cpu offline logic flush cache too early, which
>> potentially break cache coherency. wbinvd should be the last ops
>> before cpu going into dead, otherwise cache may be dirty, i.e,
>> something like setting an A bit on page tables. Pointed out by Arjan
>> van de Ven. 
> 
> The position still seems a bit arbitrary. In the first hunk below,
> why is it safe to wbinvd() outside the for-loop and before reading
> cx->entry_method, but not before reading from processor_powers[]? It
> would be neater if we could put the wbinvd() in a wrapper function
> for calling *dead_idle. 
> 
>  -- Keir

we did experiment, if did wbinvd at current position (at play_dead), sometimes 
it bring strange issue when repeatly cpu offline/online.
so for cpu dead, the near wbinvd to last step, the safer.
wbinvd would better be the last ops before cpu dead, to avoid potential cache 
coherency break.

In fact, it can do wbinvd inside loop, but as cpu_offline_3.patch said, 
at Xen 7400 when hyperthreading, the offlined thread may be spuriously waken up 
by its brother, and frequently waken inside the dead loop. 
In such case, considering heavy workload of wbinvd, we add a light-weight 
clflush instruction inside loop.

Thanks,
Jinsong


> 
>> Signed-off-by: Liu, Jinsong <jinsong.liu@xxxxxxxxx>
>> 
>> diff -r 2dc3c1cc1bba xen/arch/x86/acpi/cpu_idle.c
>> --- a/xen/arch/x86/acpi/cpu_idle.c Mon Mar 07 05:31:46 2022 +0800
>> +++ b/xen/arch/x86/acpi/cpu_idle.c Thu Mar 10 23:40:51 2022 +0800
>> @@ -562,11 +562,14 @@ static void acpi_dead_idle(void)
>>      if ( (cx = &power->states[power->count-1]) == NULL )         
>> goto default_halt; 
>> 
>> +    /*
>> +     * cache must be flashed as the last ops before cpu going into
>> dead, +     * otherwise, cpu may dead with dirty data breaking cache
>> coherency, +     * leading to strange errors.
>> +     */
>> +    wbinvd();
>>      for ( ; ; )
>>      {
>> -        if ( !power->flags.bm_check && cx->type == ACPI_STATE_C3 )
>> -            ACPI_FLUSH_CPU_CACHE();
>> -
>>          switch ( cx->entry_method )
>>          {
>>              case ACPI_CSTATE_EM_FFH:
>> @@ -584,6 +587,7 @@ static void acpi_dead_idle(void)      }
>> 
>>  default_halt:
>> +    wbinvd();
>>      for ( ; ; )
>>          halt();
>>  }
>> diff -r 2dc3c1cc1bba xen/arch/x86/domain.c
>> --- a/xen/arch/x86/domain.c Mon Mar 07 05:31:46 2022 +0800
>> +++ b/xen/arch/x86/domain.c Thu Mar 10 23:40:51 2022 +0800
>> @@ -93,6 +93,12 @@ static void default_idle(void)
>> 
>>  static void default_dead_idle(void)
>>  {
>> +    /*
>> +     * cache must be flashed as the last ops before cpu going into
>> dead, +     * otherwise, cpu may dead with dirty data breaking cache
>> coherency, +     * leading to strange errors.
>> +     */
>> +    wbinvd();
>>      for ( ; ; )
>>          halt();
>>  }
>> @@ -100,7 +106,6 @@ static void play_dead(void)
>>  static void play_dead(void)
>>  {
>>      local_irq_disable();
>> -    wbinvd();
>> 
>>      /*
>>       * NOTE: After cpu_exit_clear, per-cpu variables are no longer
>> accessible,


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel