This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


Re: [Xen-devel] Cpupools and pdata_alloc

To: George Dunlap <George.Dunlap@xxxxxxxxxxxxx>
Subject: Re: [Xen-devel] Cpupools and pdata_alloc
From: Juergen Gross <juergen.gross@xxxxxxxxxxxxxx>
Date: Wed, 12 May 2010 06:52:49 +0200
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx, Keir Fraser <keir.fraser@xxxxxxxxxxxxx>
Delivery-date: Tue, 11 May 2010 21:53:47 -0700
Dkim-signature: v=1; a=rsa-sha256; c=simple/simple; d=ts.fujitsu.com; i=juergen.gross@xxxxxxxxxxxxxx; q=dns/txt; s=s1536b; t=1273639585; x=1305175585; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; z=Message-ID:=20<4BEA3421.6040300@xxxxxxxxxxxxxx>|Date:=20 Wed,=2012=20May=202010=2006:52:49=20+0200|From:=20Juergen =20Gross=20<juergen.gross@xxxxxxxxxxxxxx>|MIME-Version: =201.0|To:=20George=20Dunlap=20<George.Dunlap@xxxxxxxxxxx om>|CC:=20xen-devel@xxxxxxxxxxxxxxxxxxx,=20=0D=0A=20Keir =20Fraser=20<keir.fraser@xxxxxxxxxxxxx>|Subject:=20Re:=20 [Xen-devel]=20Cpupools=20and=20pdata_alloc|References:=20 <AANLkTin3dC9sCDP1-zwKog6BZtwaHAmkt53NqwAZtiNw@xxxxxxxxxx .com>=09<4BE8E1B1.2030305@xxxxxxxxxxxxxx>=20<AANLkTilmTmE TcKO4aBLseQULUOUwTqP9QXOI3YPgq3YE@xxxxxxxxxxxxxx> |In-Reply-To:=20<AANLkTilmTmETcKO4aBLseQULUOUwTqP9QXOI3YP gq3YE@xxxxxxxxxxxxxx>|Content-Transfer-Encoding:=207bit; bh=n6g5uOYZZz6OLOl4guYpQHr14i1FC2iLnXujU+hRmy8=; b=l705hi2A/FCSB6ycNK+xxiUmJRyW+rdTvS8kYlzznknSaD5QXa+XjghG cyG2j0a6aWdHqA+JWxTB2gZgD71CEHlz+15Qt4ZtyE2RnwyB4goHXyP28 hZE5ZNLQqD3BLDS+q8ZLgKFNhNBs2cbvAioO0AlaXgi64sjgjLQKg8H8a eTLjk7B58JxIPN3pdVNeXVzNnNZWexCKq9Ofwj2qS5SWCb/I7noR+tDW/ k60Ng8Wz7uYdukfS4PlC0RP1IXNIX;
Domainkey-signature: s=s1536a; d=ts.fujitsu.com; c=nofws; q=dns; h=X-SBRSScore:X-IronPort-AV:Received:X-IronPort-AV: Received:Received:Message-ID:Date:From:Organization: User-Agent:MIME-Version:To:CC:Subject:References: In-Reply-To:Content-Type:Content-Transfer-Encoding; b=kOzXaYudv80ivZaFNdjp5K44esEQKfSPJCoDLSge6oBf7MP3nRtDczKE ssVvxr8Nv32iDczEPyVCGGHFyTH8CgJsLhCWAkeOvgJwjNK6NNUgdBz7A tCC5h2oCckEuJvYC1mm9M6VOmBQpGhhHmhmx7I5DAjgw09kmACZ5BPKkD cCE1Lmxi0EbX2RC3A5S111C9l5OnKWnp3V/wDmd+sm5LFOptbiDUwEAyA +2Zfx+4AMzy1jkGLmBLe5RC7W17dj;
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <AANLkTilmTmETcKO4aBLseQULUOUwTqP9QXOI3YPgq3YE@xxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Organization: Fujitsu Technology Solutions
References: <AANLkTin3dC9sCDP1-zwKog6BZtwaHAmkt53NqwAZtiNw@xxxxxxxxxxxxxx> <4BE8E1B1.2030305@xxxxxxxxxxxxxx> <AANLkTilmTmETcKO4aBLseQULUOUwTqP9QXOI3YPgq3YE@xxxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv: Gecko/20100411 Iceowl/1.0b1 Icedove/3.0.4
On 05/11/2010 07:25 PM, George Dunlap wrote:
On Mon, May 10, 2010 at 11:48 PM, Juergen Gross
<juergen.gross@xxxxxxxxxxxxxx>  wrote:
No. It happens when idle vcpus are allocated. At this time there is no
existing, all physical cpus are marked as "free", e.g. they are in no pool
Dom0 vcpus are allocated in Pool-0. This pool is created after allocation of
the idle vcpus.

Yeah, I spent some time tracing through the init code yesterday and
figured that out.  So it appears that at init time, regarding cpupools
and schedulers:
* init_idle_domain() calls schedule_init(), which calls ops->init()
with a sort of "default" ops pointer.  It also calls sched_init_vcpu()
for idle domain's vcpu 0, which will call ops->alloc_pdata for cpu 0.
* smp_prepare_cpus will eventually call do_boot_cpu for each online
cpu.  do_boot_cpu will initialize the idle_domain vcpu for that cpu,
which will call ops->alloc_pdata for that cpu (again, with cpu 0
  - At this point, all online cpus have had alloc_pdata called, albeit
for the "default" ops structure in the scheduler
* cpupool_create will create cpupool 0, calling sched_init(), which
calls ops->init with the cpupool0 ops structure.
* cpupool0_cpu_assign will then un-assign all online cpus from the
"default" ops structure and re-assign them into cpupool 0.
Re-assigning looks like this:
  - First call alloc_pdata and then alloc_vdata for the new cpupool ops
structure, for the physical cpu and idle vcpu respectively.
  - Ticks will be disabled on the old ops structure, then resumed on
the new ops structure
  - The idle vcpu is added to the new pool
  - Calls free_vdata and free_pdata on the old cpupool ops structure
for the idle vcpu and physical cpu, respectively.

Now all online cpus have idle vcpus and pdatas initialized, and set up
for cpupool 0.

Is that a pretty accurate picture?

Yes, this sounds correct.


Juergen Gross                 Principal Developer Operating Systems
TSP ES&S SWE OS6                       Telephone: +49 (0) 89 3222 2967
Fujitsu Technology Solutions              e-mail: juergen.gross@xxxxxxxxxxxxxx
Domagkstr. 28                           Internet: ts.fujitsu.com
D-80807 Muenchen                 Company details: ts.fujitsu.com/imprint.html

Xen-devel mailing list

<Prev in Thread] Current Thread [Next in Thread>