Re: [Xen-ia64-devel] RE: [Xen-devel] Help? Red Hat fails, Suse/Debian both work fine

[Alex Williamson <alex.williamson@xxxxxx>]

   A little bit of data in case it sparks some debug ideas.  Enabling
tracing in xenstored shows that the difference between the working and
non-working case are exhibited pretty quickly.  There's definitely
communication occurring through the mmap'd kmem page, but it's missing
some important chunks.  For instance, here's the first few messages
logged in the working case:

CREATE connection 0x6000000000022678
IN  0x6000000000022678 07:15:03 DIRECTORY (device )
OUT 0x6000000000022678 07:15:03 ERROR (ENOENT )
IN  0x6000000000022678 07:15:03 DIRECTORY (backend )
OUT 0x6000000000022678 07:15:03 ERROR (ENOENT )
IN  0x6000000000022678 07:15:03 WATCH (device A000000100D20520 )
CREATE watch 0x60000000000229a8
OUT 0x6000000000022678 07:15:03 WATCH (OK )
IN  0x6000000000022678 07:15:03 WATCH (backend A000000100D20500 )
CREATE watch 0x60000000000228c8
OUT 0x6000000000022678 07:15:03 WATCH_EVENT (device A000000100D20520 )
OUT 0x6000000000022678 07:15:03 WATCH (OK )
OUT 0x6000000000022678 07:15:03 WATCH_EVENT (backend A000000100D20500 )
CREATE connection 0x6000000000022748
IN  0x6000000000022748 07:15:03 WATCH (@introduceDomain domlist )
CREATE watch 0x60000000000234c8
OUT 0x6000000000022748 07:15:03 WATCH (OK )

In the failing case, I see:

CREATE connection 0x6000000000026388
IN  0x6000000000026388 07:07:47 DIRECTORY (device )
OUT 0x6000000000026388 07:07:47 ERROR (ENOENT )
CREATE connection 0x60000000000265d8
IN  0x60000000000265d8 07:07:47 WATCH (@introduceDomain domlist )
CREATE watch 0x6000000000026d78
OUT 0x60000000000265d8 07:07:47 WATCH (OK )

Note that only the lines dealing with A000000100D20520 are missing.

The next absent chunk comes after about 300 lines of identical trace
between the working and failing case.  This block is missing when it
fails:

OUT 0x6000000000022678 07:15:14 WATCH_EVENT (backend/vbd/1/769 A000000100D20500 
)
IN  0x6000000000022678 07:15:14 DIRECTORY (backend/vbd/1/769 )
OUT 0x6000000000022678 07:15:14 WATCH_EVENT (backend/vbd/1/769/domain 
A000000100D20500 )
OUT 0x6000000000022678 07:15:14 WATCH_EVENT (backend/vbd/1/769/frontend 
A000000100D20500 )
OUT 0x6000000000022678 07:15:14 WATCH_EVENT (backend/vbd/1/769/dev 
A000000100D20500 )
OUT 0x6000000000022678 07:15:14 WATCH_EVENT (backend/vbd/1/769/state 
A000000100D20500 )
OUT 0x6000000000022678 07:15:14 WATCH_EVENT (backend/vbd/1/769/params 
A000000100D20500 )
OUT 0x6000000000022678 07:15:14 WATCH_EVENT (backend/vbd/1/769/mode 
A000000100D20500 )
OUT 0x6000000000022678 07:15:14 WATCH_EVENT (backend/vbd/1/769/frontend-id 
A000000100D20500 )
OUT 0x6000000000022678 07:15:14 WATCH_EVENT (backend/vbd/1/769/type 
A000000100D20500 )
OUT 0x6000000000022678 07:15:14 DIRECTORY (domain frontend dev state params 
mode frontend-id type )
IN  0x6000000000022678 07:15:14 READ (backend/vbd/1/769/state )
OUT 0x6000000000022678 07:15:14 READ (1)
IN  0x6000000000022678 07:15:14 READ (backend/vbd/1/769/frontend-id )
OUT 0x6000000000022678 07:15:14 READ (1)
IN  0x6000000000022678 07:15:14 READ (backend/vbd/1/769/frontend )
OUT 0x6000000000022678 07:15:14 READ (/local/domain/1/device/vbd/769)
IN  0x6000000000022678 07:15:14 DIRECTORY (/local/domain/1/device/vbd/769 )
OUT 0x6000000000022678 07:15:14 DIRECTORY (virtual-device backend-id state 
backend )
IN  0x6000000000022678 07:15:14 READ (backend/vbd/1/769/frontend-id )
OUT 0x6000000000022678 07:15:14 READ (1)
IN  0x6000000000022678 07:15:14 READ (backend/vbd/1/769/frontend )
OUT 0x6000000000022678 07:15:14 READ (/local/domain/1/device/vbd/769)
IN  0x6000000000022678 07:15:14 DIRECTORY (/local/domain/1/device/vbd/769 )
OUT 0x6000000000022678 07:15:14 DIRECTORY (virtual-device backend-id state 
backend )
IN  0x6000000000022678 07:15:14 WATCH (backend/vbd/1/769/physical-device 
E000000008E31C90 )
CREATE watch 0x600000000000cad8
OUT 0x6000000000022678 07:15:14 WATCH (OK )
IN  0x6000000000022678 07:15:14 READ (backend/vbd/1/769/state )
OUT 0x6000000000022678 07:15:14 WATCH_EVENT (backend/vbd/1/769/physical-device 
E000000008E31C90 )
OUT 0x6000000000022678 07:15:14 READ (1)
IN  0x6000000000022678 07:15:14 WRITE (backend/vbd/1/769/state 2)
OUT 0x6000000000022678 07:15:14 WATCH_EVENT (backend/vbd/1/769/state 
A000000100D20500 )
OUT 0x6000000000022678 07:15:14 WRITE (OK )
IN  0x6000000000022678 07:15:14 WATCH (/local/domain/1/device/vbd/769/state 
E00000001D3AB1A0 )
CREATE watch 0x600000000000cda8
OUT 0x6000000000022678 07:15:14 WATCH (OK )
IN  0x6000000000022678 07:15:14 DIRECTORY (backend/vbd/1/769/domain )
OUT 0x6000000000022678 07:15:14 WATCH_EVENT 
(/local/domain/1/device/vbd/769/state E00000001D3AB1A0 )
OUT 0x6000000000022678 07:15:14 DIRECTORY ()
IN  0x6000000000022678 07:15:14 DIRECTORY (backend/vbd/1/769/frontend )
OUT 0x6000000000022678 07:15:14 DIRECTORY ()
IN  0x6000000000022678 07:15:14 DIRECTORY (backend/vbd/1/769/dev )
OUT 0x6000000000022678 07:15:14 DIRECTORY ()
IN  0x6000000000022678 07:15:14 DIRECTORY (backend/vbd/1/769/state )
OUT 0x6000000000022678 07:15:14 DIRECTORY ()
IN  0x6000000000022678 07:15:14 DIRECTORY (backend/vbd/1/769/params )
OUT 0x6000000000022678 07:15:14 DIRECTORY ()
IN  0x6000000000022678 07:15:14 DIRECTORY (backend/vbd/1/769/mode )
OUT 0x6000000000022678 07:15:14 DIRECTORY ()
IN  0x6000000000022678 07:15:14 DIRECTORY (backend/vbd/1/769/frontend-id )
OUT 0x6000000000022678 07:15:14 DIRECTORY ()
IN  0x6000000000022678 07:15:14 DIRECTORY (backend/vbd/1/769/type )
OUT 0x6000000000022678 07:15:14 DIRECTORY ()
IN  0x6000000000022678 07:15:14 READ (backend/vbd/1/769/physical-device )
OUT 0x6000000000022678 07:15:14 ERROR (ENOENT )
IN  0x6000000000022678 07:15:14 DIRECTORY (backend/vbd/1/769/state )
OUT 0x6000000000022678 07:15:14 DIRECTORY ()
IN  0x6000000000022678 07:15:14 READ (/local/domain/1/device/vbd/769/state )
OUT 0x6000000000022678 07:15:14 READ (1)

If I mmap and dump the kva page in a test program, it looks like it's
working correctly.  I can see and decode the send and receive rings and
see the index values.  Thanks,

        Alex

-- 
Alex Williamson                             HP Linux & Open Source Lab


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel