WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

[Xen-devel] Re: [PATCH 2/2] build: make linux download more flexible

To: Keir Fraser <Keir.Fraser@xxxxxxxxxxxx>
Subject: [Xen-devel] Re: [PATCH 2/2] build: make linux download more flexible
From: Horms <horms@xxxxxxxxxxxx>
Date: Thu, 29 Jun 2006 10:27:23 +0900
Cc: xen-devel <xen-devel@xxxxxxxxxxxxxxxxxxx>
Delivery-date: Thu, 29 Jun 2006 02:46:31 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxx
In-reply-to: <20060629012424.GA14402@xxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <20060628020414.GB28692@xxxxxxxxxxxx> <c515687a58314585f17bfe345db4592b@xxxxxxxxxxxx> <20060629000044.GW30330@xxxxxxxxxxxx> <20060629012424.GA14402@xxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Mutt/1.5.11+cvs20060403
* Allow LINUX_REPO to specify the URL of the repository for the linux kernel,
  - Defaults is: http://www.kernel.org/pub/linux/kernel/     
* Allow LINUX_REPO to be overridden in the environment
* LINUX_REPO it should be the path, trimmed to give a structure
  equivalent to http://www.kernel.org/pub/linux/kernel/
  - The kernel tarball will be downloaded from
    $(LINUX_REPO)/v$(KERNEL_MAJOR).$(KERNEL_MINOR)/
    e.g. http://www.kernel.org/pub/linux/kernel/v2.6
    Where KERNEL_MAJOR and KERNEL_MINOR are calculated at runtime
  - The path of the tarball can be overridden using LINUX_REPO_KERNEL_PATH 
    in the environment, in which case the tarball will be downloaded from
    $(LINUX_REPO)/v$(LINUX_REPO_KERNEL_PATH)/
  - If needed kernel patches will be downloaded from
    $(LINUX_REPO)/v$(KERNEL_MAJOR).$(KERNEL_MINOR)/snapshots
    if present, else
    $(LINUX_REPO)/v$(KERNEL_MAJOR).$(KERNEL_MINOR)/testing
    This will depend on the version of the patch
  - The path of the tarball can be overridden using LINUX_REPO_KERNEL_PATH 
  in the environment, in which case the patch will be downloaded from
    $(LINUX_REPO)/v$(LINUX_REPO_PATCH_PATH)/
  - The intention of LINUX_REPO_KERNEL_PATH and LINUX_REPO_PATCH_PATH
    is to allow for sites that don't follow the directory hierarchy of
    kernel.org, while simplifying the presumably common case where they do
* Previously KERNEL_REPO had similar properties to LINUX_REPO, but
  it was not as flexible, had a somewhat presumptuous name and could
  not be overridden in the environment. It has been removed
* Allow DOWNLOAD_PATH to set a search path for previously downloaded files
  - Use the first element of the path as the directory to save
    downloaded files
  - Default is LINUX_SRC_PATH if set in environment, else .:..
  - Note that kclean-tarball, makedistlcean-* and friends 
    intentionally only remove tarballs in the top level xen directory,
    regardless of the value of this variable. This is to allow
    more persistent storage of tarballs in a specified location,
    which cam be manually cleaned if desired.
* Fix bug introduced by xen-unstable.hg:9931:39fa9a75d84b whereby
  the downloaded kernel source is not removed on mrproper
* Remove the local tarball, if any, on mrproper

Signed-Off-By: Horms <horms@xxxxxxxxxxxx>

 Config.mk                     |   12 ++++++++++-
 Makefile                      |   14 +++++++++++++
 buildconfigs/Rules.mk         |   44 ++++++++++++++++++++++++++++++++---------
 buildconfigs/mk.linux-2.6-xen |    4 ++-
 4 files changed, 63 insertions(+), 11 deletions(-)

--- from-0001/Config.mk
+++ to-work/Config.mk   2006-06-29 10:11:45.000000000 +0900
@@ -57,7 +57,17 @@ LDFLAGS += $(foreach i, $(EXTRA_LIB), -L
 CFLAGS += $(foreach i, $(EXTRA_INCLUDES), -I$(i))
 
 # Choose the best mirror to download linux kernel
-KERNEL_REPO = http://www.kernel.org
+DOWNLOAD_PATH_DEFAULT := .:..
+LINUX_REPO_DEFAULT    := http://www.kernel.org/pub/linux/kernel/
+ifdef LINUX_SRC_PATH
+DOWNLOAD_PATH  ?= $(LINUX_SRC_PATH) # Compatibility
+else
+DOWNLOAD_PATH  ?= $(DOWNLOAD_PATH_DEFAULT)
+endif
+DOWNLOAD_DIR   := $(firstword $(subst :, ,$(DOWNLOAD_PATH)))
+LINUX_REPO     ?= $(LINUX_REPO_DEFAULT)
+# LINUX_REPO_KERNEL_PATH Set in buildconfigs/Rules.mk, if not in environment
+# LINUX_REPO_PATCH_PATH  Set in buildconfigs/Rules.mk, if not in environment
 
 # If ACM_SECURITY = y, then the access control module is compiled
 # into Xen and the policy type can be set by the boot policy file
--- from-0002/Makefile
+++ to-work/Makefile    2006-06-29 10:11:45.000000000 +0900
@@ -175,6 +175,20 @@ help:
        @echo '                     install into prefix/lib/python<VERSION>'
        @echo '                     instead of <PREFIX>/lib/python'
        @echo '                     true if set to non-empty value, false 
otherwise'
+       @echo '  LINUX_REPO=URL   - Base URL to download linux kernel from'
+       @echo '                     Default: "$(LINUX_REPO_DEFAULT)"'
+       @echo '  LINUX_REPO_KERNEL_PATH=DIRECTORY'
+       @echo '                   - LINUX_REPO assumes that the URL has the 
same directory'
+       @echo '                     structure as $(LINUX_REPO_DEFAULT)'
+       @echo '                     and the rest of the path to the tarball is 
derived.'
+       @echo '                     If set, override the derivation.'
+       @echo '                     Default: ""'
+       @echo '  LINUX_REPO_PATCHL_PATH=DIRECTORY'
+       @echo '                   - LINUX_REPO assumes that the URL has the 
same directory'
+       @echo '                     structure as $(LINUX_REPO_DEFAULT)'
+       @echo '                     and the rest of the path to the patches is 
derived.'
+       @echo '                     If set, override the derivation.'
+       @echo '                     Default: ""'
 
 # Use this target with extreme care!
 .PHONY: uninstall
--- from-0002/buildconfigs/Rules.mk
+++ to-work/buildconfigs/Rules.mk       2006-06-29 10:11:45.000000000 +0900
@@ -22,32 +22,58 @@ LINUX_SERIES        ?= 2.6
 LINUX_VER      ?= $(shell grep "^LINUX_VER" buildconfigs/mk.linux-2.6-xen | 
sed -e 's/.*=[ ]*//')
 
 # Setup Linux search path
-LINUX_SRC_PATH ?= .:..
-vpath linux-%.tar.bz2 $(LINUX_SRC_PATH)
-vpath patch-%.bz2 $(LINUX_SRC_PATH)
+vpath linux-%.tar.bz2 $(DOWNLOAD_PATH)
+vpath patch-%.bz2 $(DOWNLOAD_PATH)
 
 # download a pristine Linux kernel tarball if there isn't one in LINUX_SRC_PATH
+ifndef LINUX_REPO_KERNEL_PATH
 linux-%.tar.bz2: override _LINUX_VDIR = $(word 1,$(subst ., ,$*)).$(word 
2,$(subst ., ,$*))
+linux-%.tar.bz2: LINUX_REPO_KERNEL_PATH = v$(_LINUX_VDIR)
+endif
 linux-%.tar.bz2:
        @echo "Cannot find $@ in path $(LINUX_SRC_PATH)"
-       wget $(KERNEL_REPO)/pub/linux/kernel/v$(_LINUX_VDIR)/$@ -O./$@
+       wget $(LINUX_REPO)/$(LINUX_REPO_KERNEL_PATH)/$@ -O$(DOWNLOAD_DIR)/$@ \
+               || { rm -f $(DOWNLOAD_DIR)/$@; false; }
 
+ifndef LINUX_REPO_PATCH_PATH
 patch-%.bz2: override _LINUX_VDIR = $(word 1,$(subst ., ,$(*F))).$(word 
2,$(subst ., ,$(*F)))
 patch-%.bz2: override _LINUX_XDIR = $(if $(word 3,$(subst -, 
,$(*F))),snapshots,testing)
+patch-%.bz2: LINUX_REPO_PATCH_PATH = v$(_LINUX_VDIR)/$(_LINUX_XDIR)
+endif
 patch-%.bz2:
        @echo "Cannot find $(@F) in path $(LINUX_SRC_PATH)"
-       wget 
$(KERNEL_REPO)/pub/linux/kernel/v$(_LINUX_VDIR)/$(_LINUX_XDIR)/$(@F) -O./$@
+       wget $(LINUX_REPO)/$(LINUX_REPO_PATCH_PATH)/$(@F) \
+               -O$(DOWNLOAD_DIR)/$@ || { rm -f $(DOWNLOAD_DIR)/$@; false; }
 
 pristine-%: pristine-%/.valid-pristine
        @true
+  
+# unpack-pristine and patch-pristine targets are run in a submake
+# so that vpath is re-evaluated. This is important for the case
+# where the files are dowloaded this time around, and they are downloaded
+# into an alternate directory, as specified by DOWNLOAD_PATH
+
+unpack-pristine-%-X: %
+       tar -C $(UNPACK_DIR) -jxf $<
+
+patch-pristine-%-X: %
+       echo $<
+       bzcat $< | patch -d $(UNPACK_DIR) -p1
+
+.PHONY: patch-pristine-dummy
+patch-pristine-dummy-X:
 
-pristine-%/.valid-pristine: %.tar.bz2
+pristine-%/.valid-pristine: $(LINUX_TARBALL) $(LINUX_PATCHES)
        rm -rf tmp-pristine-$* $(@D)
        mkdir -p tmp-pristine-$*
-       tar -C tmp-pristine-$* -jxf $<
-       -@rm -f tmp-pristine-$*/pax_global_header
+       @# firstword is used to get rid of any whitespace
+       $(MAKE) UNPACK_DIR=tmp-pristine-$* \
+               unpack-pristine-$(firstword $(LINUX_TARBALL))-X
+       -rm -f tmp-pristine-$*/pax_global_header
        mv tmp-pristine-$*/* $(@D)
-       @rm -rf tmp-pristine-$*
+       rm -rf tmp-pristine-$*
+       $(MAKE) KERNELS=linux-2.6-xen UNPACK_DIR=pristine-$* \
+               $(patsubst %, patch-pristine-%-X, dummy $(LINUX_PATCHES))
        touch $(@D)/.hgskip
        touch $@ # update timestamp to avoid rebuild
 
--- from-0003/buildconfigs/mk.linux-2.6-xen
+++ to-work/buildconfigs/mk.linux-2.6-xen       2006-06-29 10:14:41.000000000 
+0900
@@ -1,5 +1,7 @@
 LINUX_SERIES = 2.6
 LINUX_VER    = 2.6.16.13
+LINUX_TARBALL = linux-2.6.16.13.tar.bz2 # Only one makes sense
+#LINUX_PATCHES = patch-2.6.17-rcN.bz2 # Usually zero or one patch
 
 EXTRAVERSION ?= xen
 
@@ -57,4 +59,4 @@ delete: 
 
 .PHONY: mrpropper
 mrpropper:
-       rm -f linux-$(LINUX_VER).tar.bz2
+       rm -f $(LINUX_TARBALL) $(LINUX_PATCHES)

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel