aboutsummaryrefslogtreecommitdiffstats
path: root/include/download.mk
Commit message (Collapse)AuthorAgeFilesLines
* build: add PKG_SOURCE_URL_FILE supportKamil Wcislo2019-11-231-0/+1
| | | | | | | | | | | | | | | It seems that there is a missing PKG_SOURCE_URL_FILE support. This little fix adds the support for packages to change the name of the downloaded file. Sometimes it is desirable to change the downloaded archive file name, like for mitigating name conflicts for different packages (some files on the server could be named like, e.g. 2018-01-01.tar.gz) or for the cases that there is no name for the file in the URL (e.g. http://someserver.com/download). Signed-off-by: Kamil Wcislo <kamil.wcislo@lpnplant.io> Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net> (cherry picked from commit 09c428ec6bca27bdb31e2f6fef85b07787cead30)
* build: remove sgid permission from tarEneas U de Queiroz2019-04-061-1/+2
| | | | | | | | | | Otherwise tar will keep the sgid bit when running from a sgid-set directory, resulting in a different file being generated. Signed-off-by: Eneas U de Queiroz <cote2004-github@yahoo.com> Signed-off-by: Christian Lamparter <chunkeey@gmail.com> [reworked commit message, removed DMARC]
* download.mk: enable DownloadMethod/github_archiveYousong Zhou2018-07-051-8/+14
| | | | Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
* scripts/dl_github_archive.py: rename from download.pyYousong Zhou2018-07-051-7/+7
| | | | | | | | | | | - Make the code more GitHub-specific - Requires mirror hash to work with .gitattributes - Use different API depending on whether PKG_SOURCE_VERSION is a complete commit id or other ref types like tags - Fix removing symbolic link - pre-clean dir_untar for possible leftovers from previous run Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
* download.mk: add more commentsYousong Zhou2018-07-051-2/+5
| | | | Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
* download.mk: restore the old dl_method implementationYousong Zhou2018-06-291-1/+17
| | | | | | | | Seems like the python download.py dl_method call causes serious performance regression for fresh "make defconfig" as reported in FS#1621. GitHub tarball download will also be disabled with this Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
* build: download code from github using archive APIYousong Zhou2018-06-271-31/+43
| | | | | | | | | | | | | | | | | | | | | | | | | | A new python script scripts/download.py is added to fetch tarballs using GitHub archive API [1], then repack in a reproducible way same as the current DownloadMethod/git GitHub imposes a 60 reqs/hour rate limit on unauthenticated API access[2]. This affects fetching commit date for feeding tar --mtime= argument. However, observation indicates that archive download is NOT subject to this limit at the moment. In the rare cases where download fails because of this, we will falback to using DownloadMethod/git The missing piece in the GitHub API is that it cannot provide in the tarball dependent submodules's source code. In that case, the implementation will also fallback to using DownloadMethod/git [1] Get archive link, https://developer.github.com/v3/repos/contents/#get-archive-link [2] Rate limiting, https://developer.github.com/v3/#rate-limiting v2 <- v1: - allow passing multiple urls with --urls argument - add commit ts cache. can be helpful on retry Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
* downloads.mk: introduce name-agnostic PROJECT_GIT variableJo-Philipp Wich2018-01-101-2/+4
| | | | | | | | | | Introduce a name-agnostic PROJECT_GIT variable poiting to https://git.openwrt.org/ and declare LEDE_GIT and OPENWRT_GIT as aliases to it. After some transition time we can drop this alias variables. Signed-off-by: Jo-Philipp Wich <jo@mein.io>
* build: remove @ as it's causing an errorPhilip Prindeville2017-09-201-1/+1
| | | | | | | | Since $(DownloadMethod/unknown) is being invoked in the expansion of $(call locked ...) anyway, you can't have an @ because the shell doesn't know what to do with it. Signed-off-by: Philip Prindeville <philipp@redfish-solutions.com>
* build: let make check warn about use of legacy PKG_MD5SUM variable in feedsFelix Fietkau2017-01-131-3/+0
| | | | | | | The variable rename change has been merged in OpenWrt now, so it's possible to convert the feeds as well. Signed-off-by: Felix Fietkau <nbd@nbd.name>
* build: use mkhash to replace various quirky md5sum/openssl callsFelix Fietkau2017-01-051-1/+1
| | | | Signed-off-by: Felix Fietkau <nbd@nbd.name>
* build: add defaults for PKG_SOURCE, PKG_SOURCE_SUBDIR, PKG_VERSIONFelix Fietkau2016-12-221-0/+6
| | | | | | | | This makes it easier to unify versioning of git based package downloads. PKG_SOURCE_DATE along with an 8-character abbreviation of the git hash is used as PKG_VERSION, PKG_RELEASE should be used like normal packages. Signed-off-by: Felix Fietkau <nbd@nbd.name>
* build: add FIXUP option for make checkFelix Fietkau2016-12-171-1/+18
| | | | | | | | This will attempt to automatically fix common mistakes like using MD5 instead of SHA256, using the MD5SUM variable instead of HASH, or even a missing mirror file hash. Signed-off-by: Felix Fietkau <nbd@nbd.name>
* build: implement make check and make package/X/checkFelix Fietkau2016-12-171-9/+60
| | | | | | | | | | | | | | | | This is intended to be used for a wide array of package sanity checks. The first check that is implemented is for the hash of downloaded files. It checks: - Missing hash - Use of SHA256 instead of MD5 - dl/<file> hash not matching hash in makefile - deprecated MD5SUM variable The deprecated MD5SUM variable check is skipped for feeds/ until OpenWrt is updated as well Signed-off-by: Felix Fietkau <nbd@nbd.name>
* include/package.mk: sync default value for hash fallback with mirror hashFelix Fietkau2016-12-161-1/+1
| | | | | | Simplifies dealing with PKG_HASH vs PKG_MD5SUM Signed-off-by: Felix Fietkau <nbd@nbd.name>
* build: remove duplicate Download/default definition from include/host-build.mkFelix Fietkau2016-12-161-0/+13
| | | | Signed-off-by: Felix Fietkau <nbd@nbd.name>
* build: replace MD5SUM variables with HASHFelix Fietkau2016-12-161-2/+4
| | | | | | | Since we've switched to preferring SHA256 over MD5, the old variable name is misleading. Packages using the old name remain compatible. Signed-off-by: Felix Fietkau <nbd@nbd.name>
* download.mk: improve download tarball reproducibilityFelix Fietkau2016-12-131-1/+1
| | | | | | | Store only numeric user/group id. Group 0 is 'root' on linux and 'wheel' on BSD and Mac OS X. Signed-off-by: Felix Fietkau <nbd@nbd.name>
* download.mk: remove code duplication in $(TAR) callFelix Fietkau2016-12-131-9/+12
| | | | Signed-off-by: Felix Fietkau <nbd@nbd.name>
* download.mk: use $(error) instead of a regular shell errorFelix Fietkau2016-12-131-1/+1
| | | | | | Useful for further rework Signed-off-by: Felix Fietkau <nbd@nbd.name>
* include/download.mk: Allow specify DownloadMethod specific optionsFlorian Fainelli2016-12-041-1/+2
| | | | | | | | This is going to be used to migrate the hand rolled git clone for the kernel into using the git download method. The kernel uses custom options that we may have to pass down. Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
* include/download.mk: Use -7e compression instead of -6 by defaultDaniel Engberg2016-10-051-1/+1
| | | | | | Adds a slightly higher compression level to xz by default which roughly raises memory usage from 100MiB to about 200MiB during compression, about 10MiB for decompression. (Source: xz manpage) Signed-off-by: Daniel Engberg <daniel.engberg.lists@pyret.net>
* include/download.mk: generate reproducable SCM tarballsJo-Philipp Wich2016-10-051-4/+9
| | | | | | | | | | | | | | | | | Apply a number of changes to the tarball generation in order to produce identical files on different systems: 1) Use an explicit `gzip -cn` to avoid storing file mtime in the gzip header 2) Instruct `tar` to unconditionally use uid and gid 0 for archive members 3) Instruct `tar` to sort archive members by file name 4) For SCMs that do not preserve file modification times like Git or Mercurial, use the date of the last commit to the repository and pass it as `--mtime` value to `tar` After these changes, locally produced tarballs generated from SCM checkouts should be identical on any system, simplifying the mirroring of cache archives. Signed-off-by: Jo-Philipp Wich <jo@mein.io>
* download.mk: use HTTPS for git.lede-project.orgJo-Philipp Wich2016-06-011-1/+1
| | | | | | | | Some Git versions have issues following the HTTP->HTTPS redirect and since the keyring package is fetched from this host, switching to HTTPS is a sensible choice anyway. Signed-off-by: Jo-Philipp Wich <jo@mein.io>
* include: add lede git server urlJohn Crispin2016-04-291-0/+1
| | | | Signed-off-by: John Crispin <john@phrozen.org>
* download: add @GITHUB download facilityÁlvaro Fernández Rojas2016-04-031-0/+1
| | | | | | Define a new alias (@GITHUB) for downloading raw files from github repos. Signed-off-by: Álvaro Fernández Rojas <noltari@gmail.com>
* download: add @GITHUB download facilityJohn Crispin2016-04-091-1/+1
| | | | | | | | Define a new alias (@GITHUB) for downloading raw github repository files Signed-off-by: Álvaro Fernández Rojas <noltari@gmail.com> SVN-Revision: 49138
* buildroot: improve git submodule handling for packagesFelix Fietkau2016-02-291-2/+3
| | | | | | | | | | | | | | | | | | | Move the `--recursive` switch from `git clone` to `git submodule` so that submodules are cloned for upstream branches where the PKG_SOURCE_VERSION commit-ish has a different .gitmodules configuration than the repository default. This is, for example, required when the master branch for a source package does not use submodules, but its topic branch for OpenWRT does. This changes the buildroot dependency from git-1.6.2 to git 1.7.12.2, which was released September 2012. Signed-off-by: Darik Horn <dajhorn@vanadac.com> Signed-off-by: Karl Palsson <karlp@tweak.net.au> SVN-Revision: 48830
* download.mk: fix packed checkout mirroring supportJo-Philipp Wich2016-02-171-1/+1
| | | | | | | | | | | Changeset r48416 broke the downloading of mirrored, packed scm checkouts. Fix this by removing the "@" sign in front of the download command which is now executed as part of a larger shell command under flock. Signed-off-by: Jo-Philipp Wich <jow@openwrt.org> SVN-Revision: 48733
* build: download.mk: bugfix download.pl argument call orderFelix Fietkau2016-01-211-1/+1
| | | | | | | | | Change wrong download.pl argument call order. See download.pl argument list. Signed-off-by: Florian Eckert <Eckert.Florian@googlemail.com> SVN-Revision: 48427
* build: add locking for downloads (fixes race conditions with multiple variants)Felix Fietkau2016-01-201-1/+1
| | | | | | Signed-off-by: Felix Fietkau <nbd@openwrt.org> SVN-Revision: 48416
* build: add @APACHE download facilityFelix Fietkau2016-01-171-1/+1
| | | | | | | | | | The Apache Software Foundation offers diverse download mirros. For packaging Apache software a new alias @APACHE is defined. Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de> SVN-Revision: 48270
* build: add a variable pointing to the main openwrt git repositories (useful ↵Felix Fietkau2016-01-041-0/+2
| | | | | | | | if we want to support using a mirror later) Signed-off-by: Felix Fietkau <nbd@openwrt.org> SVN-Revision: 48117
* build: add support for choosing a different url filename part than the ↵Felix Fietkau2015-11-221-2/+3
| | | | | | | | output file Signed-off-by: Felix Fietkau <nbd@openwrt.org> SVN-Revision: 47591
* include/download.mk: Add download mirrors for tools from GNU Savannah (bug ↵John Crispin2014-10-081-1/+1
| | | | | | | | | | | #15184) I defined a new download method @SAVANNAH in include/download.mk and scripts/download.pl, and converted quilt and qemu to use that method. Signed-off-by: Hannu Nyman <hannu.nyman@iki.fi> SVN-Revision: 42840
* build: add support for packing version control downloads with xzFelix Fietkau2014-04-101-0/+1
| | | | | | Signed-off-by: Felix Fietkau <nbd@openwrt.org> SVN-Revision: 40446
* Incorrect checkout of git submodulesJo-Philipp Wich2013-10-101-1/+1
| | | | | | | | | | | When checking out git packages, buildroot doesn't seem to track the revisions correctly of any submodules referenced by that project. As a result, the submodule stays at whatever revision was referenced by the head of the master branch. Running a 'git submodule update' after the checkout fixes this problem. Signed-off-by: Owen Kirby <osk@exegin.com> SVN-Revision: 38359
* include: download.mk: correct tar argument order when packing checkoutsJo-Philipp Wich2012-12-021-2/+2
| | | | | | | GNU TAR v1.26 will fail with "no such file or directory" errors when the output file does not immediately follow the -f switch. SVN-Revision: 34460
* download.mk: refuse to download mirror tarballs from packages using version ↵Felix Fietkau2012-06-061-1/+1
| | | | | | control if no mirror md5sum was provided SVN-Revision: 32079
* Enable recursive download of git sources.Jens Muecke2012-03-171-2/+2
| | | | SVN-Revision: 30967
* add support for md5sum checks for mirrored tarballs of packages with version ↵Felix Fietkau2011-03-211-1/+2
| | | | | | control source urls SVN-Revision: 26260
* download.mk: properly quote download urlsFelix Fietkau2011-03-131-1/+1
| | | | SVN-Revision: 26129
* download.mk: support https:// downloadsJo-Philipp Wich2010-12-181-1/+1
| | | | SVN-Revision: 24688
* add support for darcs repositoriesFlorian Fainelli2010-10-241-1/+18
| | | | | | | | | | | | | | | | | | | | This patch adds support for darcs repositories (as sources of packages). It does *not* add support for darcs:// URI scheme because such a scheme do not exist (AFAIK). You must therefore manually set PKG_SOURCE_PROTO to darcs in your Makefile (and use a regular http:// URI). You also have to set PKG_SOURCE_VERSION to a string matching a tag contained in the repository, and PKG_SOURCE_SUBDIR to something sensible ($(PACKAGE_NAME)-$(PACKAGE_SOURCE_VERSION) for instance). Same rationale as for the previous patch ("useless to most, but small and straightforward so why no add it?). Best regards, Signed-off-by: Gabriel Kerneis <kerneis@pps.jussieu.fr> SVN-Revision: 23615
* download.mk: fix bzr download methodNicolas Thill2010-03-291-2/+2
| | | | SVN-Revision: 20582
* the svn fallback code added in r20322 is evaluated with each make invocation ↵Jo-Philipp Wich2010-03-231-7/+3
| | | | | | and produces a lot of spurious "svn: broken pipe" error - replace it with a more efficient version SVN-Revision: 20377
* fix svn download with older svn client versionsFlorian Fainelli2010-03-201-1/+7
| | | | SVN-Revision: 20322
* accept unknown server certificates by default on svn downloadFlorian Fainelli2010-03-191-1/+1
| | | | | | | Some packages sources need to be checked out from server with unknown certificates, silently accept by default in order not to stop builds, thanks rhk. SVN-Revision: 20299
* Updated post r19769, integrates the missing dl_method bzr hunk from #6370, ↵Travis Kemen2010-03-141-12/+16
| | | | | | uses the export subcommand for the vcs' that provide a usable version (hg archive doesn't work remotely, git archive isn't enabled on github), from #6700 SVN-Revision: 20222
* this patch adds support for bzr in include/download.mk, and fixes whitespace.Travis Kemen2010-02-201-18/+33
| | | | | | -Raphael SVN-Revision: 19769
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-18 19:29:13 +0000