build: download code from github using archive API

A new python script scripts/download.py is added to fetch tarballs using
GitHub archive API [1], then repack in a reproducible way same as the
current DownloadMethod/git

GitHub imposes a 60 reqs/hour rate limit on unauthenticated API
access[2].  This affects fetching commit date for feeding tar --mtime=
argument.  However, observation indicates that archive download is NOT
subject to this limit at the moment.  In the rare cases where download
fails because of this, we will falback to using DownloadMethod/git

The missing piece in the GitHub API is that it cannot provide in the
tarball dependent submodules's source code.  In that case, the
implementation will also fallback to using DownloadMethod/git

 [1] Get archive link, https://developer.github.com/v3/repos/contents/#get-archive-link
 [2] Rate limiting, https://developer.github.com/v3/#rate-limiting

v2 <- v1:

 - allow passing multiple urls with --urls argument
 - add commit ts cache.  can be helpful on retry

Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>

0 files changed, 0 insertions, 0 deletions