aboutsummaryrefslogtreecommitdiffstats
path: root/target/imagebuilder
Commit message (Collapse)AuthorAgeFilesLines
* build,ib: add STRIP_ABI option for manifestPaul Spooren2021-03-161-1/+2
| | | | | | | | | | | | The ImageBuilder `make manifest` prints all installed packages. This function can be used to create a list of package and corresponding package versions before attempting image creation. When called with `--strip-abi` OPKG can automatically strip attached ABIVersions from package names. Make this function accessible for the ImageBuilder by adding a `STRIP_ABI` variable. Signed-off-by: Paul Spooren <mail@aparcar.org>
* imagebuilder: invoke bundle-libraries.sh w/o buildroot dirs in $PATHJo-Philipp Wich2020-12-291-1/+4
| | | | | | | | | | Invoke bundle-libraries.sh with any buildroot related directory entries removed from $PATH to avoid picking up cross versions of utilities like ldd which will not properly work when used against host executables. This should fix executable bundling for glibc-target imagebuilders. Signed-off-by: Jo-Philipp Wich <jo@mein.io>
* imagebuilder: fix partition signatureMatthew Gyurgyik2020-11-241-0/+2
| | | | | | | | | When building images with the imagebuilder, the partition signature never changes. The signature is generated by hashing SOURCE_DATE_EPOCH and LINUX_VERMAGIC which are undefined. Prepopulate these variables, as done by the SDK. Signed-off-by: Matthew Gyurgyik <matthew@gyurgyik.io>
* imagebuilder: fix main entry makefilePaulo Machado2020-11-231-1/+1
| | | | | | | Remove a syntax error from ImageBuider Makefile Acked-by: Paul Spooren <mail@aparcar.org> Signed-off-by: Paulo Machado <pffmachado@yahoo.com>
* imagebuilder: add package signature verificationPaul Spooren2020-11-192-2/+39
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The ImageBuilder downloads pre-built packages and adds them to images. This process uses `opkg` which has the capability to verify package list signatures via `usign`, as enabled per default on running OpenWrt devices. Until now this was disabled for ImageBuilders because neither the `opkg` keys nor the `opkg-add` script was present during first packagelist update. To harden the ImageBuilder against *drive-by-download-attacks* both keys and verification script are added to the ImageBuilder allowing `opkg` to verify downloaded package indices. This commit adds `opkg-add` to the ImageBuilder scripts folder. The keys folder is added to ImageBuilder $TOPDIR to have an obvious place for users to store their own keys. The `option check_signature` is appended to the repositories.conf file. All of the above only happens if the Buildbot runs with the SIGNATURE_CHECK option. The keys stored in the ImageBuilder keys/ are the same as included in the openwrt-keyring package. To avoid the chicken-egg problem of downloading and verifying a package, containing signing keys, the keys are added during the ImageBuilder generation. They are same as in shipped images (stored at `/etc/opkg/keys/`). To allow a local package feed in which the user can add additional packages, a local set of `usign` and `ucert` keys is generated, same as building OpenWrt from source. The private key signs the local repository inside the packages/ folder. The local public key is added to the keys/ folder to be considered by `opkg` when updating repositories. This way a local package feed can be modified while requiring `opkg` to check signatures for remote feed, making HTTPS optional. The new option `ADD_LOCAL_KEY` allows to add the local key inside the created images, adding the advantage that sysupgrades can validate the ImageBuilders local key. Signed-off-by: Paul Spooren <mail@aparcar.org>
* imagebuilder: fix sstripPaul Spooren2020-11-091-1/+1
| | | | | | | | Without an absolute path to staging_dir/host/bin/sstrip the Makefile tries to run a host installed version of sstrip, which is likely not available. Signed-off-by: Paul Spooren <mail@aparcar.org>
* build,IB: reload packages/ only if existingPaul Spooren2020-10-301-1/+4
| | | | | | | | | | | With the fix of external kmod feeds it is possible to ship the ImageBuilder without any packages except the pseudo packages kernel and libc. Therefore the local package feeds becomes optional. This commit adds a check to the package_reload function to only run if the local feed is existing. Signed-off-by: Paul Spooren <mail@aparcar.org>
* build,IB: include kmods only in local buildsPaul Spooren2020-10-302-12/+16
| | | | | | | | | | The buildbots generate a kmod archive which should be used instead of a local copy. This is possible due to the introduction of a kernelversion specific feed. This commit adds the ability of using only signed package feeds. Signed-off-by: Paul Spooren <mail@aparcar.org>
* imagebuilder: add missing libfakeroot filesPaul Spooren2020-09-291-0/+1
| | | | | | | The `libfakeroot` files are currently missing in the ImageBuilder. As `fakeroot` is always built, copy those files unconditionally. Signed-off-by: Paul Spooren <mail@aparcar.org>
* build: add whatdepends target to imagebuilderPaul Spooren2020-08-311-1/+9
| | | | | | | | | | | | | | | | The package manager `opkg` offers the function `whatdepends` to print packages that depend on a specific package. This feature is useful when used in a CI to not only build an upgraded package but all packages with a dependency. Usage: make whatdepends PACKAGE=libipset The resulting list can be fed into a SDK building all packages and warn if anything fails. Signed-off-by: Paul Spooren <mail@aparcar.org>
* build: SDK/IB reproducible and faster compressionPaul Spooren2020-08-231-1/+4
| | | | | | | | | | Both IB and SDK now use the same logic for packing. This commit add reproducible multithread compression to the SDK and corrects the file mtime for both. Previously all files where just copied over from the build system, generating random mtimes. Signed-off-by: Paul Spooren <mail@aparcar.org>
* imagebuilder: Remove json_info_files/ before buildPaul Spooren2020-06-111-0/+1
| | | | | | | | | | | | | | The folder `json_info_files` contains multiple JSON files which describe created firmware images. The folder is not removed between builds as the ImageBuilder does not use `image.mk`. Not removing the JSON files result in a merged `profiles.json` file containing entries for outdated or non-existing images. This commit adds the `json_info_files/` cleanup step to the ImageBuilder Makefile. Signed-off-by: Paul Spooren <mail@aparcar.org>
* imagebuilder: pass IB=1 on checking requirementsPaul Spooren2020-06-111-1/+1
| | | | | | | | | | | The patch 4a1a58a3 build, imagebuilder: Do not require libncurses-dev was supposed to remove libncurses as a requirement for the ImageBuilder. However as the IB=1 is only exported during building, not for checking requirements, it did never actually work. This commit export IB=1 to the requirement check. Signed-off-by: Paul Spooren <mail@aparcar.org>
* build: refactor JSON info files to `profiles.json`Paul Spooren2020-04-031-0/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | JSON info files contain machine readable information of built profiles and resulting images. These files were added in commit 881ed09ee6e2 ("build: create JSON files containing image info"). They are useful for firmware wizards and script checking for reproducibility. Currently all JSON files are stored next to the built images, resulting in up to 168 individual files for the ath79/generic target. This patch refactors the JSON creation to store individual per image (not per profile) files in $(BUILD_DIR)/json_info_files and create an single overview file called `profiles.json` in the target directory. Storing per image files and not per profile solves the problem of parallel file writes. If a profiles sysupgrade and factory image are finished at the same time both processes would write to the same JSON file, resulting in randomly broken outputs. Some target like x86/64 do not use the image code yet, resulting in missing JSON files. If no JSON info files were created, no `profiles.json` files is created as it would be empty anyway. As before, this creation is enabled by default only if `BUILDBOT` is set. Tested via buildroot & ImageBuilder on ath79/generic, imx6 and x86/64. Signed-off-by: Paul Spooren <mail@aparcar.org> [json_info_files dir handling in Make, if case refactoring] Signed-off-by: Petr Štetiar <ynezz@true.cz>
* target/imagebuilder: use multi-thread support for xz compressionMartin Schiller2019-10-091-1/+1
| | | | | | | | | | | | | | | | | | | | | | This speeds up the packing of the imagebuilder a lot: imagebuilder-T0.tar.xz real 0m25.199s user 2m45.967s sys 0m1.218s imagebuilder-T1.tar.xz real 2m02.543s user 2m02.418s sys 0m1.653s imagebuilder-T2.tar.xz real 1m03.684s user 1m59.931s sys 0m0.587s imagebuilder-T3.tar.xz real 0m48.033s user 2m02.904s sys 0m0.637s imagebuilder-T4.tar.xz real 0m38.963s user 2m15.521s sys 0m0.783s imagebuilder-T5.tar.xz real 0m37.994s user 2m21.461s sys 0m0.919s imagebuilder-T6.tar.xz real 0m39.524s user 2m48.115s sys 0m1.279s imagebuilder-T7.tar.xz real 0m34.061s user 2m45.097s sys 0m1.174s imagebuilder-T8.tar.xz real 0m27.286s user 2m55.449s sys 0m1.329s imagebuilder-T9.tar.xz real 0m25.205s user 2m44.894s sys 0m1.208s To keep the output reproducible in any case, we enforce a minimum amount of 2 threads. Signed-off-by: Martin Schiller <ms@dev.tdt.de> [refactored into reusable NPROC var, more verbose commit message] Signed-off-by: Petr Štetiar <ynezz@true.cz>
* imagebuilder: fix `make info` for empty SUPPORTED_DEVICESPaul Spooren2019-08-141-1/+2
| | | | | | | | | For x86/64 (maybe more) target the SUPPORTED_DEVICES variable is empty which causes the `&&` junction to fail, producing a non zero exit code. Tested-by: Paul Spooren <mail@aparcar.org> Fixed-by: Jo-Philipp Wich <jo@mein.io> Signed-off-by: Paul Spooren <mail@aparcar.org>
* imagebuilder: new DISABLED_SERVICES make variableRichard Musil2019-05-151-2/+4
| | | | | | | | | | | | Adds a new variable DISABLED_SERVICES to ImageBuilder Makefile, which defines a list of services (installed as /etc/init.d/*) to be disabled during the build of a custom image (normally all are enabled). It comes handy when a particular service should not be run under normal circumstances, but should be ready in the image for situations when it might be needed. Signed-off-by: Richard Musil <risa2000x@gmail.com>
* IB: include SUPPORTED_DEVICES in 'make info' outputDaniel Golle2019-03-111-2/+3
| | | | Signed-off-by: Daniel Golle <daniel@makrotopia.org>
* ib: show current revision based on $(REVISION)Paul Spooren2019-03-081-0/+1
| | | | | | | | This is useful in for the attendedsyupsgrade server (asu) to distinguish between snapshot version. Currently asu can't tell devices requesting a snapshot build if the same build is already installed. Signed-off-by: Paul Spooren <mail@aparcar.org>
* ib: show unified target based on $(TARGETID)Paul Spooren2019-03-081-1/+1
| | | | | | | Instead of showing a slightly more readable target like "ar71xx (Generic)" print the more generic format "ar71xx/genric" Signed-off-by: Paul Spooren <mail@aparcar.org>
* ib: display whether profile comes with image metadataDaniel Golle2019-03-061-1/+1
| | | | | | | | Having image metadata (and signature) appended is a condition for semi-automated sysupgrade, hence IB needs to be able to tell which images will end up with metadata. Signed-off-by: Daniel Golle <daniel@makrotopia.org>
* imagebuilder: manifest function show stderrPaul Spooren2018-08-061-2/+2
| | | | | | | | This really simplifies debugging, if a package is not found or a feed is not reachable, a proper stderr is printed. Currently it would only say `_call_manifest` failed. Signed-off-by: Paul Spooren <mail@aparcar.org>
* imagebuilder: add function to show manifestPaul Spooren2018-07-301-2/+27
| | | | | | | | | | | | | | | | | | | | | | | | | Tested with 18.06.0-rc2/ar71xx/generic/tl-wdr4300-v1, image & list This PR is based on the work of @fewckert[1] with slight improvements. Add function `manifest` to show the manifest of the produced image, before actually building it. The manifest contains an orderd list of package name and version. This is usefull to check package dependencies but also determine a unique and reproducible image name before building the package. The sysupgrade server[2] builds images on request with individual package selection. To distignish between created images which contain differnt packages, the EXTRA_IMAGE_NAME is set to a shortend hash of the manifest's content. So far the image was renamed afterwards as the manifests content was unknown, however this corrupts the signed sha256sums. This patch allows a clean solution as to dtermine the manifest in advance and set the EXTRA_IMAGE_NAME accordingly. [1]: https://github.com/lede-project/source/pull/1591 [2]: https://github.com/aparcar/attendedsysupgrade-server Signed-off-by: Paul Spooren <mail@aparcar.org>
* imagebuilder: remove split patch dirs from imagebuilder archiveMatthias Schiffer2018-03-071-2/+3
| | | | Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
* imagebuilder: reuse rootfs preparation from rootfs.mkMatthias Schiffer2018-03-071-37/+9
| | | | | | | | | | | | In addition to removing redundant code, this fixes various issues in IB-generated images that have been fixed in prepare_rootfs before, including better handling of CONFIG_CLEAN_IPKG and enabling of initscripts from FILES. We also reuse the opkg macro and remove --force-... flags that have been removed from rootfs.mk as well. Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
* imagebuilder: fix reference to removed VERSION_SED variableJo-Philipp Wich2018-02-021-1/+1
| | | | | Fixes: ff8e9a4ecb ("treewide: combine VERSION_SED and VERSION_SED_SCRIPT") Signed-off-by: Jo-Philipp Wich <jo@mein.io>
* merge: etc: update remaining filesZoltan HERPAI2017-12-081-1/+1
| | | | Signed-off-by: Zoltan HERPAI <wigyori@uid0.hu>
* merge: targets: update image generation and targetsZoltan HERPAI2017-12-081-1/+1
| | | | Signed-off-by: Zoltan HERPAI <wigyori@uid0.hu>
* imagebuilder: don't rewrite package list outputDaniel Golle2017-06-171-1/+1
| | | | | | | | No longer rewrite opkg list output in package_list function, remove the awk call in the pipe (which was intended for a single specific use-case). Signed-off-by: Daniel Golle <daniel@makrotopia.org>
* imagebuilder: clean package_listDaniel Golle2017-06-171-6/+6
| | | | | | | | | | | commit 19ac879954 (imagebuilder: add package_list function) introduced a new function 'package_list' to the imagebuilder Makefile. Unfortunately the package list was poluted by stdout noise of the Makefile itself as well as opkg. Redirect those outputs to stderr to make sure that the package_list returned doesn't contain progress info output but really only packages. Signed-off-by: Daniel Golle <daniel@makrotopia.org>
* imagebuilder: add package_list functionPaul Spooren2017-06-151-7/+14
| | | | | | | | | The imagebuilder can now list all available packages by using make package_list. This is usefull for scripts to retrieve a list of all packages with versions (and size) Signed-off-by: Paul Spooren <paul@spooren.de> [daniel@makrotopia.org: fixed commit message]
* imagebuilder: fix bundling of DTS sourcesJo-Philipp Wich2017-05-141-3/+3
| | | | | | | | | | | | | | | Refer to LINUX_KARCH instead of ARCH when bundling DTS files in the image builder tarball. While we're at it, also dereference symbolic links when copying as some kernel architectures contain symbolic links in their DTS directories. This fixes aarch64 imagebuilders such as brcm2708/bcm2710 ones in particular as the kernel refers to "aarch64" as "arm64" internally. Ref: https://forum.lede-project.org/t/lede-image-builder-problem/3680 Signed-off-by: Jo-Philipp Wich <jo@mein.io>
* build: get rid of host.mkFelix Fietkau2017-02-262-3/+1
| | | | | | Defined required host related variables in toplevel.mk instead Signed-off-by: Felix Fietkau <nbd@nbd.name>
* build: add buildbot specific config option for setting defaultsFelix Fietkau2017-02-141-1/+2
| | | | | | | | | | This can be used to tweak the buildbot behavior without having to change buildbot's configuration. It will also allow us to add more aggressive clean steps (e.g. on toolchain changes), which would break developers' workflows if enable by default. Signed-off-by: Felix Fietkau <nbd@nbd.name>
* imagebuilder: make submake invocations less verboseJo-Philipp Wich2017-02-021-4/+4
| | | | | | | Use silent make invocations for sub-makes like build_image or checksum to avoid bloating the IB output with non-status info. Signed-off-by: Jo-Philipp Wich <jo@mein.io>
* build: introduce STAGING_DIR_IMAGEJo-Philipp Wich2017-01-271-0/+3
| | | | | | | | | | | Introduce a new location STAGING_DIR_IMAGE which is intended to be used by bootloader iamges and similar image-related artifacts. This directory is guaranteed to be persistent across kernel upgrades which might involve a removal of KERNEL_BUILD_DIR and is guranteed to be bundled with the image builder. Signed-off-by: Jo-Philipp Wich <jo@mein.io>
* imagebuilder: properly escape single quotes in device titlesJo-Philipp Wich2017-01-261-1/+1
| | | | | | | | | | | | | The name "Plat'Home OpenBlocks AX3" causes the imagebuilders "make info" command to fail with: bash: -c: line 0: syntax error near unexpected token `(' bash: -c: line 0: `echo; [...]' Makefile:99: recipe for target '_call_info' failed Properly escape single quotes to avoid breaking the echo commands. Signed-off-by: Jo-Philipp Wich <jo@mein.io>
* imagebuilder: remove existing debug kernel imageFelix Fietkau2017-01-131-0/+1
| | | | | | Reduces tarball size and improves build time Signed-off-by: Felix Fietkau <nbd@nbd.name>
* imagebuilder: remove existing root filesystem imagesFelix Fietkau2017-01-131-0/+1
| | | | | | Reduces tarball size and improves build time Signed-off-by: Felix Fietkau <nbd@nbd.name>
* build: rework library bundlingJo-Philipp Wich2017-01-101-4/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Rework the bundle-libraries.sh implementation to use a more robust approach for executing host binaries through the shipped ELF loader and libraries. The previous approach relied on symlinks pointing to a wrapper script which caused various issues, especially with multicall binaries as the original argv[0] name was not preserved through the ld.so invocation. Another down- side was the fact that the actual binaries got moved into another directory which caused executables to fail looking up resources with paths relative to the executable location. The new library wrapper implements the following improvements: - Instead of symlinks pointing to a common wrapper, each ELF executable is now replaced by a unqiue shell script which retains the original program name getting called - Instead of letting ld.so invoke the ELF executable directly, launch the final ELF binary through a helper program which fixes up the argv[0] argument for the target program - Support sharing a common location for the bundled libraries instead of having one copy in each directory containing wrapped binaries Finally modify the SDK build to wrap the staging_dir and toolchain binaries which allows to use the SDK on systems with a different glibc version. Signed-off-by: Jo-Philipp Wich <jo@mein.io>
* target/imagebuilder: Switch to xz compression instead of bz2Daniel Engberg2016-10-151-4/+4
| | | | | | | Switch to xz compression instead of using bz2. Saves about 20% of total size (ar71xx) Signed-off-by: Daniel Engberg <daniel.engberg.lists@pyret.net>
* target/{sdk,imagebuild}: Fix for symlink-treeDaniel Dickinson2016-09-301-1/+1
| | | | | | | | | With symlink tree some directories are just symlinked which means IB and SDK end up with a symlink instead of an actual directory; this fixes the missing files by dereferencesing the directories instead of copying the symlinks. Signed-off-by: Daniel Dickinson <lede@cshore.thecshore.com>
* build: add checksum targetJo-Philipp Wich2016-08-011-0/+6
| | | | | | | | | | | | | | | Add a new "checksum" make target which generates an sha256sums file over the image files produced in bin/targets/ and automatically call it during make world after the package index generation. The advantage of this new target is that it is guaranteed to run after the images, the SDK and the ImageBuilder archives have been generated to ensure that they all end up in the checksum file. Fixes FS#51. Uses sed to postprocess the OpenSSL digest output into an sha256sum command compatible format. Signed-off-by: Jo-Philipp Wich <jo@mein.io>
* build: remove obsolete variables from opkg commandFelix Fietkau2016-07-291-3/+0
| | | | Signed-off-by: Felix Fietkau <nbd@nbd.name>
* imagebuilder: strip DEVICE_ prefix from profiles (FS#55)Felix Fietkau2016-07-211-3/+5
| | | | Signed-off-by: Felix Fietkau <nbd@nbd.name>
* build: split scripts/metadata.pl into target-metadata.pl and package-metadata.plFelix Fietkau2016-06-071-1/+1
| | | | Signed-off-by: Felix Fietkau <nbd@nbd.name>
* IB/SDK/toolchain: use lower cases filenamesAlexander Couzens2016-06-011-1/+1
| | | | Signed-off-by: Alexander Couzens <lynxis@fe80.eu>
* IB/SDK/toolchain: use VERSION_DIST_SANITIZED instead of VERSION_DISTAlexander Couzens2016-06-011-1/+1
| | | | | | VERSION_DIST can contains spaces which produces problems when used as file name Signed-off-by: Alexander Couzens <lynxis@fe80.eu>
* imagebuilder: Fix sorting package list breaks opkg dependency handling for ↵Daniel Dickinson2016-05-181-1/+1
| | | | | | | | | | | | | | | | provides When imagebuild sorts package lists it breaks opkg's ability to realize that a providers for a Provides has already been installed, when the sort results in the provider being later in the list of packages that a package which depends on a Provides (and hence the provider is not yet installed for opkg to realize the provider was available doesn't not handle the case of a package that is to be installed satisfying a dependency, only one that is already installed (or which it schedules to be installed, which in the absence of an installed provider is whichever provider happens to be the default) Signed-off-by: Daniel Dickinson <openwrt@daniel.thecshore.com>
* build: remove profile kernel/build system config override supportFelix Fietkau2016-05-151-1/+0
| | | | | | It has been unused for years Signed-off-by: Felix Fietkau <nbd@nbd.name>
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-05 11:40:06 +0000