Skip to content
This repository was archived by the owner on Aug 5, 2022. It is now read-only.

Conversation

pohly
Copy link
Contributor

@pohly pohly commented Jun 13, 2017

Imports and adapts to latest OE-core master, plus uses all the Yocto Compatible 2.0 support that recently landed in OE-core + bitbake master:

  • optional includes
  • oe.utils.all/any_distro_features()
  • OE-core distrooverrides.bbclass
  • BBFILES_DYNAMIC for .bbappends which may or may not have a .bb

Also split up the Yocto Compatible 2.0 selftest, to make it easier to run just the signature checks.

@pohly pohly force-pushed the yocto-compat-2.0-enhancements branch from 70c234e to 9ea203f Compare June 13, 2017 15:51
@pohly
Copy link
Contributor Author

pohly commented Jun 13, 2017

test this please

I had not caught all _refkit -> _df-refkit renames.

@pohly
Copy link
Contributor Author

pohly commented Jun 13, 2017

test this please

@pohly
Copy link
Contributor Author

pohly commented Jun 13, 2017

retest this please

Seems like an unrelated problem (race?):
https://ostroproject.org/jenkins/job/intel-iot-refkit_pull-requests/1410/consoleFull

DEBUG: SITE files ['endian-little', 'common-linux', 'common-glibc', 'bit-64', 'x86_64-linux', 'common']
00:37:57.075 | DEBUG: Executing shell function do_install_ptest_base
00:37:57.075 | cp: cannot stat '/srv/jenkins/workspace/ci-2017-06-13_12-39-06-build-1410/build/tmp-glibc/work/corei7-64-refkit-linux/util-linux/2.30-r0/util-linux-2.30/tests/ts/tailf': No such file or directory
00:37:57.075 | WARNING: /srv/jenkins/workspace/ci-2017-06-13_12-39-06-build-1410/build/tmp-glibc/work/corei7-64-refkit-linux/util-linux/2.30-r0/temp/run.do_install_ptest_base.160077:1 exit 1 from 'cp -pR /srv/jenkins/workspace/ci-2017-06-13_12-39-06-build-1410/build/tmp-glibc/work/corei7-64-refkit-linux/util-linux/2.30-r0/util-linux-2.30/tests/ts/$d /srv/jenkins/workspace/ci-2017-06-13_12-39-06-build-1410/build/tmp-glibc/work/corei7-64-refkit-linux/util-linux/2.30-r0/image/usr/lib/util-linux/ptest/tests/ts/'
00:37:57.075 | ERROR: Function failed: do_install_ptest_base (log file is located at /srv/jenkins/workspace/ci-2017-06-13_12-39-06-build-1410/build/tmp-glibc/work/corei7-64-refkit-linux/util-linux/2.30-r0/temp/log.do_install_ptest_base.160077)
00:37:57.075 NOTE: recipe util-linux-2.30-r0: task do_install_ptest_base: Failed

@pohly pohly force-pushed the yocto-compat-2.0-enhancements branch from 9ea203f to 685d575 Compare June 14, 2017 06:41
@pohly
Copy link
Contributor Author

pohly commented Jun 14, 2017

The util-linux 2.30 upgrade from OE-core master-next was broken. Trying again without it.

retest this please

test this please

@pohly pohly closed this Jun 14, 2017
@pohly pohly reopened this Jun 14, 2017
@pohly
Copy link
Contributor Author

pohly commented Jun 14, 2017

retest this please

test this please

@pohly
Copy link
Contributor Author

pohly commented Jun 14, 2017

OE-core test branch hadn't been pushed.

retest this please

@pohly
Copy link
Contributor Author

pohly commented Jun 14, 2017

test this please

@pohly pohly force-pushed the yocto-compat-2.0-enhancements branch from 685d575 to 715bd5a Compare June 14, 2017 07:29
@pohly pohly force-pushed the yocto-compat-2.0-enhancements branch from 6e2cc0f to 5b68993 Compare June 15, 2017 13:35
@pohly pohly changed the title TEST BUILD: OE-core master + patch and bitbake master OE-core master update + Yocto Compatible 2.0 enhancements Jun 15, 2017
@pohly pohly removed the test-build label Jun 15, 2017
@pohly pohly force-pushed the yocto-compat-2.0-enhancements branch from 5b68993 to e014adf Compare June 15, 2017 13:40
@pohly pohly mentioned this pull request Jun 15, 2017
@pohly pohly force-pushed the yocto-compat-2.0-enhancements branch 2 times, most recently from 9b39234 to 02bb422 Compare June 16, 2017 10:14
@pohly
Copy link
Contributor Author

pohly commented Jun 16, 2017

Rebased onto current master, fixed ovmf (in OE-core), fixed upm (in refkit).

@pohly pohly force-pushed the yocto-compat-2.0-enhancements branch 4 times, most recently from 0bfc25e to 7ec73b2 Compare June 16, 2017 17:16
@pohly
Copy link
Contributor Author

pohly commented Jun 16, 2017

@mythi: do we really want packagegroup-sensors in refkit-image-common? It drags in mraa, upm, nodejs, and the latter is what currently fails in the "test_common_poky_config" test, because it builds refkit-image-common with the Poky distro config and thus gcc7, which is known to fail for nodejs.

We run an increasing number of tests (installer test, soon my system update tests) with refkit-image-common. Reducing the content of that image would speed up testing and local development.

@mythi
Copy link
Contributor

mythi commented Jun 17, 2017 via email

@mythi
Copy link
Contributor

mythi commented Jun 17, 2017 via email

The new "bbappends" hierarchy can be used to provide .bbappends for
recipes in layers which may or may not be present. This relies on the
BBFILES_DYNAMIC support in bitbake master.

The complete hierarchy for all layers that are currently used by the
refkit distro is set up in advance with empty directories where
.bbappends need to go, to make it easier for developers to add
.bbappends correctly.

.bbappends outside of the "bbappends" hierarchy are ignored.

While technically not required because OE-core is a hard dependency,
also hosting the .bbappends that modified OE-core under the new
"bbappends" hierarchy makes more sense (all modifications to other
layers in one place).

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
@pohly pohly force-pushed the yocto-compat-2.0-enhancements branch from 7ec73b2 to 2f70ecf Compare June 17, 2017 19:28
@pohly
Copy link
Contributor Author

pohly commented Jun 17, 2017

Rebased (again).

Instead of reducing the content of meta-refkit-common, I restricted the test_common_poky_config to OE-core + meta-refkit-core. That's the part that should work. Additional layers will only work when using our distro config, because that's where we enabled workarounds for known problems.

Reducing meta-refkit-common can and should still be done separately.

The upm + gcc7 incompatibility gets resolved via backporting a fix (problem still occurs while sensors are in meta-refkit-common). Someone more familiar with upm should do a major version upgrade.

@pohly
Copy link
Contributor Author

pohly commented Jun 17, 2017

@ipuustin can you please review and merge, if it passes?

Note that I have not done any manual checking whether I resolved conflicts with you recent firewall changes correctly. Extra care is needed to ensure that I moved them correctly into stand-alone .bbappends.

pohly added 10 commits June 17, 2017 23:25
Several config changes were in refkit-config.inc merely because we
couldn't use .bbappends for layers other than OE-core. Now we can
make the changes in optional .bbappends.

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
We had to use "inherit" to include optional changes. Now "require" is
also supported without parameter, so we can use that instead of
abusing the class inheritance mechanism.

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Cooperation of ConnMan and systemd is getting enhanced in OE-core
"systemd: enable resolved and networkd" by introducing additional
symlinks.

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Several components are known to have problems (for example, upm =
eclipse-upm/upm#573 and nodejs) with
gcc7. We have to continue using gcc 6.x until those problems have been
sorted out.

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
We cannot guarantee that all layers actually work with OE-core master
and its Poky distro. For example, when OE-core switched to gcc7,
nodejs and upm failed to build.

Therefore we now limit testing in that build configuration to OE-core
and meta-refkit-core, without any of the additional layers.

As an additional benefit, the test now ensures that meta-refkit-core
is actually usable for image building and booting when just using
OE-core.

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
gcc7 correctly errors out because the if check does not make
sense (using enums in boolean context).

An upm version update might also make sense; for now the goal is
to get the code ready for OE-core master with gcc7 with minimal
changes.

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
The new OE-core master has its own distrooverrides.bbclass with a
slightly different API (different variable for listing overrides, all
overrides have a df- for "distro feature" prefix).

* openembedded-core b51383e...1d9a88f (28):
  > ovmf: fix secureboot PACKAGECONFIG + OpenSSL update
  > package_ipk: Clean up Source entry in ipk packages
  > mtools-native: fix Upstream-Status
  > package_manager: flush installed_pkgs file before oe-pkgdata-util uses it
  > distrooverrides.bbclass: DISTRO_FEATURES as overrides
  > Revert "bitbake.conf: DISTRO_FEATURES as overrides"
  > python-3.5: Move bz2.py, lzma.py and _compression.py from python3-misc to python3-compression
  > attr: Fix build failure when image includes man-pages pkg
  > externalsrc: verify that EXTERNALSRC/EXTERNALSRC_BUILD are absolute paths
  > adwaita-icon-theme: add a patch to speed up the do_install() task
  > runqemu: change terminal settings for valid tty's
  > grub: switch from ftp to https
  > libid3tag: switch from ftp to http
  > libglu: switch from ftp to https
  > mesa: switch from ftp to https
  > mesa-demos: switch from ftp to https
  > dhcp: switch from ftp to http
  > libpcre: switch from ftp to https
  > image.bbclass: fix setting of vardeps flag
  > image.bbclass: improve setup of flags
  > image.bbclass: cleanup: add new variable "task"
  > staging.bbclass: fix typo
  > connman: correct the systemd boot in read only rootfs
  > systemd: enable resolved and networkd
  > selftest/sstatetests: Fix potential failure on uniprocessor machines
  > gcc_7.1.bb: disable thumb on armv5t
  > gcc: Add recipes for gcc-7
  > ovmf: Update to latest

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
The helper function makes the intention of the code more explicit.

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
The docstring is only special when it appears directly at
the start of the method body.

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
It is useful to run signature checks separately from the slower image
build tests. "bitbake -r refkit_poky" still runs both, as before, but
now refkit_poky.TestRefkitPokySignatures and
refkit_poky.TestRefkitPokyBuilds can be used to run those subsets of
the tests.

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
@@ -0,0 +1,4 @@
# Temporary workaround (needs to be fixed in meta-security once the
# necessary patch "net-tools: enable native and nativesdk variant"
# is in OE-core): swtpm_setup.sh needs netstat command.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Appears to be in oe-core (openembedded/openembedded-core@508163b). Can this be removed?

@pohly
Copy link
Contributor Author

pohly commented Jun 19, 2017 via email

layers[layer] = path
# meta-poky should not be active. We expect it next to the meta-refkit layer.
if not 'meta-poky' in layers:
layers['meta-poky'] = os.path.join(os.path.dirname(layers['meta-refkit']), 'meta-yocto', 'meta-poky')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should you check if the layer was actually there?

@pohly
Copy link
Contributor Author

pohly commented Jun 19, 2017 via email

@ipuustin ipuustin merged commit b937c96 into intel:master Jun 19, 2017
@pohly pohly mentioned this pull request Jun 19, 2017
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants