Any messages to the user which are echo-ed out in the test bootstrap will generally display on a command-line interface.
The *nix specific `"\n"` line ending will be ignored on Windows, making the messages less readable.
For new lines in CLI messages, `PHP_EOL` should be used instead.
This was already done in a few places in the script, but not consistently so. Fixed now.
Follow-up to [UT882], [UT890], [44723], [45020], [48592], [49535], [51560].
Props jrf.
See #53363.
Built from https://develop.svn.wordpress.org/trunk@51581
git-svn-id: http://core.svn.wordpress.org/trunk@51192 1a063a9b-81f0-0310-95a4-ce76da25c4cd
This replaces instances of `assertTrue( is_dir( ... ) )` with `assertDirectoryExists()` to use native PHPUnit functionality.
The `assertDirectoryExists()` method was introduced in PHPUnit 5.6. As the minimum supported PHPUnit version has been raised to PHPUnit 5.7.21, it can now be used.
Follow-up to [51543], [51574], [51579].
Props jrf.
See #53363.
Built from https://develop.svn.wordpress.org/trunk@51580
git-svn-id: http://core.svn.wordpress.org/trunk@51191 1a063a9b-81f0-0310-95a4-ce76da25c4cd
This replaces instances of `assertTrue( is_file( ... ) )` followed by `assertTrue( is_readable( ... ) )` with `assertFileIsReadable()` to use native PHPUnit functionality.
The `assertFileIsReadable()` method was introduced in PHPUnit 5.6. As the minimum supported PHPUnit version has been raised to PHPUnit 5.7.21, it can now be used.
Follow-up to [51543], [51574].
Props jrf.
See #53363.
Built from https://develop.svn.wordpress.org/trunk@51579
git-svn-id: http://core.svn.wordpress.org/trunk@51190 1a063a9b-81f0-0310-95a4-ce76da25c4cd
**composer.json**:
Remove the PHPUnit dependency in favor of allowing the PHPUnit Polyfills library to manage the supported PHPUnit version. This automatically now widens the supported PHPUnit versions to 5.7.21 to 9.5.8 (current).
Letting the PHPUnit Polyfills handle the version constraints for PHPUnit prevents potential version conflicts in the future, as well as allows WordPress to benefit straight away when a new PHPUnit version would be released and the PHPUnit Polyfills package adds support for that PHPUnit version.
**Test Bootstrap**
Update the supported version number for PHPUnit 5.x, as the minimum PHPUnit 5.x version supported by the PHPUnit Polyfills is PHPUnit 5.7.21, and remove the PHPUnit maximum.
**.gitignore and svn:ignore:**
Add the PHPUnit cache file to the list of files to be ignored.
Since PHPUnit 8, PHPUnit has a built-in caching feature which creates a `.phpunit.result.cache` file. This file should not be committed.
Follow-up to [40536], [40853], [44701], [51559-51573].
Props jrf.
See #46149.
Built from https://develop.svn.wordpress.org/trunk@51574
git-svn-id: http://core.svn.wordpress.org/trunk@51185 1a063a9b-81f0-0310-95a4-ce76da25c4cd
Now that the tests can run PHPUnit cross-version and Composer will be used to install the test suite in CI, we could switch out the local copies of the [https://github.com/johnkary/phpunit-speedtrap PHPUnit speedtrap] package in favor of using the Composer package, which would prevent us having to make the WP local copies of the class compatible with later PHPUnit versions.
The SpeedTrap test listener was introduced to identify slow tests and take action on these to make them faster.
In practice, however, no notable action was ever taken based on the output of the test listener in all the years it was in place.
With that in mind, it was decided to remove the SpeedTrap test listeners without replacement.
If – at a future date – contributors would want to take action to speed up slow tests anyway, they can:
* Either add the package to their local install and use the output they receive locally to identify slow tests.
* Or use the PHPUnit native `@small` annotations in combination with the PHPUnit `PHP_Invoker` package as described in the PHPUnit documentation to [https://phpunit.readthedocs.io/en/stable/risky-tests.html#test-execution-timeout run tests with time limits].
Follow-up to [35214], [35226], [35767], [44701], [51559-51572].
Props jrf.
See #46149.
Built from https://develop.svn.wordpress.org/trunk@51573
git-svn-id: http://core.svn.wordpress.org/trunk@51184 1a063a9b-81f0-0310-95a4-ce76da25c4cd
The PHPUnit native `TestCase::getAnnotations()` method is used to check for WP flavored deprecation notices, however, this method was not covered by the backward compatibility promise for PHPUnit (and was annotated as excluded).
The method has been removed as part of an internal refactor in commit [68582043e1sebastianbergmann/phpunit@6858204], which is included in PHPUnit 9.5.0.
For now, a workaround is put in place, but it is recommended that the WP `expectDeprecated()` method should be reevaluated in a future iteration.
Follow-up to [51559-51571].
Props jrf.
See #46149.
Built from https://develop.svn.wordpress.org/trunk@51572
git-svn-id: http://core.svn.wordpress.org/trunk@51183 1a063a9b-81f0-0310-95a4-ce76da25c4cd
Most of the aliasing in this `compat.php` file is redundant as PHPUnit 5.7.21+ contains a forward compatibility layer for these classes anyway (= PHPUnit provides both the namespaced and underscore named versions of these classes in PHPUnit 5.7.21+).
All the same, the file and the aliases are left in place for the time being, as plugins/themes using the WP test suite as the basis for their integration tests may rely on it, though WP itself should not really need it anymore, save for maybe one or two classes.
Follow-up to [51559-51569].
Props jrf.
See #46149.
Built from https://develop.svn.wordpress.org/trunk@51570
git-svn-id: http://core.svn.wordpress.org/trunk@51181 1a063a9b-81f0-0310-95a4-ce76da25c4cd
> PHPUnit 8.0.0 introduced a `void` return type declaration to the "fixture" methods – `setUpBeforeClass()`, `setUp()`, `tearDown()` and `tearDownAfterClass()`. As the `void` return type was not introduced until PHP 7.1, this makes it more difficult to create cross-version compatible tests when using fixtures, due to signature mismatches.
>
> The `Yoast\PHPUnitPolyfills\TestCases\TestCase` overcomes the signature mismatch by having two versions. The correct one will be loaded depending on the PHPUnit version being used.
>
> When using this TestCase, if an individual test, or another TestCase which extends this TestCase, needs to overload any of the "fixture" methods, it should do so by using a snake_case variant of the original fixture method name, i.e. `set_up_before_class()`, `set_up()`, `assert_pre_conditions()`, `assert_post_conditions()`, `tear_down()`, and `tear_down_after_class()`.
>
> The snake_case methods will automatically be called by PHPUnit.
>
> > IMPORTANT: The snake_case methods should not call the PHPUnit parent, i.e. do not use `parent::setUp()` from within an overloaded `set_up()` method. If necessary, DO call `parent::set_up()`.
Reference: https://github.com/Yoast/PHPUnit-Polyfills#testcases
This commit renames all declared fixture methods, and calls to parent versions of those fixture methods, from camelCase to snake_case.
Follow-up to [51559-51567].
Props jrf, hellofromTonya, johnbillion, netweb, dd32, pputzer, SergeyBiryukov.
See #46149.
Built from https://develop.svn.wordpress.org/trunk@51568
git-svn-id: http://core.svn.wordpress.org/trunk@51179 1a063a9b-81f0-0310-95a4-ce76da25c4cd
> PHPUnit 8.0.0 introduced a `void` return type declaration to the "fixture" methods – `setUpBeforeClass()`, `setUp()`, `tearDown()` and `tearDownAfterClass()`. As the `void` return type was not introduced until PHP 7.1, this makes it more difficult to create cross-version compatible tests when using fixtures, due to signature mismatches.
>
> The `Yoast\PHPUnitPolyfills\TestCases\TestCase` overcomes the signature mismatch by having two versions. The correct one will be loaded depending on the PHPUnit version being used.
>
> When using this TestCase, if an individual test, or another TestCase which extends this TestCase, needs to overload any of the "fixture" methods, it should do so by using a snake_case variant of the original fixture method name, i.e. `set_up_before_class()`, `set_up()`, `assert_pre_conditions()`, `assert_post_conditions()`, `tear_down()`, and `tear_down_after_class()`.
>
> The snake_case methods will automatically be called by PHPUnit.
>
> > IMPORTANT: The snake_case methods should not call the PHPUnit parent, i.e. do not use `parent::setUp()` from within an overloaded `set_up()` method. If necessary, DO call `parent::set_up()`.
Reference: https://github.com/Yoast/PHPUnit-Polyfills#testcases
This commit:
* Lets the `PHPUnit_Adapter_TestCase` extend the `Yoast\PHPUnitPolyfills\TestCases\TestCase`, which makes this solution for the `void` return type available to the WordPress test suite.
* Removes the individual import and trait `use` statements for the Polyfill traits. These are no longer necessary as the `Yoast\PHPUnitPolyfills\TestCases\TestCase` already includes those.
Follow-up to [51559-51566].
Props jrf, hellofromTonya, johnbillion, netweb, SergeyBiryukov.
See #46149.
Built from https://develop.svn.wordpress.org/trunk@51567
git-svn-id: http://core.svn.wordpress.org/trunk@51178 1a063a9b-81f0-0310-95a4-ce76da25c4cd
The old manner of testing these is soft deprecated as of PHPUnit 8.4, hard deprecated as of PHPUnit 9.0 and will be removed in PHPUnit 10.0.
These dedicated methods introduced in PHPUnit 8.4 should be used as an alternative:
* `expectDeprecation()`
* `expectDeprecationMessage()`
* `expectDeprecationMessageMatches()`
* `expectNotice()`
* `expectNoticeMessage()`
* `expectNoticeMessageMatches()`
* `expectWarning()`
* `expectWarningMessage()`
* `expectWarningMessageMatches()`
* `expectError()`
* `expectErrorMessage()`
* `expectErrorMessageMatches()`
These new PHPUnit methods are all polyfilled by the PHPUnit Polyfills and switching to these will future-proof the tests some more.
References:
* https://github.com/sebastianbergmann/phpunit/blob/8.4.3/ChangeLog-8.4.md#840---2019-10-04
* https://github.com/sebastianbergmann/phpunit/issues/3775
Follow-up to [51559-51562].
Props jrf.
See #46149.
Built from https://develop.svn.wordpress.org/trunk@51563
git-svn-id: http://core.svn.wordpress.org/trunk@51174 1a063a9b-81f0-0310-95a4-ce76da25c4cd
PHPUnit 6 deprecated the `setExpectedException()` method in favor of the `expectException()`, `expectExceptionMessage()`, and `expectExceptionCode()` methods.
`WP_UnitTestCase_Base::setExpectedException()` backfilled the old method. As the PHPUnit Polyfills have a polyfill for the ''new'' method, this backfill can now be simplified.
This backfill ''should'' be removed in a future iteration, but is, for now, left in place so as not to break backward compatibility for plugin/theme test suites which extend the WP native test suite for their integration tests.
Follow-up to [48996], [48997], [51559-51561].
Props jrf.
See #46149.
Built from https://develop.svn.wordpress.org/trunk@51562
git-svn-id: http://core.svn.wordpress.org/trunk@51173 1a063a9b-81f0-0310-95a4-ce76da25c4cd
As things were, the inheritance order of the abstract test classes was as follows:
{{{
WP_UnitTestCase (PHPUnit adapter layer)
extends WP_UnitTestCase_Base (base test class)
extends PHPUnit\Framework\TestCase (PHPUnit native class)
}}}
Concrete (child) test classes, as well as more specific abstract TestCases, are/were expected to extend the `WP_UnitTestCase`.
This order is not optimal as it means that the `WP_UnitTestCase_Base` class would not be able to benefit from any polyfills and/or shims in the PHPUnit adapter layer.
With that in mind, this commit changes the inheritance to:
{{{
WP_UnitTestCase (empty class, left in place to not break BC for plugin/theme integration tests)
extends WP_UnitTestCase_Base (base test class)
extends PHPUnit_Adapter_TestCase (PHPUnit adapter layer)
extends PHPUnit\Framework\TestCase (PHPUnit native class)
}}}
The new order allows for the `WP_UnitTestCase_Base` to also benefit from the PHPUnit adapter layer.
For backward compatibility reasons the `WP_UnitTestCase`, which all test classes are (were) expected to extend, is left in place, though it is now an empty class and explicitly `abstract`.
Follow-up to [51559], [51560].
Props jrf, hellofromTonya, johnbillion, netweb, SergeyBiryukov.
See #46149.
Built from https://develop.svn.wordpress.org/trunk@51561
git-svn-id: http://core.svn.wordpress.org/trunk@51172 1a063a9b-81f0-0310-95a4-ce76da25c4cd
This commit:
* Removes the PHPUnit 7 specific `TestCase`.
* Removes all existing polyfills from the PHPUnit 5.x `TestCase`.
* Imports all polyfill traits from the PHPUnit Polyfills package into the `WP_UnitTestCase` class and updates the DocBlock to reflect the actual function of the class.
* Note: The list of polyfills needs to be verified and updated after each new release of the PHPUnit Polyfills package. Alternatively (recommended), one of the built-in `TestCase` classes from the PHPUnit Polyfills package can be used instead.
* Moves the `require` for the WP `abstract-testcase.php` to the `bootstrap.php` file.
* Adds a `require_once` for the PHPUnit Polyfills autoloader to the `bootstrap.php` file.
* Note: while this isn't _strictly_ necessary when the tests are run via Composer, having the include in the bootstrap allows for the tests to also be run via a PHPUnit Phar, providing contributors with more flexibility.
Follow-up to [51559].
Props jrf, hellofromTonya, johnbillion, netweb, SergeyBiryukov.
See #46149.
Built from https://develop.svn.wordpress.org/trunk@51560
git-svn-id: http://core.svn.wordpress.org/trunk@51171 1a063a9b-81f0-0310-95a4-ce76da25c4cd
The PHPUnit Polyfills package is an add-on for PHPUnit, which works around common issues for writing PHPUnit cross-version compatible tests.
Features:
* It offers a full set of polyfills for assertions and expectations introduced in PHPUnit since PHPUnit 4.8.
* It offers two generic TestCases which include these polyfills, but also solve the `void` return type issue for the fixtures methods.
* It offers a PHPUnit cross-version solution for the changes to the PHPUnit `TestListener` implementation.
* Supports PHPUnit 4.8 – current.
* Supports and is compatible with PHP 5.4 – current.
The package has no outside dependencies, other than PHPUnit, is actively maintained and endorsed by the maintainer of PHPUnit itself (the only package of its kind which has ever been endorsed).
Props jrf, hellofromTonya, johnbillion, netweb, SergeyBiryukov.
See #46149.
Built from https://develop.svn.wordpress.org/trunk@51559
git-svn-id: http://core.svn.wordpress.org/trunk@51170 1a063a9b-81f0-0310-95a4-ce76da25c4cd
Since branch filtering happens prior to the workflow run being created, filtering the branches that `workflow_run` will fire on for this workflow should cut down on the number of skipped “Slack Notifications” runs listed in the Actions section of the repository.
This also removes the check for a `skipped` outcome in the requesting workflow. Workflows for push events resulting from WordPress Core commits are never skipped.
See #52644.
Built from https://develop.svn.wordpress.org/trunk@51558
git-svn-id: http://core.svn.wordpress.org/trunk@51169 1a063a9b-81f0-0310-95a4-ce76da25c4cd
This updates the conditional check for the Slack notifications workflow to check the event of the original workflow and not the one requested through the `workflow_run:completed` event.
The run triggering the event is accessible at `github.event.workflow_run`.
Props Clorith.
See #52644.
Built from https://develop.svn.wordpress.org/trunk@51555
git-svn-id: http://core.svn.wordpress.org/trunk@51166 1a063a9b-81f0-0310-95a4-ce76da25c4cd
Previously the tests were run via a PHPUnit Phar file for PHP 5.6–7.4, with PHP 8.0 using a Composer-installed version of PHPUnit.
Running the tests via a Phar without the need for a `composer install` is (marginally) faster in overall build time, however, this commit is part of a larger chain of changes which will make the test suite PHPUnit cross-version compatible.
With an eye on those upcoming changes, which will allow us to run the tests on the most appropriate PHPUnit version for each supported PHP version, it is opportune to switch to using a Composer-installed version of PHPUnit for all PHP versions supported by WordPress. Previously this was not possible without additional conditional `update` commands being run, due to the `composer.lock` file being in place and being locked at PHPUnit 7.5.20.
Switching over to using the Composer-installed PHPUnit version, with that PHPUnit version adjusting based on the PHP version, allows for some minor simplifications in the GitHub Actions script.
This means we need additional measures to make sure that the Composer cache file does not go too far out of date as that would significantly slow down the builds.
By adding a "Last Monday" date to the cache key, in combination with the pre-existing OS, PHP version and the hash of the `composer.json` file, we can guarantee that:
1. There will be a cache created for each OS/PHP combination.
2. These caches will be replaced whenever a change is made to the `composer.json` file.
3. These caches will be replaced every Monday of each week ensuring that the cache file does not go too far out of date.
Note: The NPM script `test:php` is now no longer needed during the builds. However, to prevent breaking the workflow of contributors who may be used to having the command available, the command remains available.
In a future iteration we may be able to replace the caching of the Composer dependencies with the Composer cache action as offered on the GitHub marketplace, which would further simplify the script.
Follow-up to [42960], [46290], [47881], [48957], [49037], [51543], [51544].
Props jrf, desrosj.
Fixes#47381.
Built from https://develop.svn.wordpress.org/trunk@51545
git-svn-id: http://core.svn.wordpress.org/trunk@51156 1a063a9b-81f0-0310-95a4-ce76da25c4cd
This prevents the classes from being loaded automatically via the `autoload-dev` directives when a Composer-installed PHPUnit 5.x or 6.x version is used, as that would break the test run.
It is expected that this autoloader will be removed soon, as it should no longer be needed when the PHPUnit version constraints are widened.
Notes:
* The autoloader file will be loaded from the Test bootstrap.
* The autoloader will always be registered and directed to queue itself _before_ the Composer autoload file (which will already have been registered).
* The autoloader will only actually load the WP copies of the files/classes when PHP 8.0 in combination with PHPUnit 7.x is detected. In all other cases, the autoloader will bow out, which effectively then defers to the Composer autoload file to load the files as shipped with the installed PHPUnit version.
Follow-up to [48957], [49037], [51543].
Props jrf.
See #47381.
Built from https://develop.svn.wordpress.org/trunk@51544
git-svn-id: http://core.svn.wordpress.org/trunk@51155 1a063a9b-81f0-0310-95a4-ce76da25c4cd
This makes it easier to run unit tests against multiple different PHP versions.
There is currently no reason to have a `composer.lock` file as:
* External runtime dependencies are not managed via Composer.
* Managed updates of the non-runtime dependencies can be done by locking the version used in the `composer.json` file to a precise version instead of using a `composer.lock` file.
* Having the `composer.lock` file in place makes it a lot more difficult to run the tests against all supported PHP versions.
With these considerations in mind, the lock file is now removed from version control and added to `.gitignore` and `svn:ignore`.
Version constraints for the current dev dependencies are adjusted accordingly:
* PHPUnit now explicitly declares in its version constraints that PHPUnit 5.x, 6.x, and 7.x are supported. The minimum supported version for PHPUnit 5.x has been raised from 5.4 to 5.7, which in practice was already the version used for running the tests on PHP 5.6.
* PHPCompatibilityWP is effectively updated to version 2.1.2 with the positive impact that a few new constants polyfilled in WP 5.8 are now accounted for (excluded from being flagged).
* PHP_CodeSniffer is declared as an explicit dependency to ensure that updates to it will always be explicitly managed instead of inherited.
* The DealerDirect Composer plugin is effectively updated to version 0.7.1 without impact.
Follow-up to [42960], [46290], [47881], [48957].
Props jrf, johnbillion, desrosj, ayeshrajans, aristath, hellofromTonya, SergeyBiryukov.
See #47381.
Built from https://develop.svn.wordpress.org/trunk@51543
git-svn-id: http://core.svn.wordpress.org/trunk@51154 1a063a9b-81f0-0310-95a4-ce76da25c4cd
Add a check to `wp_add_iframed_editor_assets_html()` confirming the edit post screen is using the block-editor before including block-editor specific JavaScript. For the classic and other editors the function returns early without any output.
Props swissspidy, desrosj.
Fixes#53696.
Built from https://develop.svn.wordpress.org/trunk@51540
git-svn-id: http://core.svn.wordpress.org/trunk@51151 1a063a9b-81f0-0310-95a4-ce76da25c4cd
Prevent the bulk-select option from displaying when adding a new menu. This also prevents the option from displaying when an administrator first visits the menu page and no menus are set.
Props dlh, sabernhardt.
Fixes#53654.
Built from https://develop.svn.wordpress.org/trunk@51539
git-svn-id: http://core.svn.wordpress.org/trunk@51150 1a063a9b-81f0-0310-95a4-ce76da25c4cd
This expands Slack notifications to include success, cancelled, and “fixed” GitHub Action workflow run outcomes in addition to failures.
A “fixed” outcome occurs when the previous run for a workflow failed and the current one succeeds. This matches the behavior that was native to TravisCI by setting `on_success` for notifications to `change`.
The message details and where each outcome is posted is controlled by Slack workflows.
The Slack notification logic has also been pulled into a separate workflow to prevent repeating code in every workflow.
See #52644.
Built from https://develop.svn.wordpress.org/trunk@51535
git-svn-id: http://core.svn.wordpress.org/trunk@51146 1a063a9b-81f0-0310-95a4-ce76da25c4cd
This fixes the "Deprecated: Return type of `WP_Block_List::[METHODNAME]()` should be compatible with `ArrayAccess::[METHODNAME](): type`" warnings on PHP 8.1.
PHP native interfaces now have declared return types and methods in classes implementing these interfaces need to either have the return type declared (in a covariant compatible manner with the PHP native interface method declaration), or need to silence the deprecation warning using the `#[ReturnTypeWillChange]` attribute.
Follow-up to [51517], [51529], [51530], [51531].
Props jrf.
See #53635.
Built from https://develop.svn.wordpress.org/trunk@51532
git-svn-id: http://core.svn.wordpress.org/trunk@51143 1a063a9b-81f0-0310-95a4-ce76da25c4cd
This fixes the "Deprecated: Return type of `WP_REST_Request::[METHODNAME]($offset)` should be compatible with `ArrayAccess::[METHODNAME](): type`" warnings on PHP 8.1.
PHP native interfaces now have declared return types and methods in classes implementing these interfaces need to either have the return type declared (in a covariant compatible manner with the PHP native interface method declaration), or need to silence the deprecation warning using the `#[ReturnTypeWillChange]` attribute.
Follow-up to [51517], [51529], [51530].
Props jrf.
See #53635.
Built from https://develop.svn.wordpress.org/trunk@51531
git-svn-id: http://core.svn.wordpress.org/trunk@51142 1a063a9b-81f0-0310-95a4-ce76da25c4cd
This fixes the "Deprecated: Return type of `WP_Hook::[METHODNAME]()` should be compatible with `ArrayAccess::[METHODNAME](): type`" warnings on PHP 8.1.
PHP native interfaces now have declared return types and methods in classes implementing these interfaces need to either have the return type declared (in a covariant compatible manner with the PHP native interface method declaration), or need to silence the deprecation warning using the `#[ReturnTypeWillChange]` attribute.
Follow-up to [51517], [51529].
Props jrf.
See #53635.
Built from https://develop.svn.wordpress.org/trunk@51530
git-svn-id: http://core.svn.wordpress.org/trunk@51141 1a063a9b-81f0-0310-95a4-ce76da25c4cd
