This changeset introduces a filter-based Robots API, providing central control over the `robots` meta tag.
* Introduces `wp_robots()` function which should be called anywhere a `robots` meta tag should be included.
* Introduces `wp_robots` filter which allows adding or modifying directives for the `robots` meta tag. The `wp_robots()` function is entirely filter-based, i.e. if no filter is added to `wp_robots`, no directives will be present, and therefore the entire `robots` meta tag will be omitted.
* Introduces the following `wp_robots` filter functions which replace similar existing functions that were manually rendering a `robots` meta tag:
* `wp_robots_noindex()` replaces `noindex()`, which has been deprecated.
* `wp_robots_no_robots()` replaces `wp_no_robots()`, which has been deprecated.
* `wp_robots_sensitive_page()` replaces `wp_sensitive_page_meta()`, which has been deprecated. Its rendering of the `referrer` meta tag has been moved to another new function `wp_strict_cross_origin_referrer()`.
Migration to the new functions is straightforward. For example, a call to `add_action( 'wp_head', 'wp_no_robots' )` should be replaced with `add_filter( 'wp_robots', 'wp_robots_no_robots' )`.
Plugins and themes that render their own `robots` meta tags are encouraged to switch to rely on the `wp_robots` filter in order to use the central management layer now provided by WordPress core.
Props adamsilverstein, flixos90, timothyblynjacobs, westonruter.
See #51511.
Built from https://develop.svn.wordpress.org/trunk@49992
git-svn-id: http://core.svn.wordpress.org/trunk@49693 1a063a9b-81f0-0310-95a4-ce76da25c4cd
