Upstream
/
WordPress
mirror of https://github.com/WordPress/WordPress.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

More performance improvements to metadata lazyloading. 

Comment and term meta lazyloading for `WP_Query` loops, introduced in 4.4,
depended on filter callback methods belonging to `WP_Query` objects. This meant
storing `WP_Query` objects in the `$wp_filter` global (via `add_filter()`),
requiring that PHP retain the objects in memory, even when the local variables
would typically be expunged during normal garbage collection. In cases where a
large number of `WP_Query` objects were instantiated on a single pageload,
and/or where the contents of the `WP_Query` objects were quite large, serious
performance issues could result.

We skirt this problem by moving metadata lazyloading out of `WP_Query`. The
new `WP_Metadata_Lazyloader` class acts as a lazyload queue. Query instances
register items whose metadata should be lazyloaded - such as post terms, or
comments - and a `WP_Metadata_Lazyloader` method will intercept comment and
term meta requests to perform the cache priming. Since `WP_Metadata_Lazyloader`
instances are far smaller than `WP_Query` (containing only object IDs), and
clean up after themselves far better than the previous `WP_Query` methods (bp
only running their callbacks a single time for a given set of queued objects),
the resource use is decreased dramatically.

See [36525] for an earlier step in this direction.

Props lpawlik, stevegrunwell, boonebgorges.
Fixes #35816.
Built from https://develop.svn.wordpress.org/trunk@36566


git-svn-id: http://core.svn.wordpress.org/trunk@36533 1a063a9b-81f0-0310-95a4-ce76da25c4cd
This commit is contained in:
Boone Gorges 2016-02-17 22:58:26 +00:00
parent da5b3a55c4
commit 571e14f897
8 changed files with 263 additions and 96 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

164
wp-includes/class-wp-metadata-lazyloader.php Normal file
View File

@ -0,0 +1,164 @@
<?php
/**
* Lazyloader for object metadata.
*
* When loading many objects of a given type, such as posts in a WP_Query loop, it often makes
* sense to prime various metadata caches at the beginning of the loop. This means fetching all
* relevant metadata with a single database query, a technique that has the potential to improve
* performance dramatically in some cases.
*
* In cases where the given metadata may not even be used in the loop, we can improve performance
* even more by only priming the metadata cache for affected items the first time a piece of metadata
* is requested - ie, by lazyloading it. So, for example, comment meta may not be loaded into the
* cache in the comments section of a post until the first time get_comment_meta() is called in the
* context of the comment loop.
*
* WP uses the WP_Metadata_Lazyloader class to queue objects for metadata cache priming. The class
* then detects the relevant get_*_meta() function call, and queries the metadata of all queued objects.
*
* Do not access this class directly. Use the wp_metadata_lazyloader() function.
*
* @since 4.5.0
*/
class WP_Metadata_Lazyloader {
/**
* Pending objects queue.
*
* @since 4.5.0
* @var array
*/
protected $pending_objects;
/**
* Settings for supported object types.
*
* @since 4.5.0
* @var array
*/
protected $settings = array();
/**
* Constructor.
*
* @since 4.5.0
*/
public function __construct() {
$this->settings = array(
'term' => array(
'filter' => 'get_term_metadata',
'callback' => array( $this, 'lazyload_term_meta' ),
),
'comment' => array(
'filter' => 'get_comment_metadata',
'callback' => array( $this, 'lazyload_comment_meta' ),
),
);
}
/**
* Add objects to the metadata lazyload queue.
*
* @since 4.5.0
*
* @param string $object_type Type of object whose meta is to be lazyloaded. Accepts 'term' or 'comment'.
* @param array $object_ids Array of object IDs.
* @return bool|WP_Error True on success, WP_Error on failure.
*/
public function queue_objects( $object_type, $object_ids ) {
if ( ! isset( $this->settings[ $object_type ] ) ) {
return new WP_Error( 'invalid_object_type', __( 'Invalid object type' ) );
}
$type_settings = $this->settings[ $object_type ];
if ( ! isset( $this->pending_objects[ $object_type ] ) ) {
$this->pending_objects[ $object_type ] = array();
}
foreach ( $object_ids as $object_id ) {
// Keyed by ID for faster lookup.
if ( ! isset( $this->pending_objects[ $object_type ][ $object_id ] ) ) {
$this->pending_objects[ $object_type ][ $object_id ] = 1;
}
}
add_filter( $type_settings['filter'], $type_settings['callback'] );
/**
* Fires after objects are added to the metadata lazyload queue.
*
* @since 4.5.0
*
* @param array $object_ids Object IDs.
* @param string $object_type Type of object being queued.
* @param WP_Metadata_Lazyloader $lazyloader The lazyloader object.
*/
do_action( 'metadata_lazyloader_queued_objects', $object_ids, $object_type, $this );
}
/**
* Reset lazyload queue for a given object type.
*
* @since 4.5.0
*
* @param string $object_type Object type. Accepts 'comment' or 'term'.
* @return bool|WP_Error True on success, WP_Error on failure.
*/
public function reset_queue( $object_type ) {
if ( ! isset( $this->settings[ $object_type ] ) ) {
return new WP_Error( 'invalid_object_type', __( 'Invalid object type' ) );
}
$type_settings = $this->settings[ $object_type ];
$this->pending_objects[ $object_type ] = array();
remove_filter( $type_settings['filter'], $type_settings['callback'] );
}
/**
* Lazyloads term meta for queued terms.
*
* This method is public so that it can be used as a filter callback. As a rule, there
* is no need to invoke it directly.
*
* @since 4.5.0
* @access public
*
* @param mixed $check The `$check` param passed from the 'get_term_metadata' hook.
* @return mixed In order not to short-circuit `get_metadata()`. Generally, this is `null`, but it could be
* another value if filtered by a plugin.
*/
public function lazyload_term_meta( $check ) {
if ( ! empty( $this->pending_objects['term'] ) ) {
update_termmeta_cache( array_keys( $this->pending_objects['term'] ) );
// No need to run again for this set of terms.
$this->reset_queue( 'term' );
}
return $check;
}
/**
* Lazyload comment meta for queued comments.
*
* This method is public so that it can be used as a filter callback. As a rule, there is no need to invoke it
* directly, from either inside or outside the `WP_Query` object.
*
* @since 4.5.0
*
* @param mixed $check The `$check` param passed from the 'get_comment_metadata' hook.
* @return mixed The original value of `$check`, so as not to short-circuit `get_comment_metadata()`.
*/
public function lazyload_comment_meta( $check ) {
if ( ! empty( $this->pending_objects['comment'] ) ) {
update_meta_cache( 'comment', array_keys( $this->pending_objects['comment'] ) );
// No need to run again for this set of comments.
$this->reset_queue( 'comment' );
}
return $check;
}
}

5
wp-includes/comment-template.php
View File

@ -1393,9 +1393,6 @@ function comments_template( $file = '/comments.php', $separate_comments = false
*/
$wp_query->comments = apply_filters( 'comments_array', $comments_flat, $post->ID );
// Set up lazy-loading for comment metadata.
add_action( 'get_comment_metadata', array( $wp_query, 'lazyload_comment_meta' ), 10, 2 );
$comments = &$wp_query->comments;
$wp_query->comment_count = count($wp_query->comments);
$wp_query->max_num_comment_pages = $comment_query->max_num_pages;
@ -2030,6 +2027,8 @@ function wp_list_comments( $args = array(), $comments = null ) {
if ( null === $r['reverse_top_level'] )
$r['reverse_top_level'] = ( 'desc' == get_option('comment_order') );
wp_queue_comments_for_comment_meta_lazyload( $_comments );
if ( empty( $r['walker'] ) ) {
$walker = new Walker_Comment;
} else {

24
wp-includes/comment.php
View File

@ -468,6 +468,30 @@ function update_comment_meta($comment_id, $meta_key, $meta_value, $prev_value =
return update_metadata('comment', $comment_id, $meta_key, $meta_value, $prev_value);
}
/**
* Queue comments for metadata lazyloading.
*
* @since 4.5.0
*
* @param array $comments Array of comment objects.
*/
function wp_queue_comments_for_comment_meta_lazyload( $comments ) {
// Don't use `wp_list_pluck()` to avoid by-reference manipulation.
$comment_ids = array();
if ( is_array( $comments ) ) {
foreach ( $comments as $comment ) {
if ( $comment instanceof WP_Comment ) {
$comment_ids[] = $comment->comment_ID;
}
}
}
if ( $comment_ids ) {
$lazyloader = wp_metadata_lazyloader();
$lazyloader->queue_objects( 'comment', $comment_ids );
}
}
/**
* Sets the cookies used to store an unauthenticated commentator's identity. Typically used
* to recall previous comments by this commentator that are still held in moderation.

17
wp-includes/meta.php
View File

@ -851,6 +851,23 @@ function update_meta_cache($meta_type, $object_ids) {
return $cache;
}
/**
* Get the metadata lazyloading queue.
*
* @since 4.5.0
*
* @return WP_Metadata_Lazyloader $lazyloader Metadata lazyloader queue.
*/
function wp_metadata_lazyloader() {
static $wp_metadata_lazyloader;
if ( null === $wp_metadata_lazyloader ) {
$wp_metadata_lazyloader = new WP_Metadata_Lazyloader();
}
return $wp_metadata_lazyloader;
}
/**
* Given a meta query, generates SQL clauses to be appended to a main query.
*

37
wp-includes/post.php
View File

@ -5947,6 +5947,43 @@ function wp_delete_auto_drafts() {
}
}
/**
* Queue posts for lazyloading of term meta.
*
* @since 4.5.0
*
* @param array $posts Array of WP_Post objects.
*/
function wp_queue_posts_for_term_meta_lazyload( $posts ) {
$post_type_taxonomies = $term_ids = array();
foreach ( $posts as $post ) {
if ( ! ( $post instanceof WP_Post ) ) {
continue;
}
if ( ! isset( $post_type_taxonomies[ $post->post_type ] ) ) {
$post_type_taxonomies[ $post->post_type ] = get_object_taxonomies( $post->post_type );
}
foreach ( $post_type_taxonomies[ $post->post_type ] as $taxonomy ) {
// Term cache should already be primed by `update_post_term_cache()`.
$terms = get_object_term_cache( $post->ID, $taxonomy );
if ( false !== $terms ) {
foreach ( $terms as $term ) {
if ( ! isset( $term_ids[ $term->term_id ] ) ) {
$term_ids[] = $term->term_id;
}
}
}
}
}
if ( $term_ids ) {
$lazyloader = wp_metadata_lazyloader();
$lazyloader->queue_objects( 'term', $term_ids );
}
}
/**
* Update the custom taxonomies' term counts when a post's status is changed.
*

109
wp-includes/query.php
View File

@ -3605,11 +3605,6 @@ class WP_Query {
if ( $this->posts )
$this->posts = array_map( 'get_post', $this->posts );
if ( $q['update_post_term_cache'] ) {
add_filter( 'get_term_metadata', array( $this, 'lazyload_term_meta' ), 10, 2 );
}
if ( ! $q['suppress_filters'] ) {
/**
* Filter the raw post results array, prior to status checks.
@ -3738,7 +3733,7 @@ class WP_Query {
// If comments have been fetched as part of the query, make sure comment meta lazy-loading is set up.
if ( ! empty( $this->comments ) ) {
add_filter( 'get_comment_metadata', array( $this, 'lazyload_comment_meta' ), 10, 2 );
wp_queue_comments_for_comment_meta_lazyload( $this->comments );
}
if ( ! $q['suppress_filters'] ) {
@ -3770,6 +3765,10 @@ class WP_Query {
$this->posts = array();
}
if ( $q['update_post_term_cache'] ) {
wp_queue_posts_for_term_meta_lazyload( $this->posts );
}
return $this->posts;
}
@ -4834,106 +4833,32 @@ class WP_Query {
}
/**
* Lazy-loads termmeta for located posts.
*
* As a rule, term queries (`get_terms()` and `wp_get_object_terms()`) prime the metadata cache for matched
* terms by default. However, this can cause a slight performance penalty, especially when that metadata is
* not actually used. In the context of a `WP_Query` instance, we're able to avoid this potential penalty.
* `update_object_term_cache()`, called from `update_post_caches()`, does not 'update_term_meta_cache'.
* Instead, the first time `get_term_meta()` is called from within a `WP_Query` loop, the current method
* detects the fact, and then primes the metadata cache for all terms attached to all posts in the loop,
* with a single database query.
*
* This method is public so that it can be used as a filter callback. As a rule, there is no need to invoke it
* directly, from either inside or outside the `WP_Query` object.
* Lazyload term meta for posts in the loop.
*
* @since 4.4.0
* @access public
* @deprecated 4.5.0 See wp_queue_posts_for_term_meta_lazyload().
*
* @param mixed $check The `$check` param passed from the 'get_term_metadata' hook.
* @param int $term_id ID of the term whose metadata is being cached.
* @return mixed In order not to short-circuit `get_metadata()`. Generally, this is `null`, but it could be
* another value if filtered by a plugin.
* @param mixed $check
* @param int $term_id
* @return mixed
*/
public function lazyload_term_meta( $check, $term_id ) {
// We can only lazyload if the entire post object is present.
$posts = array();
foreach ( $this->posts as $post ) {
if ( $post instanceof WP_Post ) {
$posts[] = $post;
}
}
if ( ! empty( $posts ) ) {
// Fetch cached term_ids for each post. Keyed by term_id for faster lookup.
$term_ids = array();
foreach ( $posts as $post ) {
$taxonomies = get_object_taxonomies( $post->post_type );
foreach ( $taxonomies as $taxonomy ) {
// Term cache should already be primed by 'update_post_term_cache'.
$terms = get_object_term_cache( $post->ID, $taxonomy );
if ( false !== $terms ) {
foreach ( $terms as $term ) {
if ( ! isset( $term_ids[ $term->term_id ] ) ) {
$term_ids[ $term->term_id ] = 1;
}
}
}
}
}
/*
* Only update the metadata cache for terms belonging to these posts if the term_id passed
* to `get_term_meta()` matches one of those terms. This prevents a single call to
* `get_term_meta()` from priming metadata for all `WP_Query` objects.
*/
if ( isset( $term_ids[ $term_id ] ) ) {
update_termmeta_cache( array_keys( $term_ids ) );
remove_filter( 'get_term_metadata', array( $this, 'lazyload_term_meta' ), 10, 2 );
}
}
// If no terms were found, there's no need to run this again.
if ( empty( $term_ids ) ) {
remove_filter( 'get_term_metadata', array( $this, 'lazyload_term_meta' ), 10, 2 );
}
_deprecated_function( __METHOD__, '4.5.0' );
return $check;
}
/**
* Lazy-load comment meta when inside of a `WP_Query` loop.
*
* This method is public so that it can be used as a filter callback. As a rule, there is no need to invoke it
* directly, from either inside or outside the `WP_Query` object.
* Lazyload comment meta for comments in the loop.
*
* @since 4.4.0
* @deprecated 4.5.0 See wp_queue_comments_for_comment_meta_lazyload().
*
* @param mixed $check The `$check` param passed from the 'get_comment_metadata' hook.
* @param int $comment_id ID of the comment whose metadata is being cached.
* @return mixed The original value of `$check`, to not affect 'get_comment_metadata'.
* @param mixed $check
* @param int $comment_id
* @return mixed
*/
public function lazyload_comment_meta( $check, $comment_id ) {
// Don't use `wp_list_pluck()` to avoid by-reference manipulation.
$comment_ids = array();
if ( is_array( $this->comments ) ) {
foreach ( $this->comments as $comment ) {
$comment_ids[] = $comment->comment_ID;
}
}
/*
* Only update the metadata cache for comments belonging to these posts if the comment_id passed
* to `get_comment_meta()` matches one of those comments. This prevents a single call to
* `get_comment_meta()` from priming metadata for all `WP_Query` objects.
*/
if ( in_array( $comment_id, $comment_ids ) ) {
update_meta_cache( 'comment', $comment_ids );
remove_filter( 'get_comment_metadata', array( $this, 'lazyload_comment_meta' ), 10, 2 );
} elseif ( empty( $comment_ids ) ) {
remove_filter( 'get_comment_metadata', array( $this, 'lazyload_comment_meta' ), 10, 2 );
}
_deprecated_function( __METHOD__, '4.5.0' );
return $check;
}
}

2
wp-includes/version.php
View File

@ -4,7 +4,7 @@
*
* @global string $wp_version
*/
$wp_version = '4.5-alpha-36565';
$wp_version = '4.5-alpha-36566';
/**
* Holds the WordPress DB revision, increments when changes are made to the WordPress DB schema.

1
wp-settings.php
View File

@ -135,6 +135,7 @@ require( ABSPATH . WPINC . '/class-wp-user-query.php' );
require( ABSPATH . WPINC . '/session.php' );
require( ABSPATH . WPINC . '/meta.php' );
require( ABSPATH . WPINC . '/class-wp-meta-query.php' );
require( ABSPATH . WPINC . '/class-wp-metadata-lazyloader.php' );
require( ABSPATH . WPINC . '/general-template.php' );
require( ABSPATH . WPINC . '/link-template.php' );
require( ABSPATH . WPINC . '/author-template.php' );