Merge pull request #11850 from bitsf/doc_retention_2.0

feat(doc) update retention/replication doc for 2.0

docs/administration/configuring-replication/create-replication-rules.md

@@ -11,30 +11,26 @@ A replication endpoint must exist before you create a replication rule. To creat
    ![Add a replication rule](../../../img/replication-rule1.png)
 1. Click **New Replication Rule**.
 1. Provide a name and description for the replication rule.
-1. Select **Push-based** or **Pull-based** replication, depending on whether you want to replicate images to or from the remote registry.
+1. Select **Push-based** or **Pull-based** replication, depending on whether you want to replicate artifacts to or from the remote registry.
 
    ![Replication mode](../../../img/replication-rule2.png)
-1. For **Source resource filter**, identify the images to replicate.  
+1. For **Source resource filter**, identify the artifacts to replicate.  
 
    ![Replication filters](../../../img/replication-rule3.png)
 
-   * **Name**: Replicate resources with a given name by entering an image name or fragment.
+   * **Name**: Replicate resources with a given name by entering an artifact name or fragment.
    * **Tag**: Replicate resources with a given tag by entering a tag name or fragment.
    * **Label**: Replicate resources with a given label by using the drop-down menu to select from the available labels.
-   * **Resource**: Replicate images, charts, or both.
+   * **Resource**: Replicate artifacts, charts, or both.
 
    The name filter and tag filters support the following patterns:
 
    * **\***: Matches any sequence of non-separator characters `/`.
-   * **\*\***: Matches any sequence of characters, including path separators `/`.
-   * **?**: Matches any single non-separator character `/`.
-   * **{alt1,...}**: Matches a sequence of characters if one of the comma-separated alternatives matches. are as follows:
-   * **\***: Matches any sequence of non-separator characters `/`.
-   * **\*\***: Matches any sequence of characters, including path separators `/`.
+   * **\*\***: Matches any sequence of characters, including path separators `/`. Note that the doublestar must appear as a path component by itself. A pattern such as /path\*\* is invalid and will be treated the same as /path*, but /path\*/\*\* should achieve the desired result.
    * **?**: Matches any single non-separator character `/`.
    * **{alt1,...}**: Matches a sequence of characters if one of the comma-separated alternatives matches.
 
-   **NOTE:** You must add `library` if you want to replicate the official images of Docker Hub. For example, `library/hello-world` matches the official hello-world images.  
+   **NOTE:** You must add `library` if you want to replicate the official artifacts of Docker Hub. For example, `library/hello-world` matches the official hello-world artifacts.  
 
    Pattern | String(Match or not)
    ---------- | -------
@@ -48,13 +44,17 @@ A replication endpoint must exist before you create a replication rule. To creat
    If you do not enter a namespace, resources are placed in the same namespace as in the source registry. 
 
    ![Destination and namespaces](../../../img/replication-rule4.png)
+   
+   **NOTE:** Because of major API changes in the v2.0 release to support [OCI](https://github.com/opencontainers/distribution-spec).
+   You **can not** replicate from harbor 1.x to 2.0, and you **can not** replicate artifacts with **manifest list** from 2.0 to 1.x. 
+   
 1. Use the Trigger Mode drop-down menu to select how and when to run the rule.
    * **Manual**: Replicate the resources manually when needed. **Note**: Deletion operations are not replicated. 
    * **Scheduled**: Replicate the resources periodically by defining a cron job. **Note**: Deletion operations are not replicated. 
-   * **Event Based**: When a new resource is pushed to the project, or an image is retagged, it is replicated to the remote registry immediately. If you select the **Delete remote resources when locally deleted**, if you delete an image, it is automatically deleted from the replication target.
+   * **Event Based**: When a new resource is pushed to the project, or an artifact is retagged, it is replicated to the remote registry immediately. If you select the **Delete remote resources when locally deleted**, if you delete an artifact, it is automatically deleted from the replication target.
 
    {{< note >}}
-   You can filter images for replication based on the labels that are applied to the images. However, changing a label on an image does not trigger replication. Event-based replication is limited to pushing, retagging, and deleting images.
+   You can filter artifacts for replication based on the labels that are applied to the artifacts. However, changing a label on an artifact does not trigger replication. Event-based replication is limited to pushing, retagging, and deleting artifacts.
    {{< /note >}}
 
    ![Trigger mode](../../../img/replication-rule5.png)

docs/working-with-projects/working-with-images/create-tag-retention-rules.md

@@ -3,7 +3,7 @@ title: Create Tag Retention Rules
 weight: 80
 ---
 
-A repository can rapidly accumulate a large number of image tags, many of which might not be required after a given time or once they have been superseded by a subsequent image build. These excess tags can obviously consume large quantities of storage capacity. As a Harbor system administrator, you can define rules that govern how many tags of a given repository to retain, or for how long to retain certain tags. 
+A repository can rapidly accumulate a large number of artifacts, many of which might not be required after a given time or once they have been superseded by a subsequent artifact build. These excess artifacts can obviously consume large quantities of storage capacity. As a Harbor system administrator, you can define rules that govern how many artifacts of a given repository to retain, or for how long to retain certain artifacts. 
 
 ## How Tag Retention Rules Work
 
@@ -13,60 +13,70 @@ A tag retention rule has 3 filters that are applied sequentially, as described i
 
 |Order|Filter|Description|
 |---|---|---|
-|First|Repository or repositories|Identify the repository or repositories on which to apply the rule. You can identify repositories that either have a certain name or name fragment, or that do not have that name or name fragment. Wild cards (for example `*repo`, `repo*`, and `**`) are permitted. The repository filter is applied first to mark the repositories to which to apply the retention rule. The identified repositories are earmarked for further matching based on the tag criteria. No action is taken on the nonspecified repositories at this stage.|
+|First|Repository or repositories|Identity the tags on which to apply the rule. You can identify repositories that either have a certain name or name fragment, or that do not have that name or name fragment. Wild cards (for example `*repo`, `repo*`, and `**`) are permitted. The repository filter is applied first to mark the repositories to which to apply the retention rule. The identified repositories are earmarked for further matching based on the tag criteria. No action is taken on the nonspecified repositories at this stage.|
 |Second|Quantity to retain|Set which tags to retain either by specifying a maximum number of tags, or by specifying a maximum period for which to retain tags.|
-|Third|Tags to retain|Identify the tag or tags on which to apply the rule. You can identify tags that either have a certain name or name fragment, or that do not have that name or name fragment. Wild cards (for example `*tag`, `tag*`, and `**`) are permitted.|
+|Third|Tags to retain|Identify the tag or tags on which to apply the rule. You can identify tags that either have a certain name or name fragment, or that do not have that name or name fragment. Wild cards (for example `*tag`, `tag*`, and `**`) are permitted. Use the checkbox to select whether untagged artifacts should be captured as part of the set of artifacts eligible for tag retention.|
 
 For information about how the `**` wildcard is applied, see https://github.com/bmatcuk/doublestar#patterns.
 
 ### Example 1
 
 - You have 5 repositories in a project, repositories A to E.
-  - Repository A has 100 image tags, all of which have been pulled in the last week.
-  - Repositories B to E each have 6 images, none of which have been pulled in the last month.
+  - Repository A has 102 artifacts with 2 untagged, all of which have been pulled in the last week.
+  - Repositories B to E each have 7 artifacts with 1 untagged artifact, none of which have been pulled in the last month.
 - You set the repository filter to `**`, meaning that all repositories in the project are included.
-- You set the retention policy to retain the 10 most recently pulled images in each repository.
-- You set the tag filter to `**`, meaning that all tags in the repository are included.
+- You set the retention policy to retain the 10 most recently pulled artifacts in each repository.
+- You set the tag filter to `**`, keep "untagged artifacts" unchecked, meaning that all artifacts with at least one tag in the repository are included.
 
-In this example the rule retains the 10 most recently pulled images in repository A, and all 6 of the images in each of the 4 repositories B to E. So, a total of 34 image tags are retained in the project. In other words, the rule does not treat all of the images in repositories A to E as a single pool from which to choose the 10 most recent images. So, even if the 11th to 100th tags in repository A have been pulled more recently than any of the tags in repositories B to E, all of the tags in repositories B to E are retained, because each of those repositories has fewer than 10 tags.
+In this example the rule retains the 10 most recently pulled tagged artifacts in repository A, and the 6 of the artifacts in each of the 4 repositories B to E. So, a total of 34 artifacts are retained in the project. 
+In other words, the rule does not treat all of the artifacts in repositories A to E as a single pool from which to choose the 10 most recent artifacts. 
+So, even if the 11th to 100th tags in repository A have been pulled more recently than any of the tags in repositories B to E, all of the tagged artifacts in repositories B to E are retained, because each of those repositories has fewer than 10 tags, and all untagged artifacts are deleted.
 
 ### Example 2
 
-This example uses the same project and repositories as example 1, but sets the retention policy to retain the images in each repository that have been pulled in the last 7 days.
+This example uses the same project and repositories as example 1, but sets the retention policy to retain the artifacts in each repository that have been pulled in the last 7 days.
 
-In this case, all of the images in repository A are retained because they have been pulled in the last 7 days. None of the images in repositories B to E are retained, because none of them has been pulled in the last week. In this example, 100 images are retained, as opposed to 34 images in example 1.
+In this case, all of the 100 tagged artifacts in repository A are retained because they have been pulled in the last 7 days. None of the artifacts in repositories B to E are retained, because none of them has been pulled in the last week. In this example, 100 artifacts are retained, as opposed to 34 artifacts in example 1.
+And all untagged artifacts are deleted.
+
+### Example 3
+
+This example uses the same project and repositories as example 2, but checked "untagged artifacts", then all artifacts in repository A are retained.
+
+In this case, all of the 103 artifacts in repository A are retained because they have been pulled in the last 7 days. None of the artifacts in repositories B to E are retained, because none of them has been pulled in the last week. In this example, 103 artifacts are retained, as opposed to 100 artifacts in example 2.
+And all untagged artifacts are retained.
 
 ### Tag Retention Rules and Native Docker Tag Deletion
 
-**WARNING**: Due to native Docker tag deletion behavior, there is an issue with the current retention policy implementation. If you have multiple tags that refer to the same SHA digest, and if a subset of these tags are marked for deletion by a configured retention policy, all of the remaining tags would also be deleted. This violates the retention policy, so in this case all of the tags are retained. This issue will be addressed in a future update release, so that tag retention policies can delete tags without deleting the digest and other shared tags.
+**Note**: If an artifact has several tags, and only a partial set of tags are matched via the retention policy, then the artifact and all its tags will be retained. In other words, retention is matched at the tag level but retention / deletion is carried out at the artifact level, with retention fully preserving the artifact including all of its tags. 
 
 For example, you have following tags, listed according to their push time, and all of them refer to the same SHA digest:
 
-- `harbor-1.8`, pushed 8/14/2019 12:00am
+- `harbor-1.8`, pushed 8/14/2019 01:00am
 - `harbor-release`, pushed 8/14/2019 03:00am
 - `harbor-nightly`, pushed 8/14/2019 06:00am
 - `harbor-latest`, pushed 8/14/2019 09:00am
 
-You configure a retention policy to retain the two latest tags that match `harbor-*`, so that `harbor-rc` and `harbor-latest` are deleted. However, since all tags refer to the same SHA digest, this policy would also delete the tags `harbor-1.8` and `harbor-release`, so all tags are retained.
+You configure a retention policy to retain the two latest tags that match `harbor-*`, so that `harbor-rc` and `harbor-latest` are retained. However, since all tags refer to the same SHA digest, this policy would also retain the tags `harbor-1.8` and `harbor-release`, so all tags are retained.
 
 ## Combining Rules on a Repository
 
 You can define up to 15 rules per project. You can apply multiple rules to a repository or set of repositories. When you apply multiple rules to a repository, they are applied with `OR` logic rather than with `AND` logic. In this way, there is no prioritization of application of the rules on a given repository. Rules run concurrently in the background, and the resulting sets from each rule are combined at the end of the run.
 
-### Example 3
+### Example 4
 
 This example uses the same project and repositories as examples 1 and 2, but sets two rules:
 
-- Rule 1: Retain all of the images in each repository that have been pulled in the last 7 days.
-- Rule 2: Retain a maximum number of 10 images in each repository.
+- Rule 1: Retain all of the artifacts in each repository that have been pulled in the last 7 days.
+- Rule 2: Retain a maximum number of 10 artifacts in each repository.
 
-For repository A, rule 1 retains all of the images because they have all been pulled in the last week. Rule 2 retains the 10 most recently pulled images. So, since the two rules are applied with an `OR` relationship, all 100 images are retained in repository A.
+For repository A, rule 1 retains all of the 100 tagged artifacts because they have all been pulled in the last week. Rule 2 retains the 10 most recently pulled artifacts. So, since the two rules are applied with an `OR` relationship, all 100 artifacts are retained in repository A.
 
-For repositories B-E, rule 1 will retain 0 images as no images are pulled in the last week. Rule 2 will retain all 6 images because 6 < 10. So, since the two rules are applied with an `OR` relationship, for repositories B-E, each repository will keep all 6 images.
+For repositories B-E, rule 1 will retain 0 artifacts as no artifacts are pulled in the last week. Rule 2 will retain all 6 artifacts because 6 < 10. So, since the two rules are applied with an `OR` relationship, for repositories B-E, each repository will keep all 6 artifacts.
 
-In this example, all of the images are retained.
+In this example, all of the artifacts are retained.
 
-### Example 4
+### Example 5
 
 This example uses a different repository to the previous examples.
 
@@ -80,8 +90,8 @@ This example uses a different repository to the previous examples.
   |`4.4-your_repo-prod`|`4.4-your_repo-rc`|`4.4-your_repo-release`| 
 
 - You define three tag retention rules on this repository:
-  - Retain the 10 most recently pushed image tags that start with `2`.
-  - Retain the 10 most recently pushed image tags that end with `-prod`.
+  - Retain the 10 most recently pushed artifacts that start with `2`.
+  - Retain the 10 most recently pushed artifacts that end with `-prod`.
   - Retain all tags that do not include `2.1-your_repo-prod`.
 
 In this example, the rules are applied to the following 7 tags:
@@ -94,6 +104,8 @@ In this example, the rules are applied to the following 7 tags:
 - `3.1-your_repo-prod`
 - `4.4-your_repo-prod`
 
+Because there are no untagged artifacts, checking the checkbox makes no difference.
+
 ## How Tag Retention Rules Interact with Project Quotas
 
 The Harbor system administrator can set a maximum on the number of tags that a project can contain and the amount of storage that it can consume. For information about project quotas, see [Configure Project Quotas](../../administration/configure-project-quotas/_index.md). 
@@ -103,7 +115,7 @@ If you set a quota on a project, this quota cannot be exceeded. The quota is app
 ## Configure Tag Retention Rules
 
 1. Log in to the Harbor interface with an account that has at least project administrator privileges.
-1. Go to **Projects**, select a project, and select **Tag Retention**.
+1. Go to **Projects**, select a project, select **Policy**, and select **Tag Retention**.
 
    ![Tag options](../../../img/tag-retention1.png)
 1. Click **Add Rule** to add a rule.
@@ -119,16 +131,16 @@ If you set a quota on a project, this quota cannot be exceeded. The quota is app
    - `**` to apply the rule to all of the repositories in the project. 
   
    If you selected **matching**, the rule is applied to the repositories you identified. If you selected **excluding**, the rule is applied to all of the repositories in the project except for the ones that you identified.
-1. In the **By image count or number of days** drop-down menu, define how many tags to retain or the period to retain tags.
+1. In the **By artifact count or number of days** drop-down menu, define how many tags to retain or the period to retain tags.
   ![Select retention criteria](../../../img/tag-retention3.png)
   
    |Option|Description|
    |---|---|
-   |**retain the most recently pushed # images**|Enter the maximum number of images to retain, keeping the ones that have been pushed most recently. There is no maximum age for an image.|
-   |**retain the most recently pulled # images**|Enter the maximum number of images to retain, keeping only the ones that have been pulled recently. There is no maximum age for an image.|
-   |**retain the images pushed within the last # days**|Enter the number of days to retain images, keeping only the ones that have been pushed during this period. There is no maximum number of images.|
-   |**retain the images pulled within the last # days**|Enter the number of days to retain images, keeping only the ones that have been pulled during this period. There is no maximum number of images.|
-   |**retain always**|Always retain the images identified by this rule.| 
+   |**retain the most recently pushed # artifacts**|Enter the maximum number of artifacts to retain, keeping the ones that have been pushed most recently. There is no maximum age for an artifact.|
+   |**retain the most recently pulled # artifacts**|Enter the maximum number of artifacts to retain, keeping only the ones that have been pulled recently. There is no maximum age for an artifact.|
+   |**retain the artifacts pushed within the last # days**|Enter the number of days to retain artifacts, keeping only the ones that have been pushed during this period. There is no maximum number of artifacts.|
+   |**retain the artifacts pulled within the last # days**|Enter the number of days to retain artifacts, keeping only the ones that have been pulled during this period. There is no maximum number of artifacts.|
+   |**retain always**|Always retain the artifacts identified by this rule.| 
 
 1. In the **Tags** drop-down menu, select **matching** or **excluding**.
 1. In the **Tags** text box, identify the tags on which to apply the rule.