This PR #16941 modifies the GC UI, so update the test cases.
1 Update GC web element xpath.
2 Migrate GC content from Configuration.robot to GC.robot for classification.
Signed-off-by: Yang Jiao <jiaoya@vmware.com>
Because this PR #16941 migrates CVE allowlist from Configuration System
Settings page to Configuration Security page.
Signed-off-by: Yang Jiao <jiaoya@vmware.com>
1.Add Webhook Tester to the test environment.
2.Create a new project and enable webhook.
3.Trigger different event types(Artifact pushed、Artifact
pulled、Artifact deleted、Scanning finished、Scanning stopped、Tag
retention finished、Replication finished、Quota near threshold、Quota exceed).
4.Verify that the message is correct in the Webhook Tester.
5.Moved Webhook CRUD test cases from Common.robot to Webhook.robot for
triage.
Signed-off-by: Yang Jiao <jiaoya@vmware.com>
Add test cases for these three different scenarios to ensure that Replication functions correctly after Deployment Security Policy is enabled
Signed-off-by: Yang Jiao <jiaoya@vmware.com>
1. Use root instead of ~ in notary parameter;
2. Fix tag immutability issue caused by GUI change;
3. Replace email domain name to harbor test;
4. Add login for each base image in makefile;
5. Add customize look test in nightly.
Signed-off-by: danfengliu <danfengl@vmware.com>
1. Add manifest and CNAB replication tests;
2. Duplicate ORAS and sigularity tests from API test to nightly common
tests;
3. Optimize get dns code in CI;
4. Optimize E2E dockerfile;
5. Sample image size should be cover requirement for large size like 512M.
Signed-off-by: danfengliu <danfengl@vmware.com>
Build base image step should be in build package workflow, and local base images build by new step should be removed since images have been pushed to docker hub.
Signed-off-by: danfengliu <danfengl@vmware.com>
1. Add manifest and CNAB replication tests;
2. Duplicate ORAS and sigularity tests from API test to nightly common
tests;
3. Optimize get dns code in CI;
4. Optimize E2E dockerfile;
5. Sample image size should be cover requirement for large size like 512M.
Signed-off-by: danfengliu <danfengl@vmware.com>
1. Change way for quotas verification in upgrade pipeline, prepare specific size of image, then it's an known value for verifcation;
2. Add notary key rotate test;
3. For issue brought by docker 20, clean containerd cache is the only effective way, so both dockerd and containerd should be cache cleard and restarted;
4. Upgrade E2E Dockerfile for importing readable file size package, and other issues;
5. Uncomment project level robot account test in nightly.
6. Get DNS from local setting, and set it into docker deamon config file;
Signed-off-by: danfengliu <danfengl@vmware.com>
1. Fix E2E quotas issue, push the same image but with different name;
2. Add checkpoint for robot account test;
3. Upgraded docker and containerd in E2E image;
4. Package base image sample(busybox) into E2E image, so in E2E
container, all local docker images can be cleaned up, once base image is needed for
building image, it can be loaded locally;
5. Adapt OIDC service of supporting LDAP user, and add OIDC group user
test;
6. Restart docker deamon before content trust test, both in API and UI
test;
7. Add retry for keyword "Add A Tag Immutability Rule";
8. Fix tag retention test issue, missing click angle icon, and enhance
checkpoint of dry run and real run;
9. Fix schedule test issue for wrong cron string;
10. Disable quotas verification, it's not stable for script defect;
Signed-off-by: danfengliu <danfengl@vmware.com>
1. Local image should be removed, otherwise docker 20 will not tigger
get manifest request to harbor;
2. E2E image Dockerfile update;
3. Fix nighlty test issue of tag retention, add execution refesh to get
result;
4. Fix nighlty test keyword 'Create An New Project And Go Into
Project' issue that waiting long enough time for list display;
5. Add nightly test case, in GUI, scan result will show if cve id exist in allow list configuration;
6. Move proxy cache test to schdule pipeline, it will save some time for
db pipeline.
Signed-off-by: danfengliu <danfengl@vmware.com>
1. Add retry for Get Statics Text keywords make this operation robust;
2. Replace image to one with slash in name for CNAB test, to cover more
condition;
3. Add replication rule deletion verfication in upgrade test;
4. Add non-admin user in pull operation for proxy cache test;
5. Add verification for quota display for upgrade test;
6. Add test for large size of image replication;
7. Add test checkpoint for system robot account py-test script;
Signed-off-by: Danfeng Liu (c) <danfengl@vmware.com>
1. Add GUI test for system robot account feature;
2. Fix issue of failing to catch docker pull image exception in API test.
Signed-off-by: Danfeng Liu (c) <danfengl@vmware.com>
Due to docker-hub pull request rate limitation, we will use registed account to pull image
from docker-hub, therefore add docker-hub login in API tests.
Signed-off-by: danfengliu <danfengl@vmware.com>
1. Since upgrade python to 3.7, some urllib3 warnings come out during
execution, because missing disconnection when using request by docker
library, it's not affect script execution, therefore ignore them first.
2. Replication tests failed in nightly due to new UI code, some element
locators are not valid anymore;
3. Remove switch scanner test steps, since Clair was removed.
Signed-off-by: danfengliu <danfengl@vmware.com>
1. Verify LDAP group admin in UI;
2. Update LDAP group admin need to verify the AdminRoleInAuth, get current user will return this field as true.
Signed-off-by: danfengliu <danfengl@vmware.com>
1. Fix ldap script issue, test case name should be in lowercase;
2. Add manifest list checkout point for proxy cache issue;
3. Add P2P Preheat policy CRUD Test.
Signed-off-by: danfengliu <danfengl@vmware.com>
1. Upgrade robot-framework to 3.1 and Selenium library to 4.4.0.
2. Fix a registry issue for clear filter text input.
Signed-off-by: danfengliu <danfengl@vmware.com>
1. Fix issue of keyword Go Into Repo, the verification logic could be more strict;
2. Add API E2E pytest of GC with untag flag enabled;
3. Add sleep in test_user_view_logs.py for delete log ocurred;
4. Test Case - Tag CRUD is not stable. Although add button was clicked, but the tag was'nt added successfully.
Signed-off-by: danfengliu <danfengl@vmware.com>
1. Manifest list can be pulled by ctr;
2. ui-test missing key checkpoint, fix it by add checking scan detail result;
3. add tag retension untag image test;
Signed-off-by: danfengliu <danfengl@vmware.com>
1. Add Clair should be default scanner in upgrade test;
2. Add tag retention verification in upgrade test;
3. Add tag retention verification in upgrade test;
4. Add tag Immutability verification in upgrade test;
5. Add webhook verification in upgrade test;
6. Add CVE whitelist in upgrade test;
Signed-off-by: Danfeng Liu (c) <danfengl@vmware.com>
1. Add Multi-Scanner test file for harbor is deployed with multi scanners;
2. Modify notary doc description;
3. Add paragraph for e2e-api-python-based-scripting-guide;
4. Fix delete project issue;
5. Remove count qoutas in nightly and modify some of it;
6. Add Trivy in git hub offline action;
Signed-off-by: danfengliu <danfengl@vmware.com>
1. Add notary tool in E2E Dockerfile;
2. Add push docker manifest list test in nightly;
3. Modify document for e2e user guide;
4. Add CNAB tool in E2E Dockerfile;
Signed-off-by: danfengliu <danfengl@vmware.com>
Signed-off-by: Yogi_Wang <yawang@vmware.com>
1.add case for trivy
2.vunerbility refresh bug
3.scan mutiple artifact
4.fix global search bug
5.disable delete tag btn when remove immutable tag
6.cancel selectRow when add label or remove label;fix #11195
7.fix cron tootip
