Commit Graph

192 Commits

Author SHA1 Message Date
Daniel Jiang
08f9ffa000 Reenable token auth for cli
Docker CLI fails if it's not logged in upon seeing "basic" realm challenging while pinging the "/v2" endpoint. (#11266)
Some CLI will send HEAD to artifact endpoint before pushing (#11188)(#11271)

To fix such problems, this commit re-introduce the token auth flow to the CLIs.

For a HEAD request to "/v2/xxx" with no "Authoirzation" header, the v2_auth middleware populates the
"Www-Authenticate" header to redirect it to token endpoint with proper
requested scope.

It also adds security context to based on the content of the JWT which has the claims of the registry.
So a request from CLI carrying a token signed by the "/service/token" will have proper permissions.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-04-04 00:05:58 +08:00
DQ
cd69339014 Fix API TEST for chart Version
Fix api test for chart b/c revert the api

Signed-off-by: DQ <dengq@vmware.com>
2020-04-01 11:55:22 +08:00
Wenkai Yin(尹文开)
4faff18b2d
Merge pull request #11339 from ywk253100/200328_limit_offset
Add "order by" clause to avoid the duplicat rows
2020-03-30 17:14:44 +08:00
Wenkai Yin
fb975d902c Add "order by" clause to avoid the duplicat rows
Add "order by" clause to avoid the duplicat rows: https://www.postgresql.org/docs/9.6/queries-limit.html

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-03-30 16:42:43 +08:00
Ren Maosheng
759c759b58 Adding tests for native docker registry APIs exposed by Harbor
Adding a test for listing repositories
Adding a test for list image tags for specified repo.

Signed-off-by: Ren Maosheng <renmaosheng@gmail.com>
2020-03-29 08:27:13 -07:00
danfengliu
77e3b0d828 Fix nightly ldap test cases
1. Modify robot account test, checkout "Never Expired" button;
2. Modify OIDC get user API;
3. Modify verification for docker pull command;

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-03-27 20:24:43 +08:00
Wang Yan
eccb8aa708
append pull permission for push policy (#11303)
Fixes #11225
As registry changes to basic auth, the push action lost the pull permission.
Add it in the robot security context.

Signed-off-by: wang yan <wangyan@vmware.com>
2020-03-27 17:10:04 +08:00
danfengliu
6ccaaf1efa Remove some content of verification for artifact addtion
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-03-26 10:03:30 +08:00
Wang Yan
dc6eec8a73
Enable API logs test case (#11142)
1, enable user view log api test case
2, update project logs api permission check
3, use project ctl instead in permission check base method

Signed-off-by: wang yan <wangyan@vmware.com>
2020-03-19 14:56:37 +08:00
jwangyangls
1d435bc246
Merge pull request #11086 from danfengliu/add-api-test-of-add-addition
Modify api test for test step of add addition
2020-03-18 20:12:46 +08:00
Wang Yan
b4e941e961
drop table access log in migration (#11118)
Use the audit log instead, the access log table should be dropped after migration

Signed-off-by: wang yan <wangyan@vmware.com>
2020-03-18 19:04:38 +08:00
danfengliu
77e9fc38c7 Modify api test for test step of add addition
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-03-18 17:32:10 +08:00
danfengliu
995ce30c58 Modify API test for scan image since harbor v2 API presented
1. System level Scan All;
2. Scan An Image Artifact

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-03-18 10:15:07 +08:00
Wenkai Yin
e33b2984ce Add create/delete tag API test case
Add create/delete tag API test case

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-03-16 21:32:34 +08:00
danfengliu
843b05c2d3 Add script of push cnab bunlde API test
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-03-16 17:37:16 +08:00
danfengliu
42956c74bb
Merge pull request #11018 from danfengliu/add-verification-for-helm-api-test
Add verfication for helm API test
2020-03-16 10:17:37 +08:00
Wenkai Yin
c6940e8184 United error response format for management APIs (legacy and v2.0 APIs)
United error response format for management APIs (legacy and v2.0 APIs)

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-03-13 22:00:08 +08:00
He Weiwei
28dcb5ad59
test(quota): enable quota test case in API robot (#11039)
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-03-12 11:50:30 +08:00
danfengliu
f8811102da Add verfication for helm API test
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-03-11 15:05:16 +08:00
Wang Yan
f49994d81d
enable garbage colloection api test case (#11000)
Signed-off-by: wang yan <wangyan@vmware.com>
2020-03-11 11:19:29 +08:00
danfengliu
e49ac2f9e9 Add API test python script - Push chart file by Helm3 chart command line
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-03-05 14:58:36 +08:00
danfengliu
2d6e18a895 Add API test case of pushing index by docker manifest
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-03-02 06:31:59 +00:00
wang yan
79cf21f82f add tag controller
use the tag controller to handle CRUD of tags, especially the delete scenario, it could validate
the immutable and signature. And move the code of tag handling from artifact controller to tag controller

Signed-off-by: wang yan <wangyan@vmware.com>
2020-02-28 11:42:10 +08:00
danfengliu
c283a02e5f Update existing API tests for API V2.0
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-02-26 21:38:39 +08:00
danfengliu
4933bb634f Upgrade repository API tests to V2.0
Enable _xsrf in cookies in swagger.yaml, so that scripts don't have to handle it.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-02-24 18:15:25 +08:00
He Weiwei
88fcacd4b7
feat(middleware): add blob middlewares (#10710)
1. Add middleware to record the accepted blob size for stream blob
upload.
2. Add middleware to create blob and associate it with project after blob upload
complete.
3. Add middleware to sync blobs, create blob for manifest and associate blobs
with the manifest after put manifest.
4. Add middleware to associate blob with project after mount blob.
5. Cleanup associations for the project when artifact deleted.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-02-20 23:20:34 +08:00
Ziming
0bc32410f3
Merge pull request #10742 from bitsf/oci_tag_retention
requirement(oci) implement tag retention for oci
not include ChartClient yet
2020-02-20 20:31:49 +08:00
danfengliu
03668ad372 Build python swagger client for V2.0
Add v2 swagger.yaml python library.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-02-20 18:06:54 +08:00
Ziming Zhang
94e23dc954 requirement(oci) implement tag retention for oci
Change-Id: Ib36660835d2666b35124e66254c33b5fc19aaf77
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2020-02-20 00:43:20 +08:00
Wenkai Yin
94787ea60d Bump up the version of legacy APIs to v2.0
Bump up the version of legacy APIs to v2.0

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-14 13:16:30 +08:00
danfengliu
d1b5bd5d9c Fix project quotas API test issue
In project quotas API test, pull images from goharbor namespace instead of library:
1. Replace image source in API test;
2. Modify criteria for verify project configuration modification.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-02-06 16:43:08 +08:00
danfengliu
66eff99c7f Fix description issue of test in robot account API test and issue of Helm3 test
1. Fix issue that test step descriton was mismatch with test step;
2. Wrong helm command was used in Helm3 test, replace helm with helm3;
3. In API test, images were pulled from docker-hub registry, images size changed sometime, so we like to use internal registry.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-02-04 17:26:52 +08:00
Daniel Jiang
2064a1cd6d Switch to basic authentication for registry
1. Add basic authorizer for registry which modify the request
to add basic authorization header to request based on configuration.
2. Set basic auth header for proxy when accessing registry
3. Switche the registry to use basic auth by default and use the basic
authorizer to access Harbor.
4. Make necessary change to test cases, particularly
"test_robot_account.py" and "docker_api.py", because the error is
changed after siwtched to basic auth from token auth.  #10604 is opened
to track the follow up work.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-01-31 21:46:47 +09:00
Daniel Jiang
5f8acc3896 Add middlewares for permission checking for v2 API
When the registry shifts from token auth to basic auth, we'll use the middleware to check permission.
This commit add middlewares for populate the artifact info and check
permission based on request to /v2/* api via security context

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-01-27 12:53:15 +08:00
Ziming Zhang
9f1d538b5f feat(cicd): try fix sometimes docker push time out
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I67193a7c5b80d169237a884895627f68a1f65933
2020-01-20 11:57:03 +08:00
Ziming Zhang
45113ea8e1 feat(cicd) use a smaller docker image for test
Change-Id: Ie8f365e7271bfda24ae965aaca0e55d1099c1d68
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2020-01-17 13:09:31 +08:00
wang yan
a0f3709b3c add expiration data time when to create a robot account
Update API of creating robot accout, user can specify expiration time per account.

Signed-off-by: wang yan <wangyan@vmware.com>
2020-01-03 13:47:06 +08:00
stonezdj
6313a55219 Fix admin permission not revoked when removed from LDAP admin group
Seperate the HasAdminRole(In DB) with the privileges from external auth, and use user.HasAdminPrivilege to check

Signed-off-by: stonezdj <stonezdj@gmail.com>
2019-12-20 13:12:22 +08:00
Steven Zou
4b335b79d5
Merge pull request #9693 from steven-zou/fix/issue_#9606_update_API_swagger
update API swagger file
2019-11-06 11:06:02 +08:00
Steven Zou
0ebeaa10df update API swagger file
- add scanners mgmt related API
- add scan related API
  - trigger scan
  - get report
  - get log text stream
  - get scan all metrics
- update the scan_overview in the tag getting API
- fix #9606

Signed-off-by: Steven Zou <szou@vmware.com>
2019-11-05 22:20:14 +08:00
Wenkai Yin
e20b538141 Add e2e test case for health check API
Add e2e test case for health check API

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-10-29 13:48:06 +08:00
wang yan
3e826c4e80 update query in the immutable delete manifest middleware
Signed-off-by: wang yan <wangyan@vmware.com>
2019-10-22 19:37:20 +08:00
He Weiwei
8964a8697a build(clair): internal clair adapter when install with clair
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-10-17 12:00:51 +08:00
He Weiwei
4ce72e37c4 fix(robot): robot account improvement for policies
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-09-27 03:07:58 +00:00
Ziming Zhang
afe81a8b3b adjust wait job timeout
Change-Id: I8f32f814158d4a7418c39edb7a781879db17a4d7
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2019-09-20 16:18:03 +08:00
Wang Yan
6b5fd36bb3 add e2e test case for project quota
1. Create a new user(UA);
2. Create a new private project(PA) by user(UA);
3. Add user(UA) as a member of project(PA) with project-admin role;
4. Push an image to project(PA) by user(UA), then check the project quota usage;
5. Check quota change
6. Delete image, the quota should be changed to 0.

Signed-off-by: wang yan <wangyan@vmware.com>
2019-09-15 22:16:11 +08:00
Ziming Zhang
722e45b20b add swagger for tag retention
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I0f3ed8085e231868de74c273ba85946826181d5b
2019-09-06 17:27:20 +08:00
danfengliu
841ac24ebc
Add API test case sign image back to travis, despite codacy block it when pulling a pr. (#8617)
Signed-off-by: wang yan <wangyan@vmware.com>
2019-08-19 13:35:45 +08:00
danfengliu
2aa6f1aed7
Merge branch 'master' into add-sign-api-back-to-travis-despite-codacy-1 2019-08-16 16:28:49 +08:00
wang yan
ddb913f8d0 Add API test case sign image back to travis, despite codacy block it when pulling a pr.
Signed-off-by: wang yan <wangyan@vmware.com>
2019-08-16 01:26:36 -07:00
Daniel Jiang
e4a78ba039 API test for project level CVE whitelist
This commit update test case to cover project level CVE whitelist.
It also fixes the swagger doc to add missing attributes

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-08-15 11:37:22 +08:00
Daniel Jiang
d01597ba23 API test for system level CVE whitelist
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-08-14 14:59:58 +08:00
danfengliu
1b0931c034 Since harbor 1.8 released, API test for replication was not update yet, so in this PR, replicaiton test will be adapt for harbor 1.8.
Signed-off-by: danfengliu <danfengl@vmware.com>
2019-08-08 17:06:37 +08:00
danfengliu
3eeb8cf355 After helm cli test was added to nightly, ca was get by reading from local file, but in helm nighly test, ca file was not in local, it has to be got from harbor api.
Signed-off-by: danfengliu <danfengl@vmware.com>
2019-08-06 11:28:54 +08:00
danfengliu
9019864446
In API test, there is a dead loop in checking GC status, it's fixed in this PR and add log inmformation in the loop. (#7643)
Signed-off-by: danfengliu <danfengl@vmware.com>
2019-05-06 17:24:33 +08:00
danfengliu
e08c2e757e This API test script was blocked by a swagger error, now this error was fixed, so robot account script can be finished now. In swagger.yaml, robot account can be updated in "disbled" status, it's added into script. (#7636)
Signed-off-by: danfengliu <danfengl@vmware.com>
2019-05-06 11:34:11 +08:00
stonezdj
41a574e55c Fix issue 6450 Test LDAP server error without save configuration
Signed-off-by: stonezdj <stonezdj@gmail.com>
2019-04-18 14:24:21 +08:00
Wang Yan
a6af9e9972
Support well-formatted error returned from the REST APIs. (#6957)
Signed-off-by: wang yan <wangyan@vmware.com>
2019-04-17 16:43:06 +08:00
danfengliu
08b346ee1e
Add API test case for retag, create a user and 2 projects by this new user, push image to 1st project, and retag this image to 2nd project. (#7300)
Signed-off-by: danfengliu <danfengl@vmware.com>
2019-04-11 15:05:46 +08:00
Yan
8d3946a0e2
Refactor scan all api (#7120)
* Refactor scan all api

This commit is to let scan all api using admin job to handle schedule
management. After the PR, GC and scan all share unified code path.

Signed-off-by: wang yan <wangyan@vmware.com>

* update admin job api code according to review comments

Signed-off-by: wang yan <wangyan@vmware.com>

* Update test code and comments per review

Signed-off-by: wang yan <wangyan@vmware.com>
2019-03-22 17:52:21 +08:00
De Chen
41e290a17f
Merge pull request #6877 from yeya24/patch/fixtypo
fix some typos
2019-03-12 20:07:15 +08:00
danfengliu
eaedd89c25 add api test case for robot user, and modify swagger.yaml for wrong type of return value. (#6900)
Signed-off-by: danfengliu <danfengl@vmware.com>
2019-02-18 13:47:16 +08:00
wang yan
c77b387c53 Upgrade registry binary to v2.7.0
Signed-off-by: wang yan <wangyan@vmware.com>
2019-02-13 10:24:08 +08:00
yeya24
628b94db19 fix some typos
Signed-off-by: yeya24 <ben.ye@daocloud.io>
2019-02-05 12:03:52 +08:00
danfengliu
70016cc0eb
Change a small docker image to pull and make timeout value longer (#6543)
Signed-off-by: danfengliu <danfengl@vmware.com>
2018-12-18 10:21:03 +08:00
danfengliu
7c57864c99 Remove 2 redundant functions, pull_harbor_image_successfully and pull_harbor_image_unsuccessfully.
Signed-off-by: danfengliu <danfengl@vmware.com>
2018-12-04 13:46:07 +08:00
danfengliu
3356368bff Add API test case assign system admin role. (#6413)
Signed-off-by: danfengliu <danfengl@vmware.com>
2018-12-04 12:26:12 +08:00
danfengliu
b4c7663f5f
Add API test case scan all images and test case API test case list helm charts. (#6388)
Signed-off-by: danfengliu <danfengl@vmware.com>
2018-12-03 17:05:06 +08:00
Wenkai Yin
9d5cf57373 Check the existence of name when creating replication rule and fix bugs in testing library (#6381)
1. Fix #5102 by checking the existence of name when creating/editing replication rule
2. Add unique constraint to the name of replication policy and target
3. Fix bugs of testing library

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2018-11-30 13:32:20 +08:00
danfengliu
04fd4d444d
Add API test case for user view logs. (#6370)
Add API test case for user view logs.
Signed-off-by: danfengliu <danfengl@vmware.com>
2018-11-29 18:27:53 +08:00
danfengliu
c4bf65162c
add api test case for garbage collection (#6366)
Add API test case for garbage collection, and add swagger.yaml, GC and chart feature were updated in swagger.yaml.
Signed-off-by: danfengliu <danfengl@vmware.com>
2018-11-27 19:17:41 +08:00
danfengliu
ad77098acf
add test case project level policy content trust. (#6309)
Add test case project level policy content trust.
Signed-off-by: danfengliu <danfengl@vmware.com>
2018-11-21 12:19:28 +08:00
danfengliu
5ceebcbb10
Add tese case of manage project memebers and modify (#6304)
Move admin_client to testutil.py as global parameter.
Change create_repository function name to push_image_to_project to make it reasonable.
Signed-off-by: danfengliu <danfengl@vmware.com>
2018-11-20 14:24:13 +08:00
danfengliu
dcd34a37a9 add test case scan a not scanned image
Modify for codacy issues.
Signed-off-by: danfengliu <danfengl@vmware.com>
2018-11-16 15:35:15 +08:00
danfengliu
1cc04183cb add test case for edit project creation;
fixed some space and import issues coddcy;
remove a lib;

Signed-off-by: danfengliu <danfengl@vmware.com>
2018-11-15 10:32:21 +08:00
danfengliu
90fda71e4a Recovery repository.py for calling wrong create function (#6268)
Signed-off-by: danfengliu <danfengl@vmware.com>
2018-11-13 12:53:46 +08:00
danfengliu
6289275118 Modify script call create project (#6263)
Signed-off-by: danfengliu <danfengl@vmware.com>
2018-11-12 23:39:16 +08:00
danfengliu
205a658afb Only modify lib-project for seq of return (#6261)
Signed-off-by: danfengliu <danfengl@vmware.com>
2018-11-12 16:59:55 +08:00
danfengliu
ce8a585858 modfiy repo
Signed-off-by: danfengliu <danfengl@vmware.com>
2018-11-08 10:33:14 +08:00
danfengliu
9265754936 modify codacy errors in target
Signed-off-by: danfengliu <danfengl@vmware.com>
2018-11-08 10:33:14 +08:00
danfengliu
a6151175e3 add lib for target
Signed-off-by: danfengliu <danfengl@vmware.com>
2018-11-08 10:33:14 +08:00
danfengliu
e6f5f62c53 Modify for codacy again
Signed-off-by: danfengliu <danfengl@vmware.com>
2018-11-08 10:33:14 +08:00
danfengliu
0b84ec84d9 Modify for codacy
Signed-off-by: danfengliu <danfengl@vmware.com>
2018-11-08 10:33:14 +08:00
danfengliu
660bfdf677 add test case of add a replication rule
Signed-off-by: danfengliu <danfengl@vmware.com>
2018-11-08 10:33:14 +08:00
danfengliu
1979d17550 add tc add-sys-label-to-tag and del-repo
Signed-off-by: danfengliu <danfengl@vmware.com>
2018-11-02 17:16:43 +08:00
danfengliu
894e0591bb add new api tests (#6127)
Add API test libraries which were based on swagger doc,
it's including user and project , also add one scenario
test case which is test_add_member_to_private_project to
verify a user can see a project when the uesr is the member of it.
Delete 2 old test case.Delete a quota.

Signed-off-by: danfengliu <danfengl@vmware.com>
2018-10-31 13:13:31 +08:00
Yan
002e5a2b70
Enable api tests in travis (#5900)
This commit is to enable parallel run jobs with travis, and add API for DB and API for LDAP in travis.

Signed-off-by: wang yan <wangyan@vmware.com>
2018-09-19 17:49:43 +08:00
Wenkai Yin
f71f02deee Fix module not found error in API test (#5647)
Reload the python path to fix the module not found error in API E2E test

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2018-08-17 10:55:19 +08:00
Wenkai Yin
95c0c259ec Add E2E API test cases for replication based on labels (#5639)
This commit creates a new E2E API test case to verify the replication based on label

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2018-08-16 18:36:27 +08:00
stonezdj
94f852b826 Fix testcase related to LDAP user can not see any logs 2018-08-10 16:17:18 +08:00
Yan
9c86d0b86f
Update ldap user pwd for nightly (#5571) 2018-08-09 16:38:27 +08:00
stonezdj
edcb2fbf05 Add Harbor API Test based on python API
Correct swagger.yaml format
Add make task swagger_client to gen and install python client.
Add LDAP related Test
2018-08-01 10:43:23 +08:00