Upstream/harbor
1
0
Fork 0
mirror of https://github.com/goharbor/harbor.git synced 2024-11-18 08:15:16 +01:00
Code Issues Projects Releases Wiki Activity
12,014 Commits 49 Branches 316 Tags 351 MiB
80a9c688fc
Commit Graph

873 Commits

Author SHA1 Message Date
Ziming Zhang
e757899b49 add special error/log for not delete immutable tag in tag retention job 
Change-Id: I3440f3b888bf8c65afc75d04253eea41f20eef0e
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2019-10-24 15:25:48 +08:00
Steven Zou
7fad103e46 - fix API test cases failures 
Signed-off-by: Steven Zou <szou@vmware.com>

- fix scan report dao bug
 2019-10-23 20:44:01 +08:00
wang yan
a6ad1b2db8 update code per review comments 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-10-23 20:05:51 +08:00
wang yan
2fa85aefca fix per comments 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-10-23 18:45:30 +08:00
wang yan
5996189bb0 update per comments and fix govet error 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-10-23 18:45:30 +08:00
wang yan
22b4ea0f89 Enable robot account bypass policy check 
1, the commit is for internal robot to bypass policy check, like vul and signature checking.
2, add a bool attribute into registry token, decode it in the harbor core and add the status into request context.
3, add a bool attribut for robot API controller, but API will not use it.y

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-10-23 18:45:30 +08:00
Steven Zou
962bafb7ce fix go imports order issues 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-10-23 09:34:47 +08:00
Steven Zou
370a364c29 fix code conflict and rebase with master 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-10-22 18:39:37 +08:00
Steven Zou
dff1ee07fc improve the scan controlling 
- add LCM control to the robot account generated for scanning
- improve the scan webhook
- remove reprots when related artifact is deleted
- update report manager/scan controller and other components to support above cases
- add artifact manager/comtroller to list artifacts

Signed-off-by: Steven Zou <szou@vmware.com>
 2019-10-22 18:24:46 +08:00
Wang Yan
71bb8815bf
Merge pull request #9461 from reasonerjt/rm-validator-cve 
Remove validation for item in CVE whitelist
 2019-10-21 14:52:08 +08:00
Daniel Jiang
1a9cebd5e8 Remove validation for item in CVE whitelist 
To contain various vulnerabilities in the CVE whitelist, this commit
removes the validation.
Fixes #9242

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-10-18 15:08:32 +08:00
Wenkai Yin(尹文开)
5c5e475da4
Merge pull request #9415 from steven-zou/fix/pluggable_scanner_policy_check 
do changes to let the vul policy check compatible with new framework
 2019-10-18 09:39:20 +08:00
Wang Yan
51d3134e4f
Merge pull request #9427 from wy65701436/immutable-middleware 
add immutable tag middleware
 2019-10-17 20:28:34 +08:00
Steven Zou
0f16913635 rebase: resolve the code confilcts with master 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-10-17 17:42:41 +08:00
wang yan
da02b820ad add immutable tag middleware 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-10-17 16:58:11 +08:00
He Weiwei
8964a8697a build(clair): internal clair adapter when install with clair 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-10-17 12:00:51 +08:00
Steven Zou
f18afc0a3f do changes to let the vul policy check compatiable with new framework 
- update the scan/scanner controller
- enhance the report summary generation
- do changes to the vulnerable handler
- remove the unused clair related code
- add more UT cases
- update the scan web hook event
- drop the unsed tables/index/triggers in sql schema

Signed-off-by: Steven Zou <szou@vmware.com>
 2019-10-16 23:15:26 +08:00
Wenkai Yin(尹文开)
372875ad64
Merge pull request #9393 from wy65701436/immutable-match 
add immutable match
 2019-10-15 18:51:43 +08:00
wang yan
ae4c698ab4 use only 1 repo and tag selector to match 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-10-15 16:44:17 +08:00
wang yan
5e8f7297f5 fix list robot account API return an internal error 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-10-15 12:29:44 +08:00
wang yan
a3546478eb add immutable match 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-10-14 16:21:32 +08:00
wang yan
7c16cbfeef fix scan controller test introduced by api change 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-10-14 14:39:46 +08:00
wang yan
25f638a989 Merge branch 'master' of https://github.com/goharbor/harbor into robot-invisiable 2019-10-14 14:35:45 +08:00
wang yan
c597e46756 do not expose visible attribute of robot account to user 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-10-14 14:11:09 +08:00
Steven Zou
6e8e601c8d make robot account with new robot controller 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-10-12 16:34:08 +08:00
Steven Zou
a86afd6ebc Merge branch 'master' into feature/pluggable_scanner_s3_merge 2019-10-12 15:18:06 +08:00
wang yan
3e81bd7f1d add visible attribute to robot account 
The commit is to make robot controller could create invisible robot account for internal use

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-10-12 00:51:48 +08:00
wang yan
6f6f113f0f refactor robot api 
1, add API controller for robot account, make it callable internally
2, add Manager to handler dao releate operation

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-10-11 17:26:18 +08:00
Steven Zou
58afd8e14b [stage3] support pluggable scanner 
- implement scan controller
- add scan resource and update role bindings
- update registration model and related interfaces

Signed-off-by: Steven Zou <szou@vmware.com>

- implement scan API to do scan/get report/get log
- update repository rest API to produce scan report summary
- update scan job hook handler
- update some UT cases

- update robot account making content
- hidden credential in the job log

Commnet scan related API test cases which will be re-activate later
fix #8985

fix the issues found by codacy

Signed-off-by: Steven Zou <szou@vmware.com>
 2019-10-11 12:53:02 +08:00
wang yan
8317100cda continue refactor API 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-10-09 23:06:35 +08:00
wang yan
7c4fd79b5c refactor immutable dao code to align the new structure under pkg 
1, add manager
2, move model dao to /pkg/dao

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-09-26 20:35:58 +08:00
stonezdj
cc22a175b9 Add immutable tag API 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-09-25 15:53:56 +08:00
Steven Zou
a73f896f23
Merge pull request #9154 from steven-zou/feature/pluggable_scanner_s2 
[stage2]support pluggable scanner
 2019-09-23 21:12:27 +08:00
Steven Zou
d616bc3509 add scan report CRUD supporting and 
- change error collection in scan job
- add dead client checking in client pool
- change key word type to interface{} for q.Query
- update bearer authorizer
- add required UT cases

Signed-off-by: Steven Zou <szou@vmware.com>
 2019-09-23 16:21:39 +08:00
Steven Zou
0c19eba8c2 [stage2]support pluggable scanner 
- add scanner rest API v1 spec
- implement v1 client which is used to talk to scanner adapter
- adjust data/orm models
- adjust code package structure

Signed-off-by: Steven Zou <szou@vmware.com>

- implement scan client which is used to talk to scanner adapter
- implement scan job which take the work of communicating with scanner
- update scanner mgmt API routes
- add corresponding UT cases
 2019-09-23 09:37:54 +08:00
Ziming Zhang
f6ac2687fa fix dayspl/daysps test 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I3c654e305f774c2329109888f68e708efeca721e
 2019-09-20 18:32:58 +08:00
Steven Zou
4c4897aef1
Merge pull request #9134 from steven-zou/feature/pluggable_scanners 
support pluggable scanner
 2019-09-19 16:08:24 +08:00
Steven Zou
e324a4d623 support pluggable scanner 
- add DAO layer for scanner registration
- add CURD manager for scanner registration
- add API controller for plug scanner
- add REST APIs for CURD of plug scanner
- add migration sql:0011_1.10.0
- add scan interface definition (no implementations)
- add related UT cases with testify

fix #8979 #8990

Signed-off-by: Steven Zou <szou@vmware.com>
 2019-09-18 21:56:45 +08:00
wang yan
42a5db83b2 refactor selector of retention 
extract select from pkg/retention, move it to pkg/artselector to make it usable by immutable tag

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-09-18 16:38:41 +08:00
Ziming Zhang
722e45b20b add swagger for tag retention 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I0f3ed8085e231868de74c273ba85946826181d5b
 2019-09-06 17:27:20 +08:00
Daniel Jiang
76a79869df The default item list should be empty list,not null 
This commit make sure that the "items" in response of project level
CVE_whitelist is not null, even when it's null in the DB the API will
return an empty list

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-09-04 17:59:56 +08:00
Ziming Zhang
22e7f22b30 tag retention fix duration and show trigger 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I7f93dd4c1b93b8d7e6b2c2a14fd4ccf128945def
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2019-08-30 11:48:11 +08:00
Wenkai Yin(尹文开)
881884786d
Merge pull request #8883 from bitsf/tag_retention_log 
enhance tag retention log
 2019-08-30 10:48:01 +08:00
Ziming Zhang
247ef1b884 enhance tag retention log 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: Id5946c2dd53868fdd5f5abbcca45e453bc9ba4fd
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2019-08-29 18:40:00 +08:00
Ziming Zhang
8fb6e2f65b verify permission of GetRetentionExecTaskLog 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I4bf4ddf3d3ed6f07a4618e242e2f3774996716d6
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2019-08-29 15:01:53 +08:00
Wang Yan
03309680ff
Merge pull request #8873 from bitsf/tag_retention_log_time 
show empty if time is 0
 2019-08-29 12:39:22 +08:00
Daniel Jiang
9fa70db866
Merge pull request #8867 from bitsf/tag_retention_same_digest_tag 
Tag retention same digest tag
 2019-08-29 12:26:44 +08:00
Ziming Zhang
57a34a7d8d show empty if time is 0 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I5acc13714dbb1dc1d8bf3f1c6c620a7d54f86910
 2019-08-29 12:10:22 +08:00
Ziming Zhang
6137065eba prevent retained tag with same digest deleted by other tag 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I527ec87f48348c6607b1c9529c90f5ad804bd8f6
 2019-08-29 10:21:13 +08:00
Ziming Zhang
4248f7a45e prevent retained tag with same digest deleted by other tag 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I37a3ef0d4cff7214e35076453494d89f0ad84d43
 2019-08-28 18:27:35 +08:00
Ziming Zhang
8290af6d25 prevent retained tag with same digest deleted by other tag 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: Ie5f77db58e71636cd9aaf322bb3f83a0090f8877
 2019-08-28 18:24:55 +08:00
Ziming Zhang
063f2bfee9 prevent retained tag with same digest deleted by other tag 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I6a69b405b454ca0350677204e06bfa2b24616b33
 2019-08-28 17:48:42 +08:00
Ziming
94138137d5
add valid for rule (#8846) 
Change-Id: I82215a0cf1ec32a253c8db9bfafe7e25b26c9ad9
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2019-08-28 16:58:49 +08:00
Steven Zou
a80969e7af fix rule logic bug described in #8817 (#8818) 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-08-23 18:49:44 +08:00
Ziming Zhang
52b9fbc8f3 fix tag retention model verify 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: Ifb237bbed4c6d7272ace62da7603e027c1481aef
 2019-08-23 15:48:33 +08:00
Wang Yan
e7488e37b6
Merge pull request #8788 from bitsf/tag_retention_validate_model 
add tag retention model test
 2019-08-23 13:57:51 +08:00
Ziming Zhang
39db65e90f add tag retention model test 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I3f8b06b994024fe6052b0dee87600ed932edaaff
 2019-08-23 13:24:28 +08:00
Wenkai Yin
661470e7bc Add status revision to retention task to handle retrying 
Add status revision to retention task to handle retrying

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-08-22 20:04:25 +08:00
He Weiwei
311028be37
test: fix TestIsNegative (#8783) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-08-22 15:27:49 +08:00
Ziming Zhang
06e9467b06 1. remove rule none 
2. change rule orders
3. remove laber selector

Change-Id: Idc18a27cb0267f5f5c80a04b381e4a5dc6998508
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2019-08-21 20:24:33 +08:00
Steven Zou
217252a097
Merge pull request #8675 from ywk253100/190814_retention_task 
Handle the retention task status updating in concurrency
 2019-08-20 17:07:21 +08:00
Wenkai Yin(尹文开)
29ab93ad9c
Merge pull request #8714 from ywk253100/190816_scheduler 
Fix bug found in scheduler
 2019-08-20 14:11:04 +08:00
He Weiwei
75772aae11
refactor(quota): new error types for quota checking (#8726) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-08-19 19:00:29 +08:00
Ziming
0c79352c9c
fix tag retention schedule none error (#8715) 
Change-Id: I04cad9d4c520db751bfa413bb139317563716501
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2019-08-19 12:25:54 +08:00
Daniel Jiang
b3abd0316b
Merge pull request #8713 from reasonerjt/fix-8702 
Avoid overwriting system CVE whitelist by mistake
 2019-08-19 01:42:58 +08:00
Wenkai Yin
5c286d799f Fix bug found in scheduler 
The scheduler hook handler doesn't parse the job status struct when handling the hook. This commit fixes it.

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-08-16 13:54:57 +08:00
Daniel Jiang
30bb2ddcdf Avoid overwriting system CVE whitelist by mistake 
Fixes #8702
Also enforce the code to mitigate the potential risk.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-08-16 13:28:16 +08:00
Ziming
c279b7f3e9
fix retention rule compute error (#8664) 
Change-Id: I16d7284b17508885e136f2d9ea5651978ba4a6d8
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2019-08-15 20:12:59 +08:00
Wenkai Yin
48b067f596 Handle the retention task status updating in concurrency 
Compare the status code when updating retention task status to avoid the concurrent issue

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-08-15 18:36:18 +08:00
guanxiatao
e7fafd1941 webhook policy, job, event support 
Signed-off-by: guanxiatao <guanxiatao@corp.netease.com>
 2019-08-07 20:30:26 +08:00
Steven Zou
f3ba25f656
Merge pull request #8536 from bitsf/tag_retention_task_num 
add task retain num
 2019-08-07 17:39:39 +08:00
Wenkai Yin
216ef269b3 Populate pull/push time properties to the returning data when listing tags 
Populate pull/push time properties to the returning data when listing tags

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-08-07 11:47:05 +08:00
Wenkai Yin(尹文开)
556e07f0c4
Merge pull request #8551 from nlowe/feat/retention/GH-8548-Pulled-in-the-last-N-Days 
Retention: New Evaluator: Pulled within the last N Days
 2019-08-07 11:37:46 +08:00
Wenkai Yin(尹文开)
a00b1aab8d
Merge pull request #8581 from ywk253100/190806_retention_disable_chart 
Comment the related code for chart retention
 2019-08-07 11:04:43 +08:00
Nathan Lowe
9a7df265ce
Retention: New Evaluator: Pulled within the last N Days 
Signed-off-by: Nathan Lowe <public@nlowe.me>
 2019-08-06 22:28:28 -04:00
Wenkai Yin(尹文开)
ec4fa753d7
Merge pull request #8552 from nlowe/feat/retention/GH-8549-Pushed-in-the-last-N-Days 
Retention: New Evaluator: Retain Images pushed within N days
 2019-08-07 09:55:05 +08:00
Steven Zou
bbc7d1cb24
Merge pull request #8562 from bitsf/tag_retention_nothing_rule 
retain nothing rule
 2019-08-06 20:15:19 +08:00
Steven Zou
92c2cfa35a
Merge pull request #8529 from bitsf/tag_retention_disable_rule 
add disable rule feature for tag retention
 2019-08-06 20:05:28 +08:00
Wenkai Yin
0ac4ae687d Comment the related code for chart retention 
As we don't provide the capability for chart rerention currently, this commit comments related code

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-08-06 19:07:38 +08:00
Ziming Zhang
498a813299 retain nothing rule 
Change-Id: I4e7a4ecb40fe39b80e41a6d9bf8b5fb3968a41af
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2019-08-06 15:10:27 +08:00
Ziming Zhang
57e7854beb add disable rule feature for tag retention 
Change-Id: I335f1fb3e1273c945dda85999a0218440092be12
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2019-08-06 15:05:20 +08:00
Nathan Lowe
0db203a395
Fix failing test for the index 
Signed-off-by: Nathan Lowe <public@nlowe.me>
 2019-08-05 23:56:15 -04:00
Nathan Lowe
9f88e78381
Register the new evaluator with the index package 
Signed-off-by: Nathan Lowe <public@nlowe.me>
 2019-08-05 20:05:33 -04:00
Ziming Zhang
026aee75d9 add task retain num 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I2f8b89454fe3bb9b56af237048c9e2b90783f434
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2019-08-05 17:57:09 +08:00
Nathan Lowe
14eff4d9f2
Add comment to make the linter happy 
Signed-off-by: Nathan Lowe <public@nlowe.me>
 2019-08-05 00:30:29 -04:00
Nathan Lowe
71ada8b21d
Retention: New Evaluator: Retain Images pushed within N days 
Signed-off-by: Nathan Lowe <public@nlowe.me>
 2019-08-04 20:12:38 -04:00
Steven Zou
3b45840bb0
Merge pull request #8544 from bitsf/tag_retention_fix_pagenum 
add total page num for tag retention
 2019-08-02 18:18:21 +08:00
Ziming Zhang
decffdd6a4 add total page num for tag retention 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I58a250dbb643f6949c1e34aa3a84a01dc3e0b285
 2019-08-02 14:43:52 +08:00
Wenkai Yin
1aa5e631d7 Remove the retention job for deleting repository 
Remove the retention job for deleting repository

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-08-02 10:27:19 +08:00
Steven Zou
75707adeb9
Merge pull request #8509 from ywk253100/190731_retention 
Fix retention bug

Urgent case, need to merge now
 2019-07-31 20:33:37 +08:00
Wenkai Yin
05934d1818 Fix retention bug 
Fix retention bug that passing the wrong rules to retention job

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-07-31 19:53:04 +08:00
Steven Zou
49c1b2d4f1 fix bug: dry run is missing when deleting ropo 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-07-31 17:14:08 +08:00
Steven Zou
a63d3b91e3
Merge pull request #8487 from bitsf/tag_retention_fix 
fix tag_retention bug
 2019-07-30 22:35:07 +08:00
Ziming Zhang
98994ce459 fix tag_retention bug 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I8e645c351817afede442a1bf29dd7ad6da31ad06
 2019-07-30 19:03:39 +08:00
Wenkai Yin
61f54cc4a9 Refactor retention launcher 
Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-07-30 16:10:33 +08:00
Steven Zou
63d16d4b6b
Merge pull request #8475 from steven-zou/fix/add_del_job 
add retention job for deleting repository
 2019-07-30 14:19:46 +08:00
Steven Zou
f29d50e6dd fix ut case failures 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-07-30 11:56:17 +08:00
Wenkai Yin(尹文开)
b90874b06a
Merge pull request #8457 from bitsf/tag_retention_api_fix 
fix tag retention api bug
 2019-07-30 10:24:28 +08:00
Steven Zou
1e7f51615f fix conflicts in runtime/bootstrap.go 2019-07-30 09:52:22 +08:00
Steven Zou
3ca5116a53 add retention job for deleting repository 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-07-30 09:45:19 +08:00
Ziming
a5c31d7160
Merge pull request #8456 from ywk253100/190729_scheduler 
Register the scheduler job and fix bugs
 2019-07-30 08:57:10 +08:00
Ziming
81950980bd
fix ci 2019-07-30 08:33:32 +08:00
Steven Zou
4bf7f7b3e4
Merge pull request #8445 from steven-zou/fix/tag_retention 
refactor index registering processes
 2019-07-30 07:58:14 +08:00
Wenkai Yin
94c1cf8ac9 Register the scheduler job and fix bugs 
1. Register the scheduler job
2. Fix retention bugs

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-07-30 07:37:47 +08:00
Ziming Zhang
e8f9f58550 fix tag retention api bug 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I26bb4aade70f35132a345a945fa19d35e8c92571
 2019-07-29 19:29:03 +08:00
Ziming Zhang
7cc29f0b90 fix tag retention api bug 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I2c295090de34d133671c01422a6812fa426e9be7
 2019-07-29 18:36:51 +08:00
Steven Zou
fa678a12cc fix the OR processor failure issue 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-07-29 17:43:20 +08:00
Ziming Zhang
b0f6000f16 fix tag retention api bug 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: Id01b3a00ee0fa18cb5b9e4a74bb61a76d7d29709
 2019-07-29 17:40:38 +08:00
Wenkai Yin(尹文开)
7fd06edccf
Merge pull request #8451 from ywk253100/190729_retention_task 
Add property "repository" to retention task
 2019-07-29 17:18:42 +08:00
Steven Zou
640fcdaee3 enhance the label selector 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-07-29 16:31:47 +08:00
Ziming Zhang
ba47b4c00f get execution status on the fly 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: Iefcf8946d7a2c7a27bc22bd326ee9723b4b79c66
 2019-07-29 14:48:39 +08:00
Steven Zou
6479a22a08 refacor index registering processes 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-07-29 14:01:47 +08:00
Wenkai Yin
a55860d2df Add property "repository" to retention task 
Add property "repository" to retention task

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-07-29 13:47:18 +08:00
Ziming
5a390d2fa0
Merge pull request #8429 from ywk253100/190726_retention_chartserver 
Only listing chart repositories when chart server is enabled
 2019-07-28 22:52:34 +08:00
Ziming
67b65814d9
Merge pull request #8434 from ywk253100/190727_retention 
Fix bugs found in running the retention workflow
 2019-07-28 22:52:16 +08:00
Wenkai Yin
ab792019db Fix bugs found in running the retention workflow 
Fix bugs found in running the retention workflow

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-07-28 10:36:57 +08:00
Wenkai Yin
9235dcff54 Only listing chart repositories when chart server is enabled 
Only listing chart repositories when chart server is enabled

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-07-28 09:08:41 +08:00
Ziming Zhang
83b045f5ec add ut for tag retention controller 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I1469ee13675537ec389a068e4bc29e457b402fa4
 2019-07-26 17:55:11 +08:00
Steven Zou
3a7066346b
Merge pull request #8418 from ywk253100/190725_retention_task_log 
Implement getting the retention task log
 2019-07-26 17:51:05 +08:00
Wenkai Yin
c49a09edb4 Implement getting the retention task log 
Implement getting the retention task log

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-07-26 16:19:37 +08:00
Wenkai Yin(尹文开)
d45674960f
Merge pull request #8417 from goharbor/project-quota-dev 
Add feature project quota dev
 2019-07-26 15:41:09 +08:00
wang yan
676b922c95 merge with latest master code with quota branch 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-07-26 01:32:45 +08:00
Wenkai Yin
2e9521ad45 Support to stop one execution of retention 
Support to stop one execution of retention

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-07-25 16:48:00 +08:00
Steven Zou
584dcd8571 support dry run of retention 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-07-25 12:53:19 +08:00
wang yan
4763864dae merge with latest master code with quota feature branch 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-07-24 08:47:05 -07:00
Ziming
43c2af9857 map retention with policy (#8313) 
Signed-off-by: Ziming Zhang <zziming@vmware.com>

Implement the API and controller of tag retention
 - API handler
 - retention controller
 - dao
 2019-07-24 17:22:26 +08:00
Wenkai Yin
1adf5c976e Create retention client with the configurations got from jobservice config 
As the retention client is only used in job which running in jobservice, the configurations that needed to init the client should be read from jobservice

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-07-24 13:10:10 +08:00
Wenkai Yin(尹文开)
c4836727be
Merge pull request #8377 from steven-zou/feature/tag_retention-logs 
print rich logs in the retention job
 2019-07-24 09:54:19 +08:00
Steven Zou
09c56bfa08 print rich logs in the retention job 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-07-24 09:02:51 +08:00
He Weiwei
ce58c58c01 feat(quota,api): quota support for create project API 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-07-24 01:02:51 +08:00
Wenkai Yin
03cc8046eb Implement the task management in retention manager 
Implement the task management in retention manager

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-07-23 19:23:04 +08:00
Wenkai Yin
7362fae7cc Implement a common scheduler 
Implement a common scheduler that can be used globally

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-07-23 17:20:31 +08:00
Steven Zou
ccad9c0a29 support stop in the retention job and register it 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-07-23 11:05:22 +08:00
Steven Zou
6401314a3d
Merge pull request #8356 from steven-zou/feature/tag_retention-ut 
add missing UT cases for index and builder components
 2019-07-22 12:47:38 +08:00
Nathan Lowe
a15be60946
Retention: Implement Rule: Always Keep 
Signed-off-by: Nathan Lowe <public@nlowe.me>
 2019-07-21 22:16:47 -04:00
Steven Zou
01d3243156 add missing UT cases for index and builder components 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-07-22 10:05:50 +08:00
Steven Zou
deb021eb48 rebase and fix name conflicts among packages and vars 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-07-19 22:37:54 +08:00
Steven Zou
c76c5d7a1f refactor packages to resolve the cycle importing issue 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-07-19 18:55:08 +08:00
Wenkai Yin
ddbde72d2e Move the pkg manager/controller declaration to the api package 
Move the pkg manager/controller declaration to the api package to avoid the dependency cycle

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-07-19 17:21:58 +08:00
Steven Zou
f0ea62caa9 Merge branch 'feature/tag_retention' into feature/tag_retention-performer 2019-07-19 16:00:13 +08:00
Steven Zou
c2b5d95e02 implement retain action performer 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-07-19 15:59:21 +08:00
Wenkai Yin
5f1d2bd644 Fix package import cycle issue 
Fix package import cycle issue

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-07-19 13:50:55 +08:00
Steven Zou
24190291b6 update doublestar pattern and the failed launcher UT cases 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-07-18 14:50:02 +08:00
Steven Zou
0994e5efc9
Merge pull request #8321 from nlowe/feat/retention/GH-6657-last-active-n 
Retention: Implement Evaluator: Retain Latest K 'Active'
 2019-07-18 12:03:41 +08:00
Steven Zou
c8d2a2384a
Merge pull request #8289 from nlowe/feat/retention/GH-6659-keep-most-recently-pulled 
Retention: Implement Evaluator: Keep N Most Recently Pulled
 2019-07-18 11:27:00 +08:00
Steven Zou
2d62b2eb33
Merge pull request #8317 from steven-zou/feature/retention-selectors 
implement label and regexp selectors
 2019-07-18 11:25:08 +08:00
Nathan Lowe
040b235472
Retention: Implement Evaluator: Retain Latest K 'Active' 
Signed-off-by: Nathan Lowe <public@nlowe.me>
 2019-07-17 23:14:29 -04:00
Steven Zou
746d082e2e Merge branch 'master' into feature/tag_retention 2019-07-18 10:40:49 +08:00
Steven Zou
be8a9c0446 implement label and regexp selectors 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-07-18 10:32:47 +08:00
Nathan Lowe
b61f45e038
Retention: Implement Evaluator: Keep N Most Recently Pulled 
Signed-off-by: Nathan Lowe <public@nlowe.me>
 2019-07-17 14:29:37 -04:00
Wenkai Yin(尹文开)
a64e089773
Merge pull request #8210 from stonezdj/http_group_dao2 
Add HTTP group support
 2019-07-17 15:22:36 +08:00
Wenkai Yin
d6c6231e08 Implement the retention client 
Implement the retention client

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-07-17 14:04:29 +08:00
Steven Zou
0b2f94b0dd
Merge pull request #8256 from goharbor/feat/retention/GH-6661-retain-newer-than-n-days 
Retention: Implement Evaluator: Retain if created less than x days ago
 2019-07-17 00:04:40 +08:00
Steven Zou
3243d5bce1
Merge pull request #8255 from goharbor/feat/retention/GH-6660-keep-most-recent-n 
Retention: Implement Filter: Keep Latest K
 2019-07-17 00:04:02 +08:00
Ziming Zhang
815901ea33 fix 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I3f2d3c7f1e32b4983c31c23d9753f04239e3c82f
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2019-07-16 19:24:40 +08:00
stonezdj
bb2ae7c093 Add HTTP group feature 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-07-16 15:38:46 +08:00
Ziming Zhang
c22c38994a retention api 
Change-Id: I70f2c34d6bb96ecf4cb5359e2b1ab2dbb99fdbf9
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2019-07-16 15:06:37 +08:00
Nathan Lowe
d7e6b1b621
Sort artifacts before processing and port tests to github.com/stretchr/testify/suite 
Signed-off-by: Nathan Lowe <public@nlowe.me>
 2019-07-14 22:45:36 -04:00
Nathan Lowe
f03ed9b14c
Port tests to github.com/stretchr/testify/suite 
Signed-off-by: Nathan Lowe <public@nlowe.me>
 2019-07-14 22:38:26 -04:00
Wenkai Yin(尹文开)
44ad142d86
Merge pull request #8260 from ywk253100/190709_launcher 
Implement the launcher
 2019-07-11 18:21:04 +08:00
Wenkai Yin
91b050a01b Implement the launcher 
The commit implements the launcher for tag retention

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-07-11 17:18:46 +08:00
Steven Zou
69ff8ff8ba
Merge pull request #8277 from steven-zou/feature/tag_retentio_more_evaluators 
add more rule evaluators
 2019-07-11 14:09:29 +08:00
Steven Zou
07250469ac add more rule evaluators 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-07-11 10:50:10 +08:00
Steven Zou
52ca6de22a refactor the processor interface 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-07-11 00:23:19 +08:00
Steven Zou
9756195496 implement retention job 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-07-10 14:01:17 +08:00
Nathan Lowe
8b174825c1
Retention: Implement Evaluator: Retain if created less than x days ago 
Signed-off-by: Nathan Lowe <public@nlowe.me>
 2019-07-09 23:46:26 -04:00
Nathan Lowe
4ecbe749e4
Retention: Implement Filter: Keep Latest K 
Signed-off-by: Nathan Lowe <public@nlowe.me>
 2019-07-09 23:24:56 -04:00
Steven Zou
185f4f3861 refine or.processor implementation and also add UT case 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-07-09 22:10:09 +08:00
Steven Zou
3409065438 implement policy builder 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-07-09 17:43:19 +08:00
Steven Zou
c36afcd07d add index for rule,selector and performer 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-07-09 15:10:53 +08:00
Steven Zou
24ee32d7d1 define the base interfaces for the tag retention feature 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-07-09 09:40:59 +08:00
Daniel Jiang
8f5f0031c7 Enable project level CVE whitelist 
This commit update the project API to support "reuse_sys_cve_whitelist"
setting in project metadata and "cve_whitelist" in project request.
Also modify the interceptor to support project level CVE whitelist if
the reuse flag is false.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-07-08 18:55:54 +08:00
Daniel Jiang
bba4b2a6a4 Apply CVE white list in interceptor 
Interceptor will filter the vulnerability in whitelist while calculating
the serverity of an image and determine whether or not to block client
form pulling it.

It will use the system level whitelist in this commit, another commit
will switch to project level whitelist based on setting in a project.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-07-03 14:13:00 +08:00