Commit Graph

178 Commits

Author SHA1 Message Date
wang yan
f9996663d8 update immutable rule API
1, unify disable and enable
2, fix update rule error

Signed-off-by: wang yan <wangyan@vmware.com>
2019-10-25 14:11:07 +08:00
Steven Zou
868851cc51
Merge pull request #9571 from steven-zou/fix/support_internal_addr_scanner
support using internal registry addr to perform scan
2019-10-24 20:52:27 +08:00
Wang Yan
d18678a48d
Merge pull request #9506 from wy65701436/token-sevice
Enable robot account to support scan pull case
2019-10-24 19:52:33 +08:00
Steven Zou
cb59ba3bbc support using internal registry addr to perform scan
- do changes to the sql schema
- add `UseInternalAddr` and `Immutable` properties to scanner registration
- support multiple authentication type
  - basic
  - bearer token

Signed-off-by: Steven Zou <szou@vmware.com>
2019-10-24 18:28:35 +08:00
Ziming
13499fb60b
Merge pull request #9491 from bitsf/tag_retention_webhook
implement log for tag retention immutable tags
2019-10-24 17:06:00 +08:00
Ziming Zhang
e757899b49 add special error/log for not delete immutable tag in tag retention job
Change-Id: I3440f3b888bf8c65afc75d04253eea41f20eef0e
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2019-10-24 15:25:48 +08:00
Steven Zou
7fad103e46 - fix API test cases failures
Signed-off-by: Steven Zou <szou@vmware.com>

- fix scan report dao bug
2019-10-23 20:44:01 +08:00
wang yan
a6ad1b2db8 update code per review comments
Signed-off-by: wang yan <wangyan@vmware.com>
2019-10-23 20:05:51 +08:00
wang yan
2fa85aefca fix per comments
Signed-off-by: wang yan <wangyan@vmware.com>
2019-10-23 18:45:30 +08:00
wang yan
5996189bb0 update per comments and fix govet error
Signed-off-by: wang yan <wangyan@vmware.com>
2019-10-23 18:45:30 +08:00
wang yan
22b4ea0f89 Enable robot account bypass policy check
1, the commit is for internal robot to bypass policy check, like vul and signature checking.
2, add a bool attribute into registry token, decode it in the harbor core and add the status into request context.
3, add a bool attribut for robot API controller, but API will not use it.y

Signed-off-by: wang yan <wangyan@vmware.com>
2019-10-23 18:45:30 +08:00
Steven Zou
962bafb7ce fix go imports order issues
Signed-off-by: Steven Zou <szou@vmware.com>
2019-10-23 09:34:47 +08:00
Steven Zou
370a364c29 fix code conflict and rebase with master
Signed-off-by: Steven Zou <szou@vmware.com>
2019-10-22 18:39:37 +08:00
Steven Zou
dff1ee07fc improve the scan controlling
- add LCM control to the robot account generated for scanning
- improve the scan webhook
- remove reprots when related artifact is deleted
- update report manager/scan controller and other components to support above cases
- add artifact manager/comtroller to list artifacts

Signed-off-by: Steven Zou <szou@vmware.com>
2019-10-22 18:24:46 +08:00
Wang Yan
71bb8815bf
Merge pull request #9461 from reasonerjt/rm-validator-cve
Remove validation for item in CVE whitelist
2019-10-21 14:52:08 +08:00
Daniel Jiang
1a9cebd5e8 Remove validation for item in CVE whitelist
To contain various vulnerabilities in the CVE whitelist, this commit
removes the validation.
Fixes #9242

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-10-18 15:08:32 +08:00
Wenkai Yin(尹文开)
5c5e475da4
Merge pull request #9415 from steven-zou/fix/pluggable_scanner_policy_check
do changes to let the vul policy check compatible with new framework
2019-10-18 09:39:20 +08:00
Wang Yan
51d3134e4f
Merge pull request #9427 from wy65701436/immutable-middleware
add immutable tag middleware
2019-10-17 20:28:34 +08:00
Steven Zou
0f16913635 rebase: resolve the code confilcts with master
Signed-off-by: Steven Zou <szou@vmware.com>
2019-10-17 17:42:41 +08:00
wang yan
da02b820ad add immutable tag middleware
Signed-off-by: wang yan <wangyan@vmware.com>
2019-10-17 16:58:11 +08:00
He Weiwei
8964a8697a build(clair): internal clair adapter when install with clair
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-10-17 12:00:51 +08:00
Steven Zou
f18afc0a3f do changes to let the vul policy check compatiable with new framework
- update the scan/scanner controller
- enhance the report summary generation
- do changes to the vulnerable handler
- remove the unused clair related code
- add more UT cases
- update the scan web hook event
- drop the unsed tables/index/triggers in sql schema

Signed-off-by: Steven Zou <szou@vmware.com>
2019-10-16 23:15:26 +08:00
Wenkai Yin(尹文开)
372875ad64
Merge pull request #9393 from wy65701436/immutable-match
add immutable match
2019-10-15 18:51:43 +08:00
wang yan
ae4c698ab4 use only 1 repo and tag selector to match
Signed-off-by: wang yan <wangyan@vmware.com>
2019-10-15 16:44:17 +08:00
wang yan
5e8f7297f5 fix list robot account API return an internal error
Signed-off-by: wang yan <wangyan@vmware.com>
2019-10-15 12:29:44 +08:00
wang yan
a3546478eb add immutable match
Signed-off-by: wang yan <wangyan@vmware.com>
2019-10-14 16:21:32 +08:00
wang yan
7c16cbfeef fix scan controller test introduced by api change
Signed-off-by: wang yan <wangyan@vmware.com>
2019-10-14 14:39:46 +08:00
wang yan
25f638a989 Merge branch 'master' of https://github.com/goharbor/harbor into robot-invisiable 2019-10-14 14:35:45 +08:00
wang yan
c597e46756 do not expose visible attribute of robot account to user
Signed-off-by: wang yan <wangyan@vmware.com>
2019-10-14 14:11:09 +08:00
Steven Zou
6e8e601c8d make robot account with new robot controller
Signed-off-by: Steven Zou <szou@vmware.com>
2019-10-12 16:34:08 +08:00
Steven Zou
a86afd6ebc Merge branch 'master' into feature/pluggable_scanner_s3_merge 2019-10-12 15:18:06 +08:00
wang yan
3e81bd7f1d add visible attribute to robot account
The commit is to make robot controller could create invisible robot account for internal use

Signed-off-by: wang yan <wangyan@vmware.com>
2019-10-12 00:51:48 +08:00
wang yan
6f6f113f0f refactor robot api
1, add API controller for robot account, make it callable internally
2, add Manager to handler dao releate operation

Signed-off-by: wang yan <wangyan@vmware.com>
2019-10-11 17:26:18 +08:00
Steven Zou
58afd8e14b [stage3] support pluggable scanner
- implement scan controller
- add scan resource and update role bindings
- update registration model and related interfaces

Signed-off-by: Steven Zou <szou@vmware.com>

- implement scan API to do scan/get report/get log
- update repository rest API to produce scan report summary
- update scan job hook handler
- update some UT cases

- update robot account making content
- hidden credential in the job log

Commnet scan related API test cases which will be re-activate later
fix #8985

fix the issues found by codacy

Signed-off-by: Steven Zou <szou@vmware.com>
2019-10-11 12:53:02 +08:00
wang yan
8317100cda continue refactor API
Signed-off-by: wang yan <wangyan@vmware.com>
2019-10-09 23:06:35 +08:00
wang yan
7c4fd79b5c refactor immutable dao code to align the new structure under pkg
1, add manager
2, move model dao to /pkg/dao

Signed-off-by: wang yan <wangyan@vmware.com>
2019-09-26 20:35:58 +08:00
stonezdj
cc22a175b9 Add immutable tag API
Signed-off-by: stonezdj <stonezdj@gmail.com>
2019-09-25 15:53:56 +08:00
Steven Zou
a73f896f23
Merge pull request #9154 from steven-zou/feature/pluggable_scanner_s2
[stage2]support pluggable scanner
2019-09-23 21:12:27 +08:00
Steven Zou
d616bc3509 add scan report CRUD supporting and
- change error collection in scan job
- add dead client checking in client pool
- change key word type to interface{} for q.Query
- update bearer authorizer
- add required UT cases

Signed-off-by: Steven Zou <szou@vmware.com>
2019-09-23 16:21:39 +08:00
Steven Zou
0c19eba8c2 [stage2]support pluggable scanner
- add scanner rest API v1 spec
- implement v1 client which is used to talk to scanner adapter
- adjust data/orm models
- adjust code package structure

Signed-off-by: Steven Zou <szou@vmware.com>

- implement scan client which is used to talk to scanner adapter
- implement scan job which take the work of communicating with scanner
- update scanner mgmt API routes
- add corresponding UT cases
2019-09-23 09:37:54 +08:00
Ziming Zhang
f6ac2687fa fix dayspl/daysps test
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I3c654e305f774c2329109888f68e708efeca721e
2019-09-20 18:32:58 +08:00
Steven Zou
4c4897aef1
Merge pull request #9134 from steven-zou/feature/pluggable_scanners
support pluggable scanner
2019-09-19 16:08:24 +08:00
Steven Zou
e324a4d623 support pluggable scanner
- add DAO layer for scanner registration
- add CURD manager for scanner registration
- add API controller for plug scanner
- add REST APIs for CURD of plug scanner
- add migration sql:0011_1.10.0
- add scan interface definition (no implementations)
- add related UT cases with testify

fix #8979 #8990

Signed-off-by: Steven Zou <szou@vmware.com>
2019-09-18 21:56:45 +08:00
wang yan
42a5db83b2 refactor selector of retention
extract select from pkg/retention, move it to pkg/artselector to make it usable by immutable tag

Signed-off-by: wang yan <wangyan@vmware.com>
2019-09-18 16:38:41 +08:00
Ziming Zhang
722e45b20b add swagger for tag retention
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I0f3ed8085e231868de74c273ba85946826181d5b
2019-09-06 17:27:20 +08:00
Daniel Jiang
76a79869df The default item list should be empty list,not null
This commit make sure that the "items" in response of project level
CVE_whitelist is not null, even when it's null in the DB the API will
return an empty list

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-09-04 17:59:56 +08:00
Ziming Zhang
22e7f22b30 tag retention fix duration and show trigger
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I7f93dd4c1b93b8d7e6b2c2a14fd4ccf128945def
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2019-08-30 11:48:11 +08:00
Wenkai Yin(尹文开)
881884786d
Merge pull request #8883 from bitsf/tag_retention_log
enhance tag retention log
2019-08-30 10:48:01 +08:00
Ziming Zhang
247ef1b884 enhance tag retention log
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: Id5946c2dd53868fdd5f5abbcca45e453bc9ba4fd
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2019-08-29 18:40:00 +08:00
Ziming Zhang
8fb6e2f65b verify permission of GetRetentionExecTaskLog
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I4bf4ddf3d3ed6f07a4618e242e2f3774996716d6
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2019-08-29 15:01:53 +08:00