Commit Graph

90 Commits

Author SHA1 Message Date
Steven Zou
ebc5d2482b do improvements to the scan all job
- update scan all job to avoid sending too many HTTP requets
- update scan controller to support scan options
- update the db schema of the scan report to introduce requester
- introduce scan all metrics to report the overall progress of scan all job
- fix the status updating bug in scan report
- enhance the admin job status updats
- add duplicate checking before triggering generic admin job
- update the db scheme of admin job

fix #9705
fix #9722
fix #9670

Signed-off-by: Steven Zou <szou@vmware.com>
2019-11-05 15:12:07 +08:00
wang yan
f9996663d8 update immutable rule API
1, unify disable and enable
2, fix update rule error

Signed-off-by: wang yan <wangyan@vmware.com>
2019-10-25 14:11:07 +08:00
Steven Zou
cb59ba3bbc support using internal registry addr to perform scan
- do changes to the sql schema
- add `UseInternalAddr` and `Immutable` properties to scanner registration
- support multiple authentication type
  - basic
  - bearer token

Signed-off-by: Steven Zou <szou@vmware.com>
2019-10-24 18:28:35 +08:00
He Weiwei
bf6a14c9ad
feat(role): introduce a limited guest role (#9403)
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-10-20 14:21:28 +08:00
Steven Zou
f18afc0a3f do changes to let the vul policy check compatiable with new framework
- update the scan/scanner controller
- enhance the report summary generation
- do changes to the vulnerable handler
- remove the unused clair related code
- add more UT cases
- update the scan web hook event
- drop the unsed tables/index/triggers in sql schema

Signed-off-by: Steven Zou <szou@vmware.com>
2019-10-16 23:15:26 +08:00
wang yan
25f638a989 Merge branch 'master' of https://github.com/goharbor/harbor into robot-invisiable 2019-10-14 14:35:45 +08:00
wang yan
3e81bd7f1d add visible attribute to robot account
The commit is to make robot controller could create invisible robot account for internal use

Signed-off-by: wang yan <wangyan@vmware.com>
2019-10-12 00:51:48 +08:00
Steven Zou
9fd8b6306c refactor code to reflect code review comments
- refactor the db schema \
- refactor  permission checking in API handlers \

to follow the latest code/interface changes

Signed-off-by: Steven Zou <szou@vmware.com>
2019-10-11 18:07:47 +08:00
Steven Zou
58afd8e14b [stage3] support pluggable scanner
- implement scan controller
- add scan resource and update role bindings
- update registration model and related interfaces

Signed-off-by: Steven Zou <szou@vmware.com>

- implement scan API to do scan/get report/get log
- update repository rest API to produce scan report summary
- update scan job hook handler
- update some UT cases

- update robot account making content
- hidden credential in the job log

Commnet scan related API test cases which will be re-activate later
fix #8985

fix the issues found by codacy

Signed-off-by: Steven Zou <szou@vmware.com>
2019-10-11 12:53:02 +08:00
DQ
2bc11200d6 Move db change to new migration file
DB should move to new migration file cause 1.9 is already released

Signed-off-by: DQ <dengq@vmware.com>
2019-09-25 15:57:03 +08:00
stonezdj(Daojun Zhang)
ec559b0585
Merge pull request #9123 from stonezdj/immutable_tags
Add DAO for immutable tags
2019-09-23 21:46:07 +08:00
stonezdj
29d2bcce99 Add DAO for immutable tags
Signed-off-by: stonezdj <stonezdj@gmail.com>
2019-09-23 16:45:07 +08:00
Steven Zou
d616bc3509 add scan report CRUD supporting and
- change error collection in scan job
- add dead client checking in client pool
- change key word type to interface{} for q.Query
- update bearer authorizer
- add required UT cases

Signed-off-by: Steven Zou <szou@vmware.com>
2019-09-23 16:21:39 +08:00
Steven Zou
0c19eba8c2 [stage2]support pluggable scanner
- add scanner rest API v1 spec
- implement v1 client which is used to talk to scanner adapter
- adjust data/orm models
- adjust code package structure

Signed-off-by: Steven Zou <szou@vmware.com>

- implement scan client which is used to talk to scanner adapter
- implement scan job which take the work of communicating with scanner
- update scanner mgmt API routes
- add corresponding UT cases
2019-09-23 09:37:54 +08:00
Steven Zou
e324a4d623 support pluggable scanner
- add DAO layer for scanner registration
- add CURD manager for scanner registration
- add API controller for plug scanner
- add REST APIs for CURD of plug scanner
- add migration sql:0011_1.10.0
- add scan interface definition (no implementations)
- add related UT cases with testify

fix #8979 #8990

Signed-off-by: Steven Zou <szou@vmware.com>
2019-09-18 21:56:45 +08:00
DQ
ea5c27fcd5 Enhance: Upgrade encrypt alg to sha256
previous sha1 will still used for old password

Signed-off-by: DQ <dengq@vmware.com>
2019-09-09 21:48:21 +08:00
Wenkai Yin(尹文开)
5da4286ef4 Hard delete project metadata (#8856)
Hard delete project metadata

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-08-29 12:14:39 +08:00
Wang Yan
87893abc5e
Merge pull request #8829 from ywk253100/190822_retry_status
Add status revision to handle retrying in replication task
2019-08-28 10:55:13 +08:00
Wenkai Yin
7924f37d86 Add status revision to handle retrying in replication task
Add status revision to handle retrying in replication task

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-08-27 14:17:11 +08:00
wang yan
28f7b7a64e fix #8839: update bolb size type to bigint
and extend the length of content type to 1024

Signed-off-by: wang yan <wangyan@vmware.com>
2019-08-27 10:43:29 +08:00
Wenkai Yin
661470e7bc Add status revision to retention task to handle retrying
Add status revision to retention task to handle retrying

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-08-22 20:04:25 +08:00
Steven Zou
217252a097
Merge pull request #8675 from ywk253100/190814_retention_task
Handle the retention task status updating in concurrency
2019-08-20 17:07:21 +08:00
Daniel Jiang
f10fb67d6d
Merge pull request #8662 from stonezdj/email_sec2
Set default email to null if not provided
2019-08-20 09:01:50 +08:00
stonezdj
5fa8eb7854 Set default email to null if not provided
Signed-off-by: stonezdj <stonezdj@gmail.com>
2019-08-19 15:20:44 +08:00
wang yan
6e11ecc6fc Update codes per review comments
Signed-off-by: wang yan <wangyan@vmware.com>
2019-08-16 14:58:52 +08:00
Wenkai Yin
48b067f596 Handle the retention task status updating in concurrency
Compare the status code when updating retention task status to avoid the concurrent issue

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-08-15 18:36:18 +08:00
Wang Yan
54a39c7159
Merge pull request #8597 from heww/size-quota
refactor(quota,middleware): implement size quota by quota interceptor
2019-08-09 15:44:33 +08:00
He Weiwei
e62c29123d refactor(quota,middleware): implement size quota by quota interceptor
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-08-08 23:55:54 +00:00
guanxiatao
7e40e335b7 add sql schema
Signed-off-by: guanxiatao <guanxiatao@corp.netease.com>

Signed-off-by: guanxiatao <guanxiatao@corp.netease.com>
2019-08-07 21:35:46 +08:00
Steven Zou
f3ba25f656
Merge pull request #8536 from bitsf/tag_retention_task_num
add task retain num
2019-08-07 17:39:39 +08:00
Ziming Zhang
026aee75d9 add task retain num
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I2f8b89454fe3bb9b56af237048c9e2b90783f434
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2019-08-05 17:57:09 +08:00
wang yan
6987825ffd Delete the nono scan all schedule in migration
Signed-off-by: wang yan <wangyan@vmware.com>
2019-08-01 16:59:13 +08:00
Wenkai Yin(尹文开)
7fd06edccf
Merge pull request #8451 from ywk253100/190729_retention_task
Add property "repository" to retention task
2019-07-29 17:18:42 +08:00
Ziming Zhang
ba47b4c00f get execution status on the fly
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: Iefcf8946d7a2c7a27bc22bd326ee9723b4b79c66
2019-07-29 14:48:39 +08:00
Wenkai Yin
a55860d2df Add property "repository" to retention task
Add property "repository" to retention task

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-07-29 13:47:18 +08:00
Wenkai Yin(尹文开)
d45674960f
Merge pull request #8417 from goharbor/project-quota-dev
Add feature project quota dev
2019-07-26 15:41:09 +08:00
Wenkai Yin
2e9521ad45 Support to stop one execution of retention
Support to stop one execution of retention

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-07-25 16:48:00 +08:00
wang yan
4763864dae merge with latest master code with quota feature branch
Signed-off-by: wang yan <wangyan@vmware.com>
2019-07-24 08:47:05 -07:00
Ziming
43c2af9857 map retention with policy (#8313)
Signed-off-by: Ziming Zhang <zziming@vmware.com>

Implement the API and controller of tag retention
 - API handler
 - retention controller
 - dao
2019-07-24 17:22:26 +08:00
He Weiwei
ce58c58c01 feat(quota,api): quota support for create project API
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-07-24 01:02:51 +08:00
Wenkai Yin
03cc8046eb Implement the task management in retention manager
Implement the task management in retention manager

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-07-23 19:23:04 +08:00
Wenkai Yin
7362fae7cc Implement a common scheduler
Implement a common scheduler that can be used globally

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-07-23 17:20:31 +08:00
He Weiwei
9c9b8d3a6d Merge branch 'master' into project-quota-dev 2019-07-19 10:02:51 +08:00
Steven Zou
746d082e2e Merge branch 'master' into feature/tag_retention 2019-07-18 10:40:49 +08:00
Ziming Zhang
815901ea33 fix
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I3f2d3c7f1e32b4983c31c23d9753f04239e3c82f
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2019-07-16 19:24:40 +08:00
stonezdj
bb2ae7c093 Add HTTP group feature
Signed-off-by: stonezdj <stonezdj@gmail.com>
2019-07-16 15:38:46 +08:00
wang yan
24c3753581 add dao of artifact
Signed-off-by: wang yan <wangyan@vmware.com>

Add dao for quota

Signed-off-by: He Weiwei <hweiwei@vmware.com>

fix govet

Signed-off-by: wang yan <wangyan@vmware.com>
2019-07-08 23:42:50 +08:00
Daniel Jiang
4aca812ff2 API for system level vulnerability whitelist
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-06-26 23:35:40 +08:00
Wang Yan
e7f5954c71
Fix migrator scan all schedule from v1.7.0 (#7763)
In v1.7.0, it creates an record for scan all but without cron string, just update the record with the cron in properties.

Signed-off-by: wang yan <wangyan@vmware.com>
2019-05-10 17:08:52 +08:00
Wenkai Yin
5ce57a24ac
Merge pull request #7739 from ywk253100/190508_upgrade
Fixes #7693, the filters of replication policy is lost after upgrade
2019-05-09 20:16:13 +08:00