Commit Graph

10252 Commits

Author SHA1 Message Date
danfengliu
f57dc13081 Add py-test for system robot account
In this py-test, add a new system robot account, and verify all project privileges.

Signed-off-by: danfengliu <danfengl@vmware.com>
2021-01-08 12:01:53 +08:00
He Weiwei
4a326aa8b0 chore: delete records of scan_report
The report in previous scan_report records not work well the
vulnerabilities stored in the schema table, so delete the scan_report
records.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-01-08 03:39:11 +00:00
He Weiwei
755c6490f9
feat: remove duplicate CVE in scan report and summary (#13918)
1. Remove the duplicate CVE records in the report/summary for the image
index.
2. Add scanner field in the scan overview for the API.

Closes #13913

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-01-08 11:00:43 +08:00
Wenkai Yin(尹文开)
4580aeff3b
Merge pull request #13910 from mmpei/official-feature-artifact-hub-replicate
Support artifact hub replication using new API
2021-01-08 10:15:55 +08:00
He Weiwei
9402077695
feat: save summary for the scan all execution (#13931)
Compute the summary info for the scan all and save it to the extra attrs
of the execution.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-01-08 10:10:31 +08:00
Qian Deng
642d56041d
Add san for notary cert (#13928)
Signed-off-by: DQ <dengq@vmware.com>
2021-01-08 01:00:34 +08:00
Will Sun
4ea881564e
Modify UI to match scanner upgrading (#13915)
Signed-off-by: AllForNothing <sshijun@vmware.com>
2021-01-07 17:16:52 +08:00
Wang Yan
0cf43d766c
enable system resource access (#13826)
1, introduce & define the system resources.
2, replace the IsSysAdmin judge method.
3, give the robot the system access capability.

Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-01-07 15:45:04 +08:00
Steven Zou
be98748ca7
Merge pull request #13896 from ywk253100/210105_replication
Fix replication filter bug
2021-01-07 13:45:46 +08:00
Wenkai Yin(尹文开)
3408f0b577
Merge pull request #13909 from ywk253100/210106_task
Provide a mechanism to sweep the execution/task records in task manager
2021-01-07 13:44:21 +08:00
Will Sun
bd585481c7
Fix tag retention ui loading issue (#13905)
Signed-off-by: AllForNothing <sshijun@vmware.com>
2021-01-07 10:20:12 +08:00
peimingming
28714f8b70 Support artifact hub replication using new API
Signed-off-by: peimingming <peimingming@corp.netease.com>
2021-01-06 20:46:04 +08:00
stonezdj(Daojun Zhang)
2ae57cdeb5
Merge pull request #13897 from stonezdj/210105_add_quay_proxy
Add  quay registry to proxy cache
2021-01-06 20:27:27 +08:00
He Weiwei
ed31cf9417
feat: return scan report and summary by header (#13898)
Add X-Accept-Vulnerabilities header to the list/get artifact and get
artifact vulnerability addition APIs, and these APIs will traverse the
mime types in this header and return the first report and summary found
from the mime type.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-01-06 17:54:36 +08:00
stonezdj
6b8fb8431d Add quay registry to proxy cache
Update env.jinja to add quay

Signed-off-by: stonezdj <stonezdj@gmail.com>
2021-01-06 17:22:57 +08:00
danfengliu
511bd86930
Merge pull request #13849 from danfengliu/add-gui-test-for-system-robot-account
Ad  nightly GUI test for system robot account
2021-01-06 16:59:05 +08:00
danfengliu
b5607850af
Merge pull request #13834 from danfengliu/clear-local-useless-images-in-nightly
Clear local images in nightly test
2021-01-06 16:55:22 +08:00
Wenkai Yin
fb871dbbe8 Provide a mechanism to sweep the execution/task records in task manager
Provide a mechanism to sweep the execution/task records in task manager
Fixes #13888

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-01-06 16:41:46 +08:00
He Weiwei
e33ff932d1
test: fix the data race in the test of scan controller (#13899)
Closes #13876

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-01-06 16:17:47 +08:00
Danfeng Liu (c)
ae05e3fdb0 Add Nightly GUI test for system robot account
1. Add GUI test for system robot account feature;
2. Fix issue of failing to catch docker pull image exception in API test.

Signed-off-by: Danfeng Liu (c) <danfengl@vmware.com>
2021-01-06 15:47:04 +08:00
stonezdj(Daojun Zhang)
b748852ee8
Merge pull request #13805 from stonezdj/201218_add_content_type_length
Add content type and length in header
2021-01-06 15:02:34 +08:00
Will Sun
a1a5ef5e20
support quay registry as proxy cache (#13902)
Signed-off-by: AllForNothing <sshijun@vmware.com>
2021-01-06 14:15:15 +08:00
stonezdj
1d50be31aa Refactor Manifest cache process
Separate manifest, manifest list and image index.

Signed-off-by: stonezdj <stonezdj@gmail.com>
2021-01-06 14:01:02 +08:00
Daniel Jiang
06e993ff76
Remove scanner-pull from system admin's permission (#13901)
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-01-06 12:35:58 +08:00
Wenkai Yin
d474750e9f Fix replication filter bug
This commit fixes the bug of replication filter, see #13593 for more detail
Fixes #13593

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-01-06 09:36:31 +08:00
Will Sun
839c36c876
Fix bugs for robot account UI (#13894)
Signed-off-by: AllForNothing <sshijun@vmware.com>
2021-01-05 22:28:56 +08:00
Wenkai Yin(尹文开)
17400acea8
Merge pull request #13853 from bitsf/retention_task_manager
feat(retention) refactor task manager
2021-01-05 18:53:09 +08:00
Ziming Zhang
8faa76a1b6 feat(retention) refactor task manager
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2021-01-05 12:08:03 +08:00
shaobo
434758b71e reform job status compare method
Signed-off-by: shaobo <shaobo@caicloud.io>
2021-01-04 21:34:53 +08:00
Daniel Jiang
d0152cb446
Merge pull request #13872 from reasonerjt/token-scope
Refine the token scope generation
2021-01-04 11:16:59 +08:00
danfengliu
076ebb5ee8 Clear local images in nightly test
Remove local image right after pushing operation, then pull operation can be triggered by docker 20.

Signed-off-by: danfengliu <danfengl@vmware.com>
2021-01-04 10:25:02 +08:00
Wang Yan
0271efd3f7
enable visible when to list/create robot (#13840)
1, enable the visible attribute when to create/list robots
2, rename package name from robot2 to robot

Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-01-04 10:24:31 +08:00
Daniel Jiang
eb75123638 Refine the token scope generation
This commit directly maps the actoin permission in security context to
the scope generated by the token service in harbor-core.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-01-03 23:12:04 +08:00
Daniel Pacak
bd46af691c
chore(trivy): Bump up Trivy scanner adapter to v0.17.0 (#13639)
The adapter implements Pluggable Scanners API v1.1
and ships with Trivy v0.14.0.

There's also a tiny change in the way Trivy settings
are displayed in the scanner metadata response, i.e.
instead of com.github.aquasecurity.trivy.debugMode
it prints env.SCANNER_TRIVY_DEBUG_MODE. It makes it
explicit which env is use to set this parameter.

Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
2020-12-25 15:34:00 +08:00
Wenkai Yin(尹文开)
19ad8ad68d
Merge pull request #13823 from reasonerjt/inst-cert-home-dir
Replace tilde in install_cert.sh
2020-12-25 10:25:51 +08:00
prahaladdarkin
a890b28e1e
Store vulnerability data from scanner into a relational format (#13616)
feat: Store vulnerability report from scanner into a relational format

Convert vulnerability report JSON obtained  from scanner into a relational format describe in:https://github.com/goharbor/community/pull/145

Signed-off-by: prahaladdarkin <prahaladd@vmware.com>
2020-12-25 08:47:46 +08:00
Wang Yan
47841a04b9
fix quota cannot be updated after gc (#13844)
Signed-off-by: Wang Yan <wangyan@vmware.com>
2020-12-24 20:10:57 +08:00
Will Sun
75da08303a
Improve scan all page
Signed-off-by: AllForNothing <sshijun@vmware.com>
2020-12-24 17:36:03 +08:00
Wenkai Yin
923a538570 Provide the function to update extra attributes in the task manager
Provide the function to update extra attributes in the task manager

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-12-24 15:52:37 +08:00
He Weiwei
7b4c4b76e7
feat: add trigger to the metrics of the scan all job (#13838)
Add the trigger to the metrics of the scan all job so that the customer
can know who trigger the latest scan all job.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-12-24 14:30:43 +08:00
Wang Yan
7a8a8fa104
upgrade go version to v1.15.6 (#13836)
Signed-off-by: wang yan <wangyan@vmware.com>
2020-12-23 18:53:09 +08:00
Wenkai Yin(尹文开)
d32a243508
Use the same transaction to remove the schedule to avoid dead lock in database (#13820)
Use the same transaction to remove the schedule to avoid dead lock in database
Fixes #13808

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-12-23 15:37:51 +08:00
Will Sun
28734d7ac4
Fix a save button bug on system setting page (#13830)
Signed-off-by: AllForNothing <sshijun@vmware.com>
2020-12-23 13:00:45 +08:00
Daniel Jiang
c660727877
Merge pull request #13800 from reasonerjt/authproxy-redirect
Add handler to handle redirect via authproxy
2020-12-23 03:00:18 +08:00
stonezdj(Daojun Zhang)
bc0b6b43ed
Merge pull request #13791 from reasonerjt/oidc-redirect-extra-parm
Add extra parms when forming redirect URI for OIDC
2020-12-22 21:45:53 +08:00
Daniel Jiang
7321e3547d Add handler to handle redirect via authproxy
This commit add a handler to handle the request to
"/c/authproxy/redirect".  Harbor is configured to authenticate against
an authproxy, if a request with query string `?token=xxxx`
is sent to this URI, the handler will do tokenreview according to the
setting of authproxy and simulate a `login` workflow based on the result
of token review.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-12-22 18:59:17 +08:00
He Weiwei
3831e82b20
refactor: remove code of admin job (#13819)
Remove code of admin job as it's not needed by scan all/gc now.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-12-22 11:48:16 +08:00
He Weiwei
8fa03e3739
refactor: remove code of scan all job (#13821)
Remove code of scan all job as it's implemented by execution now.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-12-22 11:39:18 +08:00
Daniel Jiang
9d99dfa82b Replace tilde in install_cert.sh
This commit fixes #13287 to remove the usage of tilde as the $HOME is not available in some
cases.  More details see #13287

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-12-21 20:39:34 +08:00
stonezdj
aa3002e7a5 Add content type and length in header
Fixes #13740
Update ManifestExist to return Descriptor instead of digest
For docker 20.10 or containerd, it HEAD the manifest before pull, then
it GET the manifest with digest, add logic to handle this scenario and
correlate the tag between the digest in proxy cache

Signed-off-by: stonezdj <stonezdj@gmail.com>
2020-12-21 20:21:28 +08:00