Commit Graph

609 Commits

Author SHA1 Message Date
Aaron Dewes
caee762b51
Upgrade to distribution (registry) v3 alpha (#19784)
* registryctl/api/registry/blob: fix dropped test error (#19721)

Signed-off-by: Lars Lehtonen <lars.lehtonen@gmail.com>

* Remove robot account update quota permission (#19819)

Signed-off-by: Yang Jiao <yang.jiao@broadcom.com>
Co-authored-by: Yang Jiao <yang.jiao@broadcom.com>

* Cache image list with digest key (#19801)

fixes #19429

Signed-off-by: stonezdj <daojunz@vmware.com>
Co-authored-by: stonezdj <daojunz@vmware.com>

* Add quota permissions testcase (#19822)

Signed-off-by: Yang Jiao <yang.jiao@broadcom.com>
Co-authored-by: Yang Jiao <yang.jiao@broadcom.com>

* deprecate gosec in makefile (#19828)

remove the unused the part from makefile

Signed-off-by: wang yan <wangyan@vmware.com>

* Add verification that robot account duration is not 0 (#19829)

Signed-off-by: Yang Jiao <yang.jiao@broadcom.com>

* fix artifact page bug (#19807)

* fix artifact page bug

* update testcase

* Upgrade to distribution (registry) v3 alpha

This includes all the benefits of the v3 distribution, but also all breaking changes.

Most notably, Image Manifest v2 Schema v1 support has been dropped, as well as the `oss` and `swift` storage drivers.

Currently, this still relies on v2's github.com/docker/distribution/registry/client/auth/challenge, because that code has been removed from the public API in v3.

Signed-off-by: Aaron Dewes <aaron.dewes@protonmail.com>

---------

Signed-off-by: Lars Lehtonen <lars.lehtonen@gmail.com>
Signed-off-by: Yang Jiao <yang.jiao@broadcom.com>
Signed-off-by: stonezdj <daojunz@vmware.com>
Signed-off-by: wang yan <wangyan@vmware.com>
Signed-off-by: Aaron Dewes <aaron.dewes@protonmail.com>
Co-authored-by: Lars Lehtonen <lars.lehtonen@gmail.com>
Co-authored-by: Yang Jiao <72076317+YangJiao0817@users.noreply.github.com>
Co-authored-by: Yang Jiao <yang.jiao@broadcom.com>
Co-authored-by: stonezdj(Daojun Zhang) <stonezdj@gmail.com>
Co-authored-by: stonezdj <daojunz@vmware.com>
Co-authored-by: Wang Yan <wangyan@vmware.com>
Co-authored-by: ShengqiWang <124650040+ShengqiWang@users.noreply.github.com>
2024-01-26 22:48:06 +01:00
Yang Jiao
6793da72e3
Fix project metadata validate bug (#19746)
Signed-off-by: Yang Jiao <yang.jiao@broadcom.com>
Co-authored-by: Yang Jiao <yang.jiao@broadcom.com>
2023-12-20 15:28:07 +08:00
MinerYang
b38de22054
bump golang 1.21.5 & fix golangci-lint error (#19722)
bump golang 1.21.5

update golangci-lint && fix revive error

fix white space lint

Signed-off-by: yminer <yminer@vmware.com>
2023-12-19 09:41:26 +08:00
Wang Yan
7b40dc6f46
fix the landing accessory data (#19661)
Fix the keywords when to list accessories belong to the subject manifest.

Signed-off-by: wang yan <wangyan@vmware.com>
2023-12-05 14:14:32 +08:00
Wang Yan
e6900301ce
fix system label resource (#19621)
Make sure robot can use rbac.Resource(label) to access system level label

Signed-off-by: wang yan <wangyan@vmware.com>
2023-11-27 13:34:17 +08:00
Wang Yan
062d144d22
add permission validation for robot creating and updating. (#19598)
* add permission validation for robot creating and updating.

It is not allowed to create an new robot with the access outside the predefined scope.

Signed-off-by: wang yan <wangyan@vmware.com>

* Fix robot testcase and update robot permission metadata (#167)

1. Fix robot testcase
2. update robot permission metadata

Signed-off-by: Yang Jiao <jiaoya@vmware.com>
Signed-off-by: wang yan <wangyan@vmware.com>

---------

Signed-off-by: wang yan <wangyan@vmware.com>
Signed-off-by: Yang Jiao <jiaoya@vmware.com>
Co-authored-by: Yang Jiao <72076317+YangJiao0817@users.noreply.github.com>
2023-11-22 12:51:03 +08:00
stonezdj(Daojun Zhang)
307a3121aa
Return empty result when no scanner configured (#19577)
fixes #19534

Signed-off-by: stonezdj <daojunz@vmware.com>
2023-11-17 07:29:19 +00:00
stonezdj(Daojun Zhang)
3f6c0298fe
Add oidc provider name to systeminfo API (#19575)
fixes #13198

Signed-off-by: stonezdj <daojunz@vmware.com>
2023-11-15 11:00:39 +08:00
Wang Yan
5c02fd807e
add permission api (#19543)
The permission api targets to return the full set of permissons for robot to use.
And only system and project admin have the access

Signed-off-by: wang yan <wangyan@vmware.com>
2023-11-09 09:47:07 +08:00
stonezdj(Daojun Zhang)
c9214256e5
Skip to validate username when update user profile (#19552)
fixes #19528

Signed-off-by: stonezdj <daojunz@vmware.com>
2023-11-08 13:29:30 +08:00
stonezdj(Daojun Zhang)
b337f51e7e
Replace comma in username to avoid casbin issue (#19505)
Check username when creating user by API
   Replace comma with underscore in username for OnboardUser
   Fixes #19356

Signed-off-by: stonezdj <daojunz@vmware.com>
2023-11-02 03:50:50 +00:00
MinerYang
193b00b9e6
bump golang to 1.21.3 (#19504)
update ut mock

update go mod

Signed-off-by: yminer <yminer@vmware.com>
2023-10-31 15:01:23 +08:00
imeoer
13f4fca96e
subject: fix missing media type recognition for nydus (#19453)
Nydus images are compatible with both docker scheme v2 and OCI image spec v1
types of config media type:

  - application/vnd.oci.image.config.v1+json
  - application/vnd.docker.container.image.v1+json

The patch fixes the media type recognition on the accessory type check.

Signed-off-by: Yan Song <yansong.ys@antgroup.com>
Co-authored-by: MinerYang <yminer@vmware.com>
2023-10-19 10:53:13 +08:00
Wang Yan
f9ab185ed0
fix issue 19392 (#19437)
Needs to set the repo when to handle the accessory before subject manifest.

Signed-off-by: wang yan <wangyan@vmware.com>
2023-10-12 15:15:42 +08:00
Wang Yan
3338f27189
support accessory in either order (#19375)
In certain cases, the OCI client may push the subject artifact and accessory in either order.
Therefore, it is necessary to handle situations where the client pushes the accessory ahead of the subject artifact.

Signed-off-by: wang yan <wangyan@vmware.com>
2023-09-23 02:42:58 +08:00
MinerYang
6fd4a2bd37
update default processor for unknwon type config (#19372)
update OCI-Subject header

Signed-off-by: yminer <yminer@vmware.com>

update default processor & add ut for artifact icon

update ut coverage
2023-09-21 15:19:42 +08:00
Wang Yan
4051b2b302
fix user resource (#19366)
fix the user resrouce defination of user api

Signed-off-by: wang yan <wangyan@vmware.com>
2023-09-18 14:16:25 +08:00
rongfu.leng
d55823d888
remove chart-museum related to logic (#18722)
Signed-off-by: lengrongfu <1275177125@qq.com>
Co-authored-by: Wang Yan <wangyan@vmware.com>
2023-09-11 15:15:02 +08:00
MinerYang
06139c1396
change Referrers Content-Type to application/vnd.oci.image.index.v1+json (#19212)
Signed-off-by: yminer <yminer@vmware.com>
Co-authored-by: Wang Yan <wangyan@vmware.com>
2023-08-25 11:54:55 +08:00
Chlins Zhang
52e66155d4
log: change log level to reduce the noise logs (#19146)
1. Change some logs level to reduce the noise.
2. Wrap the go-redis.Nil error as ErrNotFound to avoid confusing

Signed-off-by: chlins <chenyuzh@vmware.com>
2023-08-17 11:11:04 +08:00
Shengwen YU
88c6018950
fix: cron string validation (#19071)
fix: cron string validation (the 1st field of a cron string must be 0 when there are 6 fields)

Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
2023-08-09 10:22:54 +08:00
Wang Yan
bef4ccc782
fixes 19043 (#19054)
fixes #19043
Skip the policy checking when to pull the notation signature

Signed-off-by: wang yan <wangyan@vmware.com>
2023-07-31 15:11:21 +08:00
Wang Yan
db82d6e082
fix signer name (#19001)
Append the signer name to the error message which will be returned by the client

Signed-off-by: wang yan <wangyan@vmware.com>
2023-07-25 21:30:32 +08:00
Wang Yan
ce893635aa
support nydus as a accessory (#18953)
Recognize nydus image(with subject) as a kind of accessory and built the releationship with subject manifest

Signed-off-by: wang yan <wangyan@vmware.com>
2023-07-20 15:54:23 +08:00
Chlins Zhang
970bdab936
fix: correct the operator in the webhook payload (#18906)
Fix the incorrect or meaningless operator in the webhook payload.

Fixes: #18438

Signed-off-by: chlins <chenyuzh@vmware.com>
2023-07-19 15:40:29 +08:00
stonezdj(Daojun Zhang)
d4aa9b13c4
Add vulnerability search API (#18924)
use q.Query to pass all query conditions

Signed-off-by: stonezdj <daojunz@vmware.com>
2023-07-19 10:17:14 +08:00
Wang Yan
ff2b99d711
enable notary v2 policy checker (#18927)
add notary v2 pull policy, when it enables, the artifact cannot be pull without the notation signature.

Signed-off-by: wang yan <wangyan@vmware.com>
2023-07-14 13:35:56 +08:00
Wang Yan
5cce621471
add notation support (#18909)
Accept and recognize the signature of notation client

Signed-off-by: wang yan <wangyan@vmware.com>
2023-07-13 11:18:34 +08:00
stonezdj(Daojun Zhang)
93e428d0d2
Add security hub summary API (#18872)
include WithCVE, WithArtifact option

Signed-off-by: stonezdj <daojunz@vmware.com>
2023-07-12 19:18:08 +08:00
Shengwen YU
df4dc3c00b
fix: add password/secret length check to be <= 128 (#18916)
Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
2023-07-12 17:04:50 +08:00
Bin Liu
c8120d5712
API: update ScannerRegistration.properties.url format (#18799)
The format of ScannerRegistration.properties.url should be
`uri` but not `url`.

Fixes: #18798

Signed-off-by: bin liu <liubin0329@gmail.com>
Co-authored-by: Wang Yan <wangyan@vmware.com>
2023-07-05 17:57:06 +08:00
Wang Yan
dbaae9e64e
support OCI-Subject header (#18885)
fixes #18865
the response header OCI-Subject to indicate to the client that the registry processed the request's subject.

Signed-off-by: wang yan <wangyan@vmware.com>
Co-authored-by: System Administrator <root@wangyanCQFQY.vmware.com>
2023-07-05 11:25:19 +08:00
Shijun Sun
ef96c729c0
Add costomized banner message UI (#18827)
1.Fixes #18719
2.Add Banner Message item to configuration
3.Add banner_message property to systeminfo API

Signed-off-by: AllForNothing <sshijun@vmware.com>
2023-07-03 15:58:37 +08:00
Wang Yan
7435c8c5ab
add multiple deletion of GC (#18855)
User can specify the workers when to issue an GC execution, the maxium count of workers is 5.

Signed-off-by: wang yan <wangyan@vmware.com>
2023-06-29 16:22:18 +08:00
rongfu.leng
8fe578e7ab
feat: Optimize quota checking when pushing images (#17392)
Signed-off-by: lengrongfu <1275177125@qq.com>
2023-06-05 17:19:55 +08:00
Chlins Zhang
fbeeaa7537
fix: add checkpoint when enqueue scan tasks for scan all (#18680)
Fix the scanAll cannot be stopped in case of large number of artifacts,
add the checkpoint before submit scan tasks, mark the scanAll stopped
flag in the redis.

Fixes: #18044

Signed-off-by: chlins <chenyuzh@vmware.com>
2023-06-05 15:12:54 +08:00
cui fliter
e19ec9623b
Fix: fix function name in comments (#18726)
fix function name in comments

Signed-off-by: cui fliter <imcusg@gmail.com>
2023-05-31 10:26:19 +08:00
orblazer
b021dbd2d6
fix invalid access action (#18188)
Signed-off-by: orblazer <orblazer.minecraft@gmail.com>
Co-authored-by: Wang Yan <wangyan@vmware.com>
2023-05-30 17:49:01 +08:00
guangwu
b7b0e43a21
fix: import optimization (#18727)
Signed-off-by: guoguangwu <guoguangwu@magic-shield.com>
2023-05-30 13:24:08 +08:00
stonezdj(Daojun Zhang)
1b1af4a14c
Return error when proxy cache get too many request error(429) (#18728)
Add 429 too many request error in http error
  Fixes #18707

Signed-off-by: stonezdj <stonezdj@gmail.com>
2023-05-29 13:33:46 +08:00
Wang Yan
bf7c82b9a8
remove the notary from backend (#18668)
Since harbor deprecates notary since v2.9.0, this pull request targets to remove the code related with notary.

Signed-off-by: Wang Yan <wangyan@vmware.com>
2023-05-18 18:47:42 +08:00
MinerYang
ebac530b46
add goheader linter settings (#18503)
fix files for goheader linter

fix copyright 2018/2019

Signed-off-by: yminer <yminer@vmware.com>
2023-04-25 11:18:42 +08:00
pgillich
eec5342490
feat: log with trace ID (#18181)
feat: log trace ID

Implements #18029

If the Trace ID is sent to the Harbor in HTTP header or the tracing
is enabled (and the Trace ID is generated), the Trace ID will be
added to the log lines as a new field.

Signed-off-by: Peter Gillich <pgillich@gmail.com>
2023-04-17 15:03:00 +08:00
Wang Yan
fddfaa7ba5
fix copy artifact issue (#18460)
To enable the middleware to save the project_blob data, make sure to set the accessories options to true when handling the artifact copy.

Signed-off-by: Wang Yan <wangyan@vmware.com>
2023-04-03 10:58:17 +08:00
Chlins Zhang
cb0749c7ab
fix: add default payload_format for http type webhook (#18445)
1. Add migration SQL to handle the lost payload format for old policies.
2. Set payload format to 'Default' if not specified for http webhook in the API handler.
3. Fix the migration sql of notification_job

Fixes: #18401, #18453

Signed-off-by: chlins <chenyuzh@vmware.com>
2023-04-02 15:33:33 +08:00
Wang Yan
1bb3914de4
fix referrers api response issue (#18430)
1, add fitler artifactType to header when the api is called with filter
2, give an empty json body on non aritfact scenario
3, give an empty array on non accessory scenario
4, fix the artifact type filter issue

Signed-off-by: Wang Yan <wangyan@vmware.com>
2023-03-28 13:51:37 +08:00
Wang Yan
deaecf2de5
add subject artifact repo (#18394)
add suject_artifact_repo column in the table artifact_accessory

Signed-off-by: Wang Yan <wangyan@vmware.com>
2023-03-22 21:48:09 +08:00
Wang Yan
2f380495bf
revert subject id in the accessory (#18377)
DO not replact id with digest and just add digest into the accessory table

Signed-off-by: Wang Yan <wangyan@vmware.com>
2023-03-22 10:58:30 +08:00
Chlins Zhang
f21b1481bb
refactor: refactor the old goroutine execution sweep (#18361)
refactor: refactor the old goroutine execution sweep to global execution sweep job

1. Delete the old goroutine execution sweeper when create execution.(in the case of high concurrency can cause goroutine backlogs, affect the performance of core)
2. Introduce the new way to sweep executions, a global scheduled job will take the work.

Signed-off-by: chlins <chenyuzh@vmware.com>
2023-03-21 10:10:55 +08:00
Wang Yan
bb291aaa16
add middleware for artifact with subject (#18369)
As for the distribution spec 1.1, it supports client to push an manifest with subject field. By leverging this fidle, harbor could build up the linkage between the subject artifact and it's accessories.

Signed-off-by: wang yan <wangyan@vmware.com>
2023-03-19 20:09:41 +08:00