The v2 catalog API needs to filter out the empty repository and the repository which artifacts are all with no tags.
1，In v2.0.0, Harbor does not delete repository even there is no artifact, it's different with v1.10.0
2, Compares with docker distribution, it doesn't return the respository with untagged images.
Signed-off-by: wang yan <wangyan@vmware.com>
Fixes#9704
As we do want to unify error handling, so just decreprates pkg errors, use lib/errors instead for Harbor internal used errors model.
1, The lib/errors can cover all of funcs of pkg/errors, and also it has code attribute to define the http return value.
2, lib/errors can give a OCI standard error format, like {"errors":[{"code":"UNAUTHORIZED","message":"unauthorized"}]}
If you'd like to use pkg/errors, use lib/errors instead. If it cannot meet your request, enhance it.
Signed-off-by: wang yan <wangyan@vmware.com>
Customize the json output with message with err.Error(). Otherwise, the wrappged message will be lost
in the final errors object.
Signed-off-by: wang yan <wangyan@vmware.com>
Fixes#11606
As we DO NOT want to user to execute GC in the container, rename it and append the warning message.
Signed-off-by: wang yan <wangyan@vmware.com>
- priority option is supported when doing job registration
- the priority is defined by a unique priority sampler
- the default priority is 1000 (max is 10000)
Signed-off-by: Steven Zou <szou@vmware.com>
- update Job interface to introdcue MaxCurrency method for declaring the max currency of the specified job
- change the downstream jobs to implement the new interface method
- GC and sample jobs are set to 1
- other jobs are set to 0 that means unlimited
- add max currency optiot when doing job registration
- resolve issue #11586
- probably resolve issue #11281
- resolve issue #11570
Signed-off-by: Steven Zou <szou@vmware.com>
1. Remove `common/quota` package.
2. Remove functions about quota in `common/dao` package.
3. Move `Quota` and `QuotaUsage` models from `common/models` to
`pkg/quota/dao`.
4. Add `Count` and `List` methods to `quota.Controller`.
5. Use `quota.Controller` to implement quota APIs.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
fixes#11533
GC jobs will use the filter results to call registry API to delete manifest.
In the current imple, the filter function in some case does not return the deleted artifact as it's using digest as the filter condition.
Like: If one artifact is deleted, but there is another project/repo has a image with same digest with the deleted one, filter func will
not mark the deleted artifact as candidate. It results in, GC job does not call API to remove the manifest.
To fix it, update the filter to use both digest and repository name to filter candidate.
Signed-off-by: wang yan <wangyan@vmware.com>
1. Partial helm api version number clear
2. Separate swagger to get v2.0 swagger and chart swagger
3. router add chart swagger
Signed-off-by: Yogi_Wang <yawang@vmware.com>
This commit removes the EnsureArtifactDigest as its implementation is
problematic: the artifactinfo in context is immutable.
When the content trust middleware needs the digest it will retrieve it
via artifact controller.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
1. Ignore limitation when refresh quota for project.
2. Return 403 when quota errors occurred.
3. Add test for Refresh method of quota controller.
Closes#11512
Signed-off-by: He Weiwei <hweiwei@vmware.com>
Remove the URL replacing logic temporarily to make replication work and will introduce a new solution for the hairpin issue
Signed-off-by: Wenkai Yin <yinw@vmware.com>
- schedule the periodical jobs following the UTC timezone
- e.g: 5 10 10 * * * means run jobs at UTC time 10:10:05 everyday
- fix issue #11466
Signed-off-by: Steven Zou <szou@vmware.com>
* Update tags related APIs
1. Remove API for listing tags of repository
2. Add API for listing tags of artifact
3. Support filter artifact by tag name
Signed-off-by: Wenkai Yin <yinw@vmware.com>
* [OCI] modify artifact tag name check
1. switch api get tag list
2. modify artifact tag name check
Signed-off-by: Yogi_Wang <yawang@vmware.com>
Co-authored-by: Yogi_Wang <yawang@vmware.com>
1. Copy artifact will not return 409 anymore.
2. Make sure the tags of source artifact exist in the target artifact
Signed-off-by: Wenkai Yin <yinw@vmware.com>
Fixes#11241
1, remove count quota from quota manager
2, remove count in DB scheme
3, remove UI relates on quota
4, update UT, API test and UI UT.
Signed-off-by: wang yan <wangyan@vmware.com>
1. Prevent the pull action when scan report status is not successfuly.
2. Bypass the checking when no vulnerabilities not found.
3. Improve the returned message when prevented the pull action.
Closes#11202
Signed-off-by: He Weiwei <hweiwei@vmware.com>
Docker CLI fails if it's not logged in upon seeing "basic" realm challenging while pinging the "/v2" endpoint. (#11266)
Some CLI will send HEAD to artifact endpoint before pushing (#11188)(#11271)
To fix such problems, this commit re-introduce the token auth flow to the CLIs.
For a HEAD request to "/v2/xxx" with no "Authoirzation" header, the v2_auth middleware populates the
"Www-Authenticate" header to redirect it to token endpoint with proper
requested scope.
It also adds security context to based on the content of the JWT which has the claims of the registry.
So a request from CLI carrying a token signed by the "/service/token" will have proper permissions.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
* fix(scan): fix ScanStatus when merge NativeReportSummary
1. Running and success status is high priority when merge ScanStatus of
NativeReportSummary, otherwise chose the bigger status.
2. Merge scan logs of referenced artifacts when get the scan logs of
image index.
Closes#11265
Signed-off-by: He Weiwei <hweiwei@vmware.com>
* fix(portal): fix the annotation for the scan completed percent in scan overview
Signed-off-by: He Weiwei <hweiwei@vmware.com>
1. When use the helper functions of log pkg, the depth is 4 to get the
correct file and line.
2. Whe use the default logger of log pkg, the depth is 3 to get the
correct file and line.
Closes#11391
Signed-off-by: He Weiwei <hweiwei@vmware.com>
Fixes#11190, delete all of non-referenced blobs of each project in GC job, thun the quota
can be released.
Signed-off-by: wang yan <wangyan@vmware.com>
Fixes#11313Fixes#11275
1, Add more details log in GC job
2, Add type assertion for the upgrading case, the delete_untagged parameter is introduced from v2.0
3, Add UT
Signed-off-by: wang yan <wangyan@vmware.com>
1.get artifact tag from another api
2.add refresh button in artifact tag
3.fix permission change
4.some ui style
Signed-off-by: Yogi_Wang <yawang@vmware.com>
1. Add timeout when transter artifacts
2. Check 404 error when unschedule the policy
3. Add line to mark the job failure in job log
Signed-off-by: Wenkai Yin <yinw@vmware.com>
Fixes#11206
1, fix middleware doesn't work for docker pull without auth
2, fix middleware doesn't bypass scanner pull
Signed-off-by: wang yan <wangyan@vmware.com>
That was added to support core process sending request to `/v2/xxx`.
It's no longer needed after reworking the flow.
This commit removes this.
Fixes#10602, as it's not a case we need to support for now.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
Fixes#11225
As registry changes to basic auth, the push action lost the pull permission.
Add it in the robot security context.
Signed-off-by: wang yan <wangyan@vmware.com>
For pull a public resource, there is no need to login, give the access name to anonymous in the audit logs
Signed-off-by: wang yan <wangyan@vmware.com>
1. Bump up the version of API used in replicatoin scheduler job
2. Check the error message to determine whether the job exists or not in jobservice when unschedule a job
Signed-off-by: Wenkai Yin <yinw@vmware.com>
Fixes#11315
When specify no pagination in listing artifact request, the go-swagger will set the default value for them, so we need to iterate the link header to get all of artifacts
Signed-off-by: Wenkai Yin <yinw@vmware.com>
In Harbor 2.0, the replication isn't supported between instances with different versions, this commit returns the 404 error when trying to get the registry info whose version is different with the current one
Signed-off-by: Wenkai Yin <yinw@vmware.com>
Signed-off-by: Yogi_Wang <yawang@vmware.com>
1.add case for trivy
2.vunerbility refresh bug
3.scan mutiple artifact
4.fix global search bug
5.disable delete tag btn when remove immutable tag
6.cancel selectRow when add label or remove label;fix #11195
7.fix cron tootip
The function GetRegistrationByProject should not return err when Ping
return err. The return value 'registration' has 'Health' field which
shows the scanner health status.
Resolves: #11051
See also: #9788, #9807
Signed-off-by: qinshaoxuan <qinshaoxuan@baidu.com>
Fixes#11267
When caller parse an empty orlist to orm lib, it will parse the empty vaule to beego orm.
But beego will panic if the query string is empty.
Signed-off-by: wang yan <wangyan@vmware.com>
As we don't support bearer token in Harbor 2.0, the URL checking logic in auth proxy security generator should be updated
Signed-off-by: Wenkai Yin <yinw@vmware.com>
Fixes#11016
1. src/pkg/q->src/internal/q
2. src/internal->src/lib (internal is a reserved package name of golang)
3. src/api->src/controller
Signed-off-by: Wenkai Yin <yinw@vmware.com>
* replication webhook support
Signed-off-by: guanxiatao <guanxiatao@corp.netease.com>
* replication webhook support with ut fixed
Signed-off-by: guanxiatao <guanxiatao@corp.netease.com>
1.nightly: fix tag retention and immutable tag case xpath
2.nightly: fix the part of delete repo button xpath
3.nightly: fix the api version when GC
4.nightly: fix add label of artifact xpath
5.text: change delete artifact show words
Signed-off-by: Yogi_Wang <yawang@vmware.com>
The query string is encoded by UI, and we have to unescape the "=" in "q=tag=nil",
otherwise, the query doesn't work, and returns 400
Signed-off-by: wang yan <wangyan@vmware.com>
1, enable user view log api test case
2, update project logs api permission check
3, use project ctl instead in permission check base method
Signed-off-by: wang yan <wangyan@vmware.com>
1. Skip vulnerability prevention checking when artifact is not
scannable.
2. Skip vulnerability prevention checking when artifact is image index
and its type is `IMAGE` or `CNAB`.
3. Skip vulnerability prevention checking when the artifact is pulling
by the scanner.
4. Change `hasCapability` from blacklist to whitelist.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
