1. Ignore limitation when refresh quota for project.
2. Return 403 when quota errors occurred.
3. Add test for Refresh method of quota controller.
Closes#11512
Signed-off-by: He Weiwei <hweiwei@vmware.com>
Remove the URL replacing logic temporarily to make replication work and will introduce a new solution for the hairpin issue
Signed-off-by: Wenkai Yin <yinw@vmware.com>
- schedule the periodical jobs following the UTC timezone
- e.g: 5 10 10 * * * means run jobs at UTC time 10:10:05 everyday
- fix issue #11466
Signed-off-by: Steven Zou <szou@vmware.com>
* Update tags related APIs
1. Remove API for listing tags of repository
2. Add API for listing tags of artifact
3. Support filter artifact by tag name
Signed-off-by: Wenkai Yin <yinw@vmware.com>
* [OCI] modify artifact tag name check
1. switch api get tag list
2. modify artifact tag name check
Signed-off-by: Yogi_Wang <yawang@vmware.com>
Co-authored-by: Yogi_Wang <yawang@vmware.com>
1. Copy artifact will not return 409 anymore.
2. Make sure the tags of source artifact exist in the target artifact
Signed-off-by: Wenkai Yin <yinw@vmware.com>
Fixes#11241
1, remove count quota from quota manager
2, remove count in DB scheme
3, remove UI relates on quota
4, update UT, API test and UI UT.
Signed-off-by: wang yan <wangyan@vmware.com>
1. Prevent the pull action when scan report status is not successfuly.
2. Bypass the checking when no vulnerabilities not found.
3. Improve the returned message when prevented the pull action.
Closes#11202
Signed-off-by: He Weiwei <hweiwei@vmware.com>
Docker CLI fails if it's not logged in upon seeing "basic" realm challenging while pinging the "/v2" endpoint. (#11266)
Some CLI will send HEAD to artifact endpoint before pushing (#11188)(#11271)
To fix such problems, this commit re-introduce the token auth flow to the CLIs.
For a HEAD request to "/v2/xxx" with no "Authoirzation" header, the v2_auth middleware populates the
"Www-Authenticate" header to redirect it to token endpoint with proper
requested scope.
It also adds security context to based on the content of the JWT which has the claims of the registry.
So a request from CLI carrying a token signed by the "/service/token" will have proper permissions.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
* fix(scan): fix ScanStatus when merge NativeReportSummary
1. Running and success status is high priority when merge ScanStatus of
NativeReportSummary, otherwise chose the bigger status.
2. Merge scan logs of referenced artifacts when get the scan logs of
image index.
Closes#11265
Signed-off-by: He Weiwei <hweiwei@vmware.com>
* fix(portal): fix the annotation for the scan completed percent in scan overview
Signed-off-by: He Weiwei <hweiwei@vmware.com>
1. When use the helper functions of log pkg, the depth is 4 to get the
correct file and line.
2. Whe use the default logger of log pkg, the depth is 3 to get the
correct file and line.
Closes#11391
Signed-off-by: He Weiwei <hweiwei@vmware.com>
Fixes#11190, delete all of non-referenced blobs of each project in GC job, thun the quota
can be released.
Signed-off-by: wang yan <wangyan@vmware.com>
Fixes#11313Fixes#11275
1, Add more details log in GC job
2, Add type assertion for the upgrading case, the delete_untagged parameter is introduced from v2.0
3, Add UT
Signed-off-by: wang yan <wangyan@vmware.com>
1.get artifact tag from another api
2.add refresh button in artifact tag
3.fix permission change
4.some ui style
Signed-off-by: Yogi_Wang <yawang@vmware.com>
1. Add timeout when transter artifacts
2. Check 404 error when unschedule the policy
3. Add line to mark the job failure in job log
Signed-off-by: Wenkai Yin <yinw@vmware.com>
Fixes#11206
1, fix middleware doesn't work for docker pull without auth
2, fix middleware doesn't bypass scanner pull
Signed-off-by: wang yan <wangyan@vmware.com>
That was added to support core process sending request to `/v2/xxx`.
It's no longer needed after reworking the flow.
This commit removes this.
Fixes#10602, as it's not a case we need to support for now.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
Fixes#11225
As registry changes to basic auth, the push action lost the pull permission.
Add it in the robot security context.
Signed-off-by: wang yan <wangyan@vmware.com>
For pull a public resource, there is no need to login, give the access name to anonymous in the audit logs
Signed-off-by: wang yan <wangyan@vmware.com>
1. Bump up the version of API used in replicatoin scheduler job
2. Check the error message to determine whether the job exists or not in jobservice when unschedule a job
Signed-off-by: Wenkai Yin <yinw@vmware.com>
Fixes#11315
When specify no pagination in listing artifact request, the go-swagger will set the default value for them, so we need to iterate the link header to get all of artifacts
Signed-off-by: Wenkai Yin <yinw@vmware.com>
In Harbor 2.0, the replication isn't supported between instances with different versions, this commit returns the 404 error when trying to get the registry info whose version is different with the current one
Signed-off-by: Wenkai Yin <yinw@vmware.com>
Signed-off-by: Yogi_Wang <yawang@vmware.com>
1.add case for trivy
2.vunerbility refresh bug
3.scan mutiple artifact
4.fix global search bug
5.disable delete tag btn when remove immutable tag
6.cancel selectRow when add label or remove label;fix #11195
7.fix cron tootip
The function GetRegistrationByProject should not return err when Ping
return err. The return value 'registration' has 'Health' field which
shows the scanner health status.
Resolves: #11051
See also: #9788, #9807
Signed-off-by: qinshaoxuan <qinshaoxuan@baidu.com>
Fixes#11267
When caller parse an empty orlist to orm lib, it will parse the empty vaule to beego orm.
But beego will panic if the query string is empty.
Signed-off-by: wang yan <wangyan@vmware.com>
As we don't support bearer token in Harbor 2.0, the URL checking logic in auth proxy security generator should be updated
Signed-off-by: Wenkai Yin <yinw@vmware.com>
Fixes#11016
1. src/pkg/q->src/internal/q
2. src/internal->src/lib (internal is a reserved package name of golang)
3. src/api->src/controller
Signed-off-by: Wenkai Yin <yinw@vmware.com>
* replication webhook support
Signed-off-by: guanxiatao <guanxiatao@corp.netease.com>
* replication webhook support with ut fixed
Signed-off-by: guanxiatao <guanxiatao@corp.netease.com>
1.nightly: fix tag retention and immutable tag case xpath
2.nightly: fix the part of delete repo button xpath
3.nightly: fix the api version when GC
4.nightly: fix add label of artifact xpath
5.text: change delete artifact show words
Signed-off-by: Yogi_Wang <yawang@vmware.com>
The query string is encoded by UI, and we have to unescape the "=" in "q=tag=nil",
otherwise, the query doesn't work, and returns 400
Signed-off-by: wang yan <wangyan@vmware.com>
1, enable user view log api test case
2, update project logs api permission check
3, use project ctl instead in permission check base method
Signed-off-by: wang yan <wangyan@vmware.com>
1. Skip vulnerability prevention checking when artifact is not
scannable.
2. Skip vulnerability prevention checking when artifact is image index
and its type is `IMAGE` or `CNAB`.
3. Skip vulnerability prevention checking when the artifact is pulling
by the scanner.
4. Change `hasCapability` from blacklist to whitelist.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
As we introduce a new table to record the relationship between the artifacts and labels, the current way to list label's resources doesn't work anymore, and the API isn't needed by any features, remove it in 2.0
Signed-off-by: Wenkai Yin <yinw@vmware.com>
* Use new query model to get audit logs
leverage the query builder to build query, remove the old style query string
Signed-off-by: wang yan <wangyan@vmware.com>
* Switch to new API for project log page
Signed-off-by: AllForNothing <sshijun@vmware.com>
Co-authored-by: AllForNothing <sshijun@vmware.com>
1. search result show artifact count
2. replication shows changes both to all
3. fix delete bug when delete some artifact
Signed-off-by: Yogi_Wang <yawang@vmware.com>
The v2auth middleware will check whether the requestor has the pull or
push permissions for the repository, and forbid the request when the
requestor does not have the permission. We need to assign repository
pulling permission to the robot account for the scanner, otherwise
scanner will be failed to pull the artifact.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
Filter the pulling manifest request from replication service so that the audit log will not record the pulling action
Signed-off-by: Wenkai Yin <yinw@vmware.com>
This commit introduces the tag deletion as a new capability for registry adapters, and currently only Harbor supports it
Signed-off-by: Wenkai Yin <yinw@vmware.com>
1. remove tag service / tag model
2. remove retag service
3. remove artifact service some function
4. remove repository service / repository model /repositoryItem model
Signed-off-by: Yogi_Wang <yawang@vmware.com>
1, enable audit logs for notifications
2, move the handler and meatadata into API
3, use the notification middleware to send out notification
Signed-off-by: wang yan <wangyan@vmware.com>
1. Use quota controller to implement the internal quota APIs.
2. The internal quota APIs can exceed the quota limitations.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
the notification is for send out the event after DB transaction complete.
It's safe to send hook as this middleware is after transaction in the response path.
Signed-off-by: wang yan <wangyan@vmware.com>
1. Before put manifest request, ensure that the requested size resource
include the blobs which are referenced by the manifest but not
associated with project.
2. After put manifest request, associate the blobs which are referenced
by the manifest but not associated with project.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
Type assertion not work when the ctx in the request changed in the next
handler, so change to use value in the ctx to decide whether to commit
tx.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
To avoid depedency loop, this commit moves the model of ArtifactInfo to
internal pacakge, so that a controller can it from context when needed.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
1. Introduce `Evaluator` interface which do the permission checking.
2. `admin`, `lazy`, `rbac`, `namespace` and `evaluartor` set are implemented the
`Evaluator` interface.
3. Move project rbac implemention from `project` to `rbac` pkg to reduce
the name conflict with project instance of model.
4. Do permission checking in security context by `Evaluator`.
5. Cache the regexp in rbac evaluator for casbin.
6. Cache evaluator in namespace evaluator to improve performance.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
