Filter the pulling manifest request from replication service so that the audit log will not record the pulling action
Signed-off-by: Wenkai Yin <yinw@vmware.com>
1, enable audit logs for notifications
2, move the handler and meatadata into API
3, use the notification middleware to send out notification
Signed-off-by: wang yan <wangyan@vmware.com>
1. Use quota controller to implement the internal quota APIs.
2. The internal quota APIs can exceed the quota limitations.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
the notification is for send out the event after DB transaction complete.
It's safe to send hook as this middleware is after transaction in the response path.
Signed-off-by: wang yan <wangyan@vmware.com>
1. Introduce `Evaluator` interface which do the permission checking.
2. `admin`, `lazy`, `rbac`, `namespace` and `evaluartor` set are implemented the
`Evaluator` interface.
3. Move project rbac implemention from `project` to `rbac` pkg to reduce
the name conflict with project instance of model.
4. Do permission checking in security context by `Evaluator`.
5. Cache the regexp in rbac evaluator for casbin.
6. Cache evaluator in namespace evaluator to improve performance.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
"-1" means the robot account is a permanent account, no expiration time set.
The ExpiresAt claim is optional, so if it's not set, it will still be considered a valid claim
Signed-off-by: wang yan <wangyan@vmware.com>
1, enable dao test for artifact trash
2, set default flush trash table to false
3, hanlder empty parameter in API call
4, add registry auth info into jobservice container
Signed-off-by: wang yan <wangyan@vmware.com>
This commit defines the API query string format and provides the builders to build query string to query model
Signed-off-by: Wenkai Yin <yinw@vmware.com>
1, add API entry for get audit logs
2. add audit log manager to hanlder CRUD
Use the new format of audit log to cover differernt resource, artifact/tag/repostory/project
Signed-off-by: wang yan <wangyan@vmware.com>
1. Add Checker to check the scannable status of the artifact.
2. Only set vulnerabilities addition link when the artifact scanable in the
project.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
use the tag controller to handle CRUD of tags, especially the delete scenario, it could validate
the immutable and signature. And move the code of tag handling from artifact controller to tag controller
Signed-off-by: wang yan <wangyan@vmware.com>
As we store the repository name in the artifact table, we can use it direclty in the code to reduce the database query
Signed-off-by: Wenkai Yin <yinw@vmware.com>
1, set harbor to readonly
2, select the candidate artifacts from Harbor DB.
3, call registry API(--delete-untagged=false) to delete manifest bases on the results of #2
4, clean keys of redis DB of registry, clean artifact trash and untagged from DB.
5, roll back readonly.
Signed-off-by: wang yan <wangyan@vmware.com>
1. Assemble scan overview to artifact when scanner enabled in the
project of the artifact.
2. Set addition link for vulnerabilities to artifact when scanner
enabled in the project of the artifact.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
1. Add middleware to record the accepted blob size for stream blob
upload.
2. Add middleware to create blob and associate it with project after blob upload
complete.
3. Add middleware to sync blobs, create blob for manifest and associate blobs
with the manifest after put manifest.
4. Add middleware to associate blob with project after mount blob.
5. Cleanup associations for the project when artifact deleted.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
* refactor(scan,scanner): move scan and scanner controllers to api pkg
Signed-off-by: He Weiwei <hweiwei@vmware.com>
* feat(scan-all-job): move artifacts query from job to notification
Move artifact query from scan all job to its notification handler to
ensure that the components in pkg will not call controllers in api.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
* Populate signature status in artifact API
This Commit add signature status into response of list artifact API.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
1. As "List" method of artifact DAO doesn't return the artifacts that referenced by other and without tag, so we introduce a new method "GetByDigest" to check the existence of artifact
2. The "Www-Authenticate" header is needed to be returned when the request is unauthorized. This is required in the OCI distribution spec and is needed when pushing CNAB
Signed-off-by: Wenkai Yin <yinw@vmware.com>
This commit add supporting for adding/removing label to/from artifacts and populates labels when listing artifacts
Signed-off-by: Wenkai Yin <yinw@vmware.com>
As we only provide the API to get artifact information via project name, repository name and digest, the digest of child artifact must be returned
Signed-off-by: Wenkai Yin <yinw@vmware.com>
1. Upgrade beego to v1.12.0
2. Add RequestID middleware to all HTTP requests.
3. Add Orm middleware to v2 and v2.0 APIs.
4. Remove OrmFilter from all HTTP requests.
5. Fix some test cases which cause panic in API controllers.
6. Enable XSRF for test cases of CommonController.
7. Imporve ReadOnly middleware.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
1, Leverage go v1.13 new error feature
2, Define genernal error OCI format, so that /v2 API could return a OCI compatible error
Signed-off-by: wang yan <wangyan@vmware.com>
1. Define the controller/manager interface for artifact and tag
2. Provide a null implementation for artifact manager
Signed-off-by: Wenkai Yin <yinw@vmware.com>
This commits does some basic init work for supporting OCI:
1. Create the artifact and tag model
2. Create database tables
Signed-off-by: Wenkai Yin <yinw@vmware.com>
This commit removes the attribute to control case-sensitivity from
authproxy setting.
The result in token review status will be used as the single source of
truth, regardless the case of the letters in group names and user names.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
This commit add an attribute to configurations, whose value is the
certificate of authproxy server. When this attribute is set Harbor will
pin to this cert when connecting authproxy.
This value will also be part of the response of systemInfo API.
This commit will be cherrypicked to 1.10 and 1.9 branch.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
This commit fixes#9869
It has some refactor to make sure the group is populated when user is
authenticated via tokenreview workflow.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
BREAKING CHANGE: the value negligible of severity in project metadata will change to none in the responses of project APIs
Signed-off-by: He Weiwei <hweiwei@vmware.com>
1. Fix name conflict when install internal clair adapter.
2. Remove all internal adapters when reinstall harbor without --with-clair flag
Signed-off-by: He Weiwei <hweiwei@vmware.com>
- update scan all job to avoid sending too many HTTP requets
- update scan controller to support scan options
- update the db schema of the scan report to introduce requester
- introduce scan all metrics to report the overall progress of scan all job
- fix the status updating bug in scan report
- enhance the admin job status updats
- add duplicate checking before triggering generic admin job
- update the db scheme of admin job
fix#9705fix#9722fix#9670
Signed-off-by: Steven Zou <szou@vmware.com>
1. Remove ping action when initialize clair adapter installed by harbor.
2. Remvoe the `IsDefault` property when initialize clair adapter that
make it switch to auto detecting.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
- add a common error pkg to support error with code and AsError check
- replace some errors in scan with coded errors
- fix#9668
Signed-off-by: Steven Zou <szou@vmware.com>
1, replace the UIVERSION file with ldflags, which is generarted by make to inject into the UI core.
2, inject additional ldflags for harbor compiler
Signed-off-by: wang yan <wangyan@vmware.com>
- add new endpoint for getting scanner candidates of specified project
- adjust the permission granting functions
- fix#9608
Signed-off-by: Steven Zou <szou@vmware.com>
1. Upgrade clair adapter to v1.0.0.
2. Make the clair adapter which installed by harbor immutable and using internal registry address.
3. Add support to build clair adapter image from binary.
4. Switch to ScannerPull action when make authorization for the scan request.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
- do changes to the sql schema
- add `UseInternalAddr` and `Immutable` properties to scanner registration
- support multiple authentication type
- basic
- bearer token
Signed-off-by: Steven Zou <szou@vmware.com>
1, the commit is for internal robot to bypass policy check, like vul and signature checking.
2, add a bool attribute into registry token, decode it in the harbor core and add the status into request context.
3, add a bool attribut for robot API controller, but API will not use it.y
Signed-off-by: wang yan <wangyan@vmware.com>
- add LCM control to the robot account generated for scanning
- improve the scan webhook
- remove reprots when related artifact is deleted
- update report manager/scan controller and other components to support above cases
- add artifact manager/comtroller to list artifacts
Signed-off-by: Steven Zou <szou@vmware.com>
To contain various vulnerabilities in the CVE whitelist, this commit
removes the validation.
Fixes#9242
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
- update the scan/scanner controller
- enhance the report summary generation
- do changes to the vulnerable handler
- remove the unused clair related code
- add more UT cases
- update the scan web hook event
- drop the unsed tables/index/triggers in sql schema
Signed-off-by: Steven Zou <szou@vmware.com>
1, add API controller for robot account, make it callable internally
2, add Manager to handler dao releate operation
Signed-off-by: wang yan <wangyan@vmware.com>
- implement scan controller
- add scan resource and update role bindings
- update registration model and related interfaces
Signed-off-by: Steven Zou <szou@vmware.com>
- implement scan API to do scan/get report/get log
- update repository rest API to produce scan report summary
- update scan job hook handler
- update some UT cases
- update robot account making content
- hidden credential in the job log
Commnet scan related API test cases which will be re-activate later
fix#8985
fix the issues found by codacy
Signed-off-by: Steven Zou <szou@vmware.com>
- change error collection in scan job
- add dead client checking in client pool
- change key word type to interface{} for q.Query
- update bearer authorizer
- add required UT cases
Signed-off-by: Steven Zou <szou@vmware.com>
- add scanner rest API v1 spec
- implement v1 client which is used to talk to scanner adapter
- adjust data/orm models
- adjust code package structure
Signed-off-by: Steven Zou <szou@vmware.com>
- implement scan client which is used to talk to scanner adapter
- implement scan job which take the work of communicating with scanner
- update scanner mgmt API routes
- add corresponding UT cases
- add DAO layer for scanner registration
- add CURD manager for scanner registration
- add API controller for plug scanner
- add REST APIs for CURD of plug scanner
- add migration sql:0011_1.10.0
- add scan interface definition (no implementations)
- add related UT cases with testify
fix#8979#8990
Signed-off-by: Steven Zou <szou@vmware.com>
This commit make sure that the "items" in response of project level
CVE_whitelist is not null, even when it's null in the DB the API will
return an empty list
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
