1, enable dao test for artifact trash
2, set default flush trash table to false
3, hanlder empty parameter in API call
4, add registry auth info into jobservice container
Signed-off-by: wang yan <wangyan@vmware.com>
1. Add Checker to check the scannable status of the artifact.
2. Only set vulnerabilities addition link when the artifact scanable in the
project.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
1, set harbor to readonly
2, select the candidate artifacts from Harbor DB.
3, call registry API(--delete-untagged=false) to delete manifest bases on the results of #2
4, clean keys of redis DB of registry, clean artifact trash and untagged from DB.
5, roll back readonly.
Signed-off-by: wang yan <wangyan@vmware.com>
* refactor(scan,scanner): move scan and scanner controllers to api pkg
Signed-off-by: He Weiwei <hweiwei@vmware.com>
* feat(scan-all-job): move artifacts query from job to notification
Move artifact query from scan all job to its notification handler to
ensure that the components in pkg will not call controllers in api.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
* Populate signature status in artifact API
This Commit add signature status into response of list artifact API.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
1. Add `NewContext` and `FromContext` funcs in security pkg.
2. Add `Name` func in `security.Context` interface to make the checking
for the `/api/internal/configurations` API clear.
3. Get the security from the context to prepare change the security
filter to middleware.
4. Remove `GetSecurityContext` in filter pkg.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
1. Add basic authorizer for registry which modify the request
to add basic authorization header to request based on configuration.
2. Set basic auth header for proxy when accessing registry
3. Switche the registry to use basic auth by default and use the basic
authorizer to access Harbor.
4. Make necessary change to test cases, particularly
"test_robot_account.py" and "docker_api.py", because the error is
changed after siwtched to basic auth from token auth. #10604 is opened
to track the follow up work.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
1. Upgrade beego to v1.12.0
2. Add RequestID middleware to all HTTP requests.
3. Add Orm middleware to v2 and v2.0 APIs.
4. Remove OrmFilter from all HTTP requests.
5. Fix some test cases which cause panic in API controllers.
6. Enable XSRF for test cases of CommonController.
7. Imporve ReadOnly middleware.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
1, Leverage go v1.13 new error feature
2, Define genernal error OCI format, so that /v2 API could return a OCI compatible error
Signed-off-by: wang yan <wangyan@vmware.com>
Seperate the HasAdminRole(In DB) with the privileges from external auth, and use user.HasAdminPrivilege to check
Signed-off-by: stonezdj <stonezdj@gmail.com>
In order to replace travis.
Implement 5 CI jobs
- UTTEST
- APITEST_DB
- APITEST_LDAP
- OFFLINE
- UI_UT
Signed-off-by: Ziming Zhang <zziming@vmware.com>
statistic API use security Context to list project rather than calling
projectmanager directly, such that the group membership will be taken
into account.
fixes#10230
It should be cherry picked to 1.9.x and 1.10.x branches
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
This commit removes the attribute to control case-sensitivity from
authproxy setting.
The result in token review status will be used as the single source of
truth, regardless the case of the letters in group names and user names.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
This commit add an attribute to configurations, whose value is the
certificate of authproxy server. When this attribute is set Harbor will
pin to this cert when connecting authproxy.
This value will also be part of the response of systemInfo API.
This commit will be cherrypicked to 1.10 and 1.9 branch.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
This commit fixes#9946, that when creating a project the minimum length
should be 1, not 2.
This commit should be cherry picked to 1.9.x and 1.10.x branch .
We need to double check if this change impacts the creation of replication
rule.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
BREAKING CHANGE: the value negligible of severity in project metadata will change to none in the responses of project APIs
Signed-off-by: He Weiwei <hweiwei@vmware.com>
This commit make case sensitivity configurable when the authentication
backend is auth proxy.
When the "http_authproxy_case_sensitive" is set to false, the name of
user/group will be converted to lower-case when onboarded to Harbor, so
as long as the authentication is successful there's no difference regardless
upper or lower case is used. It will be mapped to one entry in Harbor's
User/Group table.
Similar to auth_mode, there is limitation that once there are users
onboarded to Harbor's DB this attribute is not configurable.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
This commit fixes#9771
It compares the roles to return the one with highest permission in the
response of `GET /api/projects`.
In addition to that, it adds the role list to the response, because a
user can have multiple roles in a project.
It also removes the togglable attribute as it's not used anywhere.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
- update scan all job to avoid sending too many HTTP requets
- update scan controller to support scan options
- update the db schema of the scan report to introduce requester
- introduce scan all metrics to report the overall progress of scan all job
- fix the status updating bug in scan report
- enhance the admin job status updats
- add duplicate checking before triggering generic admin job
- update the db scheme of admin job
fix#9705fix#9722fix#9670
Signed-off-by: Steven Zou <szou@vmware.com>
The severity saved in db is lowercase but the severities in vuln pkg
begin with upper letter, this fix use func to transform project severity
value from db to vuln.Severity.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
- add a common error pkg to support error with code and AsError check
- replace some errors in scan with coded errors
- fix#9668
Signed-off-by: Steven Zou <szou@vmware.com>
1, replace the UIVERSION file with ldflags, which is generarted by make to inject into the UI core.
2, inject additional ldflags for harbor compiler
Signed-off-by: wang yan <wangyan@vmware.com>
- add new endpoint for getting scanner candidates of specified project
- adjust the permission granting functions
- fix#9608
Signed-off-by: Steven Zou <szou@vmware.com>
1, eat the unknown manifest error, and log it. The migration process will not crashed on it.
2, enable to persist DB of sync quota API.
3, add empty project support.
Signed-off-by: wang yan <wangyan@vmware.com>
- do changes to the sql schema
- add `UseInternalAddr` and `Immutable` properties to scanner registration
- support multiple authentication type
- basic
- bearer token
Signed-off-by: Steven Zou <szou@vmware.com>
As we introduce the pluggable scanner, users can add the external scanners, so we remove the Clair from the health check API
Signed-off-by: Wenkai Yin <yinw@vmware.com>
- add LCM control to the robot account generated for scanning
- improve the scan webhook
- remove reprots when related artifact is deleted
- update report manager/scan controller and other components to support above cases
- add artifact manager/comtroller to list artifacts
Signed-off-by: Steven Zou <szou@vmware.com>
1. Only show project member info when has member list permission.
2. Only show quota info when has quota read permission.
3. Add quota read permission for all roles of project.
4. Refactor permission service in portoal.
5. Clear cache when clear session.
Closes#8697
Signed-off-by: He Weiwei <hweiwei@vmware.com>
To contain various vulnerabilities in the CVE whitelist, this commit
removes the validation.
Fixes#9242
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
- update the scan/scanner controller
- enhance the report summary generation
- do changes to the vulnerable handler
- remove the unused clair related code
- add more UT cases
- update the scan web hook event
- drop the unsed tables/index/triggers in sql schema
Signed-off-by: Steven Zou <szou@vmware.com>
- refactor the db schema \
- refactor permission checking in API handlers \
to follow the latest code/interface changes
Signed-off-by: Steven Zou <szou@vmware.com>
1, add API controller for robot account, make it callable internally
2, add Manager to handler dao releate operation
Signed-off-by: wang yan <wangyan@vmware.com>
- implement scan controller
- add scan resource and update role bindings
- update registration model and related interfaces
Signed-off-by: Steven Zou <szou@vmware.com>
- implement scan API to do scan/get report/get log
- update repository rest API to produce scan report summary
- update scan job hook handler
- update some UT cases
- update robot account making content
- hidden credential in the job log
Commnet scan related API test cases which will be re-activate later
fix#8985
fix the issues found by codacy
Signed-off-by: Steven Zou <szou@vmware.com>
- change error collection in scan job
- add dead client checking in client pool
- change key word type to interface{} for q.Query
- update bearer authorizer
- add required UT cases
Signed-off-by: Steven Zou <szou@vmware.com>
This commit returns a common error message when testing the webhook endpoint and prints the detail in the log for debug
Signed-off-by: Wenkai Yin <yinw@vmware.com>
- add scanner rest API v1 spec
- implement v1 client which is used to talk to scanner adapter
- adjust data/orm models
- adjust code package structure
Signed-off-by: Steven Zou <szou@vmware.com>
- implement scan client which is used to talk to scanner adapter
- implement scan job which take the work of communicating with scanner
- update scanner mgmt API routes
- add corresponding UT cases
This commit mitigates the Session Fixation issue by making sure a new
session ID is generated each time user logs in to Harbor
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
This commit enable project admin to add group as project member when
Harbor is configured against OIDC as AuthN backend.
It populates the information of groups from ID Token based on the claim
that is set in OIDC settings.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
- add DAO layer for scanner registration
- add CURD manager for scanner registration
- add API controller for plug scanner
- add REST APIs for CURD of plug scanner
- add migration sql:0011_1.10.0
- add scan interface definition (no implementations)
- add related UT cases with testify
fix#8979#8990
Signed-off-by: Steven Zou <szou@vmware.com>
1. Sync blobs from manifest for image with foreign layers.
2. Ignore size of foreign layers when compute size quota.
3. Fix repo info of artifact when upgrade from 1.8 version.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
The foreign layer won't be counted into project quota
NOTE: the foreign layer will be dumped from the registry in the migration
Signed-off-by: wang yan <wangyan@vmware.com>
1, fix#8802, update the error formet
2, fix#8807, raise the real retag error to UI
3, fix#8832, raise the real chart error to chart client & ut
Signed-off-by: wang yan <wangyan@vmware.com>
The scheduler hook handler doesn't parse the job status struct when handling the hook. This commit fixes it.
Signed-off-by: Wenkai Yin <yinw@vmware.com>
this is for internal registry api call, the request should be intercpeted by quota middlerwares, like retag and delete.
Note: The api developer has to know that if the internal registry call in your api, please consider to use
NewRepositoryClientForLocal() to init the repository client, which can handle quota change.
Signed-off-by: wang yan <wangyan@vmware.com>
