* Raise an internal error if username claim is not found, instead of just logging a warning
* Don't remove userInfoKey for session on error when it is not required
* Rename "OIDC Username Claim" to just "Username claim"
Signed-off-by: Alvaro Iradier <airadier@gmail.com>
- Add an option in the UI to enable or disable the automatic user onboarding
- Add an option to specify the claim name where the username is retrieved from.
Signed-off-by: Alvaro Iradier <airadier@gmail.com>
To fetch vulnerability database updated time of the Clair had moved to
the Clair adapter so removes the initialization of clair db in the core.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
This commit rename the var name, text appearance, and swagger of "master" role
to "maintainer" role.
It only covers backend code.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
1, add two more attributes, update_time and status
2, add delete and fresh update time method in blob mgr & ctr.
Signed-off-by: wang yan <wangyan@vmware.com>
* move send error to source lib
Move the sendError into library in case the cycle dependency as regsitry and core are now the consumers.
Signed-off-by: wang yan <wangyan@vmware.com>
Use GenerateRandomStringWithLength function to generate secret for harbor_proxyservice
Add harbor-proxyservice secret used by proxy service
Signed-off-by: stonezdj <stonezdj@gmail.com>
Fixes#9704
As we do want to unify error handling, so just decreprates pkg errors, use lib/errors instead for Harbor internal used errors model.
1, The lib/errors can cover all of funcs of pkg/errors, and also it has code attribute to define the http return value.
2, lib/errors can give a OCI standard error format, like {"errors":[{"code":"UNAUTHORIZED","message":"unauthorized"}]}
If you'd like to use pkg/errors, use lib/errors instead. If it cannot meet your request, enhance it.
Signed-off-by: wang yan <wangyan@vmware.com>
1. Remove `common/quota` package.
2. Remove functions about quota in `common/dao` package.
3. Move `Quota` and `QuotaUsage` models from `common/models` to
`pkg/quota/dao`.
4. Add `Count` and `List` methods to `quota.Controller`.
5. Use `quota.Controller` to implement quota APIs.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
Fixes#11241
1, remove count quota from quota manager
2, remove count in DB scheme
3, remove UI relates on quota
4, update UT, API test and UI UT.
Signed-off-by: wang yan <wangyan@vmware.com>
Docker CLI fails if it's not logged in upon seeing "basic" realm challenging while pinging the "/v2" endpoint. (#11266)
Some CLI will send HEAD to artifact endpoint before pushing (#11188)(#11271)
To fix such problems, this commit re-introduce the token auth flow to the CLIs.
For a HEAD request to "/v2/xxx" with no "Authoirzation" header, the v2_auth middleware populates the
"Www-Authenticate" header to redirect it to token endpoint with proper
requested scope.
It also adds security context to based on the content of the JWT which has the claims of the registry.
So a request from CLI carrying a token signed by the "/service/token" will have proper permissions.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
Fixes#11225
As registry changes to basic auth, the push action lost the pull permission.
Add it in the robot security context.
Signed-off-by: wang yan <wangyan@vmware.com>
In Harbor 2.0, the replication isn't supported between instances with different versions, this commit returns the 404 error when trying to get the registry info whose version is different with the current one
Signed-off-by: Wenkai Yin <yinw@vmware.com>
Fixes#11016
1. src/pkg/q->src/internal/q
2. src/internal->src/lib (internal is a reserved package name of golang)
3. src/api->src/controller
Signed-off-by: Wenkai Yin <yinw@vmware.com>
* replication webhook support
Signed-off-by: guanxiatao <guanxiatao@corp.netease.com>
* replication webhook support with ut fixed
Signed-off-by: guanxiatao <guanxiatao@corp.netease.com>
1. Use quota controller to implement the internal quota APIs.
2. The internal quota APIs can exceed the quota limitations.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
1. Introduce `Evaluator` interface which do the permission checking.
2. `admin`, `lazy`, `rbac`, `namespace` and `evaluartor` set are implemented the
`Evaluator` interface.
3. Move project rbac implemention from `project` to `rbac` pkg to reduce
the name conflict with project instance of model.
4. Do permission checking in security context by `Evaluator`.
5. Cache the regexp in rbac evaluator for casbin.
6. Cache evaluator in namespace evaluator to improve performance.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
1, set harbor to readonly
2, select the candidate artifacts from Harbor DB.
3, call registry API(--delete-untagged=false) to delete manifest bases on the results of #2
4, clean keys of redis DB of registry, clean artifact trash and untagged from DB.
5, roll back readonly.
Signed-off-by: wang yan <wangyan@vmware.com>
1. Add middleware to record the accepted blob size for stream blob
upload.
2. Add middleware to create blob and associate it with project after blob upload
complete.
3. Add middleware to sync blobs, create blob for manifest and associate blobs
with the manifest after put manifest.
4. Add middleware to associate blob with project after mount blob.
5. Cleanup associations for the project when artifact deleted.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
* Populate signature status in artifact API
This Commit add signature status into response of list artifact API.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
This commit add supporting for adding/removing label to/from artifacts and populates labels when listing artifacts
Signed-off-by: Wenkai Yin <yinw@vmware.com>
This commit introduces a new wrapper authorizer which can authorize the request according to the auth scheme automatically
Signed-off-by: Wenkai Yin <yinw@vmware.com>
This commit implements the API to get build history of image with manifest version 2 and populates the addition links when listing/getting the artifact
Signed-off-by: Wenkai Yin <yinw@vmware.com>
1. Add `NewContext` and `FromContext` funcs in security pkg.
2. Add `Name` func in `security.Context` interface to make the checking
for the `/api/internal/configurations` API clear.
3. Get the security from the context to prepare change the security
filter to middleware.
4. Remove `GetSecurityContext` in filter pkg.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
1. Add basic authorizer for registry which modify the request
to add basic authorization header to request based on configuration.
2. Set basic auth header for proxy when accessing registry
3. Switche the registry to use basic auth by default and use the basic
authorizer to access Harbor.
4. Make necessary change to test cases, particularly
"test_robot_account.py" and "docker_api.py", because the error is
changed after siwtched to basic auth from token auth. #10604 is opened
to track the follow up work.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
1, Leverage go v1.13 new error feature
2, Define genernal error OCI format, so that /v2 API could return a OCI compatible error
Signed-off-by: wang yan <wangyan@vmware.com>
Seperate the HasAdminRole(In DB) with the privileges from external auth, and use user.HasAdminPrivilege to check
Signed-off-by: stonezdj <stonezdj@gmail.com>
This commit removes the attribute to control case-sensitivity from
authproxy setting.
The result in token review status will be used as the single source of
truth, regardless the case of the letters in group names and user names.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
This commit add an attribute to configurations, whose value is the
certificate of authproxy server. When this attribute is set Harbor will
pin to this cert when connecting authproxy.
This value will also be part of the response of systemInfo API.
This commit will be cherrypicked to 1.10 and 1.9 branch.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
This commit refactors the flow to populate user info and verify CLI
secret in OIDC authentication.
It will call the `userinfo` backend of OIDC backend and fallback to
using the ID token if userinfo is not supported by the backend.
It also makes sure the token will be persisted if it's refreshed during
this procedure.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
BREAKING CHANGE: the value negligible of severity in project metadata will change to none in the responses of project APIs
Signed-off-by: He Weiwei <hweiwei@vmware.com>
This commit make case sensitivity configurable when the authentication
backend is auth proxy.
When the "http_authproxy_case_sensitive" is set to false, the name of
user/group will be converted to lower-case when onboarded to Harbor, so
as long as the authentication is successful there's no difference regardless
upper or lower case is used. It will be mapped to one entry in Harbor's
User/Group table.
Similar to auth_mode, there is limitation that once there are users
onboarded to Harbor's DB this attribute is not configurable.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
This commit fixes#9771
It compares the roles to return the one with highest permission in the
response of `GET /api/projects`.
In addition to that, it adds the role list to the response, because a
user can have multiple roles in a project.
It also removes the togglable attribute as it's not used anywhere.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
- update scan all job to avoid sending too many HTTP requets
- update scan controller to support scan options
- update the db schema of the scan report to introduce requester
- introduce scan all metrics to report the overall progress of scan all job
- fix the status updating bug in scan report
- enhance the admin job status updats
- add duplicate checking before triggering generic admin job
- update the db scheme of admin job
fix#9705fix#9722fix#9670
Signed-off-by: Steven Zou <szou@vmware.com>
The severity saved in db is lowercase but the severities in vuln pkg
begin with upper letter, this fix use func to transform project severity
value from db to vuln.Severity.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
- add new endpoint for getting scanner candidates of specified project
- adjust the permission granting functions
- fix#9608
Signed-off-by: Steven Zou <szou@vmware.com>
1, the commit is for internal robot to bypass policy check, like vul and signature checking.
2, add a bool attribute into registry token, decode it in the harbor core and add the status into request context.
3, add a bool attribut for robot API controller, but API will not use it.y
Signed-off-by: wang yan <wangyan@vmware.com>
As we introduce the pluggable scanner, users can add the external scanners, so we remove the Clair from the health check API
Signed-off-by: Wenkai Yin <yinw@vmware.com>
1. Only show project member info when has member list permission.
2. Only show quota info when has quota read permission.
3. Add quota read permission for all roles of project.
4. Refactor permission service in portoal.
5. Clear cache when clear session.
Closes#8697
Signed-off-by: He Weiwei <hweiwei@vmware.com>
1) Disassociate id token from user session
2) Some OIDC providers do not return id_token in the response of refresh
request:
https://openid.net/specs/openid-connect-core-1_0.html#RefreshTokenResponse
When validating the CLI secret it will not validate the id token,
instead it will check the expiration of the access token, and try to
refresh it.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
- update the scan/scanner controller
- enhance the report summary generation
- do changes to the vulnerable handler
- remove the unused clair related code
- add more UT cases
- update the scan web hook event
- drop the unsed tables/index/triggers in sql schema
Signed-off-by: Steven Zou <szou@vmware.com>
1, add API controller for robot account, make it callable internally
2, add Manager to handler dao releate operation
Signed-off-by: wang yan <wangyan@vmware.com>
- implement scan controller
- add scan resource and update role bindings
- update registration model and related interfaces
Signed-off-by: Steven Zou <szou@vmware.com>
- implement scan API to do scan/get report/get log
- update repository rest API to produce scan report summary
- update scan job hook handler
- update some UT cases
- update robot account making content
- hidden credential in the job log
Commnet scan related API test cases which will be re-activate later
fix#8985
fix the issues found by codacy
Signed-off-by: Steven Zou <szou@vmware.com>
This commit enable project admin to add group as project member when
Harbor is configured against OIDC as AuthN backend.
It populates the information of groups from ID Token based on the claim
that is set in OIDC settings.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
This commit add the new setting "oidc_groups_claim" to Harbor's
configurations.
And add "group_claim" to OIDCSetting struct.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
We changed the listenning port of portal from 80 to 8080 to run the process as non-root user, but the change didn't update the default URL of portal in source code, this causes the health check API fail.
Signed-off-by: Wenkai Yin <yinw@vmware.com>
1. Sync blobs from manifest for image with foreign layers.
2. Ignore size of foreign layers when compute size quota.
3. Fix repo info of artifact when upgrade from 1.8 version.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
The foreign layer won't be counted into project quota
NOTE: the foreign layer will be dumped from the registry in the migration
Signed-off-by: wang yan <wangyan@vmware.com>
the project id is missing in the method, that makes GC to clean all of items,
and if quota will not compute twice for the existing manifest.
Signed-off-by: wang yan <wangyan@vmware.com>
1, fix#8802, update the error formet
2, fix#8807, raise the real retag error to UI
3, fix#8832, raise the real chart error to chart client & ut
Signed-off-by: wang yan <wangyan@vmware.com>
1. Skip overflow error when subtract resources
2. Take up resources before handle request and put it back when handle
failed for add action in quota interceptor
3. Free resources only after handle success for subtract action in quota
interceptor
Closes#8681
Signed-off-by: He Weiwei <hweiwei@vmware.com>
The way Harbor handles notification is problematic.
It currently triggers rescan, which will cause problem when there are
lot of images in the registry.
Such as #7316
This commit removes the notifier and we need to revisit the notification
to figure out how to map the notification to a particular image if need
the notification mechanism in future.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
this is for internal registry api call, the request should be intercpeted by quota middlerwares, like retag and delete.
Note: The api developer has to know that if the internal registry call in your api, please consider to use
NewRepositoryClientForLocal() to init the repository client, which can handle quota change.
Signed-off-by: wang yan <wangyan@vmware.com>
This commit fixes#8432
When querying the role of group ID, all matched roles should be returned
instead of the minimal role ID.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
[Add]:
1, size middleware for quota size
2, count middleware for quota artifact count
[Support]:
1, put, patch, mount blob
2, put manifest
[Refactor]:
1, Add handle response for middlerware
2, Remove the modifyResponse for registry proxy
3, Use the custom response writer to recored status
Signed-off-by: wang yan <wangyan@vmware.com>
If harbor is built (or `go test`'d) in a different folder than the repo
root, the call to common/utils/log/line(...) will panic with an index
out of range runtime error because the separator can't find `harbor/src`
in the path.
Signed-off-by: Nathan Lowe <public@nlowe.me>
