* add debugging env for GC time window
For debugging, the tester/users wants to run GC to delete the removed artifact immediately instead of waitting for two hours, add the env(GC_BLOB_TIME_WINDOW) to meet this.
Signed-off-by: wang yan <wangyan@vmware.com>
* Raise an internal error if username claim is not found, instead of just logging a warning
* Don't remove userInfoKey for session on error when it is not required
* Rename "OIDC Username Claim" to just "Username claim"
Signed-off-by: Alvaro Iradier <airadier@gmail.com>
- Add an option in the UI to enable or disable the automatic user onboarding
- Add an option to specify the claim name where the username is retrieved from.
Signed-off-by: Alvaro Iradier <airadier@gmail.com>
To fetch vulnerability database updated time of the Clair had moved to
the Clair adapter so removes the initialization of clair db in the core.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
This commit rename the var name, text appearance, and swagger of "master" role
to "maintainer" role.
It only covers backend code.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
Use GenerateRandomStringWithLength function to generate secret for harbor_proxyservice
Add harbor-proxyservice secret used by proxy service
Signed-off-by: stonezdj <stonezdj@gmail.com>
Update to projects API so it will not differentiate if a project
does not exist or the user doesn't have permission to access it.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
The commit fix a regression introduced by #11672 which impacts admin
adding new users.
When admin is creating new users, /c/UserExists is called by UI. We must
allow it called by admin when self-registration is turned off.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
Fixes#9704
As we do want to unify error handling, so just decreprates pkg errors, use lib/errors instead for Harbor internal used errors model.
1, The lib/errors can cover all of funcs of pkg/errors, and also it has code attribute to define the http return value.
2, lib/errors can give a OCI standard error format, like {"errors":[{"code":"UNAUTHORIZED","message":"unauthorized"}]}
If you'd like to use pkg/errors, use lib/errors instead. If it cannot meet your request, enhance it.
Signed-off-by: wang yan <wangyan@vmware.com>
1. Remove `common/quota` package.
2. Remove functions about quota in `common/dao` package.
3. Move `Quota` and `QuotaUsage` models from `common/models` to
`pkg/quota/dao`.
4. Add `Count` and `List` methods to `quota.Controller`.
5. Use `quota.Controller` to implement quota APIs.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
Fixes#11241
1, remove count quota from quota manager
2, remove count in DB scheme
3, remove UI relates on quota
4, update UT, API test and UI UT.
Signed-off-by: wang yan <wangyan@vmware.com>
Docker CLI fails if it's not logged in upon seeing "basic" realm challenging while pinging the "/v2" endpoint. (#11266)
Some CLI will send HEAD to artifact endpoint before pushing (#11188)(#11271)
To fix such problems, this commit re-introduce the token auth flow to the CLIs.
For a HEAD request to "/v2/xxx" with no "Authoirzation" header, the v2_auth middleware populates the
"Www-Authenticate" header to redirect it to token endpoint with proper
requested scope.
It also adds security context to based on the content of the JWT which has the claims of the registry.
So a request from CLI carrying a token signed by the "/service/token" will have proper permissions.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
In Harbor 2.0, the replication isn't supported between instances with different versions, this commit returns the 404 error when trying to get the registry info whose version is different with the current one
Signed-off-by: Wenkai Yin <yinw@vmware.com>
Fixes#11016
1. src/pkg/q->src/internal/q
2. src/internal->src/lib (internal is a reserved package name of golang)
3. src/api->src/controller
Signed-off-by: Wenkai Yin <yinw@vmware.com>
* replication webhook support
Signed-off-by: guanxiatao <guanxiatao@corp.netease.com>
* replication webhook support with ut fixed
Signed-off-by: guanxiatao <guanxiatao@corp.netease.com>
As we introduce a new table to record the relationship between the artifacts and labels, the current way to list label's resources doesn't work anymore, and the API isn't needed by any features, remove it in 2.0
Signed-off-by: Wenkai Yin <yinw@vmware.com>
1, enable audit logs for notifications
2, move the handler and meatadata into API
3, use the notification middleware to send out notification
Signed-off-by: wang yan <wangyan@vmware.com>
1. Use quota controller to implement the internal quota APIs.
2. The internal quota APIs can exceed the quota limitations.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
the notification is for send out the event after DB transaction complete.
It's safe to send hook as this middleware is after transaction in the response path.
Signed-off-by: wang yan <wangyan@vmware.com>
1. Introduce `Evaluator` interface which do the permission checking.
2. `admin`, `lazy`, `rbac`, `namespace` and `evaluartor` set are implemented the
`Evaluator` interface.
3. Move project rbac implemention from `project` to `rbac` pkg to reduce
the name conflict with project instance of model.
4. Do permission checking in security context by `Evaluator`.
5. Cache the regexp in rbac evaluator for casbin.
6. Cache evaluator in namespace evaluator to improve performance.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
This commit replaces beego's CSRF mechanism with gorilla's csrf library.
The criteria for requests to skip the csrf check remain the same.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
1, enable dao test for artifact trash
2, set default flush trash table to false
3, hanlder empty parameter in API call
4, add registry auth info into jobservice container
Signed-off-by: wang yan <wangyan@vmware.com>
1. Add Checker to check the scannable status of the artifact.
2. Only set vulnerabilities addition link when the artifact scanable in the
project.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
As we swtich to basic auth for /v2/* API
The CLI secret should handle /v2/* API so that OIDC user can use the
secret to do push/pull
This commit makes such change.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
1, set harbor to readonly
2, select the candidate artifacts from Harbor DB.
3, call registry API(--delete-untagged=false) to delete manifest bases on the results of #2
4, clean keys of redis DB of registry, clean artifact trash and untagged from DB.
5, roll back readonly.
Signed-off-by: wang yan <wangyan@vmware.com>
* refactor(scan,scanner): move scan and scanner controllers to api pkg
Signed-off-by: He Weiwei <hweiwei@vmware.com>
* feat(scan-all-job): move artifacts query from job to notification
Move artifact query from scan all job to its notification handler to
ensure that the components in pkg will not call controllers in api.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
* Populate signature status in artifact API
This Commit add signature status into response of list artifact API.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
1. Escape and clean request path for `legacyAPISkipper`.
2. Escape and clean request path for `MethodAndPathSkipper`.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
1. Add `NewContext` and `FromContext` funcs in security pkg.
2. Add `Name` func in `security.Context` interface to make the checking
for the `/api/internal/configurations` API clear.
3. Get the security from the context to prepare change the security
filter to middleware.
4. Remove `GetSecurityContext` in filter pkg.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
1. Add basic authorizer for registry which modify the request
to add basic authorization header to request based on configuration.
2. Set basic auth header for proxy when accessing registry
3. Switche the registry to use basic auth by default and use the basic
authorizer to access Harbor.
4. Make necessary change to test cases, particularly
"test_robot_account.py" and "docker_api.py", because the error is
changed after siwtched to basic auth from token auth. #10604 is opened
to track the follow up work.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
When the registry shifts from token auth to basic auth, we'll use the middleware to check permission.
This commit add middlewares for populate the artifact info and check
permission based on request to /v2/* api via security context
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
1. Upgrade beego to v1.12.0
2. Add RequestID middleware to all HTTP requests.
3. Add Orm middleware to v2 and v2.0 APIs.
4. Remove OrmFilter from all HTTP requests.
5. Fix some test cases which cause panic in API controllers.
6. Enable XSRF for test cases of CommonController.
7. Imporve ReadOnly middleware.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
1. Get and set orm from context.
2. Add WithTransaction decorator make func run in transaction.
3. Support nested transaction by Savepoint.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
This commit updates `prepare` and templates to populate the credential
for registry for basic authentication.
A temporary flag `registry_use_basic_auth` was added to avoid breakage.
It MUST be removed before the release.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
We'll support manifest list, so this commit removes the manifest list middleware who blocks the manifest list pushing
Signed-off-by: Wenkai Yin <yinw@vmware.com>
This commit disables XSRF check for "service/token" so that when
containerd sends `POST` it will not return 403 and containerd can
fallback to `GET` to complete the workflow.
Fixes#10305
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
This commit will make the "tokenreview" security filter onboard
user if the request carries a valid token. If the "skipsearch" flag in
http_auth setting is set to false the onboard will fail.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
1, Leverage go v1.13 new error feature
2, Define genernal error OCI format, so that /v2 API could return a OCI compatible error
Signed-off-by: wang yan <wangyan@vmware.com>
Seperate the HasAdminRole(In DB) with the privileges from external auth, and use user.HasAdminPrivilege to check
Signed-off-by: stonezdj <stonezdj@gmail.com>
In order to replace travis.
Implement 5 CI jobs
- UTTEST
- APITEST_DB
- APITEST_LDAP
- OFFLINE
- UI_UT
Signed-off-by: Ziming Zhang <zziming@vmware.com>
statistic API use security Context to list project rather than calling
projectmanager directly, such that the group membership will be taken
into account.
fixes#10230
It should be cherry picked to 1.9.x and 1.10.x branches
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
