* Move request id to requestid middleware
* fix span pass to child ctx on orm
* fix typos
* remove unused code
* add operation name to Transaction
Signed-off-by: Qian Deng <dengq@vmware.com>
This commit make sure when security context is not found the `Get`
function in user controller should not return error.
Because this func will be called by security middleware, at which point
of time the security context is not generated.
Additionally, checking security context is not necessary because the
permission checking is already done in the API handler layer.
fixes#15535
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
* Refactor common http GetTransport function signature
* Remove redendent GetHTTPTransport and similar functions
* Update Authorized function signature to meet new HTTPTransport
* Add trace for default Transport
Signed-off-by: Qian Deng <dengq@vmware.com>
1. Use ctx from http request for the readonly middleware.
2. Refactor the AuthenticateHelper to let it get orm from ctx of the http request.
3. Change to use ctx from http request for oidc and authproxy http handlers.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
Fixes#15450
Add paging function to usergroup list/search API
Fix some 500 error when adding LDAP user/group to project member
Signed-off-by: stonezdj <stonezdj@gmail.com>
1. Use jpillora/backoff to get the backoff to avoid the panic in RetryUntil.
2. Return with last err when retry timeout.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
1, for admin only, the system level robot should contains the project creation access.
2, for not admin only, the system level robot can create project.
3, for the project that created by system level robot, use the admin ID as the ownerID.
No path for project level robot to create project.
Signed-off-by: wang yan <wangyan@vmware.com>
The following information should cleanup before delete user:
Delete project member of this user.
Delete oidc_user when auth_mode is oidc_auth.
Fixes#8424
It also removes the deleted user from project member and the deleted condition in the project member query for consistency
Signed-off-by: stonezdj <stonezdj@gmail.com>
The transaction will be aborted when get errors during the execution which causes the following sqls report error.
This commit moves the re-getting artifact logic out of the second transaction to avoid the concurrent pushing issue
Signed-off-by: Wenkai Yin <yinw@vmware.com>
DB transaction failure may break the loop query of the artifacts, the
result is that not all artifacts are scanned in one scan all job. Using
a new DB connection to call the Scan method of the controller to avoid
this problem.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
When the core service cannot response the checkin request in time, duplicated execution records may be created, this commit introduces the revision column to make sure there is only one record for one schedule trigger
Signed-off-by: Wenkai Yin <yinw@vmware.com>
This commit moves more user related funcs, such as ChangePassword,
Login, ChangeUserProfile from common/dao to rely on /pkg/user and
pkg/oidc.
It also removes the code for resetting user's password as it's disabled.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
Add env CONFIG_OVERWRITE_JSON for declarative config
Init config with the json in CONFIG_OVERWRITE_JSON in main.go
Signed-off-by: stonezdj <stonezdj@gmail.com>
This commit remove some funcs from package `common/dao/user` that can be
covered by the manager in `pkg/user`.
Ideally all funcs should be replaced but the dependency relationships
are tricky for some of them I'll push other commit to clean them up.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
Remove deps to common/dao
Move Manager interface to config.go
Remove duplicate code and change format of dao.go
Signed-off-by: stonezdj <stonezdj@gmail.com>
* Refactor labl api
move to the new program model
Signed-off-by: wang yan <wangyan@vmware.com>
* continue resolve review comments
Signed-off-by: Wang Yan <wangyan@vmware.com>
This commit skips getting OIDC meta info for admin user.
It fixes the issue that admin user cannot login to portal, which was
introduced in refactor.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
Add pkg/member/dao
Add pkg/member/models
Add pkg/member/manager
Add controller/member
Remove the old project member API
Signed-off-by: stonezdj <stonezdj@gmail.com>
This commit moves the legacy apis related to users to new model.
Some funcs under common/dao are left b/c they are used by other module,
which should also be shifted to leverage managers.
We'll handle them separately.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
Improve the performance of replication by introducing a new API to check whether the blob can be mounted directly
Signed-off-by: Wenkai Yin <yinw@vmware.com>
Changes include:
1. Move core/config to controller/config
2. Change the job_service and gcreadonly to depends on lib/config instead of core/config
3. Move the config related dao, manager and driver to pkg/config
4. Adjust the invocation of the config API, most of then should provide a context parameter, when accessing system config, you can call it with background context, when accessing user config, the context should provide orm.Context
Signed-off-by: stonezdj <stonezdj@gmail.com>
* Refactor registry API
Refactor registry API
Signed-off-by: Wenkai Yin <yinw@vmware.com>
* Fix bugs of replications
1. Fix the scheduled replication doesn't work issue
2. Fix the destination name lost issue when updating replication policy
Signed-off-by: Wenkai Yin <yinw@vmware.com>
* update blob list query
Deprecate blob list parameters, and use the query for instead.
Signed-off-by: wang yan <wangyan@vmware.com>
* update per review comments
Signed-off-by: Wang Yan <wangyan@vmware.com>
1. Remove the default execution sweeper count for execution vendor.
2. Set the execution sweeper count for gc, preheat, replication,
retention to 50.
3. Disable sweep for the executions of the scan job.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
Don't ignore the NotFoundErr when handling the status hook of tasks to avoid the status out of sync
Fixes#14016
Signed-off-by: Wenkai Yin <yinw@vmware.com>
1. Register task status change function for the scan job triggered by
scan all.
2. Always to delete the robot account for the scan job after the job is
finished because the job does not retry again when it's failed.
Closes#14000
Signed-off-by: He Weiwei <hweiwei@vmware.com>
1, introduce & define the system resources.
2, replace the IsSysAdmin judge method.
3, give the robot the system access capability.
Signed-off-by: Wang Yan <wangyan@vmware.com>
feat: Store vulnerability report from scanner into a relational format
Convert vulnerability report JSON obtained from scanner into a relational format describe in:https://github.com/goharbor/community/pull/145
Signed-off-by: prahaladdarkin <prahaladd@vmware.com>
Fixes#13740
Update ManifestExist to return Descriptor instead of digest
For docker 20.10 or containerd, it HEAD the manifest before pull, then
it GET the manifest with digest, add logic to handle this scenario and
correlate the tag between the digest in proxy cache
Signed-off-by: stonezdj <stonezdj@gmail.com>
This reverts commit e6f80259
This reverts commit 918fe125
Signed-off-by: stonezdj <stonezdj@gmail.com>
Revert "Add content type and length in header"
This reverts commit ca379111
Signed-off-by: stonezdj <stonezdj@gmail.com>
Fixes#13566: Quota of dockerhub is still used in v2.1.1 after the image is cached
Cache manifest list in redis cache.
Trade off between efficiency and data integrity, it might cause the proxy cache return the full content of a manifest list instead of the actual manifest list saved in the Harbor storage, which is a part of the manifest list. but this change doesn't break any /v2/ API, just caches full manifest list.
Signed-off-by: stonezdj <stonezdj@gmail.com>
1, remove the gc to new programming model
2, move api define to harbor v2 swagger
3, leverage task & execution manager to manage gc job schedule, trigger and log.
Signed-off-by: wang yan <wangyan@vmware.com>
* update robot secret
1, use SHA256 to generate and validate robot secret instread of symmetric encryption.
2, update the patch input object
Signed-off-by: Wang Yan <wangyan@vmware.com>
* update robot secret
1, use SHA256 to generate and validate robot secret instread of symmetric encryption.
2, update the patch input object
Signed-off-by: Wang Yan <wangyan@vmware.com>
* updates on robot accounts
1, add patch method to refresh secret of a robot
2, fix robot account update issue
3, add editable attribute to handle the version 1 robot account
4, add duration for robot account
5, hide secret for get/list robot account
Signed-off-by: wang yan <wangyan@vmware.com>
* update code per review comments
1, change expirate creation func to AddDate().
2, remove the scanner duration specification, use the default value.
Signed-off-by: Wang Yan <wangyan@vmware.com>
default is true and doesn't break any existing api, and when to set it to false, the api only return the basic project infor
without meta, CVE settings and etc of the project.
Signed-off-by: wang yan <wangyan@vmware.com>
author Wang Yan <wangyan@vmware.com> 1605849192 +0800
committer Wang Yan <wangyan@vmware.com> 1606361046 +0800
update code per review comments
Signed-off-by: wang yan <wangyan@vmware.com>
1. Clean the dirty data in quota/quota_usage.
2. Ignore the fail when getting the reference of quota.
Closes#13387
Signed-off-by: He Weiwei <hweiwei@vmware.com>
1. Use the task manager to manage the underlying execution/task
2. Use the pkg/scheduler to schedule the periodical job
3. Apply the new program model
4. Migration the old data into the new data model
Signed-off-by: Wenkai Yin <yinw@vmware.com>
* refactor: remove core/promgr pkg
Remove `core/promgr` package and use `controller/project` instead of it.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
Use `project.Controller` instead of `promgr.ProjectManager` in security
implementations because we will remove `promgr` package later.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
Compare the local digest and the remote digest when pull by tag
Use HEAD request (ManifestExist) instead of GET request (GetManifest) to avoid been throttled
For manifest list, it can avoid GET request because cached manifest list maybe different with the original manifest list
Make RemoteInterface public
Fixes#13112
Signed-off-by: stonezdj <stonezdj@gmail.com>
There is requirement that show the cron type(daily, weekly, etc.) on the UI, this commit adds the support for storing the cron type in the schedule model
Signed-off-by: Wenkai Yin <yinw@vmware.com>
Add a new method "StopAndWait" which stops the execution and wait until the execution stopped or get an error
Signed-off-by: Wenkai Yin <yinw@vmware.com>
1. Abstract the "config" property(which contains labels) of config layer into the extra attributes for images
2. Try to get the author information from the "maintainer" label
fixes 12066
fixes 12734
Signed-off-by: Wenkai Yin <yinw@vmware.com>
