Commit Graph

211 Commits

Author SHA1 Message Date
Wang Yan
05d070865c
support acceleration service (#16682)
1, support acceleration service endpoints manage.
2, support auto convert.
3, support nydus as a new kinds of accessory.

Signed-off-by: wang yan <wangyan@vmware.com>
2022-04-11 11:46:59 +08:00
Wang Yan
bb6693b496
skip policy check on pull cosign signature (#16658)
When user enables the cosign policy and triggers the replication, the harbor adapter will try to  pull the cosign siguature if it has to do the further push.
In this case, it has to skip policy check.

Signed-off-by: wang yan <wangyan@vmware.com>
2022-04-07 09:59:26 +08:00
Wang Yan
7531168c92
update log with more inclusive language (#16569)
Use the deactive for instead in the error log

Signed-off-by: Wang Yan <wangyan@vmware.com>
2022-03-23 10:56:00 +08:00
chlins
5be0b93caa refactor: import go-redis to core as replacement of redigo
Signed-off-by: chlins <chenyuzh@vmware.com>
2022-03-15 20:44:44 +08:00
stonezdj
7aed95eda9 Add IP and agent in trace for failed login
Fixes #16423

Signed-off-by: stonezdj <stonezdj@gmail.com>
2022-02-28 15:17:55 +08:00
He Weiwei
abd85284d2
feat: merge local and remote tags for repo of proxy cache project in list API (#16394)
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2022-02-24 10:27:45 +08:00
Wang Yan
cd8d48794a
update pull policy code (#16237)
Reduce the DB call on pulling artifact with policy disabled

Signed-off-by: Wang Yan <wangyan@vmware.com>
2022-01-17 21:55:20 +08:00
Wang Yan
0b4f98074e
fix cosign conflict error on landing data (#16228)
Cosign client will generate the same signature to the same manifest, ignore the conflict error in middleware

Signed-off-by: Wang Yan <wangyan@vmware.com>
2022-01-17 19:09:56 +08:00
Wang Yan
063991078a
Cosign policy checker (#16187)
Enable policy checker for cosign, when it's enabled, user cannot pull artifact without cosign.

Signed-off-by: wang yan <wangyan@vmware.com>
2022-01-10 17:44:01 +08:00
Wang Yan
76b981faec
add cosign middleware (#16078)
The middleware is to land the cosign signature linkage with the subject artifact ID.

Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-12-07 15:43:10 +08:00
Qian Deng
354a2bd80d Enhance the trace related code
* Move request id to requestid middleware
* fix span pass to child ctx on orm
* fix typos
* remove unused code
* add operation name to Transaction

Signed-off-by: Qian Deng <dengq@vmware.com>
2021-09-22 04:14:43 +00:00
Qian Deng
bad913cf6d Refactor trace code
* use lib trace helper function
* add gracefull shutdown
* Add commens for new added exposed function
* Add licence on top of new created files
* Update trace library
* Update configs
* Add attribute and namespance in config

Signed-off-by: Qian Deng <dengq@vmware.com>
2021-09-18 10:58:52 +00:00
Qian Deng
a15983432c Add trace for core
* Add trace related lib
* Add trace middleware for core
* add rid for middleware

Signed-off-by: Qian Deng <dengq@vmware.com>
2021-09-18 10:58:52 +00:00
He Weiwei
d00024ab36 refactor: initialize the remote helper using ctx from http request
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-09-09 11:13:53 +00:00
He Weiwei
06f2414d1c
fix: use ctx from http request for middlewares (#15523)
1. Use ctx from http request for the readonly middleware.
2. Refactor the AuthenticateHelper to let it get orm from ctx of the http request.
3. Change to use ctx from http request for oidc and authproxy http handlers.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-09-02 19:05:35 +08:00
Wang Yan
14f7274989
support robot to list project (#15431)
1, add permission check for API of List Projects
2, add permission check for API of List Repositories
3, use the self defined query to handle both names and public query

Signed-off-by: wang yan <wangyan@vmware.com>
2021-08-17 16:35:36 +08:00
Wang Yan
f7a4401dcb
deprecate version 1 robot account (#15296)
1, deprecate support for version 1 robot support, the robotv1 cannot be used anymore.
2, reserve the /project/{id_or_name}/robots api.

After the PR, user cannot use the robotv1 to login, and do any interaction with Harbor,
but still can view & delete them with UI or API.

Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-07-13 13:39:44 +08:00
Wang Yan
b158086642
fix conformance failure (#15261)
fixes #15252

Give 404 for invalid digest when to get/head manifest

Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-07-06 15:11:13 +08:00
Wang Yan
2a1c9ec96a
update the log level of robot v1 context (#15037)
The robotv1 context uses the robot$ as a hardcoded prefix to identify robot account, it will raise error
for a valid robotv2 account with this prefix.

Update the log level to avoid the redundant logs for the default installation.

Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-06-01 18:30:40 +08:00
Daniel Jiang
238bea2066
Merge pull request #15010 from reasonerjt/v2auth-enhancement
Make v2auth more strict
2021-05-31 13:10:31 +08:00
Daniel Jiang
486554caa1
Merge pull request #15011 from reasonerjt/merge-slash-middleware
Add merge slash middleware
2021-05-31 13:09:39 +08:00
Wenkai Yin(尹文开)
4ed5fee681
Merge pull request #14949 from stonezdj/21may22_fallback_localregistry
Fall back to local registry when upstream registry is not working
2021-05-31 11:00:43 +08:00
Daniel Jiang
f4ac81b710 Add merge slash middleware
There are code in the core component to conditionally execute code based
on the pattern of url path, and different ingress controller or reverse
proxy may handle the dup slashes in the url path differently.
This commit merge dup slashes in the url paths to make things more
consistent.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-05-28 17:24:48 +08:00
Daniel Jiang
d3b8c613fd Make v2auth more strict
This commit enhances the v2auth middleware, such that any un-recognized
request sent to /v2/ will be blocked.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-05-28 16:41:48 +08:00
Wang Yan
42a9d0d905
remove common project code (#14939)
move project model from common to pkg

Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-05-25 11:01:19 +08:00
stonezdj
d7d26c0966 Fall back to local registry when upstream registry is not working
Fixes #14822
When upstream registry not working, but status might stay healthy because the health check interval is 5 minutes, if a pull request comes before registry status turns to unhealthy, the proxy cache middleware might proxy the request to the upstream registry and get a 401 error and this 401 error might translate to a http 500 error to the client eventually.

To solve this issue, it fall back all error to local registry when proxying manifest except the NotFoundError from the local registry.

Signed-off-by: stonezdj <stonezdj@gmail.com>
2021-05-24 14:29:05 +08:00
Wenkai Yin(尹文开)
0fe551274d
Merge pull request #14921 from ywk253100/210519_db
Clean up tech debt codes
2021-05-20 13:44:10 +08:00
Wenkai Yin
998e392bb4 Clean up tech debt codes
Clean up tech debt codes

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-05-20 11:47:46 +08:00
Daniel Jiang
294ee52d7a
Merge pull request #14900 from reasonerjt/rm-common-dao-getuser-onboard
Remove GetUser and Onboard from common/dao
2021-05-19 11:21:00 +08:00
Daniel Jiang
952644e23f Remove GetUser and Onboard from common/dao
Replaced by funcs in src/pkg/user and src/controller/user

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-05-18 15:28:41 +08:00
He Weiwei
0c315d8aee
refactor: remove allowlist in GetSummary of scan controller (#14836)
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-05-18 14:01:59 +08:00
Daniel Jiang
fa46b0d736
Merge pull request #14095 from jsoref/tokens
Tokens
2021-05-13 13:25:58 +08:00
stonezdj(Daojun Zhang)
0dfc801a50
Error string should not be capitalized (#14840)
Signed-off-by: stonezdj <stonezdj@gmail.com>
2021-05-12 16:53:37 +08:00
Daniel Jiang
6d0e391740 Move user related funcs from common/dao
This commit moves more user related funcs, such as ChangePassword,
Login, ChangeUserProfile from common/dao to rely on /pkg/user and
pkg/oidc.
It also removes the code for resetting user's password as it's disabled.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-05-12 01:12:57 +08:00
Daniel Jiang
c2ab1769b3
Merge pull request #14768 from reasonerjt/fix-14711
Return 401 for GET request to /v2 API for public artifacts.
2021-04-29 15:23:45 +08:00
Daniel Jiang
6a379d398c Return 401 for GET request to /v2 API for public artifacts.
This commits make sure when the request does not carry authorization
headers, the HEAD and GET will get the same response code.  This change
should be made due to #14711

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-04-28 09:14:43 +08:00
Daniel Jiang
5b526b8dc7
Remove dependencies from pkg/oidc to common/dao (#14739)
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-04-26 10:56:49 +08:00
stonezdj
60478f4990 Move common config api to lib/config
Register all config managers, and get it by getConfigManger()

Signed-off-by: stonezdj <stonezdj@gmail.com>
2021-04-13 19:43:33 +08:00
stonezdj
107e468b60 Refactor configure api to new programming model
Changes include:
1. Move core/config to controller/config
2. Change the job_service and gcreadonly to depends on lib/config instead of core/config
3. Move the config related dao, manager and driver to pkg/config
4. Adjust the invocation of the config API, most of then should provide a context parameter, when accessing system config, you can call it with background context, when accessing user config, the context should provide orm.Context

Signed-off-by: stonezdj <stonezdj@gmail.com>
2021-04-09 08:10:11 +08:00
Daniel Jiang
ad8eee8623 Add attribute admin username for authproxy
This commit adds the attribute "http_authproxy_admin_usernames", which
is string that contains usernames separated by comma, when a user logs
in and the username in the tokenreview status matches the setting of
this attribute, the user will have administrator permission.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-04-07 18:14:59 +08:00
Wenkai Yin(尹文开)
28596c3ffb
Refactor registry API (#14528)
* Refactor registry API

Refactor registry API

Signed-off-by: Wenkai Yin <yinw@vmware.com>

* Fix bugs of replications

1. Fix the scheduled replication doesn't work issue
2. Fix the destination name lost issue when updating replication policy

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-03-31 15:49:23 +08:00
Wang Yan
85254ccc22
refactor immutable rule (#14371)
Migrate immutable realted APIs to v2 swagger

Signed-off-by: wang yan <wangyan@vmware.com>
2021-03-08 17:10:12 +08:00
Daniel Jiang
e96c1cbced Switch API to ping OIDC endpoint to new model
This commit updates the API POST /api/v2.0/system/oidc/ping to new
programming model, in which the code will be generated by go-swagger.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-03-04 15:44:11 +08:00
Wang Yan
21d35f9702
update blob list query (#14195)
* update blob list query

Deprecate blob list parameters, and use the query for instead.

Signed-off-by: wang yan <wangyan@vmware.com>

* update per review comments

Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-02-09 17:08:26 +08:00
Wenkai Yin(尹文开)
120d88a0dd
Merge pull request #14116 from reasonerjt/reduce-oidc-warning
Reduce warning logs in OIDC middleware
2021-02-01 09:55:41 +08:00
DQ
15ad870262 Fix: unkonw metrics issue
Signed-off-by: DQ <dengq@vmware.com>
2021-01-29 18:07:06 +08:00
Daniel Jiang
2dd499bacf Reduce warning logs in OIDC middleware
If the request does not have bearer token in the header, do not decode
the empty string.
Fixes #12261

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-01-28 18:08:28 +08:00
Josh Soref
5be895cb39 Check return from token.DefaultTokenOptions()
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
2021-01-26 20:39:55 -05:00
DQ
28ae77e5c6 Aggregate metrics
1. Add operation id in ctx in baseapi before prepare
2. add operation id for registry proxy request
3. use url for other request

Signed-off-by: DQ <dengq@vmware.com>
2021-01-25 09:59:10 +08:00
Wang Yan
77347c54cf fix robot issues
fixes #13980
fixes #13981

1, add the robot prefix to the audit log
2, add duration maximum checking

Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-01-13 18:06:43 +08:00