Commit Graph

93 Commits

Author SHA1 Message Date
wang yan
0e175017aa continue updating code per review comments
Signed-off-by: wang yan <wangyan@vmware.com>
2020-06-23 14:42:28 +08:00
wang yan
446739f967 rebase with latest source code
Signed-off-by: wang yan <wangyan@vmware.com>
2020-06-23 13:10:57 +08:00
wang yan
c10467eb36 continue refactor
Signed-off-by: wang yan <wangyan@vmware.com>
2020-06-23 13:10:57 +08:00
Wang Yan
58b7242a25
move send error to source lib (#12175)
* move send error to source lib

Move the sendError into library in case the cycle dependency as regsitry and core are now the consumers.

Signed-off-by: wang yan <wangyan@vmware.com>
2020-06-19 01:04:50 +08:00
Wang Yan
b1793e795c
fix conformance test ErrorsCodes failure (#11961)
fixes #11945
When client calls PUT with an invalid digtest, Harbor has to give a 400 instead of 500.

Signed-off-by: wang yan <wangyan@vmware.com>
2020-05-18 17:20:23 +08:00
He Weiwei
5c5ba0b764
Merge pull request #11765 from heww/quota-webhook-enhancement
feat(quota,webhook): send quota webhook for put and mount blob
2020-04-28 09:35:57 +08:00
He Weiwei
b1c9d452ce feat(quota,webhook): send quota webhook for put and mount blob
Closes #11712

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-04-28 00:59:16 +00:00
Daniel Jiang
87f006d3a9
Merge pull request #11755 from reasonerjt/token-service-core-url
v2 auth middleware handles the ping request from internal
2020-04-27 15:04:10 +08:00
Daniel Jiang
fe587d0cc8 v2 auth middleware handles the ping request from internal
When scanner like trivy handles the auth flow to pull image, it pings
the /v2 and access the token service url in response body, by default it
will be external endpoint of Harbor.
There will be problem when Harbor is deployed on a single node with hairpinning not
supported.

This commit makes sure the address of token service in the challenge is
internal url of core component when the request is from internal.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-04-27 00:54:49 +08:00
wang yan
c4c279089a fix return code on getting non exist manifest
It's found by conformance test, it should be 404 instead of 500 when to get a non exist manifest

Signed-off-by: wang yan <wangyan@vmware.com>
2020-04-26 23:31:11 +08:00
Wenkai Yin
7553845b4d Remove the duplicated const definition
Remove the duplicated const definition for artifact type

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-04-13 17:02:23 +08:00
He Weiwei
4623cec1e5 feat(scan): revert bearer token support for scanner
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-04-11 08:45:29 +00:00
He Weiwei
f4821e96b3
Merge pull request #11527 from heww/refresh-quota-ignore-limitation
feat(quota): ignore limitation support for quota RefreshMiddleware
2020-04-10 15:29:41 +08:00
Daniel Jiang
56b404bfb7
Get digest in content trust middleware (#11554)
This commit removes the EnsureArtifactDigest as its implementation is
problematic: the artifactinfo in context is immutable.
When the content trust middleware needs the digest it will retrieve it
via artifact controller.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-04-10 14:11:56 +08:00
He Weiwei
0b26b36737 feat(quota): ignore limitation support for quota RefreshMiddleware
1. Ignore limitation when refresh quota for project.
2. Return 403 when quota errors occurred.
3. Add test for Refresh method of quota controller.

Closes #11512

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-04-08 19:17:41 +00:00
He Weiwei
20115b92c7 fix(vulnerable): fix the wrong count of vulnerabilities in message
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-04-08 09:06:31 +00:00
Wenkai Yin
7188e01569 Update APIs to only accept encoded repository name that contains slash
Update APIs to only accept encoded repository name that contains slash

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-04-07 20:57:50 +08:00
Wenkai Yin
0a372a85eb Remove "GetMyProjects" and "GetProjectRoles" in the interface "security.Context"
Fixes #11125, remove "GetMyProjects" and "GetProjectRoles" in the interface "security.Context"

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-04-07 19:53:38 +08:00
wang yan
a96d2f3746 remove the chart handling in quota
1, remove the chartmuseum controller
2, doesn't handle chartrepo url in v2 middleware

Signed-off-by: wang yan <wangyan@vmware.com>
2020-04-07 15:26:34 +08:00
Daniel Jiang
db10720e80
Merge pull request #11406 from reasonerjt/reenable-token-auth-for-cli-new
Reenable token auth for cli
2020-04-07 08:55:25 +08:00
wang yan
44825e819e deprecate quota count on artifact
Fixes #11241

1, remove count quota from quota manager
2, remove count in DB scheme
3, remove UI relates on quota
4, update UT, API test and UI UT.

Signed-off-by: wang yan <wangyan@vmware.com>
2020-04-06 16:56:11 +08:00
Daniel Jiang
e8f98259dd Make sure middleware handle scanner-pull claim for v2token
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-04-05 01:10:45 +08:00
He Weiwei
bd6c2f8870
fix(vulnerable,middleware): improve vulnerable middleware (#11407)
1. Prevent the pull action when scan report status is not successfuly.
2. Bypass the checking when no vulnerabilities not found.
3. Improve the returned message when prevented the pull action.

Closes #11202

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-04-04 00:54:57 +08:00
Daniel Jiang
08f9ffa000 Reenable token auth for cli
Docker CLI fails if it's not logged in upon seeing "basic" realm challenging while pinging the "/v2" endpoint. (#11266)
Some CLI will send HEAD to artifact endpoint before pushing (#11188)(#11271)

To fix such problems, this commit re-introduce the token auth flow to the CLIs.

For a HEAD request to "/v2/xxx" with no "Authoirzation" header, the v2_auth middleware populates the
"Www-Authenticate" header to redirect it to token endpoint with proper
requested scope.

It also adds security context to based on the content of the JWT which has the claims of the registry.
So a request from CLI carrying a token signed by the "/service/token" will have proper permissions.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-04-04 00:05:58 +08:00
wang yan
a11a70d941 move logger from common to lib
The logger is the fundamental library, so move it into lib folder
Signed-off-by: wang yan <wangyan@vmware.com>
2020-04-02 14:09:03 +08:00
DQ
9ff7d46e8f Rever chart version to original
Because chart version should consistent with previous version

Signed-off-by: DQ <dengq@vmware.com>
2020-04-01 11:55:22 +08:00
Wang Yan
34d05dae58
fix content trust middleware bypass scanner pull (#11321)
Fixes #11206
1, fix middleware doesn't work for docker pull without auth
2, fix middleware doesn't bypass scanner pull

Signed-off-by: wang yan <wangyan@vmware.com>
2020-03-31 11:12:21 +08:00
Daniel Jiang
fdb82ae4fa
Merge pull request #11349 from reasonerjt/fix-10602
Not checking for registry credentials in v2auth
2020-03-31 10:26:33 +08:00
Daniel Jiang
37f9d650bd Not checking for registry credentials in v2auth
That was added to support core process sending request to `/v2/xxx`.
It's no longer needed after reworking the flow.
This commit removes this.

Fixes #10602, as it's not a case we need to support for now.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-03-31 01:08:37 +08:00
He Weiwei
9c06c79ff4 refactor(errors): rename pkglib/error to lib/errors
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-03-29 05:19:05 +00:00
He Weiwei
033d6dac6b
fix(quota): allowed to put blob which size is zero (#11314)
Closes #11239

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-03-27 14:39:25 +08:00
He Weiwei
73f3a305ce
refactor: rename testing/api to testing/controller (#11295)
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-03-26 14:00:11 +08:00
Wenkai Yin(尹文开)
d05817c8a2
Update the URL checking logic of auth proxy security generator (#11180)
As we don't support bearer token in Harbor 2.0, the URL checking logic in auth proxy security generator should be updated

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-03-25 01:19:11 +08:00
Wenkai Yin(尹文开)
8984979bd2
Relocate/rename some packages (#11183)
Fixes #11016
1. src/pkg/q->src/internal/q
2. src/internal->src/lib (internal is a reserved package name of golang)
3. src/api->src/controller

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-03-24 20:45:45 +08:00
Wenkai Yin(尹文开)
1762bfae69
Merge pull request #11158 from ywk253100/200320_repository_api
Add "_self" suffix for repository API to avoid conflict
2020-03-23 14:45:52 +08:00
Wenkai Yin
0453709b74 Rewrite the filters with middleware mechinism
Fixes 10532,rewrite the filters with middleware mechinism

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-03-23 09:26:20 +08:00
Wenkai Yin
dca06b6ede Add "_self" suffix for repository API to avoid conflict
Add "_self" suffix for repository API to avoid conflict

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-03-20 17:08:08 +08:00
He Weiwei
21349e30af
feat(middleware,vulnerable): add image index checking for vulnerability prevention (#11084)
1. Skip vulnerability prevention checking when artifact is not
scannable.
2. Skip vulnerability prevention checking when artifact is image index
and its type is `IMAGE` or `CNAB`.
3. Skip vulnerability prevention checking when the artifact is pulling
by the scanner.
4. Change `hasCapability` from blacklist to whitelist.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-03-19 10:48:19 +08:00
He Weiwei
fe39bb6a2a
feat(quota,notification): notification for quota exceeded and warning (#11123)
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-03-18 20:24:23 +08:00
He Weiwei
f8983fe198
feat(log): track request id in the log message (#11095)
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-03-17 19:29:59 +08:00
Wang Yan
fbb3226e85
move notification handles and events metadata into api (#11085)
1, enable audit logs for notifications
2, move the handler and meatadata into API
3, use the notification middleware to send out notification

Signed-off-by: wang yan <wangyan@vmware.com>
2020-03-16 16:56:34 +08:00
Daniel Jiang
2615339f4c
Merge pull request #11076 from reasonerjt/csrf-secure-flag
make Secure flag of CSRF cookie adapt to config
2020-03-16 11:47:51 +08:00
Daniel Jiang
cbd2619035 make Secure flag of CSRF cookie adapt to config
fixes #11074

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-03-16 04:03:49 +08:00
Wang Yan
9cc6e88a65
add notification middleware (#11072)
the notification is for send out the event after DB transaction complete.
It's safe to send hook as this middleware is after transaction in the response path.

Signed-off-by: wang yan <wangyan@vmware.com>
2020-03-14 22:34:36 +08:00
He Weiwei
ec31a87884
fix(blob,quota): process blobs already in registry no but associated with project (#11071)
1. Before put manifest request, ensure that the requested size resource
include the blobs which are referenced by the manifest but not
associated with project.
2. After put manifest request, associate the blobs which are referenced
by the manifest but not associated with project.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-03-14 21:39:44 +08:00
He Weiwei
37e6fa5c92
fix(transaction): change to use value in the ctx to decide whether commit tx (#11062)
Type assertion not work when the ctx in the request changed in the next
handler, so change to use value in the ctx to decide whether to commit
tx.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-03-13 15:19:13 +08:00
Daniel Jiang
2e7eb8872e
Move ArtifactInfo to internal package (#11055)
To avoid depedency loop, this commit moves the model of ArtifactInfo to
internal pacakge, so that a controller can it from context when needed.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-03-13 11:16:13 +08:00
He Weiwei
2a243ef7a2
refactor(rbac): refactor rbac impl to improve performance (#9988)
1. Introduce `Evaluator` interface which do the permission checking.
2. `admin`, `lazy`, `rbac`, `namespace` and `evaluartor` set are implemented the
`Evaluator` interface.
3. Move project rbac implemention from `project` to `rbac` pkg to reduce
the name  conflict with project instance of model.
4. Do permission checking in security context by `Evaluator`.
5. Cache the regexp in rbac evaluator for casbin.
6. Cache evaluator in namespace evaluator to improve performance.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-03-12 23:42:53 +08:00
He Weiwei
12f16c8cec
feat(scan): support to scan image index (#11001)
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-03-12 19:30:12 +08:00
He Weiwei
89dfe24f19
feat(quota): add Request and Refresh middlewares for APIs (#10907)
1. Introduce ReqquestMiddleware and RefereshMiddleware.
2. Add request middlware to copy artifact, mount blob, put blob upload,
put manifest, upload chart verson APIs.
3. Add refresh project middleware to delete manifest, delete artifact,
delete chart version, delete repository APIs.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-03-11 22:25:40 +08:00