Commit Graph

834 Commits

Author SHA1 Message Date
DQ
115185894f Merge internal Transport and Secure Transport
Signed-off-by: DQ <dengq@vmware.com>
2020-03-18 19:22:09 +08:00
DQ
da359f609f Feat: enable mtls in core
add mtls related code in core

Signed-off-by: DQ <dengq@vmware.com>
2020-03-18 19:22:09 +08:00
Wang Yan
b4e941e961
drop table access log in migration (#11118)
Use the audit log instead, the access log table should be dropped after migration

Signed-off-by: wang yan <wangyan@vmware.com>
2020-03-18 19:04:38 +08:00
He Weiwei
7d20154db5
fix: remove old artifact model (#11112)
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-03-18 14:20:06 +08:00
He Weiwei
f8983fe198
feat(log): track request id in the log message (#11095)
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-03-17 19:29:59 +08:00
He Weiwei
60f8595034
refactor(quota): implement internal quota APIs by quota controller (#11058)
1. Use quota controller to implement the internal quota APIs.
2. The internal quota APIs can exceed the quota limitations.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-03-16 10:20:17 +08:00
Wenkai Yin
c6940e8184 United error response format for management APIs (legacy and v2.0 APIs)
United error response format for management APIs (legacy and v2.0 APIs)

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-03-13 22:00:08 +08:00
He Weiwei
2a243ef7a2
refactor(rbac): refactor rbac impl to improve performance (#9988)
1. Introduce `Evaluator` interface which do the permission checking.
2. `admin`, `lazy`, `rbac`, `namespace` and `evaluartor` set are implemented the
`Evaluator` interface.
3. Move project rbac implemention from `project` to `rbac` pkg to reduce
the name  conflict with project instance of model.
4. Do permission checking in security context by `Evaluator`.
5. Cache the regexp in rbac evaluator for casbin.
6. Cache evaluator in namespace evaluator to improve performance.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-03-12 23:42:53 +08:00
Wenkai Yin
4ccc3da99b Remove the "x-go-type" for artifact definition in swagger
Using "x-go-type" may cause the inconsistence between the swagger definition and the real data model

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-03-12 10:06:22 +08:00
He Weiwei
89dfe24f19
feat(quota): add Request and Refresh middlewares for APIs (#10907)
1. Introduce ReqquestMiddleware and RefereshMiddleware.
2. Add request middlware to copy artifact, mount blob, put blob upload,
put manifest, upload chart verson APIs.
3. Add refresh project middleware to delete manifest, delete artifact,
delete chart version, delete repository APIs.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-03-11 22:25:40 +08:00
Daniel Jiang
0f0e27179b Remove dependency on travis-ci
Github actions work fine, we no longer needs travi-ci to trigger the
tests.
This commit removes it.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-03-09 01:30:38 +08:00
Wenkai Yin
4c9b59c904 Migrate artifact data in 2.0
Abstract extra attributes and annotations for artifacts stored in database

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-28 18:09:02 +08:00
DQ
ff0c8b382c Refactor the version to variable
Signed-off-by: DQ <dengq@vmware.com>
2020-02-26 16:24:49 +08:00
Wang Yan
948d45604c Revise the GC job flow,
1, set harbor to readonly
2, select the candidate artifacts from Harbor DB.
3, call registry API(--delete-untagged=false) to delete manifest bases on the results of #2
4, clean keys of redis DB of registry, clean artifact trash and untagged from DB.
5, roll back readonly.

Signed-off-by: wang yan <wangyan@vmware.com>
2020-02-24 18:29:55 +08:00
Wenkai Yin
528f598268 Reimplement the registry client
This commit reimplements the registry client under directory src/pkg/registry and removes the useless code

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-24 14:36:26 +08:00
Wenkai Yin
bd204464f3 Remove dead code
Remove dead code

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-23 17:11:46 +08:00
Wenkai Yin
9312b788dc Upgrade the artifact table
Split the table artifact into artifact and tags, and populate related data

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-21 20:37:31 +08:00
He Weiwei
88fcacd4b7
feat(middleware): add blob middlewares (#10710)
1. Add middleware to record the accepted blob size for stream blob
upload.
2. Add middleware to create blob and associate it with project after blob upload
complete.
3. Add middleware to sync blobs, create blob for manifest and associate blobs
with the manifest after put manifest.
4. Add middleware to associate blob with project after mount blob.
5. Cleanup associations for the project when artifact deleted.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-02-20 23:20:34 +08:00
Wenkai Yin
c4d4850845 Implement copy artifact API
Copy artifact into the repository from the specified artifact

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-20 16:59:35 +08:00
Wenkai Yin
1db0077096 Implement delete/update repository API
Implement delete/update repository API

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-20 08:39:26 +08:00
Steven Zou
f1374737f6
Merge pull request #10694 from danielpacak/feature/install_with_trivy
chore(install): Add --with-trivy arg to the installation script
2020-02-19 16:27:57 +08:00
Daniel Jiang
5a6e9331fd
Artifact signature populate (#7)
* Populate signature status in artifact API

This Commit add signature status into response of list artifact API.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-02-18 01:42:56 +08:00
Wenkai Yin
93731eeb2e Support add/remove label to/from artifact
This commit add supporting for adding/removing label to/from artifacts and populates labels when listing artifacts

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-13 10:46:23 +08:00
Daniel Pacak
a642667ffc chore(install): Add --with-trivy arg to the installation script
Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
2020-02-12 23:47:56 +01:00
Wenkai Yin(尹文开)
d66c1a4a21
Merge pull request #10612 from ywk253100/200202_replication_basic_auth
Do enhancement for the registry authorizer
2020-02-11 22:09:40 +08:00
Wenkai Yin
a4ebbc6ecf Do enhancement for the registry authorizer
This commit introduces a new wrapper authorizer which can authorize the request according to the auth scheme automatically

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-11 21:21:49 +08:00
Wang Yan
65dc54b059 Merge pull request #10626 from ywk253100/200125_handle_error
Unify the method/style to handle error in handler/middleware
2020-02-10 17:47:12 +08:00
Wenkai Yin
af4dd142bc Unify the method/style to handle error in handler/middleware
This commit provides a "SendError" method to unify the way to handle error in handlers/middlewares

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-08 07:37:00 +08:00
Wenkai Yin
0f6057a22c Implement get addition API for image
This commit implements the API to get build history of image with manifest version 2 and populates the addition links when listing/getting the artifact

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-07 20:21:38 +08:00
Wenkai Yin
6087647895 Add permission check for artifact related APIs
Add permission check for artifact related APIs

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-05 15:58:39 +08:00
He Weiwei
b1437c1341
refactor(security): add NewContext and FromContext to security pkg (#10617)
1. Add `NewContext` and `FromContext` funcs in security pkg.
2. Add `Name` func in `security.Context` interface to make the checking
for the `/api/internal/configurations` API clear.
3. Get the security from the context to prepare change the security
filter to middleware.
4. Remove `GetSecurityContext` in filter pkg.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-02-03 17:43:36 +08:00
Daniel Jiang
2064a1cd6d Switch to basic authentication for registry
1. Add basic authorizer for registry which modify the request
to add basic authorization header to request based on configuration.
2. Set basic auth header for proxy when accessing registry
3. Switche the registry to use basic auth by default and use the basic
authorizer to access Harbor.
4. Make necessary change to test cases, particularly
"test_robot_account.py" and "docker_api.py", because the error is
changed after siwtched to basic auth from token auth.  #10604 is opened
to track the follow up work.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-01-31 21:46:47 +09:00
Wenkai Yin
400a47a5c5 Implement tag/artifact manager and artifact controller
1. Implement tag/artifact manager
2. Implement artifact controller
3. Onboard the artifact when pushing artifacts

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-01-08 20:19:48 +08:00
Wenkai Yin(尹文开)
56dc0bb71f
Merge pull request #10324 from wy65701436/common-error-13
add OCI error format support
2019-12-25 17:44:35 +08:00
wang yan
ebe5bb68b9 add OCI error format support
1, Leverage go v1.13 new error feature
2, Define genernal error OCI format, so that /v2 API could return a OCI compatible error

Signed-off-by: wang yan <wangyan@vmware.com>
2019-12-25 17:07:26 +08:00
stonezdj
6313a55219 Fix admin permission not revoked when removed from LDAP admin group
Seperate the HasAdminRole(In DB) with the privileges from external auth, and use user.HasAdminPrivilege to check

Signed-off-by: stonezdj <stonezdj@gmail.com>
2019-12-20 13:12:22 +08:00
Wang Yan
9405b11480
Merge pull request #10114 from julienvey/typo-registryctl
Fix typo in registryctl client log
2019-12-06 13:15:16 +08:00
Wang Yan
9016c427b9
Merge pull request #10136 from reasonerjt/rm-authproxy-case-sensitive
Get rid of case-sensitivity in authproxy setting
2019-12-05 14:26:18 +08:00
Daniel Jiang
d58f5e4bdc Get rid of case-sensitivity in authproxy setting
This commit removes the attribute to control case-sensitivity from
authproxy setting.
The result in token review status will be used as the single source of
truth, regardless the case of the letters in group names and user names.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-12-04 21:39:40 +08:00
Wenkai Yin(尹文开)
d145f4baf4
Merge pull request #10034 from ywk253100/191128_clean
Clean up admiral-related code
2019-12-04 17:33:31 +08:00
stonezdj(Daojun Zhang)
339c1d4cab
Merge pull request #10088 from reasonerjt/authproxy-cert-setting
Support pinning to authproxy server's cert
2019-12-04 14:03:27 +08:00
Wenkai Yin(尹文开)
a1712e5332
Merge pull request #10083 from MrMEEE/fix-listings-squashed
Squashed version of PR-9943
2019-12-04 09:35:20 +08:00
Julien Vey
a13f918fd0 Fix typo in registryctl client
Signed-off-by: Julien Vey <vey.julien@gmail.com>
2019-12-03 23:51:15 +01:00
Yogi_Wang
f022e89843 Modify the repository list sort and filter
Signed-off-by: Yogi_Wang <yawang@vmware.com>
2019-12-03 10:37:41 +08:00
Daniel Jiang
902598fabd Support pinning to authproxy server's cert
This commit add an attribute to configurations, whose value is the
certificate of authproxy server.  When this attribute is set Harbor will
pin to this cert when connecting authproxy.
This value will also be part of the response of systemInfo API.

This commit will be cherrypicked to 1.10 and 1.9 branch.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-12-03 07:31:26 +08:00
Martin Juhl
06594a1756 Squashed version of PR-9943
Signed-off-by: Martin Juhl <m@rtinjuhl.dk>
2019-12-02 11:59:33 +01:00
Wenkai Yin
dd2bc0ecef Clean up admiral-related code
Clean up admiral-related code as it's useless

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-11-28 17:28:54 +08:00
Daniel Jiang
4e1bac4b82
Merge pull request #9820 from reasonerjt/oidc-cli-secret-group
Populate user groups during OIDC CLI secret verification
2019-11-19 03:03:38 -08:00
Daniel Jiang
64af09d52b Populate user groups during OIDC CLI secret verification
This commit refactors the flow to populate user info and verify CLI
secret in OIDC authentication.

It will call the `userinfo` backend of OIDC backend and fallback to
using the ID token if userinfo is not supported by the backend.

It also makes sure the token will be persisted if it's refreshed during
this procedure.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-11-18 23:53:05 -08:00
He Weiwei
0c068d81f5
feat(vuln-severity): map negligible to none to match CVSS v3 ratings (#9885)
BREAKING CHANGE: the value negligible of severity in project metadata will change to none in the responses of project APIs

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-11-18 14:36:51 +08:00