Commit Graph

187 Commits

Author SHA1 Message Date
Wenkai Yin
8abb630b4c Implement query string builder
This commit defines the API query string format and provides the builders to build query string to query model

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-03-05 17:46:11 +08:00
stonezdj(Daojun Zhang)
49619e1907
Merge pull request #10939 from wy65701436/access-log-mgr
add audit logs API
2020-03-05 16:24:21 +08:00
Wang Yan
e79f4fd270
Merge pull request #10908 from wy65701436/middleware-blocker
add delete manifest middleware
2020-03-05 12:00:43 +08:00
wang yan
df237a5b17 add audit logs API
1, add API entry for get audit logs
2. add audit log manager to hanlder CRUD

Use the new format of audit log to cover differernt resource, artifact/tag/repostory/project

Signed-off-by: wang yan <wangyan@vmware.com>
2020-03-05 11:40:51 +08:00
wang yan
3bb574db35 use delete manifest to handle immutable and signature
1, Use signature manager to get signature
2, Check the immutable and signature status when deleting.
3, Remove the immutable middleware for delelte manifest

Signed-off-by: wang yan <wangyan@vmware.com>
2020-03-04 18:03:34 +08:00
He Weiwei
69119b6410
feat(addition-link): only set vuls addition link when artifact scanable (#10892)
1. Add Checker to check the scannable status of the artifact.
2. Only set vulnerabilities addition link when the artifact scanable in the
project.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-03-03 16:41:54 +08:00
Wang Yan
54227f1ba2 update chart sdk to support helm v3
Signed-off-by: wang yan <wangyan@vmware.com>
2020-03-02 11:39:09 +08:00
Wenkai Yin(尹文开)
bd0a8e9f8f
Merge pull request #10864 from ywk253100/200227_response
Set the "Link" and "location" header
2020-02-28 18:25:02 +08:00
Wenkai Yin
e3bbcb66d1 Set the "Link" and "location" header
Set the "Link" and "location" header

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-28 17:09:18 +08:00
Wenkai Yin(尹文开)
3d336bfac3
Merge pull request #10882 from wy65701436/tag-controller
add tag controller
2020-02-28 16:39:58 +08:00
wang yan
2d4fc0c4da move out the tags methods of artifact ctl
1, move the tag methods out of artifact ctl, let api to call tag ctr
2, update the ensure sequence for existing tag

Signed-off-by: wang yan <wangyan@vmware.com>
2020-02-28 15:49:39 +08:00
wang yan
79cf21f82f add tag controller
use the tag controller to handle CRUD of tags, especially the delete scenario, it could validate
the immutable and signature. And move the code of tag handling from artifact controller to tag controller

Signed-off-by: wang yan <wangyan@vmware.com>
2020-02-28 11:42:10 +08:00
Wenkai Yin
40890d2635 Add "Docker-Distribution-Api-Version" header for the 401 response of registry API
This is needed for "docker manifest" commands: https://github.com/docker/cli/issues/989

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-27 17:32:14 +08:00
Wenkai Yin
02c2647e1e Use the repository name of artifact model
As we store the repository name in the artifact table, we can use it direclty in the code to reduce the database query

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-26 13:37:09 +08:00
Wenkai Yin(尹文开)
b156c74f6a
Merge pull request #10831 from wy65701436/readonly-global
add readonly to beego middleware
2020-02-25 16:10:55 +08:00
wang yan
b336875ebf add readonly to beego middleware
Signed-off-by: wang yan <wangyan@vmware.com>
2020-02-25 15:00:39 +08:00
Wenkai Yin(尹文开)
5f9c976e95
Merge pull request #10837 from heww/scan-api
feat(scan): add scan API in v2.0
2020-02-25 14:45:19 +08:00
He Weiwei
55a21cd444 feat(scan): add scan API in v2.0
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-02-25 06:01:37 +00:00
Wenkai Yin
bb3ff0d752 Implement the API to get the specified repository
Implement the API to get the specified repository

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-25 12:02:50 +08:00
stonezdj(Daojun Zhang)
c2a77c2825
Merge pull request #10751 from ywk253100/200213_delete_useless
Remove dead code
2020-02-24 10:22:58 +08:00
Wenkai Yin(尹文开)
c446774d23
Merge pull request #10816 from reasonerjt/merge-art-mani-middleware
Merge artifactInfo and ManifestInfo
2020-02-24 08:10:32 +08:00
Daniel Jiang
46c72ae372 Merge artifactInfo and ManifestInfo
This commit gets rid of middleware info middleware, and make artifact
info the single source of truth in terms of the artifact a request
handles.  Fixes #10574

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-02-23 18:07:19 +08:00
Wenkai Yin
bd204464f3 Remove dead code
Remove dead code

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-23 17:11:46 +08:00
Wenkai Yin
ab3aad4d50 Implement the resolver for CNAB
This commit introduces a new resolver to resolver metadata for CNAB

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-23 07:23:22 +08:00
He Weiwei
f36152a560
feat(vulnerability): assemble vulnerabilities info for artifact (#10800)
1. Assemble scan overview to artifact when scanner enabled in the
project of the artifact.
2. Set addition link for vulnerabilities to artifact when scanner
enabled in the project of the artifact.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-02-22 13:29:58 +08:00
He Weiwei
88fcacd4b7
feat(middleware): add blob middlewares (#10710)
1. Add middleware to record the accepted blob size for stream blob
upload.
2. Add middleware to create blob and associate it with project after blob upload
complete.
3. Add middleware to sync blobs, create blob for manifest and associate blobs
with the manifest after put manifest.
4. Add middleware to associate blob with project after mount blob.
5. Cleanup associations for the project when artifact deleted.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-02-20 23:20:34 +08:00
Wenkai Yin
c4d4850845 Implement copy artifact API
Copy artifact into the repository from the specified artifact

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-20 16:59:35 +08:00
Wenkai Yin
1db0077096 Implement delete/update repository API
Implement delete/update repository API

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-20 08:39:26 +08:00
He Weiwei
4ad02de348
Scan reorganize (#10735)
* refactor(scan,scanner): move scan and scanner controllers to api pkg

Signed-off-by: He Weiwei <hweiwei@vmware.com>

* feat(scan-all-job): move artifacts query from job to notification

Move artifact query from scan all job to its notification handler to
ensure that the components in pkg will not call controllers in api.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-02-19 12:21:54 +08:00
Wenkai Yin
c8c944c6e8 Move the scan overview populating logic to API handler
Move the scan overview populating logic to API handler to avoid importing cycle

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-18 13:34:08 +08:00
Wenkai Yin(尹文开)
560dd8ce7b
Merge pull request #10745 from reasonerjt/artifact-signature-populate-v2
Artifact signature populate
2020-02-18 13:24:55 +08:00
Wenkai Yin(尹文开)
341cb88cba
Merge pull request #10727 from ywk253100/200214_bump_up_legacy_api_version
Bump up legacy api version to v2.0
2020-02-18 10:37:25 +08:00
Daniel Jiang
5a6e9331fd
Artifact signature populate (#7)
* Populate signature status in artifact API

This Commit add signature status into response of list artifact API.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-02-18 01:42:56 +08:00
Wenkai Yin(尹文开)
d7903fcf1b
Merge pull request #10682 from ywk253100/200126_label
Support add/remove label to/from artifact
2020-02-17 11:33:45 +08:00
Wenkai Yin
eceb9b2345 Fix bugs when pushing image(with index) and CNAB
1. As "List" method of artifact DAO doesn't return the artifacts that referenced by other and without tag, so we introduce a new method "GetByDigest" to check the existence of artifact
2. The "Www-Authenticate" header is needed to be returned when the request is unauthorized. This is required in the OCI distribution spec and is needed when pushing CNAB

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-14 15:24:14 +08:00
Wenkai Yin
94787ea60d Bump up the version of legacy APIs to v2.0
Bump up the version of legacy APIs to v2.0

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-14 13:16:30 +08:00
Wenkai Yin
93731eeb2e Support add/remove label to/from artifact
This commit add supporting for adding/removing label to/from artifacts and populates labels when listing artifacts

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-13 10:46:23 +08:00
He Weiwei
75bbf9d0bf
fix(middleware): escape and clean path for the skipper (#10674)
1. Escape and clean request path for `legacyAPISkipper`.
2. Escape and clean request path for `MethodAndPathSkipper`.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-02-11 22:29:55 +08:00
Daniel Jiang
9fb676c219 Fix nilpointer issue in v2 auth middleware
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-02-11 15:49:39 +08:00
Wang Yan
582fac5dae
Merge pull request #10687 from wy65701436/fix-catalog
fix catalog pagination issue
2020-02-11 10:40:35 +08:00
Wang Yan
e68b474dbc Merge pull request #10581 from ywk253100/100123_list_artifact
Add query string to listing artifact API to support specify only show the tagged artifacts
2020-02-10 18:12:06 +08:00
wang yan
beb7664b33 fix catalog pagenation issue
Signed-off-by: wang yan <wangyan@vmware.com>
2020-02-10 18:03:33 +08:00
Wang Yan
65dc54b059 Merge pull request #10626 from ywk253100/200125_handle_error
Unify the method/style to handle error in handler/middleware
2020-02-10 17:47:12 +08:00
Wenkai Yin
0373e08e58 Add query string to listing artifact API to support specify only show the tagged artifacts
Specify the query string "tags=NOT_NULL" to the listing artifact API to only return the tagged artifacts

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-10 14:53:51 +08:00
Wenkai Yin
af4dd142bc Unify the method/style to handle error in handler/middleware
This commit provides a "SendError" method to unify the way to handle error in handlers/middlewares

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-08 07:37:00 +08:00
Wenkai Yin
0f6057a22c Implement get addition API for image
This commit implements the API to get build history of image with manifest version 2 and populates the addition links when listing/getting the artifact

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-07 20:21:38 +08:00
Wenkai Yin(尹文开)
8a74fcb074
Merge pull request #10652 from wy65701436/fix-10552
Use controller rather than manager in the API handler and middleware
2020-02-07 19:30:51 +08:00
Wang Yan
6bad9f62ba
Merge pull request #10662 from ywk253100/200206_delete_repo
Implement repository deletion API
2020-02-07 17:51:03 +08:00
Wang Yan
5679c174c6
Merge pull request #10643 from ywk253100/200204_auth
Add permission check for artifact related APIs
2020-02-07 11:25:37 +08:00
wang yan
0fbbd674c2 Use controller rather than manager in the API handler and middleware
Signed-off-by: wang yan <wangyan@vmware.com>
2020-02-07 11:23:17 +08:00
Wenkai Yin
c267aaa474 Implement repository deletion API
Implement repository deletion API based on the new design

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-07 09:36:35 +08:00
wang yan
596a6261ca set the immutable status on getting/listting tag
Signed-off-by: wang yan <wangyan@vmware.com>
2020-02-05 16:24:13 +08:00
Wenkai Yin
6087647895 Add permission check for artifact related APIs
Add permission check for artifact related APIs

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-05 15:58:39 +08:00
Wenkai Yin
c1746cc675 Update the method called by registry handler
Use ListTags instead in the registry tag listing handler

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-04 15:16:28 +08:00
Wang Yan
fa41168e8e
Merge pull request #10587 from ywk253100/200122_route
Register registry handler with the new methods of Route
2020-02-04 14:28:15 +08:00
Wenkai Yin(尹文开)
26ade207d3
Merge pull request #10588 from ywk253100/200126_delete_get_artifact
Implement artifact/tag related API
2020-02-04 14:14:28 +08:00
Wenkai Yin
793b23a444 Implement artifact/tag related API
Implement APIs:
1. Get artifact
2. Delete artifact
3. Create tag
4. Delete tag

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-04 13:46:55 +08:00
He Weiwei
791439086d
feat(api,permission): add HasPermission, HasProjectPermission in BaseAPI (#10618)
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-02-04 08:42:00 +08:00
He Weiwei
b1437c1341
refactor(security): add NewContext and FromContext to security pkg (#10617)
1. Add `NewContext` and `FromContext` funcs in security pkg.
2. Add `Name` func in `security.Context` interface to make the checking
for the `/api/internal/configurations` API clear.
3. Get the security from the context to prepare change the security
filter to middleware.
4. Remove `GetSecurityContext` in filter pkg.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-02-03 17:43:36 +08:00
Wenkai Yin
ef3af85a5b Register registry handler with the new methods of Route
Register registry handler with the new methods of Route

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-03 16:55:08 +08:00
wang yan
661f7a7902 resolve conflict with latest code
Signed-off-by: wang yan <wangyan@vmware.com>
2020-02-03 10:34:09 +08:00
Daniel Jiang
2064a1cd6d Switch to basic authentication for registry
1. Add basic authorizer for registry which modify the request
to add basic authorization header to request based on configuration.
2. Set basic auth header for proxy when accessing registry
3. Switche the registry to use basic auth by default and use the basic
authorizer to access Harbor.
4. Make necessary change to test cases, particularly
"test_robot_account.py" and "docker_api.py", because the error is
changed after siwtched to basic auth from token auth.  #10604 is opened
to track the follow up work.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-01-31 21:46:47 +09:00
Wenkai Yin(尹文开)
a1b25e1fec
Switch to new registry API handlers (#10596)
Switch to new registry API handlers

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-01-30 21:21:07 +08:00
Wenkai Yin
d0ac70d26c Implement the get/delete handler for registry API
Implement the get/delete handler for registry API

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-01-28 11:01:17 +08:00
Wang Yan
9db84f2880 add Middleware policycheck into v2 handler
1, add content trust middleware in new v2 handler
2, add vulnable middlware in new v2 hanlder

Signed-off-by: wang yan <wangyan@vmware.com>
2020-01-28 00:24:58 +09:00
Daniel Jiang
5f8acc3896 Add middlewares for permission checking for v2 API
When the registry shifts from token auth to basic auth, we'll use the middleware to check permission.
This commit add middlewares for populate the artifact info and check
permission based on request to /v2/* api via security context

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-01-27 12:53:15 +08:00
He Weiwei
b594861658 feat(middleware): add transaction middleware for v2 and v2.0 APIs
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-01-22 05:00:39 +00:00
Wenkai Yin(尹文开)
f9d8951aed
Merge pull request #10519 from ywk253100/200116_router
Wrap the beego router and provide a unified view for users to register routes
2020-01-22 10:20:36 +08:00
Wenkai Yin(尹文开)
a774a19823
Merge pull request #10537 from ywk253100/200115_artifact_api
Implement the listing artifact API
2020-01-22 08:05:12 +08:00
Wenkai Yin
19f4bad042 Implement the listing artifact API
Implement the listing artifact API

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-01-22 07:35:29 +08:00
Wenkai Yin(尹文开)
17318aedc3
Merge pull request #10547 from ywk253100/200120_artifact
Add foreign key to avoid the concurrent issue
2020-01-21 17:44:27 +08:00
Wenkai Yin
8aeabc7717 Wrap the beego router and provide a unified view for users to register routes
Wrap the beego router and provide a unified view for users to register routes

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-01-21 17:26:20 +08:00
Wenkai Yin
7dc28bcff9 Add foreign key to avoid the concurrent issue
Add foreign key to avoid the concurrent issue when operating the artifacts, tags and references

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-01-21 16:03:51 +08:00
wang yan
8fae15b803 add immutable tag middleware into new v2 handler
Signed-off-by: wang yan <wangyan@vmware.com>
2020-01-20 22:41:08 +08:00
He Weiwei
33dfa1ea11
feat(beego): upgrade beego to v1.12 which support middleware (#10524)
1. Upgrade beego to v1.12.0
2. Add RequestID middleware to all HTTP requests.
3. Add Orm middleware to v2 and v2.0 APIs.
4. Remove OrmFilter from all HTTP requests.
5. Fix some test cases which cause panic in API controllers.
6. Enable XSRF for test cases of CommonController.
7. Imporve ReadOnly middleware.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-01-20 16:41:49 +08:00
Wenkai Yin(尹文开)
603cc0f5f3
Merge pull request #10526 from heww/send-error
feat(swagger): implement SendError for BaseAPI handler
2020-01-20 14:51:01 +08:00
wang yan
7ebb337e3f add regtoken middleware in new v2 handler
To move the regtoken middlware to new v2 handler framework, it parses the docker pull bearer token and check whether it's a scanner pull.

Signed-off-by: wang yan <wangyan@vmware.com>
2020-01-20 11:02:10 +08:00
Wang Yan
63ef743ba7
Merge pull request #10482 from wy65701436/api-list-repo
add code for catalog and list tag API
2020-01-19 15:26:13 +08:00
wang yan
986989af3c add code for catalog and list tag API
Signed-off-by: wang yan <wangyan@vmware.com>
2020-01-19 13:00:02 +08:00
wang yan
bffbc4009b add get manifest information middleware in new v2 hanlder
To add a new middleware to resolve the manifest information from request path, then pass
it into the request context for later use.

Signed-off-by: wang yan <wangyan@vmware.com>
2020-01-19 12:31:38 +08:00
He Weiwei
5daa3f4fb3 feat(swagger): implement SendError for BaseAPI handler
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-01-19 00:46:18 +00:00
wang yan
935303a476 move readonly middleware to new v2 handler
Signed-off-by: wang yan <wangyan@vmware.com>
2020-01-17 11:12:10 +08:00
Wenkai Yin(尹文开)
4f3024191b
Merge pull request #10458 from ywk253100/20013_middleware
Provide a util function to apply middlewares to handler
2020-01-14 17:11:59 +08:00
He Weiwei
c729e3b9e0 feat(swagger): generate apis v2.0 from swagger.yaml
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-01-14 02:20:18 +00:00
Wenkai Yin
c5cc7cf49e Provide a util function to apply middlewares to handler
Provide a util function to apply middlewares to handler

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-01-13 19:27:00 +08:00
Wenkai Yin
400a47a5c5 Implement tag/artifact manager and artifact controller
1. Implement tag/artifact manager
2. Implement artifact controller
3. Onboard the artifact when pushing artifacts

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-01-08 20:19:48 +08:00
Wenkai Yin
1a026a744f Add the registry handlers without the implementation detail
This commit adds several handlers to handle registry requests, some of the handlers aren't implemented yet, will be done later

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-12-24 09:12:20 +08:00