Commit Graph

52 Commits

Author SHA1 Message Date
Wenkai Yin
e45eaeec74 Fix transaction issue
More detail: // https://www.postgresql.org/message-id/002e01c04da9%24a8f95c20%2425efe6c1%40lasting.ro

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-03-04 08:01:11 +08:00
He Weiwei
69119b6410
feat(addition-link): only set vuls addition link when artifact scanable (#10892)
1. Add Checker to check the scannable status of the artifact.
2. Only set vulnerabilities addition link when the artifact scanable in the
project.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-03-03 16:41:54 +08:00
Ziming
70dcca6579
Merge pull request #10857 from bitsf/remove_pkg_art
feat(oci) remove dead code pkg/art
2020-03-03 11:50:15 +08:00
Wang Yan
54227f1ba2 update chart sdk to support helm v3
Signed-off-by: wang yan <wangyan@vmware.com>
2020-03-02 11:39:09 +08:00
Ziming Zhang
aee2c672e7 feat(oci) remove pkg/art dead code for OCI
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2020-03-02 10:34:34 +08:00
Wenkai Yin
4c9b59c904 Migrate artifact data in 2.0
Abstract extra attributes and annotations for artifacts stored in database

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-28 18:09:02 +08:00
He Weiwei
0f5a115a65
feat(artifact): add Walk method to artifact controller (#10881)
1. Add Walk method to artifact controller.
2. Only query references when artifact is image index.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-02-28 17:19:36 +08:00
wang yan
2d4fc0c4da move out the tags methods of artifact ctl
1, move the tag methods out of artifact ctl, let api to call tag ctr
2, update the ensure sequence for existing tag

Signed-off-by: wang yan <wangyan@vmware.com>
2020-02-28 15:49:39 +08:00
wang yan
79cf21f82f add tag controller
use the tag controller to handle CRUD of tags, especially the delete scenario, it could validate
the immutable and signature. And move the code of tag handling from artifact controller to tag controller

Signed-off-by: wang yan <wangyan@vmware.com>
2020-02-28 11:42:10 +08:00
Wenkai Yin
02c2647e1e Use the repository name of artifact model
As we store the repository name in the artifact table, we can use it direclty in the code to reduce the database query

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-26 13:37:09 +08:00
Wenkai Yin
528f598268 Reimplement the registry client
This commit reimplements the registry client under directory src/pkg/registry and removes the useless code

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-24 14:36:26 +08:00
Wenkai Yin
ab3aad4d50 Implement the resolver for CNAB
This commit introduces a new resolver to resolver metadata for CNAB

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-23 07:23:22 +08:00
He Weiwei
f36152a560
feat(vulnerability): assemble vulnerabilities info for artifact (#10800)
1. Assemble scan overview to artifact when scanner enabled in the
project of the artifact.
2. Set addition link for vulnerabilities to artifact when scanner
enabled in the project of the artifact.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-02-22 13:29:58 +08:00
He Weiwei
d1cef70cd1
refactor(scan,auth): remove bearer token auth support for scan job (#10781)
Harbor v2.0 has changed to use basic auth to pull image so remove bearer
token support for scan job

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-02-21 17:58:24 +08:00
He Weiwei
88fcacd4b7
feat(middleware): add blob middlewares (#10710)
1. Add middleware to record the accepted blob size for stream blob
upload.
2. Add middleware to create blob and associate it with project after blob upload
complete.
3. Add middleware to sync blobs, create blob for manifest and associate blobs
with the manifest after put manifest.
4. Add middleware to associate blob with project after mount blob.
5. Cleanup associations for the project when artifact deleted.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-02-20 23:20:34 +08:00
Ziming
0bc32410f3
Merge pull request #10742 from bitsf/oci_tag_retention
requirement(oci) implement tag retention for oci
not include ChartClient yet
2020-02-20 20:31:49 +08:00
Wenkai Yin
c4d4850845 Implement copy artifact API
Copy artifact into the repository from the specified artifact

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-20 16:59:35 +08:00
Wenkai Yin
1db0077096 Implement delete/update repository API
Implement delete/update repository API

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-20 08:39:26 +08:00
Ziming Zhang
94e23dc954 requirement(oci) implement tag retention for oci
Change-Id: Ib36660835d2666b35124e66254c33b5fc19aaf77
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2020-02-20 00:43:20 +08:00
Wang Yan
f160505686 add artifact trash manager
1, move the deleted artifact into trash
2, disable GC to delete the untagged manifest

Signed-off-by: wang yan <wangyan@vmware.com>
2020-02-19 14:52:58 +08:00
He Weiwei
4ad02de348
Scan reorganize (#10735)
* refactor(scan,scanner): move scan and scanner controllers to api pkg

Signed-off-by: He Weiwei <hweiwei@vmware.com>

* feat(scan-all-job): move artifacts query from job to notification

Move artifact query from scan all job to its notification handler to
ensure that the components in pkg will not call controllers in api.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-02-19 12:21:54 +08:00
Wenkai Yin
9d2f1d4d66 Refactor the logic of deleting artifact
Delete the child artifacts along with the parent when deleting an artifact

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-18 16:20:17 +08:00
Wenkai Yin
c8c944c6e8 Move the scan overview populating logic to API handler
Move the scan overview populating logic to API handler to avoid importing cycle

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-18 13:34:08 +08:00
Daniel Jiang
5a6e9331fd
Artifact signature populate (#7)
* Populate signature status in artifact API

This Commit add signature status into response of list artifact API.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-02-18 01:42:56 +08:00
Wang Yan
e5eb711827
Merge pull request #10720 from wy65701436/chart-details
Get addition properties for chart
2020-02-17 13:36:24 +08:00
wang yan
8029f70ae5 Get addition properties for chart
1, Get readme.md content into addition
2, Get dependency of chart
3, Get values of chart

Signed-off-by: wang yan <wangyan@vmware.com>
2020-02-17 12:19:10 +08:00
Wenkai Yin(尹文开)
d7903fcf1b
Merge pull request #10682 from ywk253100/200126_label
Support add/remove label to/from artifact
2020-02-17 11:33:45 +08:00
Wenkai Yin
eceb9b2345 Fix bugs when pushing image(with index) and CNAB
1. As "List" method of artifact DAO doesn't return the artifacts that referenced by other and without tag, so we introduce a new method "GetByDigest" to check the existence of artifact
2. The "Www-Authenticate" header is needed to be returned when the request is unauthorized. This is required in the OCI distribution spec and is needed when pushing CNAB

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-14 15:24:14 +08:00
Wenkai Yin
93731eeb2e Support add/remove label to/from artifact
This commit add supporting for adding/removing label to/from artifacts and populates labels when listing artifacts

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-13 10:46:23 +08:00
Wenkai Yin
0f6057a22c Implement get addition API for image
This commit implements the API to get build history of image with manifest version 2 and populates the addition links when listing/getting the artifact

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-07 20:21:38 +08:00
Wenkai Yin
c267aaa474 Implement repository deletion API
Implement repository deletion API based on the new design

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-07 09:36:35 +08:00
Wang Yan
72f9e02dd9
Merge pull request #10628 from wy65701436/af-immu
set the immutable status on getting/listting tag
2020-02-05 21:03:29 +08:00
wang yan
596a6261ca set the immutable status on getting/listting tag
Signed-off-by: wang yan <wangyan@vmware.com>
2020-02-05 16:24:13 +08:00
Wenkai Yin
7930e4f38a Add child artifact digest in reference model
As we only provide the API to get artifact information via project name, repository name and digest, the digest of child artifact must be returned

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-04 15:59:06 +08:00
Wang Yan
fa41168e8e
Merge pull request #10587 from ywk253100/200122_route
Register registry handler with the new methods of Route
2020-02-04 14:28:15 +08:00
Wenkai Yin
793b23a444 Implement artifact/tag related API
Implement APIs:
1. Get artifact
2. Delete artifact
3. Create tag
4. Delete tag

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-04 13:46:55 +08:00
Wenkai Yin
ef3af85a5b Register registry handler with the new methods of Route
Register registry handler with the new methods of Route

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-03 16:55:08 +08:00
Daniel Jiang
2064a1cd6d Switch to basic authentication for registry
1. Add basic authorizer for registry which modify the request
to add basic authorization header to request based on configuration.
2. Set basic auth header for proxy when accessing registry
3. Switche the registry to use basic auth by default and use the basic
authorizer to access Harbor.
4. Make necessary change to test cases, particularly
"test_robot_account.py" and "docker_api.py", because the error is
changed after siwtched to basic auth from token auth.  #10604 is opened
to track the follow up work.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-01-31 21:46:47 +09:00
Wenkai Yin
d0ac70d26c Implement the get/delete handler for registry API
Implement the get/delete handler for registry API

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-01-28 11:01:17 +08:00
Wenkai Yin(尹文开)
a774a19823
Merge pull request #10537 from ywk253100/200115_artifact_api
Implement the listing artifact API
2020-01-22 08:05:12 +08:00
Wenkai Yin
19f4bad042 Implement the listing artifact API
Implement the listing artifact API

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-01-22 07:35:29 +08:00
Wenkai Yin(尹文开)
17318aedc3
Merge pull request #10547 from ywk253100/200120_artifact
Add foreign key to avoid the concurrent issue
2020-01-21 17:44:27 +08:00
Wenkai Yin
7dc28bcff9 Add foreign key to avoid the concurrent issue
Add foreign key to avoid the concurrent issue when operating the artifacts, tags and references

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-01-21 16:03:51 +08:00
Wenkai Yin(尹文开)
8b3313a1ce
Merge pull request #10525 from ywk253100/200117_chart_resolver
Implement the resolver for helm chart
2020-01-21 15:12:57 +08:00
Wenkai Yin
f759c8fd64 Implement the resolver for helm chart
Implement the resolver for helm chart

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-01-21 14:38:53 +08:00
Wenkai Yin
1f3fcbde36 Try to parse the type of the artifact based on the media type when no resolver found
Try to parse the type of the artifact based on the media type when no resolver found, if parse failed, set it's type to unknown. This won't block the pushing for artifacts that has no resolver registered in Harbor

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-01-21 14:15:33 +08:00
Wang Yan
a27a5986ab
Merge pull request #10448 from ywk253100/200111_index
Implement the index/manifest list image resolver
2020-01-17 14:39:01 +08:00
Wenkai Yin
e91dbea76d Implement the manifest v1 image resolver
The manifest v1 image resolver populates the architecture info into the artifact

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-01-11 16:33:02 +08:00
Wenkai Yin
40d5043d5c Implement the index/manifest list image resolver
The index/manifest list image resolver populates the references of the artifact

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-01-11 16:02:16 +08:00
Wenkai Yin
df551e1310 Implement the artifact abstractor and resolver
1. Define the interface for artifact abstractor and resolver
2. Implement the artifact abstractor
3. Implement the resolver for image with manifest v2

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-01-11 15:09:08 +08:00