Commit Graph

67 Commits

Author SHA1 Message Date
DQ
ae80aa796e Upgrade Prometheus Client Version
upgrade to v1.8.0

Signed-off-by: DQ <dengq@vmware.com>
2020-11-03 14:33:11 +08:00
DQ
eb470501be Add metrics to Harbor Core
1. Add configs in prepare
 2. Add models and config items in Core
 3. Encapdulate getting metric in commom package
 4. Add a middleware for global request to collect 3 metrics

Signed-off-by: DQ <dengq@vmware.com>
2020-11-03 14:33:10 +08:00
Wang Yan
1af4c5e7cb
bump up go-digest version (#12992)
Signed-off-by: wang yan <wangyan@vmware.com>
2020-09-24 16:47:29 +08:00
Wenkai Yin
b1ddb5e2cc Implement the icon API to get the icon of artifact
Implement the icon API to get the icon of artifact

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-08-15 08:40:38 +08:00
DQ
b62bebc005 Update vendor file
Signed-off-by: DQ <dengq@vmware.com>
2020-07-21 10:16:22 +08:00
Ziming Zhang
8857e89e40 feature(redis) support redis sentinel
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2020-07-19 21:19:03 +08:00
Wang Yan
bad8f026fc
upgrade golang to v1.14.5 (#12489)
Signed-off-by: wang yan <wangyan@vmware.com>
2020-07-16 16:20:54 +08:00
wang yan
67be511a85 add middlware for put manifest
The middleware is to handle manifest(blob) status in different push manifest situation, similar with blob

Signed-off-by: wang yan <wangyan@vmware.com>
2020-07-02 14:57:04 +08:00
Phil Fenstermacher
33069e0f98
Upgrade aws-sdk-go
Versions of the Go AWS SDK newer than 1.23.13 support OIDC in EKS.
Running Harbor on EKS doesn't require keys in a configmap for the
registry to authenticate to S3 when using the newer library.

Signed-off-by: Phil Fenstermacher <pcfens@wm.edu>
2020-06-23 09:04:12 -04:00
Wang Yan
c993103e01
enable storage driver of gcs and oss (#12180)
Add go build tags for gcs and oss, otherwise these drivers cannot be registered and the error "StorageDriver is not regsited: GCS" will raise on registryctl launch under the setting of GCS storage.

These build tags are designed in the distribution, just refer to https://github.com/docker/distribution/blob/release/2.7/registry/storage/driver/gcs/gcs.go#L13

Pin the google cloud API to a old version is because distribution depends on it, otherwise go mode will use v0.17.0 that go-migrate is using as the dependency version, but this version will break the compile process with following error:
harbor/pkg/mod/google.golang.org/cloud@v0.0.0-20151119220103-975617b05ea8/storage/acl.go:65:16: invalid type assertion: v.(map[string]<inter>) (non-interface type *storage.ObjectAccessControl on left)
that's bacause another dependency google.golang.org/cloud requires the pinned version of google.golang.org/api.

The pinned package should be removed once https://github.com/docker/distribution/pull/3019 is merged, and distribution ships their v2.8.0

Signed-off-by: wang yan <wangyan@vmware.com>
2020-06-09 00:57:27 +08:00
Wang Yan
dec8397c21
Add api to delete blob and manifest (#12006)
* Add api to delete blob and manifest

Enable the capability of registry controller to delete blob and manifest

Signed-off-by: wang yan <wangyan@vmware.com>
2020-06-06 01:34:23 +08:00
Daniel Jiang
bc7ede21c7 Bump up golang-migrate
This commit bumps it up to 4.11.0
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-06-05 10:29:02 +08:00
Daniel Jiang
ae5ffce83a Update CSRF mechanism
This commit replaces beego's CSRF mechanism with gorilla's csrf library.
The criteria for requests to skip the csrf check remain the same.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-03-09 01:15:54 +08:00
Wang Yan
54227f1ba2 update chart sdk to support helm v3
Signed-off-by: wang yan <wangyan@vmware.com>
2020-03-02 11:39:09 +08:00
Wenkai Yin
ab3aad4d50 Implement the resolver for CNAB
This commit introduces a new resolver to resolver metadata for CNAB

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-23 07:23:22 +08:00
Ziming Zhang
94e23dc954 requirement(oci) implement tag retention for oci
Change-Id: Ib36660835d2666b35124e66254c33b5fc19aaf77
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2020-02-20 00:43:20 +08:00
wang yan
8029f70ae5 Get addition properties for chart
1, Get readme.md content into addition
2, Get dependency of chart
3, Get values of chart

Signed-off-by: wang yan <wangyan@vmware.com>
2020-02-17 12:19:10 +08:00
wy65701436
b2e3761f62 update beego version to v1.12.1
Signed-off-by: wang yan <wangyan@vmware.com>
2020-02-09 16:39:48 +08:00
He Weiwei
33dfa1ea11
feat(beego): upgrade beego to v1.12 which support middleware (#10524)
1. Upgrade beego to v1.12.0
2. Add RequestID middleware to all HTTP requests.
3. Add Orm middleware to v2 and v2.0 APIs.
4. Remove OrmFilter from all HTTP requests.
5. Fix some test cases which cause panic in API controllers.
6. Enable XSRF for test cases of CommonController.
7. Imporve ReadOnly middleware.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-01-20 16:41:49 +08:00
He Weiwei
c729e3b9e0 feat(swagger): generate apis v2.0 from swagger.yaml
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-01-14 02:20:18 +00:00
Steven Zou
e324a4d623 support pluggable scanner
- add DAO layer for scanner registration
- add CURD manager for scanner registration
- add API controller for plug scanner
- add REST APIs for CURD of plug scanner
- add migration sql:0011_1.10.0
- add scan interface definition (no implementations)
- add related UT cases with testify

fix #8979 #8990

Signed-off-by: Steven Zou <szou@vmware.com>
2019-09-18 21:56:45 +08:00
慕薇疯魔
0d71eb363a Aliyun ACR adapter
1. aliyun ACR registry adapter.
2. Unit test case.
3. Update go modules and vendor.

Signed-off-by: 疯魔慕薇 <kfanjian@gmail.com>
2019-07-29 18:24:03 +08:00
wang yan
4763864dae merge with latest master code with quota feature branch
Signed-off-by: wang yan <wangyan@vmware.com>
2019-07-24 08:47:05 -07:00
Steven Zou
09c56bfa08 print rich logs in the retention job
Signed-off-by: Steven Zou <szou@vmware.com>
2019-07-24 09:02:51 +08:00
He Weiwei
ce58c58c01 feat(quota,api): quota support for create project API
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-07-24 01:02:51 +08:00
wang yan
57821b1b4c Refactor interceptors code with chain
1, add a blob inteceptors for quota usage
2, add a manifest inteceptors for quota usage

Signed-off-by: wang yan <wangyan@vmware.com>
2019-07-09 11:08:11 +08:00
Ziming Zhang
072bdd101b aws driver for replication
Change-Id: I8792ffce2eaa5975359bb6159a1ba7b85926a925
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2019-06-25 19:11:27 +08:00
Daniel Jiang
737eaa396c Bump up to go 1.12.5 enable go.mod
This commit bumps up the version of Go to compile the code to v1.12.5,
and shifts to go.mod for managing depedency.
Some code from "harbor/tests" to "harbor/src/testing" to avoid depedency
loop of modules.

Note that in short term we will still vendor the dependency.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-06-11 00:42:20 +08:00
Wenkai Yin
2a463016a9 Upgrade the distribution and notary library (#7516)
* Return 404 when the log of task doesn't exist

Return 404 when the log of task doesn't exist

Signed-off-by: Wenkai Yin <yinw@vmware.com>

* Upgrade the distribution and notary library

Upgrade the distribution library to 2.7.1, the notary library to 0.6.1

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-04-28 12:00:26 +08:00
Steven Zou
fae5d1304d add more/refacotr current test cases of job service
Signed-off-by: Steven Zou <szou@vmware.com>
2019-04-22 19:24:37 +08:00
Steven Zou
c6a509b3dd exec dep ensure
Signed-off-by: Steven Zou <szou@vmware.com>
2019-04-19 21:22:37 +08:00
Steven Zou
8e734407c0 Merge branch 'master' into fix_issues_for_jobservice 2019-04-19 21:15:21 +08:00
Steven Zou
f8feaa192e add get scheduled and periodic executions APIs
Signed-off-by: Steven Zou <szou@vmware.com>
2019-04-19 13:54:23 +08:00
wang yan
d410a964a5 Pin pkg error version to v0.8.1
Signed-off-by: wang yan <wangyan@vmware.com>
2019-04-18 17:49:04 +08:00
root
b60b16230e Use the new package to do the match
Use the new package to do the match

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-04-16 16:07:30 +08:00
Daniel Jiang
9ce98f4acd Add controller to handle oidc login
The controller will redirect user to the OIDC login page based on
configuration.
Additionally this commit add some basic code to wrap `oauth2` package
and `provider` in `go-oidc`, and fixed an issue in UT to make
InMemoryDriver for config management thread-safe.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-03-28 11:29:05 +08:00
Yan
03709e4ec1
add authn proxy (#7199)
* add authn proxy docker login support

User could use the web hook token issued by k8s api server to login to harbor.
The username should add a specific prefix.

Signed-off-by: wang yan <wangyan@vmware.com>

* update code per review comments

Signed-off-by: wang yan <wangyan@vmware.com>

* Add UT for auth proxy modifier

Signed-off-by: wang yan <wangyan@vmware.com>
2019-03-27 12:37:54 +08:00
wang yan
25d1e69802 Remove unused files from vendor by dep v0.5.1
As for dep upgrade to v0.5.1, it removes all of unused files for vendor.

Signed-off-by: wang yan <wangyan@vmware.com>
2019-03-20 18:27:26 +08:00
He Weiwei
79f786ecbe Add ram pkg
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-01-04 13:17:13 +08:00
Wenkai Yin
2a40068d75 Fix cross slot issue of redis
This commit upgrdes the github.com/gocraft/work package to 0.5.1 and updates the namespace value
to fix the redis cross slot issue mentioned in #4633
2018-07-27 14:23:39 +08:00
Daniel Jiang
b9762afbdd Clean up vendor folder
Add `prune` section to Gopkg.toml to remove the unused packages from
vendor folder.
Update the reference of `semver` and `distribution` repo.
2018-07-23 14:40:25 +08:00
Steven Zou
7d782c41fc Implement the functions of getting chart version details and chart lists
getting chart list returns chart list instead of map and provides total versions info
getting chart details will return dependencies and values besides the metatdata of the chart version
2018-07-06 20:53:13 +08:00
Steven Zou
b65cc34c70 Implement a unified index.yaml entrypoint service to support namespace isolations
replace the helm package from github.com to k8s.io
implement /index.yaml service API
2018-07-05 18:07:00 +08:00
Steven Zou
54a0d10994 Define the related handlers to handler the requests of chart server/repo
add chartserver directory to put chart server code
add controller to coordinate the flow
add base/repo/manipulation handlers to handle requests
add operator to parse more details of chart version
call dep ensure
2018-07-02 19:34:04 +08:00
Daniel Jiang
c9b1962b1e Initialise Harbor DB schema in Harbor UI/adminserver container
This commit fixes #5040, the harbor-db image will only contain empty
databases, and harbor ui container will use migrate tool to run initial
SQL scripts to do initialization.  This is helpful for the case to
configure Harbor against external DB or DBaaS like RDS for HA deployment
However, this change will results some confusion as there are two tables
to track schema versions have been using alembic for migration, for this
release we'll try to use alembic to mock a `migration` table during
upgrade so the migrator will be bypassed, in future we'll consider to
consolidate to the golang based migrator.
Another issue is that the UI and adminserver containers will access DB
after start up in different congurations, can't ensure the sequence, so
both of them will try to update the schema when started up.
2018-06-28 16:22:53 +08:00
Wenkai Yin
46ec8c70fa Update packages github.com/Sirupsen/logrus, github.com/gorilla/context and github.com/garyburd/redigo to new version 2018-04-03 11:48:06 +08:00
Tan Jiang
381ecc3521 Merge with master 2018-03-26 10:37:17 +08:00
Wenkai Yin
0b7d4af4c1 Upgrade packages used by Harbor 2018-03-22 13:10:49 +08:00
Steven Zou
7a3c35d178 Implement the base framework of job service V2
add API server
add redis worker pool
add configuration loader
add bootstrap process
add vendor via dep
2018-03-08 13:16:57 +08:00
Wenkai Yin
e7b431899e Upgrade beego from 1.6.1 to 1.9.0 2018-02-05 13:07:52 +08:00