Introduce the quota update provider, improve the performance of pushing
artifacts to same project with high concurrency by implementing
optimistic lock in redis. By default the function is disabled, open it
by set env 'QUOTA_UPDATE_PROVIDER=Redis' for the core container.
Fixes: #18440
Signed-off-by: chlins <chenyuzh@vmware.com>
Since harbor deprecates notary since v2.9.0, this pull request targets to remove the code related with notary.
Signed-off-by: Wang Yan <wangyan@vmware.com>
At the time of making this commit, the package `github.com/ghodss/yaml`
is no longer actively maintained.
`sigs.k8s.io/yaml` is a permanent fork of `ghodss/yaml` and is actively
maintained by Kubernetes SIG.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Co-authored-by: Wang Yan <wangyan@vmware.com>
Co-authored-by: MinerYang <yminer@vmware.com>
bump helm.sh/helm/v3 to achieve the purpose of bump up github.com/docker/cli indirectly
Signed-off-by: yminer <yminer@vmware.com>
Signed-off-by: yminer <yminer@vmware.com>
updated-dependencies:
- dependency-name: github.com/prometheus/client_golang
- dependency-name: golang.org/x/net
- dependency-name: helm.sh/helm/v3
Signed-off-by: Wang Yan <wangyan@vmware.com>
Signed-off-by: Wang Yan <wangyan@vmware.com>
1. Use jpillora/backoff to get the backoff to avoid the panic in RetryUntil.
2. Return with last err when retry timeout.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
update the jwt model to github.com/golang-jwt/jwt
Starting from v3.2.1, the import path has changed from github.com/dgrijalva/jwt-go to github.com/golang-jwt/jwt.
Signed-off-by: Wang Yan <wangyan@vmware.com>
1. Limit API qps for the adapter.
2. Allow set qps via env.
3. Fix Tencnet SDK pagenation.
4. Fix resource filter.
Signed-off-by: fanjiankong <fanjiankong@tencent.com>
- update the resending logic in the hook agent
- use backoff lib to generate backoff duration
- remove retry cache queue
- leverage reaper to restore the sending if direct retries are all failed
- remove useless funcs/codes
fix#14545
Signed-off-by: Steven Zou <szou@vmware.com>
The version 1.8.0 of "github.com/prometheus/client_golang" bumps up lib "golang.org/x/sys" which referenced by "github.com/docker/docker@1.4.2". The changes of "golang.org/x/sys" causes the compile error in mac
Signed-off-by: Wenkai Yin <yinw@vmware.com>
1. Add configs in prepare
2. Add models and config items in Core
3. Encapdulate getting metric in commom package
4. Add a middleware for global request to collect 3 metrics
Signed-off-by: DQ <dengq@vmware.com>
Versions of the Go AWS SDK newer than 1.23.13 support OIDC in EKS.
Running Harbor on EKS doesn't require keys in a configmap for the
registry to authenticate to S3 when using the newer library.
Signed-off-by: Phil Fenstermacher <pcfens@wm.edu>
Add go build tags for gcs and oss, otherwise these drivers cannot be registered and the error "StorageDriver is not regsited: GCS" will raise on registryctl launch under the setting of GCS storage.
These build tags are designed in the distribution, just refer to https://github.com/docker/distribution/blob/release/2.7/registry/storage/driver/gcs/gcs.go#L13
Pin the google cloud API to a old version is because distribution depends on it, otherwise go mode will use v0.17.0 that go-migrate is using as the dependency version, but this version will break the compile process with following error:
harbor/pkg/mod/google.golang.org/cloud@v0.0.0-20151119220103-975617b05ea8/storage/acl.go:65:16: invalid type assertion: v.(map[string]<inter>) (non-interface type *storage.ObjectAccessControl on left)
that's bacause another dependency google.golang.org/cloud requires the pinned version of google.golang.org/api.
The pinned package should be removed once https://github.com/docker/distribution/pull/3019 is merged, and distribution ships their v2.8.0
Signed-off-by: wang yan <wangyan@vmware.com>
* Add api to delete blob and manifest
Enable the capability of registry controller to delete blob and manifest
Signed-off-by: wang yan <wangyan@vmware.com>
Fixes#9704
As we do want to unify error handling, so just decreprates pkg errors, use lib/errors instead for Harbor internal used errors model.
1, The lib/errors can cover all of funcs of pkg/errors, and also it has code attribute to define the http return value.
2, lib/errors can give a OCI standard error format, like {"errors":[{"code":"UNAUTHORIZED","message":"unauthorized"}]}
If you'd like to use pkg/errors, use lib/errors instead. If it cannot meet your request, enhance it.
Signed-off-by: wang yan <wangyan@vmware.com>
This commit replaces beego's CSRF mechanism with gorilla's csrf library.
The criteria for requests to skip the csrf check remain the same.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
1. As "List" method of artifact DAO doesn't return the artifacts that referenced by other and without tag, so we introduce a new method "GetByDigest" to check the existence of artifact
2. The "Www-Authenticate" header is needed to be returned when the request is unauthorized. This is required in the OCI distribution spec and is needed when pushing CNAB
Signed-off-by: Wenkai Yin <yinw@vmware.com>
1. Upgrade beego to v1.12.0
2. Add RequestID middleware to all HTTP requests.
3. Add Orm middleware to v2 and v2.0 APIs.
4. Remove OrmFilter from all HTTP requests.
5. Fix some test cases which cause panic in API controllers.
6. Enable XSRF for test cases of CommonController.
7. Imporve ReadOnly middleware.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
This commits does some basic init work for supporting OCI:
1. Create the artifact and tag model
2. Create database tables
Signed-off-by: Wenkai Yin <yinw@vmware.com>
- add DAO layer for scanner registration
- add CURD manager for scanner registration
- add API controller for plug scanner
- add REST APIs for CURD of plug scanner
- add migration sql:0011_1.10.0
- add scan interface definition (no implementations)
- add related UT cases with testify
fix#8979#8990
Signed-off-by: Steven Zou <szou@vmware.com>
This commit bumps up the version of Go to compile the code to v1.12.5,
and shifts to go.mod for managing depedency.
Some code from "harbor/tests" to "harbor/src/testing" to avoid depedency
loop of modules.
Note that in short term we will still vendor the dependency.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
