Commit Graph

898 Commits

Author SHA1 Message Date
Aaron Dewes
caee762b51
Upgrade to distribution (registry) v3 alpha (#19784)
* registryctl/api/registry/blob: fix dropped test error (#19721)

Signed-off-by: Lars Lehtonen <lars.lehtonen@gmail.com>

* Remove robot account update quota permission (#19819)

Signed-off-by: Yang Jiao <yang.jiao@broadcom.com>
Co-authored-by: Yang Jiao <yang.jiao@broadcom.com>

* Cache image list with digest key (#19801)

fixes #19429

Signed-off-by: stonezdj <daojunz@vmware.com>
Co-authored-by: stonezdj <daojunz@vmware.com>

* Add quota permissions testcase (#19822)

Signed-off-by: Yang Jiao <yang.jiao@broadcom.com>
Co-authored-by: Yang Jiao <yang.jiao@broadcom.com>

* deprecate gosec in makefile (#19828)

remove the unused the part from makefile

Signed-off-by: wang yan <wangyan@vmware.com>

* Add verification that robot account duration is not 0 (#19829)

Signed-off-by: Yang Jiao <yang.jiao@broadcom.com>

* fix artifact page bug (#19807)

* fix artifact page bug

* update testcase

* Upgrade to distribution (registry) v3 alpha

This includes all the benefits of the v3 distribution, but also all breaking changes.

Most notably, Image Manifest v2 Schema v1 support has been dropped, as well as the `oss` and `swift` storage drivers.

Currently, this still relies on v2's github.com/docker/distribution/registry/client/auth/challenge, because that code has been removed from the public API in v3.

Signed-off-by: Aaron Dewes <aaron.dewes@protonmail.com>

---------

Signed-off-by: Lars Lehtonen <lars.lehtonen@gmail.com>
Signed-off-by: Yang Jiao <yang.jiao@broadcom.com>
Signed-off-by: stonezdj <daojunz@vmware.com>
Signed-off-by: wang yan <wangyan@vmware.com>
Signed-off-by: Aaron Dewes <aaron.dewes@protonmail.com>
Co-authored-by: Lars Lehtonen <lars.lehtonen@gmail.com>
Co-authored-by: Yang Jiao <72076317+YangJiao0817@users.noreply.github.com>
Co-authored-by: Yang Jiao <yang.jiao@broadcom.com>
Co-authored-by: stonezdj(Daojun Zhang) <stonezdj@gmail.com>
Co-authored-by: stonezdj <daojunz@vmware.com>
Co-authored-by: Wang Yan <wangyan@vmware.com>
Co-authored-by: ShengqiWang <124650040+ShengqiWang@users.noreply.github.com>
2024-01-26 22:48:06 +01:00
MinerYang
b38de22054
bump golang 1.21.5 & fix golangci-lint error (#19722)
bump golang 1.21.5

update golangci-lint && fix revive error

fix white space lint

Signed-off-by: yminer <yminer@vmware.com>
2023-12-19 09:41:26 +08:00
Chlins Zhang
553c85eed0
fix: increase beego max memory and upload size (#19578)
1. Increase the default beego max memory and upload size from 32GB to
   128GB.
2. Support customize the two beego configs from env.

Signed-off-by: chlins <chenyuzh@vmware.com>
Co-authored-by: Wang Yan <wangyan@vmware.com>
2023-11-21 15:36:06 +08:00
stonezdj(Daojun Zhang)
3a9d68a32a
Allow POST method to request service/token in readonly mode (#19556)
fixes #18243

Signed-off-by: stonezdj <daojunz@vmware.com>
2023-11-10 05:08:31 +00:00
stonezdj(Daojun Zhang)
b337f51e7e
Replace comma in username to avoid casbin issue (#19505)
Check username when creating user by API
   Replace comma with underscore in username for OnboardUser
   Fixes #19356

Signed-off-by: stonezdj <daojunz@vmware.com>
2023-11-02 03:50:50 +00:00
Wang Yan
ed370a496b
deprecate resource label (#19349)
There is no api is using the DAO, remove it from the source code.

Signed-off-by: wang yan <wangyan@vmware.com>
2023-09-14 01:25:52 +08:00
Chlins Zhang
c7e25295fe
fix: support customize cache db for business (#19182)
Support to configure the customized redis db for cache layer and other
misc business for core, by default the behavior is same with
previous(stored in db 0).

Signed-off-by: chlins <chenyuzh@vmware.com>
2023-08-18 11:04:16 +08:00
Chlins Zhang
52e66155d4
log: change log level to reduce the noise logs (#19146)
1. Change some logs level to reduce the noise.
2. Wrap the go-redis.Nil error as ErrNotFound to avoid confusing

Signed-off-by: chlins <chenyuzh@vmware.com>
2023-08-17 11:11:04 +08:00
Wang Yan
ce893635aa
support nydus as a accessory (#18953)
Recognize nydus image(with subject) as a kind of accessory and built the releationship with subject manifest

Signed-off-by: wang yan <wangyan@vmware.com>
2023-07-20 15:54:23 +08:00
stonezdj(Daojun Zhang)
82ee5295ea
Log warning message when current user is freeze (#18937)
Signed-off-by: stonezdj <daojunz@vmware.com>
Co-authored-by: Wang Yan <wangyan@vmware.com>
2023-07-18 15:34:28 +08:00
Wang Yan
5cce621471
add notation support (#18909)
Accept and recognize the signature of notation client

Signed-off-by: wang yan <wangyan@vmware.com>
2023-07-13 11:18:34 +08:00
Wang Yan
bf7c82b9a8
remove the notary from backend (#18668)
Since harbor deprecates notary since v2.9.0, this pull request targets to remove the code related with notary.

Signed-off-by: Wang Yan <wangyan@vmware.com>
2023-05-18 18:47:42 +08:00
Eng Zer Jun
18a33c2b40
chore: replace github.com/ghodss/yaml with sigs.k8s.io/yaml (#18606)
At the time of making this commit, the package `github.com/ghodss/yaml`
is no longer actively maintained.

`sigs.k8s.io/yaml` is a permanent fork of `ghodss/yaml` and is actively
maintained by Kubernetes SIG.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Co-authored-by: Wang Yan <wangyan@vmware.com>
Co-authored-by: MinerYang <yminer@vmware.com>
2023-05-12 18:21:26 +08:00
MinerYang
ebac530b46
add goheader linter settings (#18503)
fix files for goheader linter

fix copyright 2018/2019

Signed-off-by: yminer <yminer@vmware.com>
2023-04-25 11:18:42 +08:00
Iceber Gu
3afa525f0b
Change the permissions of the *.go file from 0755 to 0644 (#17919)
Signed-off-by: Iceber Gu <wei.cai-nat@daocloud.io>
2023-04-13 09:07:36 +02:00
Chlins Zhang
62bb56daf6
refactor: refresh the execution status in async mode (#18434)
Refactor the logic for updating the status of execution when receiving
the hook from jobservice, avoid the optimistic lock due to the multiple
tasks update one execution by refreshing the status asynchronously. But
still retain the old way by specifying the flag from ENV.

Fixes: #17584

Signed-off-by: chlins <chenyuzh@vmware.com>
2023-04-04 16:37:37 +08:00
Chlins Zhang
a7cef5e24f
fix: wrap schedule/exec/task creation as orm tx (#18458)
Wrap orm tx when the scheduler try to create the task because submit job
maybe failure depends on the jobservice.

Fixes: #18452

Signed-off-by: chlins <chenyuzh@vmware.com>
2023-04-04 08:42:18 +08:00
Chlins Zhang
f21b1481bb
refactor: refactor the old goroutine execution sweep (#18361)
refactor: refactor the old goroutine execution sweep to global execution sweep job

1. Delete the old goroutine execution sweeper when create execution.(in the case of high concurrency can cause goroutine backlogs, affect the performance of core)
2. Introduce the new way to sweep executions, a global scheduled job will take the work.

Signed-off-by: chlins <chenyuzh@vmware.com>
2023-03-21 10:10:55 +08:00
Wang Yan
bb291aaa16
add middleware for artifact with subject (#18369)
As for the distribution spec 1.1, it supports client to push an manifest with subject field. By leverging this fidle, harbor could build up the linkage between the subject artifact and it's accessories.

Signed-off-by: wang yan <wangyan@vmware.com>
2023-03-19 20:09:41 +08:00
stonezdj(Daojun Zhang)
ab636fe3da
Remove go routine to reloadSetting (#18318)
Config is cached in redis
  Fixes #18189

Signed-off-by: stonezdj <daojunz@vmware.com>
2023-03-11 15:45:50 +08:00
Chlins Zhang
90db04e92d
refactor: refact the webhook API and life process (#18255)
refactor: refact the notification job API and life process

1. Introduce new APIs for webhook jobs management.
2. Refact legacy APIs for backforward compatible.
3. Migrate the webhook jobs process to unified execution/task framework.

Closes: #18210

Signed-off-by: chlins <chenyuzh@vmware.com>
2023-03-03 10:17:47 +08:00
Wang Yan
738fde7d3b
remove chartmuseum backend (#18191)
Harbor deprecates chartmuseum as of v2.8.0

Epic: https://github.com/goharbor/harbor/issues/17958

Discussion: https://github.com/goharbor/harbor/discussions/15057

Signed-off-by: Wang Yan <wangyan@vmware.com>
2023-02-16 18:11:05 +08:00
Wang Yan
277c78e212
update system artifact schedult timeout (#17902)
fixes #17887
enhance the timeout to 5m to avoid system artifact failed to schedule in the helm base installation.

Signed-off-by: Wang Yan <wangyan@vmware.com>

Signed-off-by: Wang Yan <wangyan@vmware.com>
2022-12-02 14:30:05 +08:00
Wang Yan
18a3373725
bump beego (#17801)
* bump beego

upgrade beego version from v1.10.12 to v2.0.5

1, beego v2 vserver/web refactor
2, beego v2 context refactor
3, beego v2 session refactor
4, beego v2 cache refactor
5, beego v2 orm refactor

Signed-off-by: MinerYang <yminer@vmware.com>
2022-11-24 18:07:42 +08:00
yanggang
0f4e2daf4c
Replaced io/ioutil with "os / io" package. (#17792)
Signed-off-by: yanggang <gang.yang@daocloud.io>

update typo

Signed-off-by: yminer <yminer@vmware.com>

Signed-off-by: yminer <yminer@vmware.com>
Co-authored-by: Wang Yan <wangyan@vmware.com>
2022-11-17 10:02:29 +08:00
Chlins Zhang
9c9aa58d6a
feat: support customize session timeout (#17767)
Add configuration session_timeout for API, then user can customize the
timeout from system config page or API. The timeout is 60 minutes by
default.

Signed-off-by: chlins <chenyuzh@vmware.com>

Signed-off-by: chlins <chenyuzh@vmware.com>
2022-11-15 11:30:01 +08:00
stonezdj(Daojun Zhang)
a47e175056
Remove email configuration (#17712)
Email related settings are deprecated
   Remove email ping API
   Fixes #17683

Signed-off-by: stonezdj <stonezdj@gmail.com>

Signed-off-by: stonezdj <stonezdj@gmail.com>
2022-11-14 15:03:49 +08:00
Chlins Zhang
1a1a6d2fd2
feat: log completion message when database migrated (#17684)
Signed-off-by: chlins <chenyuzh@vmware.com>
2022-10-31 10:45:05 +08:00
Maksym Trofimenko
486bdb6b99
Fix redirect url redirect_url when OIDC auth mode is enabled (#17628)
* fix redirect url for OIDC auth mode

Signed-off-by: Maksym Trofimenko <maksym@container-registry.com>

* portal lint_fix

Signed-off-by: mtrofimenko <gtpoxa@gmail.com>

* make linter happy

Signed-off-by: mtrofimenko <gtpoxa@gmail.com>

Signed-off-by: Maksym Trofimenko <maksym@container-registry.com>
Signed-off-by: mtrofimenko <gtpoxa@gmail.com>
Co-authored-by: Maksym Trofimenko <maksym@container-registry.com>
2022-10-21 14:38:13 +08:00
Wang Yan
ebab99613f
update session log level (#17300)
To avoid too much noise in the core log, udpate the level to debug.

Signed-off-by: Wang Yan <wangyan@vmware.com>
2022-08-03 13:51:26 +08:00
Adam
bf741ad381
feature: Use RegisteredClaims instead of deprecated staruct StandardClaims (#16206)
Signed-off-by: wujw39640 <wujw39640@hundsun.com>
2022-08-01 10:23:47 +08:00
MinerYang
f7ebf20d2e
Disable Nydus middleware for v2.6 (#17233)
disable nydus middleware for v2.6

Signed-off-by: yminer <yminer@vmware.com>

remove nyduus module import in main.go

Signed-off-by: yminer <yminer@vmware.com>
2022-07-27 08:41:33 +08:00
Loong Dai
09371b48e8
lint: sort imports (#17131)
* lint: add goimports

Signed-off-by: Loong Dai <loong.dai@intel.com>
2022-07-20 11:33:08 +08:00
MinerYang
efd9632e96
add nydus middleware (#17126)
Signed-off-by: yminer <yminer@vmware.com>

remove comments

Signed-off-by: yminer <yminer@vmware.com>

update ut manifest

Signed-off-by: yminer <yminer@vmware.com>

modify comment manifest

Signed-off-by: yminer <yminer@vmware.com>

updtae ut testcase

Signed-off-by: yminer <yminer@vmware.com>

fixwhitespace lint

Signed-off-by: yminer <yminer@vmware.com>

update isNydus judgement && define annotation var

Signed-off-by: yminer <yminer@vmware.com>

whitespace lint

Signed-off-by: yminer <yminer@vmware.com>
2022-07-08 09:56:10 +08:00
Shengwen YU
315d8a8993
fix: update code in compliance with golangci-lint revive (#17087)
Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
2022-07-01 14:52:47 +08:00
Wang Yan
94bf1c0ebd
Resovle the system artifact schedule issue (#17098)
Scheduling of system artifact depends on the jobservice, where gorountine is used to avoid the circular dependencies between core and jobservice.

Signed-off-by: Wang Yan <wangyan@vmware.com>
2022-07-01 14:25:45 +08:00
stonezdj(Daojun Zhang)
aa63569de2
Fix scan log mismatch issue (#17085)
Add checks in label

Signed-off-by: stonezdj <stonezdj@gmail.com>
2022-06-28 18:12:38 +08:00
Wang Yan
ea2ed27a7d
add url raw query check middleware (#17070)
The middleware can give a uniform url validation and raised error early.

Signed-off-by: Wang Yan <wangyan@vmware.com>
2022-06-24 17:57:46 +08:00
Shengwen YU
e9fca3de45
fix: refactor code for golangci-lint whitespace (#17005)
Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
2022-06-16 17:42:46 +08:00
Shengwen YU
c2a9f5de64
fix: update code for golangci-lint gosimple (#16974)
Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
2022-06-14 13:44:09 +08:00
stonezdj(Daojun Zhang)
e778ec2edf
Add audit_log forward endpoint (#16914)
add config item to set log forward endpoint
  fallback the audit log to default log when endpoint in error

Signed-off-by: stonezdj <stonezdj@gmail.com>
2022-06-10 10:59:40 +08:00
prahaladdarkin
4d062c33d1
Introduce system artifact manager cleanup job (#16879)
Signed-off-by: prahaladdarkin <prahaladd@vmware.com>
2022-06-09 14:16:50 +08:00
Shengwen YU
b43ba15f40
fix: golangci-lint errcheck (#16920)
Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
2022-06-07 17:00:36 +08:00
MinerYang
9d8e9158de
fix deadcode lint & update golangci-lint.yaml (#16896)
* fix deadcode lint & update golangci-lint.yaml

Signed-off-by: yminer <yminer@vmmware.com>

mock.go

Signed-off-by: yminer <yminer@vmmware.com>

commentfmt

Signed-off-by: yminer <yminer@vmmware.com>

mock.go update

Signed-off-by: yminer <yminer@vmmware.com>

update makefile

Signed-off-by: yminer <yminer@vmmware.com>

* update /pkg/allowlist/validator.go

Signed-off-by: yminer <yminer@vmmware.com>

Co-authored-by: yminer <yminer@vmmware.com>
2022-05-26 10:32:07 +08:00
chlins
0fd0c84446 feat(project): introduce cache manager for project (#16740)
Implement cached manager for resource project and migrate project.Mgr
to pkg.ProjectMgr.

Signed-off-by: chlins <chenyuzh@vmware.com>
2022-05-11 15:52:09 +08:00
Wang Yan
d3d4ad6a34
fix staticcheck issues (#16828)
Fix the staticcheck problems that reported by golangci-line staticcheck

Signed-off-by: Wang Yan <wangyan@vmware.com>
2022-05-10 16:07:10 +08:00
MinerYang
083b44e719
bump up astaxie/beego@v1.12.1 to beego/beego/@v1.12.7 (#16770)
Signed-off-by: yminer <yminer@vmmware.com>

goo mod tidy

Signed-off-by: yminer <yminer@vmmware.com>

Co-authored-by: yminer <yminer@vmmware.com>
2022-04-28 11:58:44 +08:00
stonezdj(Daojun Zhang)
1449988353
Delete unused files and functions (#16599)
testing/apitest/apilib is not used after harbor 2.4 refactor
 no testcase removed

Signed-off-by: stonezdj <stonezdj@gmail.com>
2022-03-29 14:19:36 +08:00
chlins
dbb8859ad8 feat: implement beego session provider
Signed-off-by: chlins <chenyuzh@vmware.com>
2022-03-24 10:33:21 +08:00
Wang Yan
7531168c92
update log with more inclusive language (#16569)
Use the deactive for instead in the error log

Signed-off-by: Wang Yan <wangyan@vmware.com>
2022-03-23 10:56:00 +08:00