Commit Graph

903 Commits

Author SHA1 Message Date
stonezdj(Daojun Zhang)
be648ea47f
Refactor scan job service make it easy to add new scan type (#20177)
Signed-off-by: stonezdj <daojunz@vmware.com>
Signed-off-by: stonezdj(Daojun Zhang) <stonezdj@gmail.com>
Co-authored-by: stonezdj <daojunz@vmware.com>
2024-04-09 16:05:30 +08:00
stonezdj(Daojun Zhang)
cea47c7db3
Add accessory type for sbom (#20208)
Signed-off-by: stonezdj <daojunz@vmware.com>
Co-authored-by: stonezdj <daojunz@vmware.com>
2024-04-02 18:11:27 +08:00
stonezdj(Daojun Zhang)
5b832c1724
Limit url to local path (#20025) 2024-02-24 01:34:30 +00:00
stonezdj(Daojun Zhang)
54819ba8cd
Limit url to local site (#20013)
Signed-off-by: stonezdj <daojunz@vmware.com>
Co-authored-by: stonezdj <daojunz@vmware.com>
2024-02-23 07:40:13 +00:00
Antoine Jouve
73c2884e58
[Token/JWT] Update to golang-jwt v5.2.0 (#19802)
* feat: update to golang-jwt v5.2.0

Signed-off-by: Antoine Jouve <ant.jouve@gmail.com>

* fix: module issues and robot claims

Signed-off-by: Antoine Jouve <ant.jouve@gmail.com>

* fix: add missing time import

Signed-off-by: Antoine Jouve <ant.jouve@gmail.com>

* feat: set jwt validation leeway to 60s

Signed-off-by: Antoine Jouve <ant.jouve@gmail.com>

* fix: update leeways that were still set to 10s

Signed-off-by: Antoine Jouve <ant.jouve@gmail.com>

* feat: update go.sum

Signed-off-by: Antoine Jouve <ant.jouve@gmail.com>

* feat: add two leeway related test cases

Signed-off-by: Antoine Jouve <ant.jouve@gmail.com>

* fix: correct jwt audience validation

Signed-off-by: Antoine Jouve <ant.jouve@gmail.com>

* fix: gofmt v2_token.go

Signed-off-by: Antoine Jouve <ant.jouve@gmail.com>

* feat: take into account review comments

Signed-off-by: Antoine Jouve <ant.jouve@gmail.com>

* feat: use a common constant to store JWT leeway

Signed-off-by: Antoine Jouve <ant.jouve@gmail.com>

---------

Signed-off-by: Antoine Jouve <ant.jouve@gmail.com>
Signed-off-by: Antoine Jouve <an-toine@users.noreply.github.com>
Co-authored-by: MinerYang <yminer@vmware.com>
2024-02-23 11:30:13 +08:00
Bin Liu
5a576174b1
skip transaction for POST /service/token (#19339)
Habor doesnot support POST /service/token endpoint,
put this request into a transaction will consume
extra database resources and generate many logs.

Signed-off-by: bin liu <liubin0329@gmail.com>
2024-01-19 07:11:32 +00:00
MinerYang
b38de22054
bump golang 1.21.5 & fix golangci-lint error (#19722)
bump golang 1.21.5

update golangci-lint && fix revive error

fix white space lint

Signed-off-by: yminer <yminer@vmware.com>
2023-12-19 09:41:26 +08:00
Chlins Zhang
553c85eed0
fix: increase beego max memory and upload size (#19578)
1. Increase the default beego max memory and upload size from 32GB to
   128GB.
2. Support customize the two beego configs from env.

Signed-off-by: chlins <chenyuzh@vmware.com>
Co-authored-by: Wang Yan <wangyan@vmware.com>
2023-11-21 15:36:06 +08:00
stonezdj(Daojun Zhang)
3a9d68a32a
Allow POST method to request service/token in readonly mode (#19556)
fixes #18243

Signed-off-by: stonezdj <daojunz@vmware.com>
2023-11-10 05:08:31 +00:00
stonezdj(Daojun Zhang)
b337f51e7e
Replace comma in username to avoid casbin issue (#19505)
Check username when creating user by API
   Replace comma with underscore in username for OnboardUser
   Fixes #19356

Signed-off-by: stonezdj <daojunz@vmware.com>
2023-11-02 03:50:50 +00:00
Wang Yan
ed370a496b
deprecate resource label (#19349)
There is no api is using the DAO, remove it from the source code.

Signed-off-by: wang yan <wangyan@vmware.com>
2023-09-14 01:25:52 +08:00
Chlins Zhang
c7e25295fe
fix: support customize cache db for business (#19182)
Support to configure the customized redis db for cache layer and other
misc business for core, by default the behavior is same with
previous(stored in db 0).

Signed-off-by: chlins <chenyuzh@vmware.com>
2023-08-18 11:04:16 +08:00
Chlins Zhang
52e66155d4
log: change log level to reduce the noise logs (#19146)
1. Change some logs level to reduce the noise.
2. Wrap the go-redis.Nil error as ErrNotFound to avoid confusing

Signed-off-by: chlins <chenyuzh@vmware.com>
2023-08-17 11:11:04 +08:00
Wang Yan
ce893635aa
support nydus as a accessory (#18953)
Recognize nydus image(with subject) as a kind of accessory and built the releationship with subject manifest

Signed-off-by: wang yan <wangyan@vmware.com>
2023-07-20 15:54:23 +08:00
stonezdj(Daojun Zhang)
82ee5295ea
Log warning message when current user is freeze (#18937)
Signed-off-by: stonezdj <daojunz@vmware.com>
Co-authored-by: Wang Yan <wangyan@vmware.com>
2023-07-18 15:34:28 +08:00
Wang Yan
5cce621471
add notation support (#18909)
Accept and recognize the signature of notation client

Signed-off-by: wang yan <wangyan@vmware.com>
2023-07-13 11:18:34 +08:00
Wang Yan
bf7c82b9a8
remove the notary from backend (#18668)
Since harbor deprecates notary since v2.9.0, this pull request targets to remove the code related with notary.

Signed-off-by: Wang Yan <wangyan@vmware.com>
2023-05-18 18:47:42 +08:00
Eng Zer Jun
18a33c2b40
chore: replace github.com/ghodss/yaml with sigs.k8s.io/yaml (#18606)
At the time of making this commit, the package `github.com/ghodss/yaml`
is no longer actively maintained.

`sigs.k8s.io/yaml` is a permanent fork of `ghodss/yaml` and is actively
maintained by Kubernetes SIG.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Co-authored-by: Wang Yan <wangyan@vmware.com>
Co-authored-by: MinerYang <yminer@vmware.com>
2023-05-12 18:21:26 +08:00
MinerYang
ebac530b46
add goheader linter settings (#18503)
fix files for goheader linter

fix copyright 2018/2019

Signed-off-by: yminer <yminer@vmware.com>
2023-04-25 11:18:42 +08:00
Iceber Gu
3afa525f0b
Change the permissions of the *.go file from 0755 to 0644 (#17919)
Signed-off-by: Iceber Gu <wei.cai-nat@daocloud.io>
2023-04-13 09:07:36 +02:00
Chlins Zhang
62bb56daf6
refactor: refresh the execution status in async mode (#18434)
Refactor the logic for updating the status of execution when receiving
the hook from jobservice, avoid the optimistic lock due to the multiple
tasks update one execution by refreshing the status asynchronously. But
still retain the old way by specifying the flag from ENV.

Fixes: #17584

Signed-off-by: chlins <chenyuzh@vmware.com>
2023-04-04 16:37:37 +08:00
Chlins Zhang
a7cef5e24f
fix: wrap schedule/exec/task creation as orm tx (#18458)
Wrap orm tx when the scheduler try to create the task because submit job
maybe failure depends on the jobservice.

Fixes: #18452

Signed-off-by: chlins <chenyuzh@vmware.com>
2023-04-04 08:42:18 +08:00
Chlins Zhang
f21b1481bb
refactor: refactor the old goroutine execution sweep (#18361)
refactor: refactor the old goroutine execution sweep to global execution sweep job

1. Delete the old goroutine execution sweeper when create execution.(in the case of high concurrency can cause goroutine backlogs, affect the performance of core)
2. Introduce the new way to sweep executions, a global scheduled job will take the work.

Signed-off-by: chlins <chenyuzh@vmware.com>
2023-03-21 10:10:55 +08:00
Wang Yan
bb291aaa16
add middleware for artifact with subject (#18369)
As for the distribution spec 1.1, it supports client to push an manifest with subject field. By leverging this fidle, harbor could build up the linkage between the subject artifact and it's accessories.

Signed-off-by: wang yan <wangyan@vmware.com>
2023-03-19 20:09:41 +08:00
stonezdj(Daojun Zhang)
ab636fe3da
Remove go routine to reloadSetting (#18318)
Config is cached in redis
  Fixes #18189

Signed-off-by: stonezdj <daojunz@vmware.com>
2023-03-11 15:45:50 +08:00
Chlins Zhang
90db04e92d
refactor: refact the webhook API and life process (#18255)
refactor: refact the notification job API and life process

1. Introduce new APIs for webhook jobs management.
2. Refact legacy APIs for backforward compatible.
3. Migrate the webhook jobs process to unified execution/task framework.

Closes: #18210

Signed-off-by: chlins <chenyuzh@vmware.com>
2023-03-03 10:17:47 +08:00
Wang Yan
738fde7d3b
remove chartmuseum backend (#18191)
Harbor deprecates chartmuseum as of v2.8.0

Epic: https://github.com/goharbor/harbor/issues/17958

Discussion: https://github.com/goharbor/harbor/discussions/15057

Signed-off-by: Wang Yan <wangyan@vmware.com>
2023-02-16 18:11:05 +08:00
Wang Yan
277c78e212
update system artifact schedult timeout (#17902)
fixes #17887
enhance the timeout to 5m to avoid system artifact failed to schedule in the helm base installation.

Signed-off-by: Wang Yan <wangyan@vmware.com>

Signed-off-by: Wang Yan <wangyan@vmware.com>
2022-12-02 14:30:05 +08:00
Wang Yan
18a3373725
bump beego (#17801)
* bump beego

upgrade beego version from v1.10.12 to v2.0.5

1, beego v2 vserver/web refactor
2, beego v2 context refactor
3, beego v2 session refactor
4, beego v2 cache refactor
5, beego v2 orm refactor

Signed-off-by: MinerYang <yminer@vmware.com>
2022-11-24 18:07:42 +08:00
yanggang
0f4e2daf4c
Replaced io/ioutil with "os / io" package. (#17792)
Signed-off-by: yanggang <gang.yang@daocloud.io>

update typo

Signed-off-by: yminer <yminer@vmware.com>

Signed-off-by: yminer <yminer@vmware.com>
Co-authored-by: Wang Yan <wangyan@vmware.com>
2022-11-17 10:02:29 +08:00
Chlins Zhang
9c9aa58d6a
feat: support customize session timeout (#17767)
Add configuration session_timeout for API, then user can customize the
timeout from system config page or API. The timeout is 60 minutes by
default.

Signed-off-by: chlins <chenyuzh@vmware.com>

Signed-off-by: chlins <chenyuzh@vmware.com>
2022-11-15 11:30:01 +08:00
stonezdj(Daojun Zhang)
a47e175056
Remove email configuration (#17712)
Email related settings are deprecated
   Remove email ping API
   Fixes #17683

Signed-off-by: stonezdj <stonezdj@gmail.com>

Signed-off-by: stonezdj <stonezdj@gmail.com>
2022-11-14 15:03:49 +08:00
Chlins Zhang
1a1a6d2fd2
feat: log completion message when database migrated (#17684)
Signed-off-by: chlins <chenyuzh@vmware.com>
2022-10-31 10:45:05 +08:00
Maksym Trofimenko
486bdb6b99
Fix redirect url redirect_url when OIDC auth mode is enabled (#17628)
* fix redirect url for OIDC auth mode

Signed-off-by: Maksym Trofimenko <maksym@container-registry.com>

* portal lint_fix

Signed-off-by: mtrofimenko <gtpoxa@gmail.com>

* make linter happy

Signed-off-by: mtrofimenko <gtpoxa@gmail.com>

Signed-off-by: Maksym Trofimenko <maksym@container-registry.com>
Signed-off-by: mtrofimenko <gtpoxa@gmail.com>
Co-authored-by: Maksym Trofimenko <maksym@container-registry.com>
2022-10-21 14:38:13 +08:00
Wang Yan
ebab99613f
update session log level (#17300)
To avoid too much noise in the core log, udpate the level to debug.

Signed-off-by: Wang Yan <wangyan@vmware.com>
2022-08-03 13:51:26 +08:00
Adam
bf741ad381
feature: Use RegisteredClaims instead of deprecated staruct StandardClaims (#16206)
Signed-off-by: wujw39640 <wujw39640@hundsun.com>
2022-08-01 10:23:47 +08:00
MinerYang
f7ebf20d2e
Disable Nydus middleware for v2.6 (#17233)
disable nydus middleware for v2.6

Signed-off-by: yminer <yminer@vmware.com>

remove nyduus module import in main.go

Signed-off-by: yminer <yminer@vmware.com>
2022-07-27 08:41:33 +08:00
Loong Dai
09371b48e8
lint: sort imports (#17131)
* lint: add goimports

Signed-off-by: Loong Dai <loong.dai@intel.com>
2022-07-20 11:33:08 +08:00
MinerYang
efd9632e96
add nydus middleware (#17126)
Signed-off-by: yminer <yminer@vmware.com>

remove comments

Signed-off-by: yminer <yminer@vmware.com>

update ut manifest

Signed-off-by: yminer <yminer@vmware.com>

modify comment manifest

Signed-off-by: yminer <yminer@vmware.com>

updtae ut testcase

Signed-off-by: yminer <yminer@vmware.com>

fixwhitespace lint

Signed-off-by: yminer <yminer@vmware.com>

update isNydus judgement && define annotation var

Signed-off-by: yminer <yminer@vmware.com>

whitespace lint

Signed-off-by: yminer <yminer@vmware.com>
2022-07-08 09:56:10 +08:00
Shengwen YU
315d8a8993
fix: update code in compliance with golangci-lint revive (#17087)
Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
2022-07-01 14:52:47 +08:00
Wang Yan
94bf1c0ebd
Resovle the system artifact schedule issue (#17098)
Scheduling of system artifact depends on the jobservice, where gorountine is used to avoid the circular dependencies between core and jobservice.

Signed-off-by: Wang Yan <wangyan@vmware.com>
2022-07-01 14:25:45 +08:00
stonezdj(Daojun Zhang)
aa63569de2
Fix scan log mismatch issue (#17085)
Add checks in label

Signed-off-by: stonezdj <stonezdj@gmail.com>
2022-06-28 18:12:38 +08:00
Wang Yan
ea2ed27a7d
add url raw query check middleware (#17070)
The middleware can give a uniform url validation and raised error early.

Signed-off-by: Wang Yan <wangyan@vmware.com>
2022-06-24 17:57:46 +08:00
Shengwen YU
e9fca3de45
fix: refactor code for golangci-lint whitespace (#17005)
Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
2022-06-16 17:42:46 +08:00
Shengwen YU
c2a9f5de64
fix: update code for golangci-lint gosimple (#16974)
Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
2022-06-14 13:44:09 +08:00
stonezdj(Daojun Zhang)
e778ec2edf
Add audit_log forward endpoint (#16914)
add config item to set log forward endpoint
  fallback the audit log to default log when endpoint in error

Signed-off-by: stonezdj <stonezdj@gmail.com>
2022-06-10 10:59:40 +08:00
prahaladdarkin
4d062c33d1
Introduce system artifact manager cleanup job (#16879)
Signed-off-by: prahaladdarkin <prahaladd@vmware.com>
2022-06-09 14:16:50 +08:00
Shengwen YU
b43ba15f40
fix: golangci-lint errcheck (#16920)
Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
2022-06-07 17:00:36 +08:00
MinerYang
9d8e9158de
fix deadcode lint & update golangci-lint.yaml (#16896)
* fix deadcode lint & update golangci-lint.yaml

Signed-off-by: yminer <yminer@vmmware.com>

mock.go

Signed-off-by: yminer <yminer@vmmware.com>

commentfmt

Signed-off-by: yminer <yminer@vmmware.com>

mock.go update

Signed-off-by: yminer <yminer@vmmware.com>

update makefile

Signed-off-by: yminer <yminer@vmmware.com>

* update /pkg/allowlist/validator.go

Signed-off-by: yminer <yminer@vmmware.com>

Co-authored-by: yminer <yminer@vmmware.com>
2022-05-26 10:32:07 +08:00
chlins
0fd0c84446 feat(project): introduce cache manager for project (#16740)
Implement cached manager for resource project and migrate project.Mgr
to pkg.ProjectMgr.

Signed-off-by: chlins <chenyuzh@vmware.com>
2022-05-11 15:52:09 +08:00