Commit Graph

66 Commits

Author SHA1 Message Date
stonezdj(Daojun Zhang)
2ea7d09412
skip to log scan sbom accessory for sbom accessory (#20290)
Avoid to log the generate SBOM failure message when the artifact is SBOM in webhook event

Signed-off-by: stonezdj <stone.zhang@broadcom.com>
2024-04-17 14:51:11 +00:00
stonezdj(Daojun Zhang)
2e7db335b3
Add auto generate SBOM on push feature (#20250)
Signed-off-by: stonezdj <daojunz@vmware.com>
Co-authored-by: stonezdj <daojunz@vmware.com>
Co-authored-by: Wang Yan <wangyan@vmware.com>
2024-04-09 09:30:53 +00:00
Taras Katkov
da3c85be5a
fix image name extraction (#18992)
* Update replication.go

It also could be 'library/bitnami/fluentd:1.13.3-debian-10-r0' so we need to split resource to only 2 parts - possible namespace and image name which may include slashes for example -
namespace: library, image: bitnami/fluentd:1.13.3-debian-10-r0

Signed-off-by: Taras Katkov <tkatkov@gmail.com>

* Update replication_test.go

Adding namespace and resource extraction tests.

Signed-off-by: Taras Katkov <tkatkov@gmail.com>

* Reformat only

Signed-off-by: Taras Katkov <tkatkov@gmail.com>

---------

Signed-off-by: Taras Katkov <tkatkov@gmail.com>
2024-03-30 13:41:50 +00:00
MinerYang
b38de22054
bump golang 1.21.5 & fix golangci-lint error (#19722)
bump golang 1.21.5

update golangci-lint && fix revive error

fix white space lint

Signed-off-by: yminer <yminer@vmware.com>
2023-12-19 09:41:26 +08:00
stonezdj(Daojun Zhang)
da949bfc3f
Delete project member when delete project (#19523)
Signed-off-by: stonezdj <daojunz@vmware.com>
2023-11-08 20:51:21 +08:00
stonezdj(Daojun Zhang)
7b0beed934
Delete tag retention rule and tag immutable rule when deleting project (#19390)
fixes #18250

Signed-off-by: stonezdj <daojunz@vmware.com>
2023-10-24 04:28:16 +00:00
Chlins Zhang
99b40bf764
feat: enhance the replication webhook payload (#19433)
Add the new filed 'references' to the replication webhook payload, which
can help user better know the replicated artifact tags or digests.
(references is the lists of the artifact tag name or digest if no tag)

Signed-off-by: chlins <chenyuzh@vmware.com>
2023-10-23 03:29:30 +00:00
Chlins Zhang
a036e4a7b0
fix: skip to delete scan reports if the digest still referenced (#19110)
fix: skip to delete scan reports if the digest still referenced by other artifacts

Avoid to delete the scan reports in case the artifact deleted but still
referenced by the other artifacts.

Signed-off-by: chlins <chenyuzh@vmware.com>
2023-08-07 14:00:26 +08:00
Chlins Zhang
94c76002a2
refactor: remove duplicated artifact deletion handler (#18959)
Remove the duplicated artifact deletion event handler and enhance the
internal artifact onDelete method.

Signed-off-by: chlins <chenyuzh@vmware.com>
2023-07-24 13:38:39 +08:00
Chlins Zhang
970bdab936
fix: correct the operator in the webhook payload (#18906)
Fix the incorrect or meaningless operator in the webhook payload.

Fixes: #18438

Signed-off-by: chlins <chenyuzh@vmware.com>
2023-07-19 15:40:29 +08:00
Chlins Zhang
a98711c0fc
fix: clean up scan executions and reports after deleting artifact (#18693)
Cleanup the associated resources(scan executions and scan reports) after
deletion of artifact.

Fixes: #18634

Signed-off-by: chlins <chenyuzh@vmware.com>
2023-05-31 14:54:52 +08:00
MinerYang
ebac530b46
add goheader linter settings (#18503)
fix files for goheader linter

fix copyright 2018/2019

Signed-off-by: yminer <yminer@vmware.com>
2023-04-25 11:18:42 +08:00
Chlins Zhang
02c51c6b70
fix: correct the retention webhook payload (#18462)
Fix the incorrect number of total and retained in the retention webhook
payload, and completes the deleted_artifacts field.

Fixes: #18428

Signed-off-by: chlins <chenyuzh@vmware.com>
2023-04-03 17:51:02 +08:00
Chlins Zhang
65e675d2e6
feat: integrate CloudEvents to webhook (#18322)
Integrate CloudEvents as payload format for webhook.

Closes: #17748

Signed-off-by: chlins <chenyuzh@vmware.com>
2023-03-14 10:44:25 +08:00
stonezdj(Daojun Zhang)
58054c0ce8
Change queue name for purge audit log and filter the old one (#18182)
Fixes #18121
 Refactor job name with VendorType prefix, make sure job queue name and vendor type in execution and task are identical

Signed-off-by: stonezdj <daojunz@vmware.com>
2023-02-20 16:03:22 +08:00
stonezdj(Daojun Zhang)
d03f0dcf2d
Skip to update pull time and pull count for scanner robot account (#17807)
Add prefix for scanner robot account
   Fixes #14638

Signed-off-by: stonezdj <daojunz@vmware.com>

# Conflicts:
#	api/v2.0/swagger.yaml
#	src/common/const.go
#	src/lib/config/metadata/metadatalist.go
2023-02-20 15:09:21 +08:00
Wang Yan
738fde7d3b
remove chartmuseum backend (#18191)
Harbor deprecates chartmuseum as of v2.8.0

Epic: https://github.com/goharbor/harbor/issues/17958

Discussion: https://github.com/goharbor/harbor/discussions/15057

Signed-off-by: Wang Yan <wangyan@vmware.com>
2023-02-16 18:11:05 +08:00
Wang Yan
18a3373725
bump beego (#17801)
* bump beego

upgrade beego version from v1.10.12 to v2.0.5

1, beego v2 vserver/web refactor
2, beego v2 context refactor
3, beego v2 session refactor
4, beego v2 cache refactor
5, beego v2 orm refactor

Signed-off-by: MinerYang <yminer@vmware.com>
2022-11-24 18:07:42 +08:00
Chlins Zhang
7f00a77d99
fix: pre check notification enable for artifact webhook (#17779)
Check the notification enable before in the artifact webhook handler,
avoid additional db query cost for notification_policy.

Signed-off-by: chlins <chenyuzh@vmware.com>
2022-11-15 09:54:36 +08:00
Chenyu Zhang
49999ab1c0
fix: replication webhook lost when src namespace different with dest (#17312)
Fix the replication webhook notification lost when the rule is
pull-based and src namespace different with dest.

Closes: #17298

Signed-off-by: chlins <chenyuzh@vmware.com>
2022-08-04 15:10:19 +08:00
Abirdcfly
2fece8c9ea
delete duplicate import and use ST1019 in golangci-lint (#17211)
Signed-off-by: Abirdcfly <fp544037857@gmail.com>
2022-07-25 14:12:00 +08:00
Loong Dai
09371b48e8
lint: sort imports (#17131)
* lint: add goimports

Signed-off-by: Loong Dai <loong.dai@intel.com>
2022-07-20 11:33:08 +08:00
Chenyu Zhang
927f9f5af3
fix: attach labels for replication event (#17108)
Attach labels for replication event, list labels by artifact ID so for
event-based replication rule filter by label can work as expect.

Closes: #17014

Signed-off-by: chlins <chenyuzh@vmware.com>
2022-07-05 19:57:13 +08:00
Shengwen YU
c2a9f5de64
fix: update code for golangci-lint gosimple (#16974)
Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
2022-06-14 13:44:09 +08:00
stonezdj(Daojun Zhang)
e778ec2edf
Add audit_log forward endpoint (#16914)
add config item to set log forward endpoint
  fallback the audit log to default log when endpoint in error

Signed-off-by: stonezdj <stonezdj@gmail.com>
2022-06-10 10:59:40 +08:00
Shengwen YU
b43ba15f40
fix: golangci-lint errcheck (#16920)
Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
2022-06-07 17:00:36 +08:00
chlins
a82f1ba63d feat(repository): introduce cache manager for repository (#16741)
Implement cache layer for resource repository and migrate repository.Mgr
to pkg.RepositoryMgr.

Signed-off-by: chlins <chenyuzh@vmware.com>
2022-05-12 10:14:35 +08:00
MinerYang
083b44e719
bump up astaxie/beego@v1.12.1 to beego/beego/@v1.12.7 (#16770)
Signed-off-by: yminer <yminer@vmmware.com>

goo mod tidy

Signed-off-by: yminer <yminer@vmmware.com>

Co-authored-by: yminer <yminer@vmmware.com>
2022-04-28 11:58:44 +08:00
chlins
8c223135e7 feat: add cache layer for artifact (#16739)
Implement cache layer for resource artifact and define common
cache manager and workflow. Also add cache related options to
configuration yaml.

Signed-off-by: chlins <chenyuzh@vmware.com>
2022-04-25 18:01:45 +08:00
chlins
de7978e1b5 feat: async update artifact pull time and repository pull count
Signed-off-by: chlins <chenyuzh@vmware.com>
2021-12-24 11:17:44 +08:00
Wang Yan
7608df4b5c
add pull update control env (#16051)
These variables are temporary solution for issue: https://github.com/goharbor/harbor/issues/16039
When user disable the pull count/time/audit log, it will decrease the database access, especially in large concurrency pull scenarios.

1, PULL_TIME_UPDATE_DISABLE : The flag to indicate if pull time is disable for pull request.
2, PULL_COUNT_UPDATE_DISABLE : The flag to indicate if pull count is disable for pull request.
3, pull audit log will not create on disabling pull time.

Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-11-26 18:13:23 +08:00
Qian Deng
354a2bd80d Enhance the trace related code
* Move request id to requestid middleware
* fix span pass to child ctx on orm
* fix typos
* remove unused code
* add operation name to Transaction

Signed-off-by: Qian Deng <dengq@vmware.com>
2021-09-22 04:14:43 +00:00
Shengwen Yu
e2e3bcca1c feat: add stop scan & stop scan-all feature
Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
2021-09-13 11:19:21 +08:00
He Weiwei
1b955cd28e fix: supply the latest tag of artifact to scan request
Closes #14416 #14299

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-06-08 10:11:10 +00:00
Wang Yan
42a9d0d905
remove common project code (#14939)
move project model from common to pkg

Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-05-25 11:01:19 +08:00
stonezdj
60478f4990 Move common config api to lib/config
Register all config managers, and get it by getConfigManger()

Signed-off-by: stonezdj <stonezdj@gmail.com>
2021-04-13 19:43:33 +08:00
stonezdj
107e468b60 Refactor configure api to new programming model
Changes include:
1. Move core/config to controller/config
2. Change the job_service and gcreadonly to depends on lib/config instead of core/config
3. Move the config related dao, manager and driver to pkg/config
4. Adjust the invocation of the config API, most of then should provide a context parameter, when accessing system config, you can call it with background context, when accessing user config, the context should provide orm.Context

Signed-off-by: stonezdj <stonezdj@gmail.com>
2021-04-09 08:10:11 +08:00
Wenkai Yin(尹文开)
28596c3ffb
Refactor registry API (#14528)
* Refactor registry API

Refactor registry API

Signed-off-by: Wenkai Yin <yinw@vmware.com>

* Fix bugs of replications

1. Fix the scheduled replication doesn't work issue
2. Fix the destination name lost issue when updating replication policy

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-03-31 15:49:23 +08:00
Wang Yan
9ef50ed430
refactor notification (#14406)
* Refactor webhook

refactor notification to new programming model

Signed-off-by: wang yan <wangyan@vmware.com>
2021-03-22 17:27:23 +08:00
Wenkai Yin
3d7fd070c7 Refeactor replication policy APIs
Refeactor replication policy APIs

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-03-17 10:57:31 +08:00
He Weiwei
e0f7778027
fix: ignore tag in the scan webhook when it's empty (#14057)
Closes #13464

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-01-22 20:08:36 +08:00
Ziming Zhang
39fb500318 feat(retention) refactor to use go swagger api
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2021-01-08 07:09:28 +00:00
prahaladdarkin
a890b28e1e
Store vulnerability data from scanner into a relational format (#13616)
feat: Store vulnerability report from scanner into a relational format

Convert vulnerability report JSON obtained  from scanner into a relational format describe in:https://github.com/goharbor/community/pull/145

Signed-off-by: prahaladdarkin <prahaladd@vmware.com>
2020-12-25 08:47:46 +08:00
He Weiwei
08580f9fec
refactor(scan): refactor scan/scan all job to task manager (#13684)
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-12-14 13:34:35 +08:00
Wenkai Yin
294385c34d Refactor the replication execution
1. Use the task manager to manage the underlying execution/task
2. Use the pkg/scheduler to schedule the periodical job
3. Apply the new program model
4. Migration the old data into the new data model

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-11-23 14:24:10 +08:00
He Weiwei
76f1afbe0d
refactor: remove core/promgr pkg (#13408)
* refactor: remove core/promgr pkg

Remove `core/promgr` package and use `controller/project` instead of it.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-11-12 15:33:13 +08:00
wang yan
b51aaac26e fix event log issue
fixes #12554
Add string method for each event, and the detail can be logged in the core.log

Signed-off-by: wang yan <wangyan@vmware.com>
2020-08-26 18:27:21 +08:00
Ted Guan
eb317fb8cb
tag retention webhook support (#12749)
Signed-off-by: guanxiatao <guanxiatao@corp.netease.com>
2020-08-19 00:07:45 +08:00
He Weiwei
df1bdc1020 refactor(project): add more methods to project controller and manager
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-07-31 17:55:35 +00:00
Steven Zou
be5858b1ed fix(preheat):fix issues of event-based preheat
- fix issue of missing handling error in the preheat event handler
- change preheat artifact logic to reduce health check times
- publish pushed events only for the tagged artifacts

Signed-off-by: Steven Zou <szou@vmware.com>
2020-07-28 18:32:26 +08:00