Commit Graph

892 Commits

Author SHA1 Message Date
Maksym Trofimenko
3a83f6d31f
Merge 5c949c1214 into d01dfd450a 2024-04-30 15:44:05 +02:00
stonezdj(Daojun Zhang)
d154c27362
Add scan type in webhook event (#20363)
fixes #20331

Signed-off-by: stonezdj <stone.zhang@broadcom.com>
2024-04-29 13:51:09 +00:00
stonezdj(Daojun Zhang)
fba4c40c65
Delete scan_report when accessory is removed (#20365)
Signed-off-by: stonezdj <stone.zhang@broadcom.com>
2024-04-27 01:56:30 +00:00
stonezdj(Daojun Zhang)
ec8d692fe6
Add scanner info and report_id to sbom_overview on listing artifact (#20358)
Add scan_status and report_id when scan has a failed task

Signed-off-by: stonezdj <stone.zhang@broadcom.com>
2024-04-25 17:00:35 +08:00
stonezdj(Daojun Zhang)
c80e9bf477
Add 422 in the swagger.yaml (#20344)
change log level with no content message
  fix time in sbom accessory
  fixes #20342 #20332 #20328

Signed-off-by: stonezdj <stone.zhang@broadcom.com>
2024-04-24 09:57:46 +08:00
stonezdj(Daojun Zhang)
d759429831
Set default capability for old scanners (#20306)
Signed-off-by: stonezdj <stone.zhang@broadcom.com>
Co-authored-by: Wang Yan <wangyan@vmware.com>
2024-04-20 10:37:30 +08:00
stonezdj(Daojun Zhang)
654aa8edcf
Add generate SBOM feature (#20251)
* Add SBOM scan feature

  Add scan handler for sbom
  Delete previous sbom accessory before the job service

Signed-off-by: stonezdj <daojunz@vmware.com>

* fix issue

Signed-off-by: stonezdj <stone.zhang@broadcom.com>

---------

Signed-off-by: stonezdj <daojunz@vmware.com>
Signed-off-by: stonezdj <stone.zhang@broadcom.com>
Co-authored-by: stonezdj <daojunz@vmware.com>
2024-04-16 13:34:19 +00:00
Wang Yan
550bf1d750
fix issue 20269 (#20274)
By default, use the nvd score as the primary score, and if it is unavailable, fallback to the redhat score.
fix #20269

Signed-off-by: wang yan <wangyan@vmware.com>
2024-04-16 16:49:52 +08:00
MinerYang
7465a29919
add SBOM icon (#20270)
Signed-off-by: yminer <yminer@vmware.com>
2024-04-12 20:12:46 +08:00
MinerYang
e9d2f50669
update mockery to v2.42.2 (#20258)
Signed-off-by: yminer <yminer@vmware.com>
2024-04-11 03:37:59 +00:00
stonezdj(Daojun Zhang)
5d7c668028
Support list artifact with_sbom_overview option (#20244)
Signed-off-by: stonezdj <stone.zhang@broadcom.com>
Co-authored-by: stonezdj <daojunz@vmware.com>
2024-04-10 14:47:45 +00:00
Wang Yan
461a5fa50d
add stop sbom scanning API (#20200)
* add stop sbom scanning API

1. [UI] support to stop sbom scanning #20200
2. add type for stop scanning api, make it able to support both vulnerability and sbom.
3. refactor the db query to support multiple extra attributes.

Signed-off-by: wang yan <wangyan@vmware.com>
Signed-off-by: xuelichao <xuel@vmware.com>
Co-authored-by: xuelichao <xuel@vmware.com>
2024-04-09 16:07:47 +08:00
stonezdj(Daojun Zhang)
be648ea47f
Refactor scan job service make it easy to add new scan type (#20177)
Signed-off-by: stonezdj <daojunz@vmware.com>
Signed-off-by: stonezdj(Daojun Zhang) <stonezdj@gmail.com>
Co-authored-by: stonezdj <daojunz@vmware.com>
2024-04-09 16:05:30 +08:00
Prima Adi Pradana
7b8a322a88
delete membership=0 in getProjectsByName (#20153)
delete membership=0 in getProjectsByName but lets getProjects still using membership=1 for reserve if getProjectsByName not found any

Signed-off-by: prima <prima101112@gmail.com>
2024-04-05 15:29:07 +00:00
stonezdj(Daojun Zhang)
b6366e03e9
Update GenAccessoryArt API to generate valid accessory for SBOM (#20214)
Signed-off-by: stonezdj <daojunz@vmware.com>
Co-authored-by: stonezdj <daojunz@vmware.com>
2024-04-03 07:59:15 +00:00
guangwu
9778176ff1
fix: close file (#20189)
Signed-off-by: guoguangwu <guoguangwug@gmail.com>
Co-authored-by: Wang Yan <wangyan@vmware.com>
2024-04-03 11:45:59 +08:00
stonezdj(Daojun Zhang)
cea47c7db3
Add accessory type for sbom (#20208)
Signed-off-by: stonezdj <daojunz@vmware.com>
Co-authored-by: stonezdj <daojunz@vmware.com>
2024-04-02 18:11:27 +08:00
James Kang
fd81e7c43e
chore: fix function names (#20159)
Signed-off-by: majorteach <csgcgl@126.com>
Co-authored-by: Wang Yan <wangyan@vmware.com>
2024-03-26 13:53:44 +08:00
stonezdj(Daojun Zhang)
80a9c688fc
panic due to mark retention task error (#20161)
panic due to mark retention task error

fixes #20129

Signed-off-by: stonezdj <daojunz@vmware.com>
2024-03-26 04:52:17 +00:00
Wang Yan
2eb5464603
add type for scanner metadata (#20108)
Signed-off-by: wang yan <wangyan@vmware.com>
2024-03-25 07:02:39 +00:00
MinerYang
a269b4f31c
Update support for artifactType for both manifest and index (#20030)
add artifact_type for artifact model to support artifactType filter

Signed-off-by: yminer <yminer@vmware.com>

add 2.11 sql schema & update index artifactType omitted

Signed-off-by: yminer <yminer@vmware.com>

update UT

update migrate sql for artifact_type

Signed-off-by: yminer <yminer@vmware.com>

remove debug line
2024-03-12 13:52:56 +00:00
Wang Yan
dbe9790147
add generate sbom object utility (#20097)
* add generate sbom object utility

Leverage the go-containerregistry to generate the oci object for sbom and add it as an accessory of the subject artifact.

Signed-off-by: wang yan <wangyan@vmware.com>

* remove vendor

Signed-off-by: wang yan <wangyan@vmware.com>

* resolve comments

Signed-off-by: wang yan <wangyan@vmware.com>

* fix ut

Signed-off-by: wang yan <wangyan@vmware.com>

* resolve comments

Signed-off-by: wang yan <wangyan@vmware.com>

* remove the todo comments

Signed-off-by: wang yan <wangyan@vmware.com>

---------

Signed-off-by: wang yan <wangyan@vmware.com>
2024-03-12 12:27:34 +08:00
Wang Yan
3782bab80a
add sbom settings for project (#20069)
Add a new switcher for sbom generation, by default is false.

Signed-off-by: wang yan <wangyan@vmware.com>
2024-03-01 14:40:02 +08:00
Maksym Trofimenko
5c949c1214 add statistics metrics collector
Signed-off-by: Maksym Trofimenko <maksym.trofimenko@gmail.com>
2024-02-25 15:23:49 +00:00
Antoine Jouve
73c2884e58
[Token/JWT] Update to golang-jwt v5.2.0 (#19802)
* feat: update to golang-jwt v5.2.0

Signed-off-by: Antoine Jouve <ant.jouve@gmail.com>

* fix: module issues and robot claims

Signed-off-by: Antoine Jouve <ant.jouve@gmail.com>

* fix: add missing time import

Signed-off-by: Antoine Jouve <ant.jouve@gmail.com>

* feat: set jwt validation leeway to 60s

Signed-off-by: Antoine Jouve <ant.jouve@gmail.com>

* fix: update leeways that were still set to 10s

Signed-off-by: Antoine Jouve <ant.jouve@gmail.com>

* feat: update go.sum

Signed-off-by: Antoine Jouve <ant.jouve@gmail.com>

* feat: add two leeway related test cases

Signed-off-by: Antoine Jouve <ant.jouve@gmail.com>

* fix: correct jwt audience validation

Signed-off-by: Antoine Jouve <ant.jouve@gmail.com>

* fix: gofmt v2_token.go

Signed-off-by: Antoine Jouve <ant.jouve@gmail.com>

* feat: take into account review comments

Signed-off-by: Antoine Jouve <ant.jouve@gmail.com>

* feat: use a common constant to store JWT leeway

Signed-off-by: Antoine Jouve <ant.jouve@gmail.com>

---------

Signed-off-by: Antoine Jouve <ant.jouve@gmail.com>
Signed-off-by: Antoine Jouve <an-toine@users.noreply.github.com>
Co-authored-by: MinerYang <yminer@vmware.com>
2024-02-23 11:30:13 +08:00
zycupup
ee6f61c502
feat: volc cr adapter (#19456)
feat: support volcEngine replication

Signed-off-by: zhuyuchen.1 <zhuyuchen.1@bytedance.com>
2024-01-19 14:15:49 +08:00
MinerYang
b38de22054
bump golang 1.21.5 & fix golangci-lint error (#19722)
bump golang 1.21.5

update golangci-lint && fix revive error

fix white space lint

Signed-off-by: yminer <yminer@vmware.com>
2023-12-19 09:41:26 +08:00
Chlins Zhang
43ccd2f09f
perf: optimize the performance of accessory query (#19557)
1. Add db index for subject_artifact_id in table artifact_accessory.
2. Optimize the SQL of excluding the accessory for artifact count.

Signed-off-by: chlins <chenyuzh@vmware.com>
2023-11-22 02:11:11 +00:00
Maksym Trofimenko
996e57b511
Feature: GDPR compliant audit logs (#17396) 2023-11-20 21:31:22 +01:00
stonezdj(Daojun Zhang)
307a3121aa
Return empty result when no scanner configured (#19577)
fixes #19534

Signed-off-by: stonezdj <daojunz@vmware.com>
2023-11-17 07:29:19 +00:00
Shengwen YU
e941f3272b
fix: sorting quota (#19538)
fix: sort Project Quotas

Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
2023-11-13 14:37:41 +08:00
Wang Yan
5c02fd807e
add permission api (#19543)
The permission api targets to return the full set of permissons for robot to use.
And only system and project admin have the access

Signed-off-by: wang yan <wangyan@vmware.com>
2023-11-09 09:47:07 +08:00
stonezdj(Daojun Zhang)
da949bfc3f
Delete project member when delete project (#19523)
Signed-off-by: stonezdj <daojunz@vmware.com>
2023-11-08 20:51:21 +08:00
Daniel Jiang
3f7c605cf5
[WIP] Remove vendor folder from harbor code base (#19508)
* Remove vendor folder from harbor code base

Signed-off-by: Daniel Jiang <jiangd@vmware.com>

* debug ut failure

* debug failure

* debug failure

---------

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2023-11-05 14:37:25 +08:00
stonezdj(Daojun Zhang)
b337f51e7e
Replace comma in username to avoid casbin issue (#19505)
Check username when creating user by API
   Replace comma with underscore in username for OnboardUser
   Fixes #19356

Signed-off-by: stonezdj <daojunz@vmware.com>
2023-11-02 03:50:50 +00:00
stonezdj(Daojun Zhang)
7b0beed934
Delete tag retention rule and tag immutable rule when deleting project (#19390)
fixes #18250

Signed-off-by: stonezdj <daojunz@vmware.com>
2023-10-24 04:28:16 +00:00
stonezdj(Daojun Zhang)
d030ab216b
Use batch to list the job id in the job queue to avoid crash redis (#19444)
fixes: #19436

Signed-off-by: stonezdj <daojunz@vmware.com>
2023-10-18 09:31:37 +00:00
stonezdj(Daojun Zhang)
663d0b81a7
Change fixed_version to package_version in query dangerous CVE sql (#19397)
Change fixed_version to package_version
  fixes #19391

Signed-off-by: stonezdj <daojunz@vmware.com>
2023-10-11 07:57:19 +00:00
Wang Yan
3338f27189
support accessory in either order (#19375)
In certain cases, the OCI client may push the subject artifact and accessory in either order.
Therefore, it is necessary to handle situations where the client pushes the accessory ahead of the subject artifact.

Signed-off-by: wang yan <wangyan@vmware.com>
2023-09-23 02:42:58 +08:00
Wang Yan
ed370a496b
deprecate resource label (#19349)
There is no api is using the DAO, remove it from the source code.

Signed-off-by: wang yan <wangyan@vmware.com>
2023-09-14 01:25:52 +08:00
stonezdj(Daojun Zhang)
7f191630e4
Remove job status track information from redis after stop (#19227)
Remove job status track information from redis after stop the job in the queue

  After stop in the queue:
  Remove key in {harbor_job_service_namespace}:job_track:inprogress
  Remove {harbor_job_service_namespace}:job_stats:<job_id>
  fixes #19211

Signed-off-by: stonezdj <daojunz@vmware.com>
2023-09-05 10:30:48 +08:00
Bin Liu
0e039b255c
exporter: add field alias for count(*) func (#18840)
Add an alias so that orm can parse the column name regardless of
the type and version of the database.

Signed-off-by: bin liu <liubin0329@gmail.com>
2023-08-30 09:05:44 +08:00
stonezdj(Daojun Zhang)
298ecbbe7d
Refine total artifact and scanned artifact (#19228)
Artifact include all accessory, child artifact
  fixes #19215

Signed-off-by: stonezdj <daojunz@vmware.com>
2023-08-22 16:03:33 +08:00
Chlins Zhang
c7e25295fe
fix: support customize cache db for business (#19182)
Support to configure the customized redis db for cache layer and other
misc business for core, by default the behavior is same with
previous(stored in db 0).

Signed-off-by: chlins <chenyuzh@vmware.com>
2023-08-18 11:04:16 +08:00
stonezdj(Daojun Zhang)
83ff2b277a
Wrong artifact scanned count (#19198)
fixes #19197

Signed-off-by: stonezdj <daojunz@vmware.com>
2023-08-17 14:12:07 +08:00
Chlins Zhang
52e66155d4
log: change log level to reduce the noise logs (#19146)
1. Change some logs level to reduce the noise.
2. Wrap the go-redis.Nil error as ErrNotFound to avoid confusing

Signed-off-by: chlins <chenyuzh@vmware.com>
2023-08-17 11:11:04 +08:00
stonezdj(Daojun Zhang)
d98699b5fc
Filter artifact without CVE from top 5 dangerous artifacts (#19187)
Fixes #19186

Signed-off-by: stonezdj <daojunz@vmware.com>
2023-08-17 09:29:23 +08:00
Shengwen YU
88c6018950
fix: cron string validation (#19071)
fix: cron string validation (the 1st field of a cron string must be 0 when there are 6 fields)

Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
2023-08-09 10:22:54 +08:00
Shengwen YU
90de9092ce
fix: add storage_limit check (#19095)
fix: add storage_limit check (add ValidateQuotaLimit as a general method to validate quota limit value)

Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
2023-08-09 09:37:07 +08:00
stonezdj(Daojun Zhang)
0e92eaea18
Sort most dangerous vulnerabilities by score and severity level (#19103)
Check it is referred by a scan_report
  Add index to report_vulnerability_record to improve performance
  Fixes #19014 #19028

Signed-off-by: stonezdj <daojunz@vmware.com>
2023-08-07 10:13:34 +08:00