Commit Graph

336 Commits

Author SHA1 Message Date
Chlins Zhang
bf4cfe9e1e
fix: add human friendly message when export cve job failure (#17726)
Add human friendly when export CVE in the condition of empty CSV file,
because this file will be stored as system artifact and pushed to
distribution, but it will leads to error when push empty blob to S3
storage driver.

Signed-off-by: chlins <chenyuzh@vmware.com>
2022-11-01 15:04:07 +08:00
Chlins Zhang
c330b8c63a
feat: introduce the copy by chunk for replication (#17602)
1. Add sql migration to alter replication policy table
2. Implement the PullBlobChunk and PushBlobChunk for the underlying v2 registry client
3. Update image transfer logic to support copy by chunk
4. Update the replication policy API handler

Signed-off-by: chlins <chenyuzh@vmware.com>
2022-11-01 11:19:17 +08:00
stonezdj(Daojun Zhang)
3050e856d9
Add check when adding project member (#17701)
Signed-off-by: stonezdj <stonezdj@gmail.com>

Signed-off-by: stonezdj <stonezdj@gmail.com>
2022-10-28 15:06:27 +08:00
Eng Zer Jun
cd0fa06a32
test: use T.Setenv to set env vars in tests (#17670)
This commit replaces `os.Setenv` with `t.Setenv` in tests. The
environment variable is automatically restored to its original value
when the test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.Setenv
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-10-25 10:56:26 +08:00
stonezdj(Daojun Zhang)
b08b5fe617
Add check on the audit log forward and skip audit log database (#17575)
fixes #17524
Signed-off-by: stonezdj <stonezdj@gmail.com>
2022-09-21 15:24:57 +08:00
Chenyu Zhang
5d7a549620
upgrade: bump up mockery to v2.14.0 (#17415)
Signed-off-by: chlins <chenyuzh@vmware.com>
2022-08-22 14:21:13 +08:00
MinerYang
daf0874ed5
project userid not found log level into debug (#17343)
Signed-off-by: yminer <yminer@vmware.com>
2022-08-08 14:20:13 +08:00
Chenyu Zhang
49999ab1c0
fix: replication webhook lost when src namespace different with dest (#17312)
Fix the replication webhook notification lost when the rule is
pull-based and src namespace different with dest.

Closes: #17298

Signed-off-by: chlins <chenyuzh@vmware.com>
2022-08-04 15:10:19 +08:00
Chenyu Zhang
70a95a9696
fix: reduce the high db cpu usage for tag retention (#17296)
1. Add two indexes to database migrations.
2. Skip refresh quota in middleware for requests from jobservice.
3. Refresh quota by self in the end of tag retention job.

Closes: #14708

Signed-off-by: chlins <chenyuzh@vmware.com>
2022-08-04 10:08:02 +08:00
Wang Yan
914d9a5526
fix copy artifact accessory quota issue (#17293)
It needs to request quota for accessory on copying artifact.

Signed-off-by: wang yan <wangyan@vmware.com>
2022-08-03 20:02:20 +08:00
MinerYang
722d1a19ea
update image extra attributes author in labels (#17291)
update image extra attributes author

Signed-off-by: yminer <yminer@vmware.com>

adding spec comment

Signed-off-by: yminer <yminer@vmware.com>
2022-08-03 11:50:54 +08:00
Chenyu Zhang
bff4e13087
fix: export cve adds resource check and project validation (#17265)
1. Add resource permission check for API handler
2. Validate export cve params project
3. Optimize friendly human message when execution status is error

Signed-off-by: chlins <chenyuzh@vmware.com>
2022-07-29 19:01:46 +08:00
Wang Yan
02eae9dede
Filter out accessory from list artifact results (#17231)
Fixed #17145
1, Filter out the accessory from the artifact list.
2, Disable the display func of the accessory interface, currently this will not impact any kind of accessory, like signature and nydus. If we'd like to introduce it, it needs to resolve the pagiation issue of artifact list.

Signed-off-by: Wang Yan <wangyan@vmware.com>
2022-07-26 00:38:05 +08:00
Abirdcfly
2fece8c9ea
delete duplicate import and use ST1019 in golangci-lint (#17211)
Signed-off-by: Abirdcfly <fp544037857@gmail.com>
2022-07-25 14:12:00 +08:00
Loong Dai
09371b48e8
lint: sort imports (#17131)
* lint: add goimports

Signed-off-by: Loong Dai <loong.dai@intel.com>
2022-07-20 11:33:08 +08:00
prahaladdarkin
3f383e3ffd
Improvements to Vulnerability Data Export functionality. (#17161)
Closes:
* https://github.com/goharbor/harbor/issues/17152
* https://github.com/goharbor/harbor/issues/17153
Better error handling in case of task executions not found in the system

Signed-off-by: prahaladdarkin <prahaladd@vmware.com>
2022-07-14 15:08:25 +08:00
prahaladdarkin
130452111b
Vulnerability scan data export functionality (#15998)
Vulnerability Scan Data (CVE) Export Functionality
Proposal - goharbor/community#174
Closes - https://github.com/goharbor/harbor/issues/17150
Changes:
* CVE Data export to CSV with filtering support.
* Implement CSV data export job for creating CSVs
* APIs to trigger CSV export job executions

Signed-off-by: prahaladdarkin <prahaladd@vmware.com>
2022-07-11 16:35:04 +08:00
Roooocky
7e4b26b220
Add new feature for supporting WebAssembly artifact (#16931)
support wasm

Signed-off-by: ln23415 <ln23415@hotmail.com>
2022-07-08 23:08:28 +08:00
MinerYang
efd9632e96
add nydus middleware (#17126)
Signed-off-by: yminer <yminer@vmware.com>

remove comments

Signed-off-by: yminer <yminer@vmware.com>

update ut manifest

Signed-off-by: yminer <yminer@vmware.com>

modify comment manifest

Signed-off-by: yminer <yminer@vmware.com>

updtae ut testcase

Signed-off-by: yminer <yminer@vmware.com>

fixwhitespace lint

Signed-off-by: yminer <yminer@vmware.com>

update isNydus judgement && define annotation var

Signed-off-by: yminer <yminer@vmware.com>

whitespace lint

Signed-off-by: yminer <yminer@vmware.com>
2022-07-08 09:56:10 +08:00
Wang Yan
bd8d66c68d
resolve robot authgen password format issue (#17134)
In some cases, the robot automatically generates passwords that do not meet confidentiality requirements.
The fix adds retry for auto generating passwords, and the timeout is 1 minute.

The requirement: the secret must longer than 8 chars with at least 1 uppercase letter, 1 lowercase letter and 1 number

Signed-off-by: Wang Yan <wangyan@vmware.com>
2022-07-08 02:05:32 +08:00
stonezdj(Daojun Zhang)
dcccb44db8
Add options to the user.Count method (#16285)
Fixes #16269, exclude the admin account by default
  Add excludeDefaultAdmin method -- exclude default admin by option
  Update authModeCanBeModified method -- the user count should be 0 without admin

Signed-off-by: stonezdj <stonezdj@gmail.com>
2022-07-06 13:57:31 +08:00
Wang Yan
6917021b41
resolve copy failure for artifact with multiple accessories (#17123)
If the artifact has more than one signaure, it will currently copy the last one. After the fix, all accessories can be copied to the target project.

Signed-off-by: wang yan <wangyan@vmware.com>
2022-07-05 20:10:22 +08:00
Chenyu Zhang
927f9f5af3
fix: attach labels for replication event (#17108)
Attach labels for replication event, list labels by artifact ID so for
event-based replication rule filter by label can work as expect.

Closes: #17014

Signed-off-by: chlins <chenyuzh@vmware.com>
2022-07-05 19:57:13 +08:00
Shengwen YU
315d8a8993
fix: update code in compliance with golangci-lint revive (#17087)
Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
2022-07-01 14:52:47 +08:00
Wang Yan
b4ef1dbfd5
Unify retention schedule/task retrieve and update (#17083)
Add not-found handling for non exist items in retention operation.

Signed-off-by: Wang Yan <wangyan@vmware.com>
2022-07-01 01:58:14 +08:00
stonezdj(Daojun Zhang)
aa63569de2
Fix scan log mismatch issue (#17085)
Add checks in label

Signed-off-by: stonezdj <stonezdj@gmail.com>
2022-06-28 18:12:38 +08:00
stonezdj(Daojun Zhang)
c639257ba7
Support stop purge audit log job (#17033)
fixes #17006

Signed-off-by: stonezdj <stonezdj@gmail.com>
2022-06-28 11:15:11 +08:00
MinerYang
aab320591f
golangci-lint fix ineffassign (#17027)
Signed-off-by: yminer <yminer@vmware.com>

delete src/lib/redis/helper.go

Signed-off-by: yminer <yminer@vmware.com>
2022-06-21 12:01:59 +08:00
Maksym Trofimenko
9a3cb4a041
GDPR compliant deletion of Users (#16859)
fixes #16697

Signed-off-by: Maksym Trofimenko <maksym@container-registry.com>
2022-06-16 14:28:15 +02:00
Shengwen YU
e9fca3de45
fix: refactor code for golangci-lint whitespace (#17005)
Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
2022-06-16 17:42:46 +08:00
Shengwen YU
c2a9f5de64
fix: update code for golangci-lint gosimple (#16974)
Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
2022-06-14 13:44:09 +08:00
stonezdj(Daojun Zhang)
e778ec2edf
Add audit_log forward endpoint (#16914)
add config item to set log forward endpoint
  fallback the audit log to default log when endpoint in error

Signed-off-by: stonezdj <stonezdj@gmail.com>
2022-06-10 10:59:40 +08:00
prahaladdarkin
4d062c33d1
Introduce system artifact manager cleanup job (#16879)
Signed-off-by: prahaladdarkin <prahaladd@vmware.com>
2022-06-09 14:16:50 +08:00
Shengwen YU
b43ba15f40
fix: golangci-lint errcheck (#16920)
Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
2022-06-07 17:00:36 +08:00
MinerYang
9b22e2943f
bumpup mockery to v1.12.3 (#16955)
Signed-off-by: yminer <yminer@vmmware.com>

Co-authored-by: yminer <yminer@vmmware.com>
2022-06-07 16:04:33 +08:00
stonezdj(Daojun Zhang)
0dc7a6806c
Add NextScheduledTime in schedule object (#16925)
Add NextSchedule in utils
  Add NextScheduledTime in ScheduleObj to display it in UI

Signed-off-by: stonezdj <stonezdj@gmail.com>
2022-06-06 22:17:41 +08:00
Chenyu Zhang
136767df15
Merge pull request #16892 from chlins/feat/cache-layer-for-project-metadata
feat(project): introduce cache layer for project_metadata (#16891)
2022-05-27 09:30:55 +08:00
MinerYang
9d8e9158de
fix deadcode lint & update golangci-lint.yaml (#16896)
* fix deadcode lint & update golangci-lint.yaml

Signed-off-by: yminer <yminer@vmmware.com>

mock.go

Signed-off-by: yminer <yminer@vmmware.com>

commentfmt

Signed-off-by: yminer <yminer@vmmware.com>

mock.go update

Signed-off-by: yminer <yminer@vmmware.com>

update makefile

Signed-off-by: yminer <yminer@vmmware.com>

* update /pkg/allowlist/validator.go

Signed-off-by: yminer <yminer@vmmware.com>

Co-authored-by: yminer <yminer@vmmware.com>
2022-05-26 10:32:07 +08:00
stonezdj
1839965c7a Add REST API for purge audit log
Add rest API for purge audit log
  Extract scheduler, execution, task from GC controller for common use

Signed-off-by: stonezdj <stonezdj@gmail.com>
2022-05-23 16:58:31 +08:00
chlins
f16cc4bda4 feat(project): introduce cache layer for project_metadata (#16891)
Implement cache layer for project_metadata and migrate metadata.Mgr to
pkg.ProjectMetaMgr.

Signed-off-by: chlins <chenyuzh@vmware.com>
2022-05-23 09:08:39 +08:00
stonezdj(Daojun Zhang)
2852a7606b
Merge pull request #16833 from stonezdj/22may10_purge_audit_log
(feat) Add job service to purge audit_log
2022-05-17 09:02:56 +08:00
stonezdj
3ac7281177 Add job service to purge audit_log
Add jobservice type PurgeAuditLog

Signed-off-by: stonezdj <stonezdj@gmail.com>
2022-05-16 21:52:58 +08:00
Chenyu Zhang
d56a505fa7
Merge pull request #16846 from chlins/feat/cache-layer-for-repo
feat(repository): introduce cache manager for repository (#16741)
2022-05-16 11:39:28 +08:00
Wang Yan
54a857f2aa
fix artifact count issue (#16851)
The count should filter out the accessory, the result should be consistent with list method.

Fixes #16816

Signed-off-by: Wang Yan <wangyan@vmware.com>
2022-05-13 11:48:20 +08:00
chlins
a82f1ba63d feat(repository): introduce cache manager for repository (#16741)
Implement cache layer for resource repository and migrate repository.Mgr
to pkg.RepositoryMgr.

Signed-off-by: chlins <chenyuzh@vmware.com>
2022-05-12 10:14:35 +08:00
Shengwen YU
ee214f0347
Merge pull request #16841 from zyyw/issue-16355-gc-update_time
fix: gc history update_time
2022-05-12 10:12:47 +08:00
Shengwen Yu
d2c4c4f71b fix: gc history update_time
Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
2022-05-11 17:17:09 +08:00
chlins
0fd0c84446 feat(project): introduce cache manager for project (#16740)
Implement cached manager for resource project and migrate project.Mgr
to pkg.ProjectMgr.

Signed-off-by: chlins <chenyuzh@vmware.com>
2022-05-11 15:52:09 +08:00
Wang Yan
d3d4ad6a34
fix staticcheck issues (#16828)
Fix the staticcheck problems that reported by golangci-line staticcheck

Signed-off-by: Wang Yan <wangyan@vmware.com>
2022-05-10 16:07:10 +08:00
MinerYang
083b44e719
bump up astaxie/beego@v1.12.1 to beego/beego/@v1.12.7 (#16770)
Signed-off-by: yminer <yminer@vmmware.com>

goo mod tidy

Signed-off-by: yminer <yminer@vmmware.com>

Co-authored-by: yminer <yminer@vmmware.com>
2022-04-28 11:58:44 +08:00