In the dry run mode, the accessories should be considered when try to simulate delete the subject manifest.
Signed-off-by: wang yan <wangyan@vmware.com>
Support to configure the customized redis db for cache layer and other
misc business for core, by default the behavior is same with
previous(stored in db 0).
Signed-off-by: chlins <chenyuzh@vmware.com>
* add details for gc dry-run
fixes 19040
Gives the make details for the dry-run
Signed-off-by: wang yan <wangyan@vmware.com>
* Update details column for gc history (#164)
1.Fixes #19038
Signed-off-by: AllForNothing <sshijun@vmware.com>
---------
Signed-off-by: wang yan <wangyan@vmware.com>
Signed-off-by: AllForNothing <sshijun@vmware.com>
Co-authored-by: Shijun Sun <30999793+AllForNothing@users.noreply.github.com>
fixes#19041
The issuse was introduced by https://github.com/goharbor/harbor/pull/17087.
To move the import to main since the anonymous imports only allowed at main in go-cilint.
Fixes it:
To add the anonymous imports at the jobservice main since the gc job needs to remove accessory when to delete an artifact
Signed-off-by: wang yan <wangyan@vmware.com>
Refactor the clean redis logic in the GC job, replace the redigo client
to the lib cache interface which can simplify operations.
Signed-off-by: chlins <chenyuzh@vmware.com>
jobservice: add DB to job logger config in readme
Job logger support file, db and stdout, the comment should
include the type of DB as well as file and stdout.
Signed-off-by: bin liu <liubin0329@gmail.com>
Show more infors in the gc history, like the sweep size and how many blobs and manifests were removed by GC.
Signed-off-by: Wang Yan <wangyan@vmware.com>
1. Change the SCAN_ALL job execution retain counts from 5 to 1(per
current design, only one report be stored for every artifact, so
retain latest 1 is enough).
2. Enable the sweep for IMAGE_SCAN job(retain latest 1).
Fixes: #18633
Signed-off-by: chlins <chenyuzh@vmware.com>
Change the bitSize from 16 to 64 in the jobservice reaper, the 16 is too
small when the redis cursor over the max value of int16.
Fixes: #18486
Signed-off-by: chlins <chenyuzh@vmware.com>
Change the JobId param type from int to string, use int will bring some
problems for encode/decode type mismatch which generate the invalid
repository name.
Fixes: #18380
Signed-off-by: chlins <chenyuzh@vmware.com>
refactor: refactor the old goroutine execution sweep to global execution sweep job
1. Delete the old goroutine execution sweeper when create execution.(in the case of high concurrency can cause goroutine backlogs, affect the performance of core)
2. Introduce the new way to sweep executions, a global scheduled job will take the work.
Signed-off-by: chlins <chenyuzh@vmware.com>
Fixes#18121
Refactor job name with VendorType prefix, make sure job queue name and vendor type in execution and task are identical
Signed-off-by: stonezdj <daojunz@vmware.com>
Remove the project filter in the scan data export job as they have been
validated by API handler, fix the oidc or ldap group users cannot export
cve.
Fixes: #18112
Signed-off-by: chlins <chenyuzh@vmware.com>
fixes#17857
It needs to clean the artifact_blob records on artifact deletion in GC
Signed-off-by: Wang Yan <wangyan@vmware.com>
Signed-off-by: Wang Yan <wangyan@vmware.com>
1. Skip to push system artifact to the distribution when the exported CSV file is empty.
2. Add status message for cve export execution.
Signed-off-by: chlins <chenyuzh@vmware.com>
Add queue manager and redis client
Update scheduler to add count and list
Signed-off-by: stonezdj <daojunz@vmware.com>
Signed-off-by: stonezdj <daojunz@vmware.com>
Add REST API to list job pool, worker, stop running task
Add jobservice handler to retrieve configuration
Add RBAC for jobservice monitoring dashboard
Add REST API to list pool, worker and stop running task
Signed-off-by: stonezdj <stonezdj@gmail.com>
Signed-off-by: stonezdj <stonezdj@gmail.com>
Add human friendly when export CVE in the condition of empty CSV file,
because this file will be stored as system artifact and pushed to
distribution, but it will leads to error when push empty blob to S3
storage driver.
Signed-off-by: chlins <chenyuzh@vmware.com>
1. Add sql migration to alter replication policy table
2. Implement the PullBlobChunk and PushBlobChunk for the underlying v2 registry client
3. Update image transfer logic to support copy by chunk
4. Update the replication policy API handler
Signed-off-by: chlins <chenyuzh@vmware.com>
This commit replaces `os.Setenv` with `t.Setenv` in tests. The
environment variable is automatically restored to its original value
when the test and all its subtests complete.
Reference: https://pkg.go.dev/testing#T.Setenv
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
