refactor: refactor the old goroutine execution sweep to global execution sweep job
1. Delete the old goroutine execution sweeper when create execution.(in the case of high concurrency can cause goroutine backlogs, affect the performance of core)
2. Introduce the new way to sweep executions, a global scheduled job will take the work.
Signed-off-by: chlins <chenyuzh@vmware.com>
Fixes#18121
Refactor job name with VendorType prefix, make sure job queue name and vendor type in execution and task are identical
Signed-off-by: stonezdj <daojunz@vmware.com>
Remove the project filter in the scan data export job as they have been
validated by API handler, fix the oidc or ldap group users cannot export
cve.
Fixes: #18112
Signed-off-by: chlins <chenyuzh@vmware.com>
fixes#17857
It needs to clean the artifact_blob records on artifact deletion in GC
Signed-off-by: Wang Yan <wangyan@vmware.com>
Signed-off-by: Wang Yan <wangyan@vmware.com>
1. Skip to push system artifact to the distribution when the exported CSV file is empty.
2. Add status message for cve export execution.
Signed-off-by: chlins <chenyuzh@vmware.com>
Add queue manager and redis client
Update scheduler to add count and list
Signed-off-by: stonezdj <daojunz@vmware.com>
Signed-off-by: stonezdj <daojunz@vmware.com>
Add REST API to list job pool, worker, stop running task
Add jobservice handler to retrieve configuration
Add RBAC for jobservice monitoring dashboard
Add REST API to list pool, worker and stop running task
Signed-off-by: stonezdj <stonezdj@gmail.com>
Signed-off-by: stonezdj <stonezdj@gmail.com>
Add human friendly when export CVE in the condition of empty CSV file,
because this file will be stored as system artifact and pushed to
distribution, but it will leads to error when push empty blob to S3
storage driver.
Signed-off-by: chlins <chenyuzh@vmware.com>
1. Add sql migration to alter replication policy table
2. Implement the PullBlobChunk and PushBlobChunk for the underlying v2 registry client
3. Update image transfer logic to support copy by chunk
4. Update the replication policy API handler
Signed-off-by: chlins <chenyuzh@vmware.com>
This commit replaces `os.Setenv` with `t.Setenv` in tests. The
environment variable is automatically restored to its original value
when the test and all its subtests complete.
Reference: https://pkg.go.dev/testing#T.Setenv
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
1. Validate export cve request params in the API handler
2. Trim space for request in the scan export job
Closes: #17326
Signed-off-by: chlins <chenyuzh@vmware.com>
1. Refact the scan export FilterProcessor interface
2. Optmize the sql template for export cve report
3. Update the process of cve export job
Signed-off-by: chlins <chenyuzh@vmware.com>
fix: fix cache layer issues (#16995,#16997,#16996,#17038)
1. Load config and initialize cache layer in jobservice(for GC)
2. Cache artifact by digest the key should contains repository name
3. Repository cache cleanup error when update
4. Skip save cache when request ctx in transaction
Signed-off-by: chlins <chenyuzh@vmware.com>
Make func defaultMgr() public
Set DefaultCfgManager to RestCfgManager and Load it in the jobservice main
config.WithNotary call DefaultMgr(), it wil get the RestCfgManager
Fixes#16418
Signed-off-by: stonezdj <stonezdj@gmail.com>
Chlins Zhang
Shengwen YU
stonezdj(Daojun Zhang)
Wang Yan
Noah Tang
Aleksandr Chebotov
Christian Zenker
MinerYang
yanggang
Eng Zer Jun
Gaius
prahaladdarkin
Loong Dai
Wenkai Yin(尹文开)