Commit Graph

13 Commits

Author SHA1 Message Date
Chlins Zhang f21b1481bb
refactor: refactor the old goroutine execution sweep (#18361)
refactor: refactor the old goroutine execution sweep to global execution sweep job

1. Delete the old goroutine execution sweeper when create execution.(in the case of high concurrency can cause goroutine backlogs, affect the performance of core)
2. Introduce the new way to sweep executions, a global scheduled job will take the work.

Signed-off-by: chlins <chenyuzh@vmware.com>
2023-03-21 10:10:55 +08:00
Loong Dai 09371b48e8
lint: sort imports (#17131)
* lint: add goimports

Signed-off-by: Loong Dai <loong.dai@intel.com>
2022-07-20 11:33:08 +08:00
chlins ed4d9a533e fix: add worker pool for execution sweep to limit workers
Signed-off-by: chlins <chenyuzh@vmware.com>
2022-01-20 10:45:00 +08:00
Wenkai Yin 43df3bf8a4 Add upgrade sql file introduced in 2.1.4
1. Add upgrade sql file introduced in 2.1.4
2. Minor improvement for task/execution to cover corner cases

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-03-15 16:48:51 +08:00
He Weiwei de97b900cf
fix: remove default execution sweeper count (#14168)
1. Remove the default execution sweeper count for execution vendor.
2. Set the execution sweeper count for gc, preheat, replication,
retention to 50.
3. Disable sweep for the executions of the scan job.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-02-05 12:15:15 +08:00
He Weiwei 9402077695
feat: save summary for the scan all execution (#13931)
Compute the summary info for the scan all and save it to the extra attrs
of the execution.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-01-08 10:10:31 +08:00
Wenkai Yin fb871dbbe8 Provide a mechanism to sweep the execution/task records in task manager
Provide a mechanism to sweep the execution/task records in task manager
Fixes #13888

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-01-06 16:41:46 +08:00
Wenkai Yin 69808f033e Tiny fixes for task manager
1. Add update time for execution
2. Add unique constraint for schedule to avoid dup records when updating policies
3. Format replication log
4. Keep the webhook handler for legacy replication jobs to avoid jobservice resending the status change request

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-12-14 17:26:32 +08:00
Wenkai Yin 294385c34d Refactor the replication execution
1. Use the task manager to manage the underlying execution/task
2. Use the pkg/scheduler to schedule the periodical job
3. Apply the new program model
4. Migration the old data into the new data model

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-11-23 14:24:10 +08:00
Wenkai Yin 1a4106a996 Tiny improvement for the task manager
Add a new method "StopAndWait" which stops the execution and wait until the execution stopped or get an error

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-09-23 15:24:26 +08:00
Wenkai Yin 0fd230c2d6 Refresh the status of execution for every status changing of task
Refresh the status of execution for every status changing of task to support filtering executions by status directly

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-08-17 17:38:55 +08:00
chlins 302e4c6659 fix(preheat): add pagination for execution and task list api
Signed-off-by: chlins <chlins.zhang@gmail.com>
2020-08-06 16:59:38 +08:00
Wenkai Yin ea20690264 Implement task and execution manager
Implement task and execution manager

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-06-23 17:10:58 +08:00