Commit Graph

20 Commits

Author SHA1 Message Date
Chlins Zhang f21b1481bb
refactor: refactor the old goroutine execution sweep (#18361)
refactor: refactor the old goroutine execution sweep to global execution sweep job

1. Delete the old goroutine execution sweeper when create execution.(in the case of high concurrency can cause goroutine backlogs, affect the performance of core)
2. Introduce the new way to sweep executions, a global scheduled job will take the work.

Signed-off-by: chlins <chenyuzh@vmware.com>
2023-03-21 10:10:55 +08:00
stonezdj(Daojun Zhang) ceb130e617
Avoid internal error in the UI when reset a schedule (#17946)
fixes #17926

Signed-off-by: stonezdj <daojunz@vmware.com>

Signed-off-by: stonezdj <daojunz@vmware.com>
2022-12-08 20:43:44 +08:00
Chenyu Zhang 24d60eb119
enhancement: add error log for create task (#17397)
Signed-off-by: chlins <chenyuzh@vmware.com>

Signed-off-by: chlins <chenyuzh@vmware.com>
2022-08-16 11:35:05 +08:00
chlins ed4d9a533e fix: add worker pool for execution sweep to limit workers
Signed-off-by: chlins <chenyuzh@vmware.com>
2022-01-20 10:45:00 +08:00
Wenkai Yin(尹文开) b2f0a1f0f5
Merge pull request #14400 from ywk253100/210226_registry
Refeactor replication policy APIs
2021-03-22 15:02:46 +08:00
Wenkai Yin 8b1817be0f Fix the consume too much CPU issue
1. Update execution status during the upgrade
2. Refine the execution sweeper

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-03-17 13:34:24 +08:00
Wenkai Yin 3d7fd070c7 Refeactor replication policy APIs
Refeactor replication policy APIs

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-03-17 10:57:31 +08:00
Wenkai Yin 43df3bf8a4 Add upgrade sql file introduced in 2.1.4
1. Add upgrade sql file introduced in 2.1.4
2. Minor improvement for task/execution to cover corner cases

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-03-15 16:48:51 +08:00
He Weiwei de97b900cf
fix: remove default execution sweeper count (#14168)
1. Remove the default execution sweeper count for execution vendor.
2. Set the execution sweeper count for gc, preheat, replication,
retention to 50.
3. Disable sweep for the executions of the scan job.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-02-05 12:15:15 +08:00
Wenkai Yin 027c80ea58 Ignore the not found error in some cases
Ignore the not found error in some cases
Fixes #14154

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-02-03 20:26:50 +08:00
Wenkai Yin 38e0f102bf Query executions with both vendor type and ID when sweep the execution records
Query executions with both vendor type and ID when sweep the execution records

Fixes #13996

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-01-15 20:23:55 +08:00
He Weiwei 9402077695
feat: save summary for the scan all execution (#13931)
Compute the summary info for the scan all and save it to the extra attrs
of the execution.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-01-08 10:10:31 +08:00
Wenkai Yin fb871dbbe8 Provide a mechanism to sweep the execution/task records in task manager
Provide a mechanism to sweep the execution/task records in task manager
Fixes #13888

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-01-06 16:41:46 +08:00
Wenkai Yin 69808f033e Tiny fixes for task manager
1. Add update time for execution
2. Add unique constraint for schedule to avoid dup records when updating policies
3. Format replication log
4. Keep the webhook handler for legacy replication jobs to avoid jobservice resending the status change request

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-12-14 17:26:32 +08:00
Wenkai Yin 294385c34d Refactor the replication execution
1. Use the task manager to manage the underlying execution/task
2. Use the pkg/scheduler to schedule the periodical job
3. Apply the new program model
4. Migration the old data into the new data model

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-11-23 14:24:10 +08:00
Wenkai Yin 1a4106a996 Tiny improvement for the task manager
Add a new method "StopAndWait" which stops the execution and wait until the execution stopped or get an error

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-09-23 15:24:26 +08:00
Wenkai Yin 0fd230c2d6 Refresh the status of execution for every status changing of task
Refresh the status of execution for every status changing of task to support filtering executions by status directly

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-08-17 17:38:55 +08:00
chlins 302e4c6659 fix(preheat): add pagination for execution and task list api
Signed-off-by: chlins <chlins.zhang@gmail.com>
2020-08-06 16:59:38 +08:00
Wenkai Yin ea20690264 Implement task and execution manager
Implement task and execution manager

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-06-23 17:10:58 +08:00
Wenkai Yin 127988b70c Define the task manager interface and data model
Define the task manager interface and data model

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-06-15 18:42:09 +08:00