Commit Graph

33 Commits

Author SHA1 Message Date
MinerYang
b38de22054
bump golang 1.21.5 & fix golangci-lint error (#19722)
bump golang 1.21.5

update golangci-lint && fix revive error

fix white space lint

Signed-off-by: yminer <yminer@vmware.com>
2023-12-19 09:41:26 +08:00
MinerYang
ebac530b46
add goheader linter settings (#18503)
fix files for goheader linter

fix copyright 2018/2019

Signed-off-by: yminer <yminer@vmware.com>
2023-04-25 11:18:42 +08:00
Chlins Zhang
5791f0c979
fix: fix error bitsize of jobservice reaper scan locks (#18487)
Change the bitSize from 16 to 64 in the jobservice reaper, the 16 is too
small when the redis cursor over the max value of int16.

Fixes: #18486

Signed-off-by: chlins <chenyuzh@vmware.com>
2023-04-07 10:59:30 +08:00
stonezdj(Daojun Zhang)
46fa91f866
Fix dangling state execution (#18272)
Add max update hour and max dangling hour setting in config.yaml
  Fixes #17611

Signed-off-by: stonezdj <daojunz@vmware.com>
2023-03-21 18:48:14 +08:00
Loong Dai
09371b48e8
lint: sort imports (#17131)
* lint: add goimports

Signed-off-by: Loong Dai <loong.dai@intel.com>
2022-07-20 11:33:08 +08:00
Shengwen YU
c2a9f5de64
fix: update code for golangci-lint gosimple (#16974)
Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
2022-06-14 13:44:09 +08:00
chlins
3d7a5064e9 fix: trim linebreaks for log parameters
Signed-off-by: chlins <chenyuzh@vmware.com>
2022-02-16 10:52:58 +08:00
chlins
b37a987073 fix: fix codeql alerts
Signed-off-by: chlins <chenyuzh@vmware.com>
2022-02-15 17:25:02 +08:00
Steven Zou
bb7f706d78 fix(js): webhook infinitely resending issue
- update the resending logic in the hook agent
- use backoff lib to generate backoff duration
- remove retry cache queue
- leverage reaper to restore the sending if direct retries are all failed
- remove useless funcs/codes

fix #14545

Signed-off-by: Steven Zou <szou@vmware.com>
2021-04-08 19:59:16 +08:00
DQ
fbe9cd88f8 Enabled Prometheus for Jobservice
* Add prom server on jobservice
* Enabeld configs in templates
* Enabeld jobservice metrics in nginx

Signed-off-by: DQ <dengq@vmware.com>
2021-03-30 08:52:59 +00:00
shaobo
434758b71e reform job status compare method
Signed-off-by: shaobo <shaobo@caicloud.io>
2021-01-04 21:34:53 +08:00
Steven Zou
85783f3c0c fix(api):update stop job api
- update StopJob() of basic worker
- update UnSchedule() of basic scheduler
- update the policy store to get more data

fix #13599 , fix #13597

Signed-off-by: Steven Zou <szou@vmware.com>
2020-12-08 23:15:05 +08:00
Steven Zou
3cd47af9a5 fix(jobservice):mismatch status issue when stopping job
- returnning nil instead of error when trying to stop a job that has been in the final status(Error/Success/Stopped)
- do enhancements to the periodic job unschedule func
- fix a UT nil ptr issue

Signed-off-by: Steven Zou <szou@vmware.com>
2020-06-03 16:18:42 +08:00
Wang Yan
8a0e8627ff
replace pkg errors with lib errors (#11605)
Fixes #9704

As we do want to unify error handling, so just decreprates pkg errors, use lib/errors instead for Harbor internal used errors model.

1, The lib/errors can cover all of funcs of pkg/errors, and also it has code attribute to define the http return value.
2, lib/errors can give a OCI standard error format, like {"errors":[{"code":"UNAUTHORIZED","message":"unauthorized"}]}

If you'd like to use pkg/errors, use lib/errors instead. If it cannot meet your request, enhance it.

Signed-off-by: wang yan <wangyan@vmware.com>
2020-04-15 22:41:45 +08:00
Steven Zou
adb305e4e1
feat(job):enable priority option (#11608)
- priority option is supported when doing job registration
- the priority is defined by a unique priority sampler
- the default priority is 1000 (max is 10000)

Signed-off-by: Steven Zou <szou@vmware.com>
2020-04-14 14:54:44 +08:00
Steven Zou
3ad5b2ba06
fix(job currency):introduce max corrency of job (#11589)
- update Job interface to introdcue MaxCurrency method for declaring the max currency of the specified job
- change the downstream jobs to implement the new interface method
  - GC and sample jobs are set to 1
  - other jobs are set to 0 that means unlimited
- add max currency optiot when doing job registration
- resolve issue #11586
  - probably resolve issue #11281
  - resolve issue #11570

Signed-off-by: Steven Zou <szou@vmware.com>
2020-04-13 18:07:54 +08:00
Steven Zou
b2434945ff fix[js_ut]:update stop job case of js
remove the status checking loop

Signed-off-by: Steven Zou <szou@vmware.com>
2020-03-10 15:54:10 +08:00
Steven Zou
bd0e401cae fix[js]:fix ut case faulure
- refactor default context creation to avoid data race
- refactor the timer interval in c_worker UT cases to avoid receieving signals at the same time

Signed-off-by: Steven Zou <szou@vmware.com>
2020-03-03 11:47:41 +08:00
Wang Yan
948d45604c Revise the GC job flow,
1, set harbor to readonly
2, select the candidate artifacts from Harbor DB.
3, call registry API(--delete-untagged=false) to delete manifest bases on the results of #2
4, clean keys of redis DB of registry, clean artifact trash and untagged from DB.
5, roll back readonly.

Signed-off-by: wang yan <wangyan@vmware.com>
2020-02-24 18:29:55 +08:00
Steven Zou
e899d659f3 fix[jobservice]:job status is hung after restart
- improve the status hook sending/resending approach
- improve the status compare and set approach
- simplify the relevant flow
- add reaper to fix the out of sync jobs
- fix #10244 , fix #9963

Signed-off-by: Steven Zou <szou@vmware.com>
2020-01-09 17:10:35 +08:00
Steven Zou
718e336302 reduce the expire time of job stats in redis DB
Signed-off-by: Steven Zou <szou@vmware.com>
2019-08-22 18:26:09 +08:00
Steven Zou
926ac44a67 fix #8525: periodic job retry issue: job stats is not found
Signed-off-by: Steven Zou <szou@vmware.com>
2019-08-21 16:30:47 +08:00
Steven Zou
ace07b0c0a fix #8745: job retry issue
Signed-off-by: Steven Zou <szou@vmware.com>
2019-08-20 16:39:44 +08:00
Wenkai Yin
8325f0769f Ignore the arguments when logging the job detail
As the arguments of a job may contain sensitive information, ignore them when logging the job detail. Fixes #8404

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-08-06 15:02:14 +08:00
Steven Zou
b9f45cd30a fix failure UT case of job service
Signed-off-by: Steven Zou <szou@vmware.com>
2019-06-21 13:31:27 +08:00
Steven Zou
e091916d54 fix UT case failure (#7734)
Signed-off-by: Steven Zou <szou@vmware.com>
2019-05-10 13:15:44 +08:00
Steven Zou
f2870272ce add get jobs API to provide powerful job stats management
Signed-off-by: Steven Zou <szou@vmware.com>
2019-04-28 22:36:13 +08:00
Steven Zou
9bcbe2907b fix go vet issues in the code
Signed-off-by: Steven Zou <szou@vmware.com>
2019-04-24 07:31:37 +08:00
Steven Zou
fae5d1304d add more/refacotr current test cases of job service
Signed-off-by: Steven Zou <szou@vmware.com>
2019-04-22 19:24:37 +08:00
Steven Zou
ea34bcc950 add retry for status switching
Signed-off-by: Steven Zou <szou@vmware.com>
2019-04-19 21:14:09 +08:00
Steven Zou
38e6df4edb fix code style issues: missing comments
Signed-off-by: Steven Zou <szou@vmware.com>
2019-04-19 14:16:08 +08:00
Steven Zou
f8feaa192e add get scheduled and periodic executions APIs
Signed-off-by: Steven Zou <szou@vmware.com>
2019-04-19 13:54:23 +08:00
Steven Zou
1f481e492c Refactor job servcie primary logic to fix related bugs
Signed-off-by: Steven Zou <szou@vmware.com>
2019-04-18 16:02:49 +08:00