fix: increase beego max memory and upload size (#19578)
1. Increase the default beego max memory and upload size from 32GB to
128GB.
2. Support customize the two beego configs from env.
(cherry picked from commit 553c85eed0)
Signed-off-by: chlins <chenyuzh@vmware.com>
Signed-off-by: stonezdj <daojunz@vmware.com>
Co-authored-by: Chlins Zhang <chenyuzh@vmware.com>
Co-authored-by: Wang Yan <wangyan@vmware.com>
Remove job status track information from redis after stop the job in the queue
After stop in the queue:
Remove key in {harbor_job_service_namespace}:job_track:inprogress
Remove {harbor_job_service_namespace}:job_stats:<job_id>
fixes#19211
Signed-off-by: stonezdj <daojunz@vmware.com>
Support to configure the customized redis db for cache layer and other
misc business for core, by default the behavior is same with
previous(stored in db 0).
Signed-off-by: chlins <chenyuzh@vmware.com>
refactor: migrate the redis command keys to scan
Refine the cache interface, migrate the Keys to Scan, change the redis
underlying keys command to scan.
Signed-off-by: chlins <chenyuzh@vmware.com>
fix: fix replication list projects with pure numberic name
Quote the project name when listing projects in the replication, resolve
the issue of pure number name of project.
Signed-off-by: chlins <chenyuzh@vmware.com>
fix accessory import issue
fixes#19041
The issuse was introduced by https://github.com/goharbor/harbor/pull/17087.
To move the import to main since the anonymous imports only allowed at main in go-cilint.
Fixes it:
To add the anonymous imports at the jobservice main since the gc job needs to remove accessory when to delete an artifact
Signed-off-by: wang yan <wangyan@vmware.com>
Return error when proxy cache get too many request error(429)
Add 429 too many request error in http error
Fixes#18707
Signed-off-by: stonezdj <stonezdj@gmail.com>
fix: improve the performance of list artifacts
1. Change the query for listing tasks of scan which can use the db
index.
2. Add the gin index for task.extra_attrs.report_uuids
Fixes: #18013
Signed-off-by: chlins <chenyuzh@vmware.com>
fix: update stopped execution status immediately instead of async update
Update the stopped execution status immediately becasue the user
experience is not good if wait long time when stop or reschedule a job.
Fixes: #18526
Signed-off-by: chlins <chenyuzh@vmware.com>
fix: clean the redis if the execution is not found
Delete the execution outdated status key in the redis when the execution
is not found.
Fixes: #18511
Signed-off-by: chlins <chenyuzh@vmware.com>
fix: fix error bitsize of jobservice reaper scan locks
Change the bitSize from 16 to 64 in the jobservice reaper, the 16 is too
small when the redis cursor over the max value of int16.
Fixes: #18486
Signed-off-by: chlins <chenyuzh@vmware.com>
Refactor the logic for updating the status of execution when receiving
the hook from jobservice, avoid the optimistic lock due to the multiple
tasks update one execution by refreshing the status asynchronously. But
still retain the old way by specifying the flag from ENV.
Fixes: #17584
Signed-off-by: chlins <chenyuzh@vmware.com>
Wrap orm tx when the scheduler try to create the task because submit job
maybe failure depends on the jobservice.
Fixes: #18452
Signed-off-by: chlins <chenyuzh@vmware.com>
Fix the incorrect number of total and retained in the retention webhook
payload, and completes the deleted_artifacts field.
Fixes: #18428
Signed-off-by: chlins <chenyuzh@vmware.com>
1. extend the width for event type label
2. add quotes for Retain image last pull time to make it more understandable
Signed-off-by: AllForNothing <sshijun@vmware.com>
1. Change the the value of CloudEvents id from requestid to uuid
2. Add additional requestid to trace to event
Fixes: #18448
Signed-off-by: chlins <chenyuzh@vmware.com>
Co-authored-by: Wang Yan <wangyan@vmware.com>
To enable the middleware to save the project_blob data, make sure to set the accessories options to true when handling the artifact copy.
Signed-off-by: Wang Yan <wangyan@vmware.com>
1. Add migration SQL to handle the lost payload format for old policies.
2. Set payload format to 'Default' if not specified for http webhook in the API handler.
3. Fix the migration sql of notification_job
Fixes: #18401, #18453
Signed-off-by: chlins <chenyuzh@vmware.com>
1. Show a app level warning if there is a stuck job
2. Chang `Replication finished` to `Replication status changed`
Signed-off-by: AllForNothing <sshijun@vmware.com>
Change the JobId param type from int to string, use int will bring some
problems for encode/decode type mismatch which generate the invalid
repository name.
Fixes: #18380
Signed-off-by: chlins <chenyuzh@vmware.com>
1, add fitler artifactType to header when the api is called with filter
2, give an empty json body on non aritfact scenario
3, give an empty array on non accessory scenario
4, fix the artifact type filter issue
Signed-off-by: Wang Yan <wangyan@vmware.com>
Shengwen YU
stonezdj(Daojun Zhang)
Shijun Sun
Chlins Zhang
Wang Yan
Danny Kulchinsky
Viktor
MinerYang
Orlix