Commit Graph

68 Commits

Author SHA1 Message Date
Wenkai Yin
1d16e18dff Remove "ng" from source code
Remove "ng" from source code

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-04-12 22:42:27 +08:00
慕薇疯魔
a555b09df0 1. Implement native docker registry adapter.
2. register the native adapter.

Signed-off-by: 慕薇疯魔 <kfanjian@gmail.com>
2019-04-12 21:08:23 +08:00
cd1989
b876a576a6 Implement dockerhub adapter
Signed-off-by: cd1989 <chende@caicloud.io>
2019-04-12 17:31:32 +08:00
Wenkai Yin
b73acde051 Support the migration for scheduled replication rule from previous version of Harbor
Support the migration for scheduled replication rule from previous version of Harbor

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-04-11 13:14:32 +08:00
Wenkai Yin
4f8e283e8e Add event based trigger and scheduled trigger
This commit implements the event based trigger and scheduled trigger in replilcation

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-03-29 13:48:34 +08:00
Wenkai Yin
258b22a9a5 Fix bug in replication
This commit fixes bugs found in the implement of replciation NG

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-03-10 20:57:59 +08:00
Wenkai Yin
772367498f Merge remote-tracking branch 'upstream/master' into 190311_sync 2019-03-11 20:34:49 +08:00
He Weiwei
053a18a490 Skip transfer blob for foreign layers
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-02-27 16:14:17 +08:00
Wenkai Yin
95888b3dc2
Merge branch 'replication_ng' into 190130_transfer_repo 2019-02-27 11:00:42 +08:00
Wenkai Yin
0c0c833dda Merge remote-tracking branch 'upstream/master' into 190220_sync 2019-02-20 17:21:38 +08:00
stonezdj
1ae5126bb4 Refactor adminserver stage 3: replace config api and change ut settings
Signed-off-by: stonezdj <stonezdj@gmail.com>
2019-02-18 14:06:19 +08:00
Wenkai Yin
f0f2e77fb4 Implement the repository transfer
This commit implements the Transfer interface for resource repository

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-02-15 17:49:35 +08:00
Wenkai Yin
759cb5467b Refine the replication job
1. Refine the replication job
2. Define the Transfer interface
3. Define the common models

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-01-29 15:48:02 +08:00
Wenkai Yin
5388eb4db6 Read ssl mode param from the adminserver to init the connection with database (#6504)
This commit adds the ssl mode as a param when establishing the connection with database

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2018-12-11 13:58:37 +08:00
peimingming
238dbc0347 Add UT and review comments and issue fix (#6144)
Signed-off-by: peimingming <peimingming@corp.netease.com>
2018-11-28 17:43:14 +08:00
peimingming
c67fdc40f5 Support store job log in DB (#6144)
Signed-off-by: peimingming <peimingming@corp.netease.com>
2018-11-28 15:09:29 +08:00
Steven Zou
e6d4c024ee Update README of job service to reflect latest updates
Signed-off-by: Steven Zou <szou@vmware.com>
2018-11-08 10:35:12 +08:00
Steven Zou
7b106d06c5 Build logger framework to support configurable loggers/sweepers/getters
Signed-off-by: Steven Zou <szou@vmware.com>
2018-11-06 09:31:31 +08:00
Wenkai Yin
af17c2c3fd
Merge pull request #6154 from wy65701436/fix-gc-redis
Fix gc jobs issues
2018-10-29 16:29:57 +08:00
wang yan
903d3906ab Fix per comments by wk
Signed-off-by: wang yan <wangyan@vmware.com>
2018-10-29 15:29:14 +08:00
Steven Zou
d32e51052e Fix the issue of duplicated periodic job scheduling
Signed-off-by: Steven Zou <szou@vmware.com>

Signed-off-by: Steven Zou <szou@vmware.com>
2018-10-29 14:52:45 +08:00
wang yan
566169f59a Fix gc jobs issues.
This commit is to fix two problems in the gc job,
1, in order to support redis cluseter, gc job should delete the keys only
for docker reigstry instead of all keys. It because that in the redis cluster,
only db 0 is supported.
2, to restore the real status of read only not just to set it to false.

Signed-off-by: wang yan <wangyan@vmware.com>
2018-10-26 18:37:20 +08:00
Steven Zou
7ca2e9df81 Enhance job service to support multiple executions rectording
- print more logs
- add execution records for periodic job
- add launch job func in job context to launch sub job
- replace copyright header with new version

Signed-off-by: Steven Zou <szou@vmware.com>
2018-10-19 11:02:09 +08:00
Steven Zou
326e0d455a Log error if failed to create logger for job
Signed-off-by: Steven Zou <szou@vmware.com>
2018-10-11 16:29:00 +08:00
Daniel Jiang
1188bd89b9 Use secure transport to access HTTP endpoint
In various parts of the code, we used insecure transport in http Client
when we assume the endpoint is http.  This causes complaints form
security scanner.  We should use secure transport in such cases.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-10-10 17:51:02 +08:00
Daniel Jiang
0699980924 Add Scan All job to job service (#5934)
This commit adds the job to scan all images on registry.
It also makes necessary change to Secret based security context, to
job service has higher permission to call the API of core service.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-09-22 13:07:32 +08:00
clouderati
587459df15 Replacing copyright notices with "Copyright Project Harbor Authors".
Signed-off-by: clouderati <35942204+clouderati@users.noreply.github.com>
2018-09-19 16:59:36 +00:00
Qian Deng
7873a0312a Rename harbor-ui to harbor-core
1. Update the nginx.conf
2. Update Makefile
3. Update docker-compose
4. Update image name
5. Rename folder ui to core
6. Change the harbor-ui's package name to core
7. Remove unused static file on harbor-core
8. Remove unused code for harbor-portal

Signed-off-by: Qian Deng <dengq@vmware.com>
2018-09-19 16:35:13 +08:00
陈德
0582db9a82 Apply consistent format for comments
Signed-off-by: 陈德 <chende@caicloud.io>
2018-09-05 16:16:31 +08:00
wang yan
aab761ac8a Fix gofmt check results
Signed-off-by: wang yan <wangyan@vmware.com>
2018-08-29 11:50:00 +08:00
Yan
fca2bb3a6b
Fix misspell checking results (#5749)
Signed-off-by: wang yan <wangyan@vmware.com>
2018-08-29 10:25:42 +08:00
Daniel Jiang
dcf4e2ee78 Update import path in go code
vmware -> goharbor

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-08-23 17:50:53 +08:00
wangyan
29d0d51403 Signed-off-by: wangyan <wangyan@vmware.com>
Add clean registry cache to gc job

To workaround the issue: https://github.com/docker/distribution/issues/2094
GC needs to clean cache before to call the docker reigstry api to delete blobs.
Otherwise, the following docker push will not be performed as docker registry
does not clean cache in GC, it thinks the image is still there, and the new
blobs will be uploaded.
2018-08-13 02:58:27 -07:00
Yan
efdb57548f
add admin job api (#5344)
It supports Harbor admin to trigger job either manual or
schedule. The job will be populated to job service to execute. 
The api includes:
1. POST /api/system/gc
2, GET /api/system/gc/:id 
3, GET /api/system/gc/:id/log
4, PUT/GET/POST /api/system/gc/schedule
2018-07-20 19:22:37 +08:00
Yan
9e65499c10
Add garbage collection job implemention, this job could (#5268)
be triggered by manual and schedule. It calls registrtctl
to do the GC job, and log the output.
2018-07-16 18:08:40 +08:00
Steven Zou
4db708096b Add retry for getting configs from admin server when doing job conetxt initialization 2018-05-10 15:26:59 +08:00
Yan
ae257433cc
Fully migrate harbor db to postgresql (#4689)
* Merge harbor db to postgres
2018-04-27 02:27:12 -07:00
Steven Zou
a17436962f Return the logger.Interface instead of the pointer of logger to avoid nil checking issue 2018-04-20 14:59:28 +08:00
Steven Zou
b5b728bee3 Fix the vulnerability issues in the job service code
'Errors unhandled' in 'jobservice/job/impl/logger/job_logger.go'
'TLS InsecureSkipVerify set true' in 'jobservice/opm/hook_client.go'
2018-04-12 15:12:39 +08:00
Steven Zou
529ad3e079
Merge pull request #4593 from vmware/add_more_log_2_js
Fix issue: failed to update the status of job if runtme error occurred
2018-04-08 14:03:34 +08:00
Steven Zou
1cb40368f5 Fix issue: failed to update the status of job if runtme error occurred 2018-04-08 10:38:47 +08:00
Tan Jiang
ff06ec05c3 Store secret in header instead of cookie 2018-04-07 22:02:06 +08:00
Tan Jiang
15580a5e8c Read the system properties from scan job context 2018-04-04 19:58:54 +08:00
Steven Zou
6c69a8cd05 Fix issue of stopping periodic job
improve op command by using cache
return 404 if no job found to stop
2018-04-02 18:08:03 +08:00
Steven Zou
fee7f6ddef Merge branch 'job_service' into switch_job_service 2018-04-02 12:06:23 +08:00
Steven Zou
013028ef4a Replace old job service package with V2 (rename packages) 2018-03-30 11:22:29 +08:00
Steven Zou
bfbb949034 Use the new job service(v2) to replace the old one 2018-03-29 23:30:03 +08:00
Wenkai Yin
9022abfc13 Fix code issues found by Gas 2018-01-29 15:17:03 +08:00
Wenkai Yin
4070ed5152 Provide a mechanism to stop pending and retrying jobs 2018-01-12 15:29:20 +08:00
Wenkai Yin
b5e7de331e Delete enabled and start_time properties of replication rule 2017-12-15 09:40:31 +08:00