Commit Graph

271 Commits

Author SHA1 Message Date
Yuan Lei
253600ad39 sync from upstream and merge the conflic
Signed-off-by: Yuan Lei <371304458@qq.com>
2019-02-27 10:52:03 +08:00
Wenkai Yin
79d34c6f80 Create dao folder in replication
This commit creates a new folder called dao under replication to hold dao codes

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-02-26 14:54:16 +08:00
Nguyen Quang Huy
eda6c47b3e add signoff for DCO gate (#6981)
Some variable name, function name is colliding with builtin function.

Signed-off-by: Nguyen Quang Huy <huynq0911@gmail.com>
2019-02-22 15:00:18 +08:00
Wenkai Yin
8894a27d2d Implement the operation controller
This commit implements the operation controller. The operation controller wraps the flow controller and execution manager to provide capabilities for the upper level

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-02-22 10:12:14 +08:00
Wenkai Yin
e3a9264162 Refactor the Scheduler interface
This commit refactors the Scheduler interface and does the corresponding changes to the flow controller

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-02-21 15:34:57 +08:00
Yuan Lei
7108b5c58c update scheduler interface to follow the new standard
Signed-off-by: Yuan Lei <371304458@qq.com>
2019-02-18 11:40:58 +08:00
Wenkai Yin
f0f2e77fb4 Implement the repository transfer
This commit implements the Transfer interface for resource repository

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-02-15 17:49:35 +08:00
Wenkai Yin
530ba1d27b Fix #6698
This commit fixes the issue #6698: cannot create a same name replication policy after deleting it

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-02-15 15:17:48 +08:00
Wenkai Yin
e12da7264a Implement the replication flow controller
1. Implement the replication flow controller
2. Implement the registry adapter management
3. Define the interface for PolicyManager, RegistryManager, ExecutionManager, TaskManager and Scheduler

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-02-14 11:34:12 +08:00
Yuan Lei
bd84f5f490 sperate the logics to two part: 1,tasks generating. 2,tasks submit
Signed-off-by: Yuan Lei <371304458@qq.com>
2019-02-13 17:47:07 +08:00
Yuan Lei
956a4c4f05 scheduler module to schedule the transfer job of resouce
Signed-off-by: Yuan Lei <371304458@qq.com>
2019-02-11 16:49:25 +08:00
Wenkai Yin
759cb5467b Refine the replication job
1. Refine the replication job
2. Define the Transfer interface
3. Define the common models

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-01-29 15:48:02 +08:00
Daniel Jiang
5d59d6fab8 Bump up golang to 1.11.2
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-01-11 14:44:32 +08:00
Steven Zou
ec2ad4d0b8
Merge pull request #6093 from cd1989/replication-record-id
Add op uuid to image replication
2018-11-30 14:54:43 +08:00
Wenkai Yin
88aa413fe6 Pass internal core URL as source in replication
Currently, we pass the external URL as the source registry and set insecure as true in replication, this causes security issue. This commit replaces the URL with the internal one and set insecure to false. Fixes #6109.

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2018-11-01 17:13:38 +08:00
Steven Zou
9bbc9233f1
Merge pull request #6147 from clouderati/fix-misspelling
Fix mispelling
2018-10-26 12:47:31 +08:00
clouderati
5c47e83bd6 Fix mispelling
* s/convertor/converter
* s/Convertor/Converter

Signed-off-by: clouderati <35942204+clouderati@users.noreply.github.com>
2018-10-25 21:29:46 +00:00
clouderati
b0d523841e Adding copyright header
Adding header to replication/models/policy.go file as it was previously
missing.

Signed-off-by: clouderati <35942204+clouderati@users.noreply.github.com>
2018-10-25 20:54:05 +00:00
陈德
1ffd9d8fba Add op uuid to image replication
Signed-off-by: 陈德 <chende@caicloud.io>
2018-10-21 23:55:57 +08:00
Daniel Jiang
b895a2d7cf Revoke the change in replicator
Revoke the change made to replicator to reset the insecure flag to true
when accessing source registry.  The reason is that the replicator will
use external URL to access source registry, setting it to false will
break replication when Harbor is hosting with self-signed certificate.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-10-11 18:29:38 +08:00
Daniel Jiang
1188bd89b9 Use secure transport to access HTTP endpoint
In various parts of the code, we used insecure transport in http Client
when we assume the endpoint is http.  This causes complaints form
security scanner.  We should use secure transport in such cases.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-10-10 17:51:02 +08:00
Daniel Jiang
b12dc3b5d8 Schedule "scan all" via jobservice
This commit leverage the jobservice to trigger "scan all" and
gets rid of the local scheduler to make the harbor-core container
stateless.
It keeps using the notifer mechanism to handle the configuration change.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-09-28 15:42:37 +08:00
clouderati
587459df15 Replacing copyright notices with "Copyright Project Harbor Authors".
Signed-off-by: clouderati <35942204+clouderati@users.noreply.github.com>
2018-09-19 16:59:36 +00:00
Qian Deng
7873a0312a Rename harbor-ui to harbor-core
1. Update the nginx.conf
2. Update Makefile
3. Update docker-compose
4. Update image name
5. Rename folder ui to core
6. Change the harbor-ui's package name to core
7. Remove unused static file on harbor-core
8. Remove unused code for harbor-portal

Signed-off-by: Qian Deng <dengq@vmware.com>
2018-09-19 16:35:13 +08:00
陈德
0582db9a82 Apply consistent format for comments
Signed-off-by: 陈德 <chende@caicloud.io>
2018-09-05 16:16:31 +08:00
wang yan
aab761ac8a Fix gofmt check results
Signed-off-by: wang yan <wangyan@vmware.com>
2018-08-29 11:50:00 +08:00
Yan
fca2bb3a6b
Fix misspell checking results (#5749)
Signed-off-by: wang yan <wangyan@vmware.com>
2018-08-29 10:25:42 +08:00
Daniel Jiang
dcf4e2ee78 Update import path in go code
vmware -> goharbor

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-08-23 17:50:53 +08:00
Wenkai Yin
4ecbf3c0f5 Update job status when failed to submit it
Set the status of replication job to "error" when the submitting to jobservice fails
2018-08-01 10:35:01 +08:00
Yan
efdb57548f
add admin job api (#5344)
It supports Harbor admin to trigger job either manual or
schedule. The job will be populated to job service to execute. 
The api includes:
1. POST /api/system/gc
2, GET /api/system/gc/:id 
3, GET /api/system/gc/:id/log
4, PUT/GET/POST /api/system/gc/schedule
2018-07-20 19:22:37 +08:00
Wenkai Yin 79628
ad62fffb0e Add label filter in replication policy API
The label filter can be used to filter images when do the replication, this commit provides the filter in replication policy
2018-05-21 15:01:17 +08:00
Steven Zou
3917512d3e
Merge pull request #4962 from ywk253100/180509_label_filter
Add label filter to filter chain
2018-05-15 20:12:34 +08:00
Wenkai Yin 79628
c1ebf0ba1e Add label filter to filter chain
Label filters are used to filter the resources according to the labels, adding the filters to the chain to enable the function.
2018-05-15 19:19:01 +08:00
Deng, Qian
43c8e9f589 Fix all tslint warnings
1. Fix tslint warnings
2. Add tslint to travis
2018-05-15 16:28:23 +08:00
Wenkai Yin
500651a5a1 Hide schedule job when listing replication jobs 2018-04-03 01:11:55 +08:00
Steven Zou
6c69a8cd05 Fix issue of stopping periodic job
improve op command by using cache
return 404 if no job found to stop
2018-04-02 18:08:03 +08:00
Steven Zou
fee7f6ddef Merge branch 'job_service' into switch_job_service 2018-04-02 12:06:23 +08:00
Steven Zou
35fe857b07 Fix nil pointer issue in replication/target 2018-04-02 11:12:20 +08:00
Wenkai Yin
dd40f187ec Move schedule replication job to new jobservice 2018-03-30 17:44:05 +08:00
Wenkai Yin
dd156ca243 Handle replication job status hook 2018-03-24 21:18:58 +08:00
Wenkai Yin
1fea594c3d Trigger replication job from UI with new jobservice 2018-03-24 18:10:50 +08:00
Wenkai Yin
838b439560 Implement filter repository and tags by label API 2018-03-21 10:51:06 +08:00
Wenkai Yin
611709a7be Add pagination support in listing replication policy API 2018-01-18 15:54:12 +08:00
Wenkai Yin
af8dbff4ce Update the value of trigger type and schedule type 2018-01-12 16:19:41 +08:00
Wenkai Yin
8e4e5e8b54 Add unit test cases for replication codes 2018-01-03 18:39:30 +08:00
Wenkai Yin
d36811ce52 Using the string trimmed the project name at the begining as the input of repository filter 2017-12-26 19:31:53 +08:00
Wenkai Yin
3be1d5a7fd Assign read-only privilege of replication policy to project admin and add stopping replication jobs API. 2017-12-25 23:49:21 +08:00
Wenkai Yin
a736cb7b09 Update the HTTP client according to the comments 2017-12-15 09:40:31 +08:00
Wenkai Yin
b5e7de331e Delete enabled and start_time properties of replication rule 2017-12-15 09:40:31 +08:00
Wenkai Yin
055ab0ba15 Refine replication schedule trigger API 2017-12-15 09:40:31 +08:00
Wenkai Yin
1d2e206ce4 Refactor replication filter chain 2017-12-15 09:40:31 +08:00
Wenkai Yin
76c88d8d49 Fix bugs of replicaton 2017-12-15 09:40:31 +08:00
Wenkai Yin
fe10c2e7f5 Create replicator to submit replication job to jobservice 2017-12-15 09:40:31 +08:00
Wenkai Yin
8b4fdfc2cc Add unit tests for replication related methods 2017-12-15 09:40:31 +08:00
Wenkai Yin
c5ccb7e53c Enable filter chain in replication 2017-12-15 09:40:30 +08:00
Wenkai Yin
594d213630 Publish replication notification for manual, scheduel and immediate trigger 2017-12-04 15:07:30 +08:00
Wenkai Yin
34f70ff3b6
Merge pull request #3693 from ywk253100/171127_schedual_trigger
Implement schedual trigger for replication
2017-11-28 13:43:01 +08:00
Wenkai Yin
4902fdf2ef Implement schedual trigger for replication 2017-11-27 16:50:35 +08:00
Wenkai Yin
6b0ee138e5 Implement immediate trigger and the methods of WatchList 2017-11-27 14:23:21 +08:00
Wenkai Yin
59c1160edd Setup/Unset trigger when CURD policies 2017-11-20 17:09:46 +08:00
Wenkai Yin
31cf6c078e Implement replication policy manager 2017-11-16 10:55:03 +08:00
Steven Zou
c2e0c8d1f2 Define the related interfaces for triggers and core controllers of replication service 2017-11-10 15:06:24 +08:00
Wenkai Yin
41c0ff66ce
Merge pull request #3534 from ywk253100/171101_poilicy
Update replication policy API to support trigger and filter
2017-11-10 14:00:17 +08:00
Wenkai Yin
ef0556d3f8
Merge pull request #3541 from ywk253100/171102_filter_chain
Implement filter chain
2017-11-10 10:42:25 +08:00
Wenkai Yin
3e3baa03f4 Merge remote-tracking branch 'upstream/replication_enhancement' into 171101_poilicy
Conflicts:
	src/replication/consts.go
2017-11-08 18:51:56 +08:00
Wenkai Yin
5cef58baa1 update according to the comments 2017-11-08 17:53:41 +08:00
Wenkai Yin
049642e894 Implement filter chain 2017-11-08 16:51:11 +08:00
Wenkai Yin
f86a4be84d Implement Convert interface 2017-11-07 14:34:12 +08:00
Steven Zou
abe3d37a33 Add registry adaptor interface and sourcer definitions 2017-11-02 17:10:15 +08:00
Wenkai Yin
82b400c049 Add replication filter implement 2017-11-01 14:59:44 +08:00
Steven Zou
e2242cffd2 define filter interfaces for replication service including:interface Filter,Converter and FilterChain and other supporting models 2017-10-27 18:16:16 +08:00