Commit Graph

730 Commits

Author SHA1 Message Date
wang yan
4763864dae merge with latest master code with quota feature branch
Signed-off-by: wang yan <wangyan@vmware.com>
2019-07-24 08:47:05 -07:00
Ziming
43c2af9857 map retention with policy (#8313)
Signed-off-by: Ziming Zhang <zziming@vmware.com>

Implement the API and controller of tag retention
 - API handler
 - retention controller
 - dao
2019-07-24 17:22:26 +08:00
He Weiwei
ce58c58c01 feat(quota,api): quota support for create project API
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-07-24 01:02:51 +08:00
Wenkai Yin
03cc8046eb Implement the task management in retention manager
Implement the task management in retention manager

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-07-23 19:23:04 +08:00
Wenkai Yin
7362fae7cc Implement a common scheduler
Implement a common scheduler that can be used globally

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-07-23 17:20:31 +08:00
He Weiwei
9c9b8d3a6d Merge branch 'master' into project-quota-dev 2019-07-19 10:02:51 +08:00
Steven Zou
746d082e2e Merge branch 'master' into feature/tag_retention 2019-07-18 10:40:49 +08:00
Daniel Jiang
e0e6a1d30b
Merge pull request #8301 from ninjadq/external_endpoint_support
Add supoort for external endpoint
2019-07-18 01:36:08 +08:00
Qian Deng
739b4723db
Merge pull request #8308 from ninjadq/upgrade_chartmuseum_2_v0_9_0
Upgrade chartmuseum from v0.8.1 to v0.9.0
2019-07-17 16:48:52 +08:00
DQ
6cf4596292 Add supoort for external endpoint
Add config item in harbor.yml
Make fowarding rule configurable

Signed-off-by: DQ <dengq@vmware.com>
2019-07-17 16:23:37 +08:00
Wenkai Yin(尹文开)
a64e089773
Merge pull request #8210 from stonezdj/http_group_dao2
Add HTTP group support
2019-07-17 15:22:36 +08:00
Qian Deng
5cd3594f20 Upgrade chartmuseum from v0.8.1 to v0.9.0
Signed-off-by: Qian Deng <dengqian0826@gmail.com>
2019-07-17 06:45:23 +00:00
Ziming Zhang
815901ea33 fix
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I3f2d3c7f1e32b4983c31c23d9753f04239e3c82f
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2019-07-16 19:24:40 +08:00
stonezdj
bb2ae7c093 Add HTTP group feature
Signed-off-by: stonezdj <stonezdj@gmail.com>
2019-07-16 15:38:46 +08:00
wang yan
105518fe12 Merge branch 'master' of https://github.com/goharbor/harbor into project-quota-dev 2019-07-16 14:51:37 +08:00
Amritanshu Pandey
bde54cf938 fix typos
Signed-off-by: Amritanshu Pandey <amp.msit@gmail.com>
2019-07-15 04:28:10 +00:00
wang yan
24c3753581 add dao of artifact
Signed-off-by: wang yan <wangyan@vmware.com>

Add dao for quota

Signed-off-by: He Weiwei <hweiwei@vmware.com>

fix govet

Signed-off-by: wang yan <wangyan@vmware.com>
2019-07-08 23:42:50 +08:00
Ronald van Butselaar
7cd29b399a Add SELinux label to all volumes inside prepare script
Signed-off-by: Ronald van Butselaar <ronald@vbutselaar.nl>
2019-06-28 16:21:39 +02:00
Steven Zou
5521b7b7ad
Merge pull request #7915 from bitsf/replication_ecr_1.9
aws driver for replication
2019-06-27 11:24:54 +08:00
Daniel Jiang
4aca812ff2 API for system level vulnerability whitelist
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-06-26 23:35:40 +08:00
Ziming Zhang
072bdd101b aws driver for replication
Change-Id: I8792ffce2eaa5975359bb6159a1ba7b85926a925
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2019-06-25 19:11:27 +08:00
stonezdj
a8cd1bca59 Change the mount target of gcs.key file
Signed-off-by: stonezdj <stonezdj@gmail.com>
2019-06-16 13:34:32 +08:00
Wenkai Yin
1ceb7a2fb9
Merge pull request #7825 from ninjadq/update_installation_doc
Update doc caused by refactor prepare
2019-05-17 10:02:47 +08:00
Qian Deng
48151f6d46 Update doc caused by refactor prepare
Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-16 23:01:12 +08:00
Wenkai Yin
64cacc99e0
Merge pull request #7750 from liqiang-fit2cloud/fix-7288
Fix issue: harbor 1.7.4 aliyun oss chartmuseum 500
2019-05-16 18:23:35 +08:00
Qian Deng
f75a60f300 Fix typo
Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-16 15:15:42 +08:00
Qian Deng
f4ac7f9b4a
Merge pull request #7816 from ninjadq/fix_typo_of_azure
fix typo of azure config
2019-05-16 10:53:22 +08:00
Qian Deng
ea889d5a50 Fix typo in azure config
Fix typo in chart azure

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-15 18:25:34 +08:00
Wenkai Yin
1a8a55855b Add "MaxMessageSize" to the config of rsyslogd
Add "MaxMessageSize" to the config of rsyslogd to avoid the mess of log file when the size of one log line > 8k

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-05-15 18:10:22 +08:00
Qian Deng
1677686140 Made logs in jobservice configurable
Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-15 10:57:42 +08:00
Qian Deng
4188c4db76
Merge pull request #7719 from ninjadq/fix_chart_relative_url_issue
Fix chart relative url issue
2019-05-15 10:02:07 +08:00
stonezdj(Daojun Zhang)
47f24cab4b
Merge pull request #7770 from ninjadq/fix_typo_in_registry_config
Typo in registry config
2019-05-14 13:58:03 +08:00
Qian Deng
f607c5177d Fix frontend failure caused by absolute path
Fix failures because front downlowd chart using relative path

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-14 13:22:06 +08:00
Qian Deng
3022b617f2 Add chart absolute url item in config
Add a config item to enable and disalbe chart_url

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-14 12:56:20 +08:00
Qian Deng
cd6c5a9f10 Enable absolute url in helm chart
assign public_url to chart-url
remove namespace merge in index.yaml

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-14 12:56:20 +08:00
Qian Deng
41e399dec0 Fix issue caused by notary default_alias
Fix notary issue

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-13 18:43:15 +08:00
Qian Deng
6db39f9c71 Typo in registry config
it should be disable not disabled

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-10 23:08:21 +08:00
Wenkai Yin
a2e7692f10 Document how to use the external database in installation guide (#7761)
Document how to use the external database in installation guide. Fixes #7189

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-05-10 17:16:58 +08:00
Wang Yan
e7f5954c71
Fix migrator scan all schedule from v1.7.0 (#7763)
In v1.7.0, it creates an record for scan all but without cron string, just update the record with the cron in properties.

Signed-off-by: wang yan <wangyan@vmware.com>
2019-05-10 17:08:52 +08:00
Qian Deng
439b44c61f Fix public url shoud not display port is it's default value (#7760)
if https port is 443 or http port is 80, then only showing url

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-10 16:53:25 +08:00
Qian Deng
eba20baba5
Merge pull request #7612 from ninjadq/fix_tls_related_issues
Fix tls related issues
2019-05-10 16:36:51 +08:00
Daniel Jiang
0b0fad04e9
Merge pull request #7757 from reasonerjt/install-sh-chk
Fix a bug of checking docker version in install.sh
2019-05-10 15:42:34 +08:00
Qian Deng
d255e66604 Remove -it in docker run
Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-10 15:31:58 +08:00
Qian Deng
286167ad74
Merge pull request #7755 from ninjadq/fix_rendering_none_in_jinja
Fix None rendered in jinja2
2019-05-10 14:59:21 +08:00
Daniel Jiang
b802f14e1f Fix a bug of checking docker version in install.sh
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-05-10 13:55:04 +08:00
Wang Yan
ef4afff8b0
Update default value of external_database (#7756)
Signed-off-by: wang yan <wangyan@vmware.com>
2019-05-10 13:41:47 +08:00
Qian Deng
f9f9661acd New type of bind volume
using long style bind volume

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-10 13:18:48 +08:00
Qian Deng
cd9932db23 Update the path of server.key and server.crt
change the path of cert key paris to prevent futrue issues.

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-10 13:18:48 +08:00
Qian Deng
3dfebed98e Enhance: Add an empty cert files if not exist
To avoid confusion error message

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-10 13:18:48 +08:00
Qian Deng
6ea0514bb7 Revert "Set default mode to https"
This reverts commit 8a308a63e2.

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-10 13:18:48 +08:00
Qian Deng
96bb638067
Merge pull request #7749 from reasonerjt/install-sh-chk
Update the version check in install.sh
2019-05-10 12:56:37 +08:00
Qian Deng
0aaccf62b2 Fix None rendered in jinja2
jinja2 render None to empty string

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-10 12:17:02 +08:00
Qian Deng
bb66358df8 Update migratrion script (#7728)
* Fix migration script

1. port is string when parsed from configparser
2. remove index and db_user in if condition

Signed-off-by: Qian Deng <dengq@vmware.com>

* Add port to public_url

Add port to public_url

Signed-off-by: Qian Deng <dengq@vmware.com>

* Customized value for notary and clair

db config in notary and clair is hardcoded

Signed-off-by: Qian Deng <dengq@vmware.com>

* Add notary and clair db config in harbor.yml

Add notary clair config to harbor.yml and fix related regression

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-10 10:44:05 +08:00
Wang Yan
774a9f8d75
Remove unused configure item cfg_expiration (#7744)
Signed-off-by: wang yan <wangyan@vmware.com>
2019-05-09 22:07:18 +08:00
Wenkai Yin
5ce57a24ac
Merge pull request #7739 from ywk253100/190508_upgrade
Fixes #7693, the filters of replication policy is lost after upgrade
2019-05-09 20:16:13 +08:00
Daniel Jiang
4c18f487ad Update the version check in install.sh
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-05-09 19:18:52 +08:00
liqiang-fit2cloud
218889acdd Fix issue: https://github.com/goharbor/harbor/issues/7288
Signed-off-by: liqiang-fit2cloud <liqiang@fit2cloud.com>
2019-05-09 18:57:57 +08:00
Qian Deng
39f2bf2dfe
Merge pull request #7639 from ninjadq/fix_chart_storage_issue
Fix chart storage keyfile issue in gcs
2019-05-09 16:26:03 +08:00
Daniel Jiang
a67cc2b8b5
Merge pull request #7640 from ninjadq/remove_env_duplicate_items
Remove duplicate env items
2019-05-09 15:35:26 +08:00
Wenkai Yin
8ca3c44b87 Fixes #7693, the filters of replication policy is lost after upgrade
Fixes #7693, the filters of replication policy is lost after upgrade

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-05-09 14:13:32 +08:00
Qian Deng
322b108acf Remove duplicate env items
some env items are duplicate in both env and config_env file

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-09 10:55:39 +08:00
Qian Deng
d0e5936665 Fix chart storage keyfile issue in gcs
Add volumn binding on docker-compose.yml

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-08 19:20:36 +08:00
Wang Yan
095f7b2ff7
add scan all and gc schedule migration (#7628)
* add scan all and gc schedule migration

Signed-off-by: wang yan <wangyan@vmware.com>

* Fix gofmt errors

Signed-off-by: wang yan <wangyan@vmware.com>

* Update code according to review comments

Signed-off-by: wang yan <wangyan@vmware.com>

* remove convertschedule return name just return value

Signed-off-by: wang yan <wangyan@vmware.com>
2019-05-08 19:11:33 +08:00
Qian Deng
3550e2eb23
Merge pull request #7624 from ninjadq/prepare_for_harbor_tile
Prepare for harbor tile
2019-05-08 17:45:38 +08:00
Qian Deng
a70202f063 Add redirect disable item
if set storage redirect disable ture, will render it in registry config file

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-08 16:05:22 +08:00
Qian Deng
c44e3bf9d1 Clean admin server related config (#7615)
Clean up the admin_server

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-08 10:56:56 +08:00
Qian Deng
30918eff61
Merge pull request #7652 from reasonerjt/uaa-ca-file
Alow user to set CA cert for UAA in harbor.yml
2019-05-07 12:36:00 +08:00
Qian Deng
bed60352bd
Merge pull request #7329 from ninjadq/migration_script_to_1_8_0
Add migration script from 1.7.0 to 1.8.0
2019-05-07 10:53:51 +08:00
Wenkai Yin
d27a6c0335 Fix a few bugs of replication (#7619)
1. handle the public/private property when creating the projects
2. extend the length of access_secret
3. update the task status by using orm functions

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-05-07 10:47:14 +08:00
Daniel Jiang
0bb2829d27 Alow user to set CA cert for UAA in harbor.yml
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-05-06 17:32:02 +08:00
Wang Yan
a1ad6374ae
add install cert for registry contoller (#7633)
Mount the ca bunlder into registry controller, and add them into os
trust store that resolves the problem of garabe collection on ca
enabled registry.

Signed-off-by: wang yan <wangyan@vmware.com>
2019-05-05 15:24:52 +08:00
Qian Deng
71104011b3 Enhance: Read prepare dir in Env fisrt
If HARBOR_BUNDLE_DIR is exist, then get dir value from this.

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-04-30 16:16:06 +08:00
Qian Deng
fdd8ba7b83
Merge pull request #7600 from reasonerjt/fix-harbor-yml-typo
Fix typo in harbor.yml
2019-04-30 11:00:58 +08:00
Daniel Jiang
b5f2f71394 Fix typo in harbor.yml
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-04-29 18:51:00 +08:00
Qian Deng
c06c3fd08d Fix cannot load external configs of database (#7591)
Fix that when loading external db config wrong varible used

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-04-29 15:34:12 +08:00
Daniel Jiang
5cd3c039a9
Merge pull request #7463 from reasonerjt/regen-cli-secret
Provide API to generate CLI secret
2019-04-22 14:54:20 +08:00
Daniel Jiang
1fdc2e6ba9 Provide API to generate CLI secret
This commit provide an API to allow a user that is onboarded via OIDC
authn update his CLI secret.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-04-22 13:34:12 +08:00
Qian Deng
f742c415ad Upgrade the version of jinja2
Prevent verneribility issue

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-04-22 10:58:44 +08:00
Daniel Jiang
71cd51c9f4
Merge pull request #7326 from ninjadq/default_config_to_https
Set default mode to https
2019-04-22 09:24:39 +08:00
Wenkai Yin
a5cc228781
Merge pull request #7420 from ywk253100/190417_revert_local_harbor
Update the migration sql
2019-04-17 19:58:31 +08:00
wang yan
e017294f71 merge with master latest
Signed-off-by: wang yan <wangyan@vmware.com>
2019-04-17 17:52:39 +08:00
Wenkai Yin
547c2337de Update the migration sql
1. Update the migration sql
2. Rename the ResourceRepository from repository to image

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-04-17 17:18:03 +08:00
Qian Deng
8a7d04ef47
Merge pull request #7387 from ninjadq/add_compatibility_in_registry
Add compatibility config
2019-04-17 16:30:04 +08:00
Qian Deng
85fdf885e8
Merge pull request #7386 from ninjadq/fix_http_scheme_on_install_sh
Fix: grep https not work
2019-04-17 16:29:35 +08:00
Qian Deng
c1e676ad99 Add migration script from 1.7.0 to 1.8.0
Add jinja2 to migrator
Add template to migrator
Add config upgrading script

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-04-17 15:14:54 +08:00
Wenkai Yin
6e0d892963 Support creating project with service account
This commit introduces a solution to workaround the restriction of project creation API: only normal users can create projects

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-04-17 14:50:32 +08:00
Qian Deng
8a308a63e2 Set default mode to https
Default mode to https
The cert file value is set to previous version style

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-04-16 15:50:12 +08:00
Qian Deng
aad63e7ae5 Add compatibility config
Add compatibility config in registry.yml

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-04-16 15:44:47 +08:00
Qian Deng
9ddfd259d3 Fix bug when rendering port in proxy
rendering 443 when https enabled
rendering 4443 when notary enabled

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-04-16 11:20:13 +08:00
Qian Deng
0e8436263f Fix: grep https not work
Add space before right braket

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-04-15 14:32:36 +08:00
cd1989
27c1bdc5e2 Fix make prepare problem
Signed-off-by: cd1989 <chende@caicloud.io>
2019-04-10 19:48:13 +08:00
Daniel Jiang
0a2343f542 Support secret for docker CLI
As CLI does not support oauth flow, we'll use secret for help OIDC user
to authenticate via CLI.
Add column to store secret and token, and add code to support
verify/refresh token associates with secret.  Such that when the user is
removed from OIDC provider the secret will no longer work.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-04-10 19:38:11 +08:00
Daniel Jiang
a243c7f05a
Merge pull request #7330 from wy65701436/reg-271
Patch regsitry v2.7.1 with fix on distribution issue 2819
2019-04-10 09:45:56 +08:00
wang yan
469473b31c Patch regsitry v2.7.1 with fix on distribution issue 2819
This commit is to build a regsitry bases on v2.7.1 code and introduces
an fix on issue #2819, this is a P0 bug on v2.7.1 which causes GCS doesn't
work well on v2.7.1

For more details, refer to https://github.com/docker/distribution/pull/2821

Signed-off-by: wang yan <wangyan@vmware.com>
2019-04-09 18:42:29 +08:00
cd1989
92b04cffd5 Fix make prepare problem
Signed-off-by: cd1989 <chende@caicloud.io>
2019-04-09 17:02:09 +08:00
Wenkai Yin
580674f3da Merge remote-tracking branch 'upstream/master' into 190409_sync 2019-04-09 17:01:09 +08:00
Qian Deng
deba378842 Enhance: Refacotr Registry config file
1. Refactor registry configs
2. cp gcs keyfile is exist

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-04-09 14:40:41 +08:00
Qian Deng
74c4e243e3 Refator the host related config
1. Refactor host config
2. Refactor certiface config
3. Add port config
4. Add log info config

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-04-09 12:55:07 +08:00
Qian Deng
fef7702e9a Enhance: Refactor the config parse logic
Refactor the config parse logic

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-04-09 12:55:07 +08:00
Qian Deng
ac1b7bb1fb Enhance: remove the reload key item
Remove the reload_config item in config file

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-04-09 12:55:07 +08:00