Commit Graph

298 Commits

Author SHA1 Message Date
He Weiwei
e2a19d8ab9
fix(build): max idle and open conn settings for external db (#8854)
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-08-29 15:04:10 +08:00
Wang Yan
39f78ae768
Merge pull request #7872 from cd1989/config-redis-pool-idletimeout
Config idle timeout for redis pool to avoid jobservice restarting
2019-08-27 14:46:01 +08:00
DQ
fe3c71094b Disable redis and db containers if external db enabled
If depend on external redis or pg. local db and redis should not start. Therefore can save some resources.

Signed-off-by: DQ <dengq@vmware.com>
2019-08-26 17:59:13 +08:00
He Weiwei
a2c8536d37 fix(build): install tzdata pkg for core and jobservice images
Closes #8314

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-08-21 14:40:29 +00:00
cd1989
db9b52d827 Config idle timeout for redis pool
Signed-off-by: cd1989 <chende@caicloud.io>
2019-08-20 17:23:32 +08:00
Daniel Jiang
f674bb4e6c
Merge pull request #8590 from ninjadq/fix_registry_log_level
Fix: registry log level rendering issue
2019-08-20 09:11:56 +08:00
Daniel Jiang
b34fda173c Bump up Clair to v2.0.9
Fixes #8584

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-08-19 16:19:29 +08:00
He Weiwei
98e1f68468 feat(configuration,db): connection pool configs for db
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-08-14 14:30:34 +08:00
Daniel Jiang
ca585f8b9c
Merge pull request #8640 from ninjadq/fix_permission_of_nginx_cert
Fix permission of nginx cert
2019-08-14 14:23:40 +08:00
Wenkai Yin(尹文开)
a6445c1ebe
Merge pull request #8472 from kofj/feature/proxy
Proxy
2019-08-13 12:27:19 +08:00
Qian Deng
b4975d8601 Fix nginx permission issue
* mount root of host
* copy file to data dir and change ownership and permission

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-08-13 02:59:27 +00:00
疯魔慕薇
3e8a73ca1e Proxy
1. Global proxy config for components.
2. Prepare proxy configure for clair, core and jobservice.

Signed-off-by: 疯魔慕薇 <kfanjian@gmail.com>
2019-08-11 00:24:18 +08:00
Yogi_Wang
53bd4d7897 Fix issue width Link to license in the about dialog should not be hardcoded to master
Signed-off-by: Yogi_Wang <yawang@vmware.com>
2019-08-09 13:26:43 +08:00
Qian Deng
a935823e3d
Merge pull request #8362 from ninjadq/non-root-contaienr
Non root contaienr
2019-08-08 17:34:25 +08:00
Jakub Onderka
8f83310022 nginx: Remove TLSv1.1 support
Signed-off-by: Jakub Onderka <jakub.onderka@gmail.com>
2019-08-07 17:51:31 +02:00
王添
94d4f9c6b6 add webhook job
Signed-off-by: 王添 <wangtian@corp.netease.com>
2019-08-07 20:56:31 +08:00
DQ
057bc34703 Fix: registry log level rendering issue
when log level is warning, the actual value of registry should be warn

Signed-off-by: DQ <dengq@vmware.com>
2019-08-07 14:35:36 +08:00
Qian Deng
dacb1fc79e Add healthcheck in Dockerfile* redis* jobservice
Signed-off-by: Qian Deng <dengq@vmware.com>
2019-08-06 13:16:12 +00:00
Qian Deng
89d6370201 Remove ruby dependency while build portal
Python is already intalled in node image. so we can use python to parse yaml file

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-08-06 05:21:47 +00:00
Qian Deng
303471563f DB container run as non-root
Signed-off-by: Qian Deng <dengq@vmware.com>
2019-08-06 05:21:47 +00:00
Qian Deng
8b7f1ae4c0 Add proxy nginx container as non-root user
Signed-off-by: Qian Deng <dengq@vmware.com>
2019-08-06 05:21:47 +00:00
Qian Deng
f8a8040c8f Add notary as non-root user
Signed-off-by: Qian Deng <dengq@vmware.com>
2019-08-06 05:21:47 +00:00
Qian Deng
29727148b3 Running job service with non-root container
job-service running with 10000:10000 user

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-08-06 05:21:45 +00:00
Qian Deng
e62a9f1e18 Running redis using non-root user
redis running with user redis

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-08-06 05:21:45 +00:00
Qian Deng
904f04fac1 Enhance: Running contaienr with non-root user
* core
* portal

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-08-06 05:21:45 +00:00
Qian Deng
96b62e5741 Make core container to non-root user
Signed-off-by: Qian Deng <dengq@vmware.com>
2019-08-06 05:21:45 +00:00
Daniel Jiang
eec4fc2798 Remove clair notifier
The way Harbor handles notification is problematic.
It currently triggers rescan, which will cause problem when there are
lot of images in the registry.
Such as #7316
This commit removes the notifier and we need to revisit the notification
to figure out how to map the notification to a particular image if need
the notification mechanism in future.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-08-06 01:58:15 +08:00
Jakub Onderka
53b5dcfece nginx.https.conf.jinja template indention fix
Signed-off-by: Jakub Onderka <jakub.onderka@gmail.com>
2019-08-01 22:24:19 +02:00
wang yan
4410cc93f9 add internal reg request handler chain
this is for internal registry api call, the request should be intercpeted by quota middlerwares, like retag and delete.
Note: The api developer has to know that if the internal registry call in your api, please consider to use
NewRepositoryClientForLocal() to init the repository client, which can handle quota change.

Signed-off-by: wang yan <wangyan@vmware.com>
2019-07-30 19:39:56 +08:00
Daniel Jiang
e0e6a1d30b
Merge pull request #8301 from ninjadq/external_endpoint_support
Add supoort for external endpoint
2019-07-18 01:36:08 +08:00
DQ
6cf4596292 Add supoort for external endpoint
Add config item in harbor.yml
Make fowarding rule configurable

Signed-off-by: DQ <dengq@vmware.com>
2019-07-17 16:23:37 +08:00
Qian Deng
5cd3594f20 Upgrade chartmuseum from v0.8.1 to v0.9.0
Signed-off-by: Qian Deng <dengqian0826@gmail.com>
2019-07-17 06:45:23 +00:00
Ziming Zhang
072bdd101b aws driver for replication
Change-Id: I8792ffce2eaa5975359bb6159a1ba7b85926a925
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2019-06-25 19:11:27 +08:00
stonezdj
a8cd1bca59 Change the mount target of gcs.key file
Signed-off-by: stonezdj <stonezdj@gmail.com>
2019-06-16 13:34:32 +08:00
Wenkai Yin
1ceb7a2fb9
Merge pull request #7825 from ninjadq/update_installation_doc
Update doc caused by refactor prepare
2019-05-17 10:02:47 +08:00
Qian Deng
48151f6d46 Update doc caused by refactor prepare
Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-16 23:01:12 +08:00
Wenkai Yin
64cacc99e0
Merge pull request #7750 from liqiang-fit2cloud/fix-7288
Fix issue: harbor 1.7.4 aliyun oss chartmuseum 500
2019-05-16 18:23:35 +08:00
Qian Deng
f4ac7f9b4a
Merge pull request #7816 from ninjadq/fix_typo_of_azure
fix typo of azure config
2019-05-16 10:53:22 +08:00
Qian Deng
ea889d5a50 Fix typo in azure config
Fix typo in chart azure

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-15 18:25:34 +08:00
Wenkai Yin
1a8a55855b Add "MaxMessageSize" to the config of rsyslogd
Add "MaxMessageSize" to the config of rsyslogd to avoid the mess of log file when the size of one log line > 8k

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-05-15 18:10:22 +08:00
Qian Deng
1677686140 Made logs in jobservice configurable
Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-15 10:57:42 +08:00
Qian Deng
4188c4db76
Merge pull request #7719 from ninjadq/fix_chart_relative_url_issue
Fix chart relative url issue
2019-05-15 10:02:07 +08:00
stonezdj(Daojun Zhang)
47f24cab4b
Merge pull request #7770 from ninjadq/fix_typo_in_registry_config
Typo in registry config
2019-05-14 13:58:03 +08:00
Qian Deng
f607c5177d Fix frontend failure caused by absolute path
Fix failures because front downlowd chart using relative path

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-14 13:22:06 +08:00
Qian Deng
3022b617f2 Add chart absolute url item in config
Add a config item to enable and disalbe chart_url

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-14 12:56:20 +08:00
Qian Deng
cd6c5a9f10 Enable absolute url in helm chart
assign public_url to chart-url
remove namespace merge in index.yaml

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-14 12:56:20 +08:00
Qian Deng
41e399dec0 Fix issue caused by notary default_alias
Fix notary issue

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-13 18:43:15 +08:00
Qian Deng
6db39f9c71 Typo in registry config
it should be disable not disabled

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-10 23:08:21 +08:00
Qian Deng
439b44c61f Fix public url shoud not display port is it's default value (#7760)
if https port is 443 or http port is 80, then only showing url

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-10 16:53:25 +08:00
Qian Deng
eba20baba5
Merge pull request #7612 from ninjadq/fix_tls_related_issues
Fix tls related issues
2019-05-10 16:36:51 +08:00
Qian Deng
d255e66604 Remove -it in docker run
Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-10 15:31:58 +08:00
Qian Deng
286167ad74
Merge pull request #7755 from ninjadq/fix_rendering_none_in_jinja
Fix None rendered in jinja2
2019-05-10 14:59:21 +08:00
Qian Deng
f9f9661acd New type of bind volume
using long style bind volume

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-10 13:18:48 +08:00
Qian Deng
cd9932db23 Update the path of server.key and server.crt
change the path of cert key paris to prevent futrue issues.

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-10 13:18:48 +08:00
Qian Deng
3dfebed98e Enhance: Add an empty cert files if not exist
To avoid confusion error message

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-10 13:18:48 +08:00
Qian Deng
0aaccf62b2 Fix None rendered in jinja2
jinja2 render None to empty string

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-10 12:17:02 +08:00
Qian Deng
bb66358df8 Update migratrion script (#7728)
* Fix migration script

1. port is string when parsed from configparser
2. remove index and db_user in if condition

Signed-off-by: Qian Deng <dengq@vmware.com>

* Add port to public_url

Add port to public_url

Signed-off-by: Qian Deng <dengq@vmware.com>

* Customized value for notary and clair

db config in notary and clair is hardcoded

Signed-off-by: Qian Deng <dengq@vmware.com>

* Add notary and clair db config in harbor.yml

Add notary clair config to harbor.yml and fix related regression

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-10 10:44:05 +08:00
Wang Yan
774a9f8d75
Remove unused configure item cfg_expiration (#7744)
Signed-off-by: wang yan <wangyan@vmware.com>
2019-05-09 22:07:18 +08:00
liqiang-fit2cloud
218889acdd Fix issue: https://github.com/goharbor/harbor/issues/7288
Signed-off-by: liqiang-fit2cloud <liqiang@fit2cloud.com>
2019-05-09 18:57:57 +08:00
Qian Deng
39f2bf2dfe
Merge pull request #7639 from ninjadq/fix_chart_storage_issue
Fix chart storage keyfile issue in gcs
2019-05-09 16:26:03 +08:00
Daniel Jiang
a67cc2b8b5
Merge pull request #7640 from ninjadq/remove_env_duplicate_items
Remove duplicate env items
2019-05-09 15:35:26 +08:00
Qian Deng
322b108acf Remove duplicate env items
some env items are duplicate in both env and config_env file

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-09 10:55:39 +08:00
Qian Deng
d0e5936665 Fix chart storage keyfile issue in gcs
Add volumn binding on docker-compose.yml

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-08 19:20:36 +08:00
Qian Deng
3550e2eb23
Merge pull request #7624 from ninjadq/prepare_for_harbor_tile
Prepare for harbor tile
2019-05-08 17:45:38 +08:00
Qian Deng
a70202f063 Add redirect disable item
if set storage redirect disable ture, will render it in registry config file

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-08 16:05:22 +08:00
Qian Deng
c44e3bf9d1 Clean admin server related config (#7615)
Clean up the admin_server

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-05-08 10:56:56 +08:00
Daniel Jiang
0bb2829d27 Alow user to set CA cert for UAA in harbor.yml
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-05-06 17:32:02 +08:00
Wang Yan
a1ad6374ae
add install cert for registry contoller (#7633)
Mount the ca bunlder into registry controller, and add them into os
trust store that resolves the problem of garabe collection on ca
enabled registry.

Signed-off-by: wang yan <wangyan@vmware.com>
2019-05-05 15:24:52 +08:00
Qian Deng
c06c3fd08d Fix cannot load external configs of database (#7591)
Fix that when loading external db config wrong varible used

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-04-29 15:34:12 +08:00
Qian Deng
f742c415ad Upgrade the version of jinja2
Prevent verneribility issue

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-04-22 10:58:44 +08:00
Qian Deng
8a7d04ef47
Merge pull request #7387 from ninjadq/add_compatibility_in_registry
Add compatibility config
2019-04-17 16:30:04 +08:00
Qian Deng
aad63e7ae5 Add compatibility config
Add compatibility config in registry.yml

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-04-16 15:44:47 +08:00
Qian Deng
9ddfd259d3 Fix bug when rendering port in proxy
rendering 443 when https enabled
rendering 4443 when notary enabled

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-04-16 11:20:13 +08:00
Daniel Jiang
a243c7f05a
Merge pull request #7330 from wy65701436/reg-271
Patch regsitry v2.7.1 with fix on distribution issue 2819
2019-04-10 09:45:56 +08:00
wang yan
469473b31c Patch regsitry v2.7.1 with fix on distribution issue 2819
This commit is to build a regsitry bases on v2.7.1 code and introduces
an fix on issue #2819, this is a P0 bug on v2.7.1 which causes GCS doesn't
work well on v2.7.1

For more details, refer to https://github.com/docker/distribution/pull/2821

Signed-off-by: wang yan <wangyan@vmware.com>
2019-04-09 18:42:29 +08:00
cd1989
92b04cffd5 Fix make prepare problem
Signed-off-by: cd1989 <chende@caicloud.io>
2019-04-09 17:02:09 +08:00
Qian Deng
deba378842 Enhance: Refacotr Registry config file
1. Refactor registry configs
2. cp gcs keyfile is exist

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-04-09 14:40:41 +08:00
Qian Deng
74c4e243e3 Refator the host related config
1. Refactor host config
2. Refactor certiface config
3. Add port config
4. Add log info config

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-04-09 12:55:07 +08:00
Qian Deng
fef7702e9a Enhance: Refactor the config parse logic
Refactor the config parse logic

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-04-09 12:55:07 +08:00
Qian Deng
ac1b7bb1fb Enhance: remove the reload key item
Remove the reload_config item in config file

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-04-09 12:55:07 +08:00
Qian Deng
15c2c9048f Fix: clair env file should using empty string is not exist
This is quick fix, further fixs will in the config refactor PR

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-04-01 18:12:34 +08:00
Qian Deng
cb846bd936 Fix: copy upstream file to nginx config file
Copy notary.upstream.conf to nginx config file

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-03-27 17:56:31 +08:00
wang yan
1ba1c5726a Upgrade node version to 10.15
To fix the issue https://lists.debian.org/debian-devel-announce/2019/03/msg00006.html,
it needs to upgrade node to 10.15, which has pitched the fix.

Signed-off-by: wang yan <wangyan@vmware.com>
2019-03-27 10:47:13 +08:00
Qian Deng
df2425a02b Fix: upgrade pyyaml version to 4.2b1
Because previous version has security issue

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-03-22 19:05:16 +08:00
Qian Deng
ba4764c61d
Merge pull request #6755 from ninjadq/refactor_prepare
Refactor the prepare script
2019-03-22 14:54:30 +08:00
Qian Deng
fcdab4d4af Fix: packaging offline in new prepare
This new prepare script now support offline packaging

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-03-22 13:56:15 +08:00
Qian Deng
0c84751a10 Enhance: Refactor the notary structure
1. Update notary template on docker-compose
2. automatic generate cert if not exist

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-03-18 10:20:42 +08:00
Qian Deng
93af296eeb Enhance: refactor the mount dirs and workflow of generate cert
mount a temp dir input for all input files and configs
generated secrets file stored in data volumns keys dir
certs file stored in data volumns nginx dir

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-03-15 15:51:11 +08:00
Che-Wei Lin
7aa00aee87 fix hostname command not found (#7045)
Signed-off-by: mycroftlin <mycroftlin@tencent.com>
2019-03-15 10:52:47 +08:00
Qian Deng
b0f158c4c8 Add migratior script
Add migrator template and script

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-03-08 16:46:14 +08:00
Qian Deng
7b7cb82f86 Enhance: refactor the format of harbor.yml
refactor the format of the harbor.yml configuration items

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-03-08 16:46:13 +08:00
Qian Deng
ab7c81dac6 Fix: the adminserver caused regression
Remove some code related to adminserver
Fix some issues by adminserver removeing

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-03-08 16:46:13 +08:00
Qian Deng
5f80fe7b8a Refacotr the prepare script base on the proposal https://github.com/goharbor/community/pull/22
Signed-off-by: Qian Deng <dengq@vmware.com>
2019-03-08 16:46:13 +08:00
stonezdj
0cba36d79f Remove everything of adminserver
Signed-off-by: stonezdj <stonezdj@gmail.com>
2019-02-22 16:34:39 +08:00
Daniel Jiang
1832699e93 Bump up the migrate tool of notary
fixes #5863
The migrate binary that we include in notary is quite out dated.
Additionally it introduced a breaking change, more details see #5863

In this commit a go program was added to workaround this issue to ensure the
migration process works, and refined bootstrap scripts and make process accordingly.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-02-21 00:36:24 -08:00
Steven Zou
056ccbac41
Merge pull request #6931 from ninjadq/bump_chart_version
Enhhance: bump chartmuseum version to 0.8.1
2019-02-20 14:51:07 +08:00
Wenkai Yin
38d3c33ce4
Merge pull request #6729 from stonezdj/ref_admin_replace_backup
Refactor adminserver stage3
2019-02-19 13:52:46 +08:00
Wenkai Yin
696264bee9 Run logrotate as user 10000 to avoid issue #6895 (#6913)
This commit fixes issue #6895 by running logrotate with user 10000

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-02-19 13:18:45 +08:00
stonezdj
c9a8de9002 copy migration script to core container instead of mount volumn
Signed-off-by: stonezdj <stonezdj@gmail.com>
2019-02-18 14:06:19 +08:00
Qian Deng
4a4ebc2fba Enhhance: bump chartmuseum version to 0.8.1
bump the version of chartmuseum to 0.8.1

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-02-15 15:19:53 +08:00