Upstream/harbor
1
0
Fork 0
mirror of https://github.com/goharbor/harbor.git synced 2024-11-29 13:45:20 +01:00
Code Issues Projects Releases Wiki Activity
11,700 Commits 53 Branches 317 Tags 335 MiB
14192dfb06
Commit Graph

163 Commits

Author SHA1 Message Date
DQ
eb470501be Add metrics to Harbor Core 
1. Add configs in prepare
 2. Add models and config items in Core
 3. Encapdulate getting metric in commom package
 4. Add a middleware for global request to collect 3 metrics

Signed-off-by: DQ <dengq@vmware.com>
 2020-11-03 14:33:10 +08:00
Ziming Zhang
ff19dd499c fix(jobservice) redis sentinel failover hang 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-08-26 10:42:44 +08:00
DQ
a251e90507 Add log denpendency ti trivy 
To void trivy can not start issue

Signed-off-by: DQ <dengq@vmware.com>
 2020-08-13 11:35:21 +08:00
DQ
d3ab9d7c6b Add internal tls configs for portal 
add related file, config, command to enabled https for portal

Signed-off-by: DQ <dengq@vmware.com>
 2020-07-31 12:10:47 +08:00
DQ
d7618a6274 Fix: beego app config port hardcode 
the port should be flexible depend on the internal tls

Signed-off-by: DQ <dengq@vmware.com>
 2020-07-27 15:35:43 +08:00
Ziming Zhang
8857e89e40 feature(redis) support redis sentinel 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-07-19 21:19:03 +08:00
He Weiwei
039aef5356 refactor: remove initialization of clair db 
To fetch vulnerability database updated time of the Clair had moved to
the Clair adapter so removes the initialization of clair db in the core.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-07-09 15:26:14 +00:00
Wenkai Yin
02690d1d04 Suport filtering registries by type in listing registry API 
Suport filtering registries by type in listing registry API

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-07-07 10:30:46 +08:00
DQ
d0ddd61ad9 Fix Amazon S3 storage not work 
The Chartmuseum S3 client need set an Env variable
Ref: https://github.com/helm/chartmuseum/issues/280

Signed-off-by: DQ <dengq@vmware.com>
 2020-06-30 15:16:18 +08:00
Wang Yan
dec8397c21
Add api to delete blob and manifest (#12006) 
* Add api to delete blob and manifest

Enable the capability of registry controller to delete blob and manifest

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-06-06 01:34:23 +08:00
Qian Deng
9e1302211b
Merge pull request #12072 from ninjadq/add_timeout_in_nginx_config 
Add timeout in nginx config
 2020-06-02 15:14:42 +08:00
He Weiwei
d97be71234 refactor(configuration): cleanup unneeded CLAIR_URL configuration in core 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-05-29 07:27:50 +00:00
DQ
278338e401 Add timount on nginx configs 
set timeout to 900

Signed-off-by: DQ <dengq@vmware.com>
 2020-05-26 16:18:35 +08:00
DQ
715685ae51 Remove tls1.1 in notary 
Signed-off-by: DQ <dengq@vmware.com>
 2020-05-26 16:11:57 +08:00
DQ
b06e19a637 Fix: GCS storage gc issue 
Mount gcs key to registryctl

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-29 15:04:16 +08:00
DQ
f70339870a Enhance: Create shared to store shared ca 
this shared ca will mount to all harbor components

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-28 02:58:11 +08:00
Daniel Jiang
2ecf0425a4 Remove the certs of notary signer 
Since `prepare` generates the certs as needed during installation, these
certs should not exist in the repo.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-04-21 13:20:19 +08:00
DQ
b728f04d0a Fix tls min version for registry 
cert,key,mintls should in the same context

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-20 19:19:15 +08:00
Daniel Pacak
5c3abee135 chore(trivy): Bump up trivy adapter to 0.9.0 
- Vendor the latest Trivy release 0.6.0
- Configure TLS 1.2 as min version when TLS is enabled
- Add more tracing to adapter config to facilitate troubleshooting

Resolves: #11544

Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
 2020-04-16 08:40:27 +02:00
DQ
42c1095216 Fix cert issue of trivy 
Trivy can't access harbor from external if https enabled so inject cert to trivy container trust

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-16 10:52:03 +08:00
DQ
75f78b64b2 Set registry tls version to 1.2 
when internal tls enabled set min version of registry to 1.2

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-13 18:13:30 +08:00
DQ
08ff622310 Remove lines not needed 
volume already defined above

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-09 20:06:51 +08:00
DQ
4a836ea975 Fix health check url 
health check url should depend on internal https

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-07 03:35:52 +00:00
DQ
cdb675bf3d Add proxy cert file to jobservice when https enabled 
jobservice may request via absolute path of url to harbor

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-04 17:44:34 +00:00
Qian Deng
a702c32346
Merge pull request #11063 from ninjadq/fix_syslog_dir_in_tpl 
Fix: fix logrotate is dir issue
 2020-04-02 11:37:29 +08:00
Steven Zou
2859cd8b69
Merge pull request #11134 from danielpacak/feat/issue_11090/trivy_skip_update_flag 
feat(trivy): Configure Trivy to skip database updates
 2020-03-19 18:13:08 +08:00
Daniel Pacak
7325105714 feat(trivy): Configure Trivy to skip database updates 
Resolves: #11090

Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
 2020-03-18 17:11:47 +01:00
DQ
b93092e012 Add tls for trivy 
Add trivy tls cert files
Add tivey tls env and config
enhance gencert

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:10 +08:00
DQ
c954969bcd Add mTLS configs 
mTLS only enabled in jobservice and registryctl

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:10 +08:00
DQ
454382149f TLS update for chart, clairadapter, registry 
Remove trustca in chartmuseum
Remove trustca in registry
Add tls in clair-adapter

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:10 +08:00
DQ
03e11c63c7 Fix docker file with secure tls change 
Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:10 +08:00
DQ
dcc6950af7 Feat: auto install ca in registry 
Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
DQ
40e67f3b14 Feat: Enable mtls for registry 
Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
DQ
da359f609f Feat: enable mtls in core 
add mtls related code in core

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
DQ
a4855cca36 Feat: update prepare to support tls 
update makefile
add model for prepare
update jinja template for prepare

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
Daniel Pacak
9c13116963 chore(trivy): Allow configuring HTTP(S) proxy 
Resolves: #11032

Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
 2020-03-16 12:26:49 +01:00
Daniel Pacak
46fb43bc25 chore: Bump up Trivy adapter to v0.4.0 
Allows configuring SCANNER_TRIVY_GITHUB_TOKEN environment variable,
which is passed to trivy executable binary when it starts scanning
a given artifact.

This is to increase GitHub requests rate limit from 60 per hours
(for anonymous requests) to 5000 when Trivy download its
vulnerabilities database.

Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
 2020-03-16 09:53:16 +01:00
DQ
1eeea6b888 Fix: fix logrotate is dir issue 
Change it to bind command

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-13 14:58:45 +08:00
Daniel Jiang
ae5ffce83a Update CSRF mechanism 
This commit replaces beego's CSRF mechanism with gorilla's csrf library.
The criteria for requests to skip the csrf check remain the same.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-03-09 01:15:54 +08:00
wang yan
2b0b7576b2 Fix gc issue on clean the artifact trash 
1, enable dao test for artifact trash
2, set default flush trash table to false
3, hanlder empty parameter in API call
4, add registry auth info into jobservice container

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-06 03:11:31 +08:00
Daniel Jiang
1823c984f7
Merge branch 'master' into redis-idle-timeout 2020-02-27 22:01:22 +08:00
stonezdj
6005101c95 Remove registry notification and change /api/ping 
Update config.yaml.jinja to remove notification
Change api/ping in core/Dockerfile

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2020-02-25 11:24:21 +08:00
Wenkai Yin
bd204464f3 Remove dead code 
Remove dead code

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-23 17:11:46 +08:00
dechen
e642a73280 Set redis idle timeout for core 
Signed-off-by: dechen <xxyydream@gmail.com>
 2020-02-23 12:31:56 +08:00
Daniel Pacak
70dda1387a chore: Configure Redis URL for Trivy adapter 
Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
 2020-02-13 17:57:02 +01:00
Daniel Pacak
a642667ffc chore(install): Add --with-trivy arg to the installation script 
Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
 2020-02-12 23:47:56 +01:00
Daniel Jiang
2064a1cd6d Switch to basic authentication for registry 
1. Add basic authorizer for registry which modify the request
to add basic authorization header to request based on configuration.
2. Set basic auth header for proxy when accessing registry
3. Switche the registry to use basic auth by default and use the basic
authorizer to access Harbor.
4. Make necessary change to test cases, particularly
"test_robot_account.py" and "docker_api.py", because the error is
changed after siwtched to basic auth from token auth.  #10604 is opened
to track the follow up work.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-01-31 21:46:47 +09:00
Daniel Jiang
a087ba02e3 Populate basic auth information for registry 
This commit updates `prepare` and templates to populate the credential
for registry for basic authentication.

A temporary flag `registry_use_basic_auth` was added to avoid breakage.
It MUST be removed before the release.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-12-31 14:50:46 +08:00
Wang Yan
2a63382236
Merge pull request #10047 from bitsf/makefile_clean 
optimize the makefile process
 2019-12-05 19:03:19 +08:00
Ziming Zhang
332f88ec8c add make clean 
Change-Id: Ibe806972a19cd69bfd90be051cdc340c4d7c6afb
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2019-12-05 14:44:07 +08:00
Wenkai Yin
dd2bc0ecef Clean up admiral-related code 
Clean up admiral-related code as it's useless

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-11-28 17:28:54 +08:00
He Weiwei
fe69a5df99 build(scanner-adapter): bump up clair adapter to v1.0.1-rc2 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-11-13 02:35:21 +00:00
Wang Yan
0b09bd14b1
Merge pull request #9756 from ninjadq/add_ignore_media_type 
Add ignore mediatypes for registry
 2019-11-08 18:34:13 +08:00
Daniel Jiang
06e4e124d8
Refine request handle process (#9760) 
* Refine request handle process

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-11-07 13:02:17 +08:00
DQ
45868107aa Add ignore mediatypes for registry 
Add these mediatypes to reduce the amount of registry event

Signed-off-by: DQ <dengq@vmware.com>
 2019-11-06 21:39:08 +08:00
wang yan
c46d7e856a add a switcher for quota sync on core launch 
As the quota sync is default called by harbor-core on every launch, and it will break the launch process if any failure throwed.

1, The commit is to provide an switcher for the system admin to bypass the quota sync.
2, In case Harbor goes into the restarting cycle.

Harbor already provides an internal API to sync quota data, in the failure case,
system admin can launch harbor and call the /api/internal/syncquota to sync quota.

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-10-31 16:17:27 +08:00
He Weiwei
b0f7404231
chore(log): log level support for clair adapter (#9640) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-10-29 16:50:26 +08:00
He Weiwei
8964a8697a build(clair): internal clair adapter when install with clair 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-10-17 12:00:51 +08:00
stonezdj(Daojun Zhang)
0fa4934679
Merge pull request #8596 from JakubOnderka/patch-4 
nginx: Remove TLSv1.1 support
 2019-10-16 11:39:55 +08:00
Qian Deng
578adaa064
Merge pull request #9240 from ninjadq/add_extra_headers_in_nginx 
Add headers in nginx config file
 2019-09-26 10:27:08 +08:00
DQ
e7394041ab Add headers in nginx config file 
extra headered added in https and http config

Signed-off-by: DQ <dengq@vmware.com>
 2019-09-24 17:50:40 +08:00
Daniel Jiang
3e5973fc6e Add Secure flag to cookie 
This commit modifies nginx configuration file to make sure the secure
flag is added to "Set-Cookie" header when Harbor is serving https

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-09-19 21:04:37 +08:00
Wang Yan
6e462baa0d
Merge pull request #8837 from ninjadq/disable_redis_n_db_container_if_use_exeternal 
Disable redis and db containers if external db enabled
 2019-09-01 17:47:28 +08:00
DQ
fe3c71094b Disable redis and db containers if external db enabled 
If depend on external redis or pg. local db and redis should not start. Therefore can save some resources.

Signed-off-by: DQ <dengq@vmware.com>
 2019-08-26 17:59:13 +08:00
cd1989
db9b52d827 Config idle timeout for redis pool 
Signed-off-by: cd1989 <chende@caicloud.io>
 2019-08-20 17:23:32 +08:00
He Weiwei
98e1f68468 feat(configuration,db): connection pool configs for db 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-08-14 14:30:34 +08:00
Daniel Jiang
ca585f8b9c
Merge pull request #8640 from ninjadq/fix_permission_of_nginx_cert 
Fix permission of nginx cert
 2019-08-14 14:23:40 +08:00
Qian Deng
b4975d8601 Fix nginx permission issue 
* mount root of host
* copy file to data dir and change ownership and permission

Signed-off-by: Qian Deng <dengq@vmware.com>
 2019-08-13 02:59:27 +00:00
疯魔慕薇
3e8a73ca1e Proxy 
1. Global proxy config for components.
2. Prepare proxy configure for clair, core and jobservice.

Signed-off-by: 疯魔慕薇 <kfanjian@gmail.com>
 2019-08-11 00:24:18 +08:00
Qian Deng
a935823e3d
Merge pull request #8362 from ninjadq/non-root-contaienr 
Non root contaienr
 2019-08-08 17:34:25 +08:00
Jakub Onderka
8f83310022 nginx: Remove TLSv1.1 support 
Signed-off-by: Jakub Onderka <jakub.onderka@gmail.com>
 2019-08-07 17:51:31 +02:00
王添
94d4f9c6b6 add webhook job 
Signed-off-by: 王添 <wangtian@corp.netease.com>
 2019-08-07 20:56:31 +08:00
Qian Deng
303471563f DB container run as non-root 
Signed-off-by: Qian Deng <dengq@vmware.com>
 2019-08-06 05:21:47 +00:00
Qian Deng
8b7f1ae4c0 Add proxy nginx container as non-root user 
Signed-off-by: Qian Deng <dengq@vmware.com>
 2019-08-06 05:21:47 +00:00
Qian Deng
904f04fac1 Enhance: Running contaienr with non-root user 
* core
* portal

Signed-off-by: Qian Deng <dengq@vmware.com>
 2019-08-06 05:21:45 +00:00
Daniel Jiang
eec4fc2798 Remove clair notifier 
The way Harbor handles notification is problematic.
It currently triggers rescan, which will cause problem when there are
lot of images in the registry.
Such as #7316
This commit removes the notifier and we need to revisit the notification
to figure out how to map the notification to a particular image if need
the notification mechanism in future.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-08-06 01:58:15 +08:00
Jakub Onderka
53b5dcfece nginx.https.conf.jinja template indention fix 
Signed-off-by: Jakub Onderka <jakub.onderka@gmail.com>
 2019-08-01 22:24:19 +02:00
wang yan
4410cc93f9 add internal reg request handler chain 
this is for internal registry api call, the request should be intercpeted by quota middlerwares, like retag and delete.
Note: The api developer has to know that if the internal registry call in your api, please consider to use
NewRepositoryClientForLocal() to init the repository client, which can handle quota change.

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-07-30 19:39:56 +08:00
DQ
6cf4596292 Add supoort for external endpoint 
Add config item in harbor.yml
Make fowarding rule configurable

Signed-off-by: DQ <dengq@vmware.com>
 2019-07-17 16:23:37 +08:00
stonezdj
a8cd1bca59 Change the mount target of gcs.key file 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-06-16 13:34:32 +08:00
Qian Deng
1677686140 Made logs in jobservice configurable 
Signed-off-by: Qian Deng <dengq@vmware.com>
 2019-05-15 10:57:42 +08:00
Qian Deng
4188c4db76
Merge pull request #7719 from ninjadq/fix_chart_relative_url_issue 
Fix chart relative url issue
 2019-05-15 10:02:07 +08:00
stonezdj(Daojun Zhang)
47f24cab4b
Merge pull request #7770 from ninjadq/fix_typo_in_registry_config 
Typo in registry config
 2019-05-14 13:58:03 +08:00
Qian Deng
3022b617f2 Add chart absolute url item in config 
Add a config item to enable and disalbe chart_url

Signed-off-by: Qian Deng <dengq@vmware.com>
 2019-05-14 12:56:20 +08:00
Qian Deng
cd6c5a9f10 Enable absolute url in helm chart 
assign public_url to chart-url
remove namespace merge in index.yaml

Signed-off-by: Qian Deng <dengq@vmware.com>
 2019-05-14 12:56:20 +08:00
Qian Deng
41e399dec0 Fix issue caused by notary default_alias 
Fix notary issue

Signed-off-by: Qian Deng <dengq@vmware.com>
 2019-05-13 18:43:15 +08:00
Qian Deng
6db39f9c71 Typo in registry config 
it should be disable not disabled

Signed-off-by: Qian Deng <dengq@vmware.com>
 2019-05-10 23:08:21 +08:00
Qian Deng
d255e66604 Remove -it in docker run 
Signed-off-by: Qian Deng <dengq@vmware.com>
 2019-05-10 15:31:58 +08:00
Qian Deng
f9f9661acd New type of bind volume 
using long style bind volume

Signed-off-by: Qian Deng <dengq@vmware.com>
 2019-05-10 13:18:48 +08:00
Qian Deng
cd9932db23 Update the path of server.key and server.crt 
change the path of cert key paris to prevent futrue issues.

Signed-off-by: Qian Deng <dengq@vmware.com>
 2019-05-10 13:18:48 +08:00
Qian Deng
bb66358df8 Update migratrion script (#7728) 
* Fix migration script

1. port is string when parsed from configparser
2. remove index and db_user in if condition

Signed-off-by: Qian Deng <dengq@vmware.com>

* Add port to public_url

Add port to public_url

Signed-off-by: Qian Deng <dengq@vmware.com>

* Customized value for notary and clair

db config in notary and clair is hardcoded

Signed-off-by: Qian Deng <dengq@vmware.com>

* Add notary and clair db config in harbor.yml

Add notary clair config to harbor.yml and fix related regression

Signed-off-by: Qian Deng <dengq@vmware.com>
 2019-05-10 10:44:05 +08:00
Wang Yan
774a9f8d75
Remove unused configure item cfg_expiration (#7744) 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-05-09 22:07:18 +08:00
Qian Deng
39f2bf2dfe
Merge pull request #7639 from ninjadq/fix_chart_storage_issue 
Fix chart storage keyfile issue in gcs
 2019-05-09 16:26:03 +08:00
Daniel Jiang
a67cc2b8b5
Merge pull request #7640 from ninjadq/remove_env_duplicate_items 
Remove duplicate env items
 2019-05-09 15:35:26 +08:00
Qian Deng
322b108acf Remove duplicate env items 
some env items are duplicate in both env and config_env file

Signed-off-by: Qian Deng <dengq@vmware.com>
 2019-05-09 10:55:39 +08:00
Qian Deng
d0e5936665 Fix chart storage keyfile issue in gcs 
Add volumn binding on docker-compose.yml

Signed-off-by: Qian Deng <dengq@vmware.com>
 2019-05-08 19:20:36 +08:00
Qian Deng
3550e2eb23
Merge pull request #7624 from ninjadq/prepare_for_harbor_tile 
Prepare for harbor tile
 2019-05-08 17:45:38 +08:00
Qian Deng
a70202f063 Add redirect disable item 
if set storage redirect disable ture, will render it in registry config file

Signed-off-by: Qian Deng <dengq@vmware.com>
 2019-05-08 16:05:22 +08:00
Qian Deng
c44e3bf9d1 Clean admin server related config (#7615) 
Clean up the admin_server

Signed-off-by: Qian Deng <dengq@vmware.com>
 2019-05-08 10:56:56 +08:00
Daniel Jiang
0bb2829d27 Alow user to set CA cert for UAA in harbor.yml 
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-05-06 17:32:02 +08:00