Commit Graph

70 Commits

Author SHA1 Message Date
DQ
234b29e170 Fix pythom yaml load to safe_load
Signed-off-by: DQ <dengq@vmware.com>
2020-12-16 14:59:06 +08:00
DQ
0c9faea294 Clean up Clair in prepare script
Signed-off-by: DQ <dengq@vmware.com>
2020-11-10 11:39:18 +08:00
DQ
eb470501be Add metrics to Harbor Core
1. Add configs in prepare
 2. Add models and config items in Core
 3. Encapdulate getting metric in commom package
 4. Add a middleware for global request to collect 3 metrics

Signed-off-by: DQ <dengq@vmware.com>
2020-11-03 14:33:10 +08:00
DQ
d3ab9d7c6b Add internal tls configs for portal
add related file, config, command to enabled https for portal

Signed-off-by: DQ <dengq@vmware.com>
2020-07-31 12:10:47 +08:00
Ziming Zhang
8857e89e40 feature(redis) support redis sentinel
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2020-07-19 21:19:03 +08:00
DQ
026e37e777 Fix chart museum absolute url issue
if absolute url is enabled return true else set it to false

Signed-off-by: DQ <dengq@vmware.com>
2020-04-26 13:04:29 +08:00
DQ
599ca98c09 Hidden veriify client cert verfiy option
Remove to avoid replication access core from external_url issue

Signed-off-by: DQ <dengq@vmware.com>
2020-04-23 10:14:36 +08:00
Daniel Pacak
5c3abee135 chore(trivy): Bump up trivy adapter to 0.9.0
- Vendor the latest Trivy release 0.6.0
- Configure TLS 1.2 as min version when TLS is enabled
- Add more tracing to adapter config to facilitate troubleshooting

Resolves: #11544

Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
2020-04-16 08:40:27 +02:00
He Weiwei
77a8c3205f fix(prepare): not accpet items of false value in external_redis
Item in yaml without value will be as None in python, which will make
the password of redis as `None` in `get_redis_configs`. This fix will
not accept items of `false value` in `external_redis` configurations.

Closes #11367

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-04-03 04:09:26 +00:00
Steven Zou
2859cd8b69
Merge pull request #11134 from danielpacak/feat/issue_11090/trivy_skip_update_flag
feat(trivy): Configure Trivy to skip database updates
2020-03-19 18:13:08 +08:00
DQ
f18a546429 Fix: return error when internal_tls_not_provided
When iinternal_tls is empty, prepare should works as usual

Signed-off-by: DQ <dengq@vmware.com>
2020-03-19 10:37:58 +08:00
Daniel Pacak
7325105714 feat(trivy): Configure Trivy to skip database updates
Resolves: #11090

Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
2020-03-18 17:11:47 +01:00
DQ
6e8d44101f Enhance: User can generate cert by their own ca key pair
User can put their ca key pair on internal cert dir and name them to `harbor_internal_ca.key` and `harbor_internal_ca.crt` we wil use them to generate other certs

Signed-off-by: DQ <dengq@vmware.com>
2020-03-18 19:22:10 +08:00
DQ
b93092e012 Add tls for trivy
Add trivy tls cert files
Add tivey tls env and config
enhance gencert

Signed-off-by: DQ <dengq@vmware.com>
2020-03-18 19:22:10 +08:00
DQ
c5d73e6a0c Add switch to https
use switch to make decision whether mTLS or server TLS

Signed-off-by: DQ <dengq@vmware.com>
2020-03-18 19:22:10 +08:00
DQ
454382149f TLS update for chart, clairadapter, registry
Remove trustca in chartmuseum
Remove trustca in registry
Add tls in clair-adapter

Signed-off-by: DQ <dengq@vmware.com>
2020-03-18 19:22:10 +08:00
DQ
dcc6950af7 Feat: auto install ca in registry
Signed-off-by: DQ <dengq@vmware.com>
2020-03-18 19:22:09 +08:00
DQ
b852605193 Feat: enable mtls in harbor replication
Signed-off-by: DQ <dengq@vmware.com>
2020-03-18 19:22:09 +08:00
DQ
40e67f3b14 Feat: Enable mtls for registry
Signed-off-by: DQ <dengq@vmware.com>
2020-03-18 19:22:09 +08:00
DQ
da359f609f Feat: enable mtls in core
add mtls related code in core

Signed-off-by: DQ <dengq@vmware.com>
2020-03-18 19:22:09 +08:00
DQ
a4855cca36 Feat: update prepare to support tls
update makefile
add model for prepare
update jinja template for prepare

Signed-off-by: DQ <dengq@vmware.com>
2020-03-18 19:22:09 +08:00
Daniel Pacak
46fb43bc25 chore: Bump up Trivy adapter to v0.4.0
Allows configuring SCANNER_TRIVY_GITHUB_TOKEN environment variable,
which is passed to trivy executable binary when it starts scanning
a given artifact.

This is to increase GitHub requests rate limit from 60 per hours
(for anonymous requests) to 5000 when Trivy download its
vulnerabilities database.

Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
2020-03-16 09:53:16 +01:00
dechen
e642a73280 Set redis idle timeout for core
Signed-off-by: dechen <xxyydream@gmail.com>
2020-02-23 12:31:56 +08:00
Daniel Pacak
70dda1387a chore: Configure Redis URL for Trivy adapter
Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
2020-02-13 17:57:02 +01:00
Daniel Pacak
a642667ffc chore(install): Add --with-trivy arg to the installation script
Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
2020-02-12 23:47:56 +01:00
Daniel Jiang
2064a1cd6d Switch to basic authentication for registry
1. Add basic authorizer for registry which modify the request
to add basic authorization header to request based on configuration.
2. Set basic auth header for proxy when accessing registry
3. Switche the registry to use basic auth by default and use the basic
authorizer to access Harbor.
4. Make necessary change to test cases, particularly
"test_robot_account.py" and "docker_api.py", because the error is
changed after siwtched to basic auth from token auth.  #10604 is opened
to track the follow up work.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-01-31 21:46:47 +09:00
Daniel Jiang
a087ba02e3 Populate basic auth information for registry
This commit updates `prepare` and templates to populate the credential
for registry for basic authentication.

A temporary flag `registry_use_basic_auth` was added to avoid breakage.
It MUST be removed before the release.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-12-31 14:50:46 +08:00
Wenkai Yin(尹文开)
d145f4baf4
Merge pull request #10034 from ywk253100/191128_clean
Clean up admiral-related code
2019-12-04 17:33:31 +08:00
Daniel Jiang
7bb71db478
Merge pull request #10003 from ninjadq/migrator_miss_component_no_proxy
Add default domainname for no_proxy
2019-12-03 10:50:32 +08:00
Wenkai Yin
dd2bc0ecef Clean up admiral-related code
Clean up admiral-related code as it's useless

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-11-28 17:28:54 +08:00
DQ
79344887b9 Fix ca bundle path join issue
CA bundle name start with '/' will break the os path join

Signed-off-by: DQ <dengq@vmware.com>
2019-11-27 18:37:45 +08:00
DQ
ed6438cf69 Add default domainname for no_proxy
All internal service and known internal hostname shuold add to no_proxy by default

Signed-off-by: DQ <dengq@vmware.com>
2019-11-27 15:10:42 +08:00
He Weiwei
b8308f41a0 fix(prepaire,clair): disable clair updaters when its interval is 0
Closes #9961

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-11-22 03:31:20 +00:00
He Weiwei
fe69a5df99 build(scanner-adapter): bump up clair adapter to v1.0.1-rc2
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-11-13 02:35:21 +00:00
DQ
80c3e76b5a check the permission of ca bundle file
CA bundle need check before use

Signed-off-by: DQ <dengq@vmware.com>
2019-11-08 15:34:17 +08:00
Wang Yan
27cb25cc04
Merge pull request #9400 from ninjadq/inject_certs_to_non_root
Inject certs to non root
2019-11-05 14:49:08 +08:00
DQ
ece321a53a Change certs's owner to 10000
Signed-off-by: DQ <dengq@vmware.com>
2019-11-04 17:38:41 +08:00
DQ
873d9f5b82 Enable https by default
1. Umcomment https related configs
2. Remove the https prepare related thing in ci

Signed-off-by: DQ <dengq@vmware.com>
2019-10-31 20:58:09 +08:00
He Weiwei
8964a8697a build(clair): internal clair adapter when install with clair
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-10-17 12:00:51 +08:00
stonezdj(Daojun Zhang)
ca97c85279
Merge pull request #8927 from ninjadq/fix_config_with_components
Add logic to read clair and notary config
2019-09-09 15:50:09 +08:00
DQ
495a257ab5 Add logic to read clair and notary config
Signed-off-by: DQ <dengq@vmware.com>
2019-09-05 12:49:32 +08:00
DQ
377739204b Update config file names
Signed-off-by: DQ <dengq@vmware.com>
2019-09-02 18:19:06 +08:00
Wang Yan
6e462baa0d
Merge pull request #8837 from ninjadq/disable_redis_n_db_container_if_use_exeternal
Disable redis and db containers if external db enabled
2019-09-01 17:47:28 +08:00
He Weiwei
e2a19d8ab9
fix(build): max idle and open conn settings for external db (#8854)
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-08-29 15:04:10 +08:00
DQ
fe3c71094b Disable redis and db containers if external db enabled
If depend on external redis or pg. local db and redis should not start. Therefore can save some resources.

Signed-off-by: DQ <dengq@vmware.com>
2019-08-26 17:59:13 +08:00
He Weiwei
98e1f68468 feat(configuration,db): connection pool configs for db
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-08-14 14:30:34 +08:00
疯魔慕薇
3e8a73ca1e Proxy
1. Global proxy config for components.
2. Prepare proxy configure for clair, core and jobservice.

Signed-off-by: 疯魔慕薇 <kfanjian@gmail.com>
2019-08-11 00:24:18 +08:00
王添
94d4f9c6b6 add webhook job
Signed-off-by: 王添 <wangtian@corp.netease.com>
2019-08-07 20:56:31 +08:00
wang yan
4410cc93f9 add internal reg request handler chain
this is for internal registry api call, the request should be intercpeted by quota middlerwares, like retag and delete.
Note: The api developer has to know that if the internal registry call in your api, please consider to use
NewRepositoryClientForLocal() to init the repository client, which can handle quota change.

Signed-off-by: wang yan <wangyan@vmware.com>
2019-07-30 19:39:56 +08:00
DQ
6cf4596292 Add supoort for external endpoint
Add config item in harbor.yml
Make fowarding rule configurable

Signed-off-by: DQ <dengq@vmware.com>
2019-07-17 16:23:37 +08:00