Commit Graph

158 Commits

Author SHA1 Message Date
He Weiwei
77a8c3205f fix(prepare): not accpet items of false value in external_redis
Item in yaml without value will be as None in python, which will make
the password of redis as `None` in `get_redis_configs`. This fix will
not accept items of `false value` in `external_redis` configurations.

Closes #11367

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-04-03 04:09:26 +00:00
Qian Deng
a702c32346
Merge pull request #11063 from ninjadq/fix_syslog_dir_in_tpl
Fix: fix logrotate is dir issue
2020-04-02 11:37:29 +08:00
DQ
dc271e1a87 Add packaging to pipenv
Signed-off-by: DQ <dengq@vmware.com>
2020-04-01 22:54:47 +08:00
DQ
d636f2ea5c Enhance help message
Provide more info in help message
Add requried opition and they will show missing option if you are not provide them instead of Exception

Signed-off-by: DQ <dengq@vmware.com>
2020-04-01 17:02:59 +08:00
DQ
b2e1905e7a Enhance: Stop upgrade when input version less then 1.9.0
The migration script should failure early when version is not supported

Signed-off-by: DQ <dengq@vmware.com>
2020-04-01 15:35:49 +08:00
Qian Deng
9e101b73a4
Merge pull request #11156 from ninjadq/migrate_config_to_harbor2
Migrate config to harbor2
2020-03-25 16:02:18 +08:00
DQ
85ec0e7820 Enhance: Refactor the migration structure
1. Refactor structure of migrate file
2. fix some previous bugs

Signed-off-by: DQ <dengq@vmware.com>
2020-03-23 21:26:28 +08:00
DQ
444678fe07 Fix: module path raise exception when it is loop
add test for loop

Signed-off-by: DQ <dengq@vmware.com>
2020-03-23 19:29:59 +08:00
DQ
e8bb977ae1 Feat: Upgrade configs to harbor 2.0
add migrate files for harbor 2.0

Signed-off-by: DQ <dengq@vmware.com>
2020-03-20 15:20:32 +08:00
DQ
1e0c9f7231 Feat: Add config migrator to prepare
deprecated migrator container and move config migration to prepare

Signed-off-by: DQ <dengq@vmware.com>
2020-03-20 03:04:10 +08:00
Steven Zou
2859cd8b69
Merge pull request #11134 from danielpacak/feat/issue_11090/trivy_skip_update_flag
feat(trivy): Configure Trivy to skip database updates
2020-03-19 18:13:08 +08:00
DQ
f18a546429 Fix: return error when internal_tls_not_provided
When iinternal_tls is empty, prepare should works as usual

Signed-off-by: DQ <dengq@vmware.com>
2020-03-19 10:37:58 +08:00
Daniel Pacak
7325105714 feat(trivy): Configure Trivy to skip database updates
Resolves: #11090

Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
2020-03-18 17:11:47 +01:00
DQ
6e8d44101f Enhance: User can generate cert by their own ca key pair
User can put their ca key pair on internal cert dir and name them to `harbor_internal_ca.key` and `harbor_internal_ca.crt` we wil use them to generate other certs

Signed-off-by: DQ <dengq@vmware.com>
2020-03-18 19:22:10 +08:00
DQ
b93092e012 Add tls for trivy
Add trivy tls cert files
Add tivey tls env and config
enhance gencert

Signed-off-by: DQ <dengq@vmware.com>
2020-03-18 19:22:10 +08:00
DQ
c954969bcd Add mTLS configs
mTLS only enabled in jobservice and registryctl

Signed-off-by: DQ <dengq@vmware.com>
2020-03-18 19:22:10 +08:00
DQ
c5d73e6a0c Add switch to https
use switch to make decision whether mTLS or server TLS

Signed-off-by: DQ <dengq@vmware.com>
2020-03-18 19:22:10 +08:00
DQ
454382149f TLS update for chart, clairadapter, registry
Remove trustca in chartmuseum
Remove trustca in registry
Add tls in clair-adapter

Signed-off-by: DQ <dengq@vmware.com>
2020-03-18 19:22:10 +08:00
DQ
03e11c63c7 Fix docker file with secure tls change
Signed-off-by: DQ <dengq@vmware.com>
2020-03-18 19:22:10 +08:00
DQ
dcc6950af7 Feat: auto install ca in registry
Signed-off-by: DQ <dengq@vmware.com>
2020-03-18 19:22:09 +08:00
DQ
b852605193 Feat: enable mtls in harbor replication
Signed-off-by: DQ <dengq@vmware.com>
2020-03-18 19:22:09 +08:00
DQ
40e67f3b14 Feat: Enable mtls for registry
Signed-off-by: DQ <dengq@vmware.com>
2020-03-18 19:22:09 +08:00
DQ
07a1d51693 Feat: enable tls in registryctlAdd tls related code in registryctl
Signed-off-by: DQ <dengq@vmware.com>
2020-03-18 19:22:09 +08:00
DQ
da359f609f Feat: enable mtls in core
add mtls related code in core

Signed-off-by: DQ <dengq@vmware.com>
2020-03-18 19:22:09 +08:00
DQ
a4855cca36 Feat: update prepare to support tls
update makefile
add model for prepare
update jinja template for prepare

Signed-off-by: DQ <dengq@vmware.com>
2020-03-18 19:22:09 +08:00
Daniel Pacak
9c13116963 chore(trivy): Allow configuring HTTP(S) proxy
Resolves: #11032

Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
2020-03-16 12:26:49 +01:00
Daniel Pacak
46fb43bc25 chore: Bump up Trivy adapter to v0.4.0
Allows configuring SCANNER_TRIVY_GITHUB_TOKEN environment variable,
which is passed to trivy executable binary when it starts scanning
a given artifact.

This is to increase GitHub requests rate limit from 60 per hours
(for anonymous requests) to 5000 when Trivy download its
vulnerabilities database.

Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
2020-03-16 09:53:16 +01:00
DQ
1eeea6b888 Fix: fix logrotate is dir issue
Change it to bind command

Signed-off-by: DQ <dengq@vmware.com>
2020-03-13 14:58:45 +08:00
Ziming Zhang
695a2559be feat(cicd) use unified version as tag name, clean more
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2020-03-09 17:13:28 +08:00
Ziming Zhang
200c352c35 feat(cicd) use unified version as tag name
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2020-03-09 15:30:03 +08:00
Daniel Jiang
ae5ffce83a Update CSRF mechanism
This commit replaces beego's CSRF mechanism with gorilla's csrf library.
The criteria for requests to skip the csrf check remain the same.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-03-09 01:15:54 +08:00
wang yan
2b0b7576b2 Fix gc issue on clean the artifact trash
1, enable dao test for artifact trash
2, set default flush trash table to false
3, hanlder empty parameter in API call
4, add registry auth info into jobservice container

Signed-off-by: wang yan <wangyan@vmware.com>
2020-03-06 03:11:31 +08:00
Daniel Jiang
1823c984f7
Merge branch 'master' into redis-idle-timeout 2020-02-27 22:01:22 +08:00
stonezdj(Daojun Zhang)
a7e5873f46
Merge pull request #10821 from stonezdj/20200224_remove_notification
Remove registry notification and change core health check url
2020-02-25 13:34:37 +08:00
Ziming Zhang
94230b5e19 feat(cicd) fix some build problem
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2020-02-25 12:05:39 +08:00
stonezdj
6005101c95 Remove registry notification and change /api/ping
Update config.yaml.jinja to remove notification
Change api/ping in core/Dockerfile

Signed-off-by: stonezdj <stonezdj@gmail.com>
2020-02-25 11:24:21 +08:00
Wenkai Yin
bd204464f3 Remove dead code
Remove dead code

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-23 17:11:46 +08:00
dechen
e642a73280 Set redis idle timeout for core
Signed-off-by: dechen <xxyydream@gmail.com>
2020-02-23 12:31:56 +08:00
Daniel Pacak
70dda1387a chore: Configure Redis URL for Trivy adapter
Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
2020-02-13 17:57:02 +01:00
Daniel Pacak
a642667ffc chore(install): Add --with-trivy arg to the installation script
Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
2020-02-12 23:47:56 +01:00
Daniel Jiang
2064a1cd6d Switch to basic authentication for registry
1. Add basic authorizer for registry which modify the request
to add basic authorization header to request based on configuration.
2. Set basic auth header for proxy when accessing registry
3. Switche the registry to use basic auth by default and use the basic
authorizer to access Harbor.
4. Make necessary change to test cases, particularly
"test_robot_account.py" and "docker_api.py", because the error is
changed after siwtched to basic auth from token auth.  #10604 is opened
to track the follow up work.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-01-31 21:46:47 +09:00
Daniel Jiang
a087ba02e3 Populate basic auth information for registry
This commit updates `prepare` and templates to populate the credential
for registry for basic authentication.

A temporary flag `registry_use_basic_auth` was added to avoid breakage.
It MUST be removed before the release.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-12-31 14:50:46 +08:00
Wang Yan
2a63382236
Merge pull request #10047 from bitsf/makefile_clean
optimize the makefile process
2019-12-05 19:03:19 +08:00
Ziming Zhang
332f88ec8c add make clean
Change-Id: Ibe806972a19cd69bfd90be051cdc340c4d7c6afb
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2019-12-05 14:44:07 +08:00
Wenkai Yin(尹文开)
d145f4baf4
Merge pull request #10034 from ywk253100/191128_clean
Clean up admiral-related code
2019-12-04 17:33:31 +08:00
Daniel Jiang
7bb71db478
Merge pull request #10003 from ninjadq/migrator_miss_component_no_proxy
Add default domainname for no_proxy
2019-12-03 10:50:32 +08:00
Wenkai Yin
dd2bc0ecef Clean up admiral-related code
Clean up admiral-related code as it's useless

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-11-28 17:28:54 +08:00
DQ
79344887b9 Fix ca bundle path join issue
CA bundle name start with '/' will break the os path join

Signed-off-by: DQ <dengq@vmware.com>
2019-11-27 18:37:45 +08:00
DQ
ed6438cf69 Add default domainname for no_proxy
All internal service and known internal hostname shuold add to no_proxy by default

Signed-off-by: DQ <dengq@vmware.com>
2019-11-27 15:10:42 +08:00
He Weiwei
b8308f41a0 fix(prepaire,clair): disable clair updaters when its interval is 0
Closes #9961

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-11-22 03:31:20 +00:00