Upstream/harbor
1
0
Fork 0
mirror of https://github.com/goharbor/harbor.git synced 2024-11-18 16:25:16 +01:00
Code Issues Projects Releases Wiki Activity
10,994 Commits 54 Branches 316 Tags 351 MiB
719155d595
Commit Graph

250 Commits

Author SHA1 Message Date
stonezdj(Daojun Zhang)
be4e6a5985
Merge pull request #13537 from stonezdj/201118_add_more_registry_type 
Add more registry type to proxy cache
 2020-11-26 11:16:16 +08:00
Ziming Zhang
d55f55aeb9 fix(chartmuseum) compatible s3 cache fail 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-11-25 17:00:16 +08:00
stonezdj
e667121a34 Add more registry type to proxy cache 
Includes: azure-acr, aws-ecr, google-gcr
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2020-11-18 10:38:07 +08:00
DQ
0c9faea294 Clean up Clair in prepare script 
Signed-off-by: DQ <dengq@vmware.com>
 2020-11-10 11:39:18 +08:00
DQ
eb470501be Add metrics to Harbor Core 
1. Add configs in prepare
 2. Add models and config items in Core
 3. Encapdulate getting metric in commom package
 4. Add a middleware for global request to collect 3 metrics

Signed-off-by: DQ <dengq@vmware.com>
 2020-11-03 14:33:10 +08:00
DQ
184e89365b Fix internal tls config upgrade issue 
internal tls config upgrade is not included in template, this pr is to add it.

Signed-off-by: DQ <dengq@vmware.com>
 2020-09-25 09:54:31 +08:00
DQ
17f3bfccb4 Fix trivy setting in upgrading script 
Signed-off-by: DQ <dengq@vmware.com>
 2020-09-08 18:15:57 +08:00
Ziming Zhang
ff19dd499c fix(jobservice) redis sentinel failover hang 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-08-26 10:42:44 +08:00
Qian Deng
5dbbfa76d3
Merge pull request #12766 from ninjadq/add_log_dependency_to_trivy 
Add log denpendency ti trivy
 2020-08-13 18:23:09 +08:00
DQ
a251e90507 Add log denpendency ti trivy 
To void trivy can not start issue

Signed-off-by: DQ <dengq@vmware.com>
 2020-08-13 11:35:21 +08:00
DQ
7ba498be5b Fix: append trivy every time run migrate 
Signed-off-by: DQ <dengq@vmware.com>
 2020-08-11 17:43:25 +08:00
Qian Deng
fbef7fd088
Merge pull request #12651 from ninjadq/add_migration_2_1_0 
Add migration 2.1.0
 2020-08-03 15:59:28 +08:00
DQ
1e32792dc5 Add migration 2.1.0 
db_max_open_comms should be 1000 if its value between 100 and 1000

Signed-off-by: DQ <dengq@vmware.com>
 2020-08-03 15:17:41 +08:00
DQ
d3ab9d7c6b Add internal tls configs for portal 
add related file, config, command to enabled https for portal

Signed-off-by: DQ <dengq@vmware.com>
 2020-07-31 12:10:47 +08:00
DQ
d7618a6274 Fix: beego app config port hardcode 
the port should be flexible depend on the internal tls

Signed-off-by: DQ <dengq@vmware.com>
 2020-07-27 15:35:43 +08:00
Ziming Zhang
8857e89e40 feature(redis) support redis sentinel 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-07-19 21:19:03 +08:00
Qian Deng
bd26c294e8
Merge pull request #12341 from ninjadq/support_multi_down_version 
Enhance: Support multi downversion in migration
 2020-07-15 23:39:11 +08:00
He Weiwei
039aef5356 refactor: remove initialization of clair db 
To fetch vulnerability database updated time of the Clair had moved to
the Clair adapter so removes the initialization of clair db in the core.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-07-09 15:26:14 +00:00
DQ
4617e0ff38 Enhance: Support multi downversion in migration 
1. Change down version to list to accept multi verstion value
2. Update search function use BFS to find migration path
2. Add test case

Signed-off-by: DQ <dengq@vmware.com>
 2020-07-07 21:36:58 +08:00
Wenkai Yin
02690d1d04 Suport filtering registries by type in listing registry API 
Suport filtering registries by type in listing registry API

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-07-07 10:30:46 +08:00
DQ
d0ddd61ad9 Fix Amazon S3 storage not work 
The Chartmuseum S3 client need set an Env variable
Ref: https://github.com/helm/chartmuseum/issues/280

Signed-off-by: DQ <dengq@vmware.com>
 2020-06-30 15:16:18 +08:00
Wang Yan
dec8397c21
Add api to delete blob and manifest (#12006) 
* Add api to delete blob and manifest

Enable the capability of registry controller to delete blob and manifest

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-06-06 01:34:23 +08:00
Qian Deng
9e1302211b
Merge pull request #12072 from ninjadq/add_timeout_in_nginx_config 
Add timeout in nginx config
 2020-06-02 15:14:42 +08:00
He Weiwei
d97be71234 refactor(configuration): cleanup unneeded CLAIR_URL configuration in core 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-05-29 07:27:50 +00:00
DQ
278338e401 Add timount on nginx configs 
set timeout to 900

Signed-off-by: DQ <dengq@vmware.com>
 2020-05-26 16:18:35 +08:00
DQ
715685ae51 Remove tls1.1 in notary 
Signed-off-by: DQ <dengq@vmware.com>
 2020-05-26 16:11:57 +08:00
DQ
b06e19a637 Fix: GCS storage gc issue 
Mount gcs key to registryctl

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-29 15:04:16 +08:00
Qian Deng
9469252e85
Merge pull request #11745 from ninjadq/mount_ca_bundle 
Enhance: Create shared to store shared ca
 2020-04-28 10:19:26 +08:00
DQ
f70339870a Enhance: Create shared to store shared ca 
this shared ca will mount to all harbor components

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-28 02:58:11 +08:00
DQ
90faf700f8 Enhance: output the stdout of gen cert script 
use popen replace check_all

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-27 10:43:22 +08:00
DQ
026e37e777 Fix chart museum absolute url issue 
if absolute url is enabled return true else set it to false

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-26 13:04:29 +08:00
DQ
599ca98c09 Hidden veriify client cert verfiy option 
Remove to avoid replication access core from external_url issue

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-23 10:14:36 +08:00
Daniel Jiang
2ecf0425a4 Remove the certs of notary signer 
Since `prepare` generates the certs as needed during installation, these
certs should not exist in the repo.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-04-21 13:20:19 +08:00
DQ
b728f04d0a Fix tls min version for registry 
cert,key,mintls should in the same context

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-20 19:19:15 +08:00
Daniel Pacak
5c3abee135 chore(trivy): Bump up trivy adapter to 0.9.0 
- Vendor the latest Trivy release 0.6.0
- Configure TLS 1.2 as min version when TLS is enabled
- Add more tracing to adapter config to facilitate troubleshooting

Resolves: #11544

Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
 2020-04-16 08:40:27 +02:00
DQ
42c1095216 Fix cert issue of trivy 
Trivy can't access harbor from external if https enabled so inject cert to trivy container trust

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-16 10:52:03 +08:00
DQ
75f78b64b2 Set registry tls version to 1.2 
when internal tls enabled set min version of registry to 1.2

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-13 18:13:30 +08:00
DQ
08ff622310 Remove lines not needed 
volume already defined above

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-09 20:06:51 +08:00
Ziming Zhang
572ebef685 feat(cicd) parameterize docker base image and external url 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-04-08 00:21:47 +08:00
DQ
4a836ea975 Fix health check url 
health check url should depend on internal https

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-07 03:35:52 +00:00
DQ
cdb675bf3d Add proxy cert file to jobservice when https enabled 
jobservice may request via absolute path of url to harbor

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-04 17:44:34 +00:00
DQ
23ed189ed4 Add SAN to gencert script 
add localhost and 127.0.0.1 to SAN

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-04 17:44:34 +00:00
He Weiwei
77a8c3205f fix(prepare): not accpet items of false value in external_redis 
Item in yaml without value will be as None in python, which will make
the password of redis as `None` in `get_redis_configs`. This fix will
not accept items of `false value` in `external_redis` configurations.

Closes #11367

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-04-03 04:09:26 +00:00
Qian Deng
a702c32346
Merge pull request #11063 from ninjadq/fix_syslog_dir_in_tpl 
Fix: fix logrotate is dir issue
 2020-04-02 11:37:29 +08:00
DQ
dc271e1a87 Add packaging to pipenv 
Signed-off-by: DQ <dengq@vmware.com>
 2020-04-01 22:54:47 +08:00
DQ
d636f2ea5c Enhance help message 
Provide more info in help message
Add requried opition and they will show missing option if you are not provide them instead of Exception

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-01 17:02:59 +08:00
DQ
b2e1905e7a Enhance: Stop upgrade when input version less then 1.9.0 
The migration script should failure early when version is not supported

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-01 15:35:49 +08:00
Qian Deng
9e101b73a4
Merge pull request #11156 from ninjadq/migrate_config_to_harbor2 
Migrate config to harbor2
 2020-03-25 16:02:18 +08:00
DQ
85ec0e7820 Enhance: Refactor the migration structure 
1. Refactor structure of migrate file
2. fix some previous bugs

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-23 21:26:28 +08:00
DQ
444678fe07 Fix: module path raise exception when it is loop 
add test for loop

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-23 19:29:59 +08:00
DQ
e8bb977ae1 Feat: Upgrade configs to harbor 2.0 
add migrate files for harbor 2.0

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-20 15:20:32 +08:00
DQ
1e0c9f7231 Feat: Add config migrator to prepare 
deprecated migrator container and move config migration to prepare

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-20 03:04:10 +08:00
Steven Zou
2859cd8b69
Merge pull request #11134 from danielpacak/feat/issue_11090/trivy_skip_update_flag 
feat(trivy): Configure Trivy to skip database updates
 2020-03-19 18:13:08 +08:00
DQ
f18a546429 Fix: return error when internal_tls_not_provided 
When iinternal_tls is empty, prepare should works as usual

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-19 10:37:58 +08:00
Daniel Pacak
7325105714 feat(trivy): Configure Trivy to skip database updates 
Resolves: #11090

Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
 2020-03-18 17:11:47 +01:00
DQ
6e8d44101f Enhance: User can generate cert by their own ca key pair 
User can put their ca key pair on internal cert dir and name them to `harbor_internal_ca.key` and `harbor_internal_ca.crt` we wil use them to generate other certs

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:10 +08:00
DQ
b93092e012 Add tls for trivy 
Add trivy tls cert files
Add tivey tls env and config
enhance gencert

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:10 +08:00
DQ
c954969bcd Add mTLS configs 
mTLS only enabled in jobservice and registryctl

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:10 +08:00
DQ
c5d73e6a0c Add switch to https 
use switch to make decision whether mTLS or server TLS

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:10 +08:00
DQ
454382149f TLS update for chart, clairadapter, registry 
Remove trustca in chartmuseum
Remove trustca in registry
Add tls in clair-adapter

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:10 +08:00
DQ
03e11c63c7 Fix docker file with secure tls change 
Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:10 +08:00
DQ
dcc6950af7 Feat: auto install ca in registry 
Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
DQ
b852605193 Feat: enable mtls in harbor replication 
Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
DQ
40e67f3b14 Feat: Enable mtls for registry 
Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
DQ
07a1d51693 Feat: enable tls in registryctlAdd tls related code in registryctl 
Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
DQ
da359f609f Feat: enable mtls in core 
add mtls related code in core

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
DQ
a4855cca36 Feat: update prepare to support tls 
update makefile
add model for prepare
update jinja template for prepare

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
Daniel Pacak
9c13116963 chore(trivy): Allow configuring HTTP(S) proxy 
Resolves: #11032

Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
 2020-03-16 12:26:49 +01:00
Daniel Pacak
46fb43bc25 chore: Bump up Trivy adapter to v0.4.0 
Allows configuring SCANNER_TRIVY_GITHUB_TOKEN environment variable,
which is passed to trivy executable binary when it starts scanning
a given artifact.

This is to increase GitHub requests rate limit from 60 per hours
(for anonymous requests) to 5000 when Trivy download its
vulnerabilities database.

Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
 2020-03-16 09:53:16 +01:00
DQ
1eeea6b888 Fix: fix logrotate is dir issue 
Change it to bind command

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-13 14:58:45 +08:00
Ziming Zhang
695a2559be feat(cicd) use unified version as tag name, clean more 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-03-09 17:13:28 +08:00
Ziming Zhang
200c352c35 feat(cicd) use unified version as tag name 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-03-09 15:30:03 +08:00
Daniel Jiang
ae5ffce83a Update CSRF mechanism 
This commit replaces beego's CSRF mechanism with gorilla's csrf library.
The criteria for requests to skip the csrf check remain the same.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-03-09 01:15:54 +08:00
wang yan
2b0b7576b2 Fix gc issue on clean the artifact trash 
1, enable dao test for artifact trash
2, set default flush trash table to false
3, hanlder empty parameter in API call
4, add registry auth info into jobservice container

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-06 03:11:31 +08:00
Daniel Jiang
1823c984f7
Merge branch 'master' into redis-idle-timeout 2020-02-27 22:01:22 +08:00
stonezdj(Daojun Zhang)
a7e5873f46
Merge pull request #10821 from stonezdj/20200224_remove_notification 
Remove registry notification and change core health check url
 2020-02-25 13:34:37 +08:00
Ziming Zhang
94230b5e19 feat(cicd) fix some build problem 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-02-25 12:05:39 +08:00
stonezdj
6005101c95 Remove registry notification and change /api/ping 
Update config.yaml.jinja to remove notification
Change api/ping in core/Dockerfile

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2020-02-25 11:24:21 +08:00
Wenkai Yin
bd204464f3 Remove dead code 
Remove dead code

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-23 17:11:46 +08:00
dechen
e642a73280 Set redis idle timeout for core 
Signed-off-by: dechen <xxyydream@gmail.com>
 2020-02-23 12:31:56 +08:00
Daniel Pacak
70dda1387a chore: Configure Redis URL for Trivy adapter 
Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
 2020-02-13 17:57:02 +01:00
Daniel Pacak
a642667ffc chore(install): Add --with-trivy arg to the installation script 
Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
 2020-02-12 23:47:56 +01:00
Daniel Jiang
2064a1cd6d Switch to basic authentication for registry 
1. Add basic authorizer for registry which modify the request
to add basic authorization header to request based on configuration.
2. Set basic auth header for proxy when accessing registry
3. Switche the registry to use basic auth by default and use the basic
authorizer to access Harbor.
4. Make necessary change to test cases, particularly
"test_robot_account.py" and "docker_api.py", because the error is
changed after siwtched to basic auth from token auth.  #10604 is opened
to track the follow up work.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-01-31 21:46:47 +09:00
Daniel Jiang
a087ba02e3 Populate basic auth information for registry 
This commit updates `prepare` and templates to populate the credential
for registry for basic authentication.

A temporary flag `registry_use_basic_auth` was added to avoid breakage.
It MUST be removed before the release.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-12-31 14:50:46 +08:00
Wang Yan
2a63382236
Merge pull request #10047 from bitsf/makefile_clean 
optimize the makefile process
 2019-12-05 19:03:19 +08:00
Ziming Zhang
332f88ec8c add make clean 
Change-Id: Ibe806972a19cd69bfd90be051cdc340c4d7c6afb
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2019-12-05 14:44:07 +08:00
Wenkai Yin(尹文开)
d145f4baf4
Merge pull request #10034 from ywk253100/191128_clean 
Clean up admiral-related code
 2019-12-04 17:33:31 +08:00
Daniel Jiang
7bb71db478
Merge pull request #10003 from ninjadq/migrator_miss_component_no_proxy 
Add default domainname for no_proxy
 2019-12-03 10:50:32 +08:00
Wenkai Yin
dd2bc0ecef Clean up admiral-related code 
Clean up admiral-related code as it's useless

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-11-28 17:28:54 +08:00
DQ
79344887b9 Fix ca bundle path join issue 
CA bundle name start with '/' will break the os path join

Signed-off-by: DQ <dengq@vmware.com>
 2019-11-27 18:37:45 +08:00
DQ
ed6438cf69 Add default domainname for no_proxy 
All internal service and known internal hostname shuold add to no_proxy by default

Signed-off-by: DQ <dengq@vmware.com>
 2019-11-27 15:10:42 +08:00
He Weiwei
b8308f41a0 fix(prepaire,clair): disable clair updaters when its interval is 0 
Closes #9961

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-11-22 03:31:20 +00:00
Daniel Jiang
2fcd174e4b
Merge pull request #9828 from wy65701436/cii-docker-base 
add base images when to build harbor assets
 2019-11-15 14:24:11 +08:00
He Weiwei
fe69a5df99 build(scanner-adapter): bump up clair adapter to v1.0.1-rc2 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-11-13 02:35:21 +00:00
wang yan
47793e77e3 update base file name ane pass base version to build file 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-11-12 19:12:49 +08:00
Wang Yan
544cc98971 add base images when to build harbor assets 
* add base images when to build harbor assets

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-11-12 15:38:51 +08:00
Wang Yan
6da183d576
Merge pull request #9800 from ninjadq/failure_earlier_of_ca_bundle_permission_check 
Failure earlier of ca bundle permission check
 2019-11-11 14:09:21 +08:00
Wang Yan
0b09bd14b1
Merge pull request #9756 from ninjadq/add_ignore_media_type 
Add ignore mediatypes for registry
 2019-11-08 18:34:13 +08:00
DQ
80c3e76b5a check the permission of ca bundle file 
CA bundle need check before use

Signed-off-by: DQ <dengq@vmware.com>
 2019-11-08 15:34:17 +08:00
Daniel Jiang
06e4e124d8
Refine request handle process (#9760) 
* Refine request handle process

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-11-07 13:02:17 +08:00