Upstream/harbor
1
0
Fork 0
mirror of https://github.com/goharbor/harbor.git synced 2024-11-17 07:45:24 +01:00
Code Issues Projects Releases Wiki Activity
9,645 Commits 49 Branches 315 Tags 351 MiB
4570a46823
Commit Graph

1012 Commits

Author SHA1 Message Date
DQ
f70339870a Enhance: Create shared to store shared ca 
this shared ca will mount to all harbor components

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-28 02:58:11 +08:00
guanxiatao
f96cfab100 Table notification_policy fix when updgrding to 2.0 
Signed-off-by: guanxiatao <guanxiatao@corp.netease.com>
 2020-04-27 20:13:46 +08:00
Wang Yan
add8dedc90
Fix the database upgrade issue (#11766) 
It's a workaround for issue https://github.com/goharbor/harbor/issues/11754

The phenomenon is the repository data is gone, but artifacts belong to the repository are still there.
To resolve it, just set the repository_id to a negative, and cannot duplicate.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-04-27 17:28:36 +08:00
DQ
90faf700f8 Enhance: output the stdout of gen cert script 
use popen replace check_all

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-27 10:43:22 +08:00
DQ
026e37e777 Fix chart museum absolute url issue 
if absolute url is enabled return true else set it to false

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-26 13:04:29 +08:00
DQ
599ca98c09 Hidden veriify client cert verfiy option 
Remove to avoid replication access core from external_url issue

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-23 10:14:36 +08:00
Daniel Jiang
2ecf0425a4 Remove the certs of notary signer 
Since `prepare` generates the certs as needed during installation, these
certs should not exist in the repo.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-04-21 13:20:19 +08:00
DQ
b728f04d0a Fix tls min version for registry 
cert,key,mintls should in the same context

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-20 19:19:15 +08:00
Qian Deng
9c7caddeae
Merge pull request #11635 from hyy0322/set-root-password-never-expire 
fix: set root password never expire
 2020-04-16 22:05:10 +08:00
Maosheng Ren
89e9ea0145
Merge pull request #11636 from danielpacak/bump_up_trivy_adapter_to_0.9.0 
chore(trivy): Bump up trivy adapter to 0.9.0
 2020-04-16 16:16:50 +08:00
Wang Yan
790064df2e
fix notification policy ugrade issue (#11627) 
Fixes #11624

All of the existing policies created v1.10 has no name, it fails the upgrade process.
When to set the unique constraint for policy name, the empty can be seen as duplicated key.

ERROR:  could not create unique index "notification_policy_name_key"
DETAIL:  Key (name)=() is duplicated.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-04-16 14:53:58 +08:00
Daniel Pacak
5c3abee135 chore(trivy): Bump up trivy adapter to 0.9.0 
- Vendor the latest Trivy release 0.6.0
- Configure TLS 1.2 as min version when TLS is enabled
- Add more tracing to adapter config to facilitate troubleshooting

Resolves: #11544

Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
 2020-04-16 08:40:27 +02:00
DQ
42c1095216 Fix cert issue of trivy 
Trivy can't access harbor from external if https enabled so inject cert to trivy container trust

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-16 10:52:03 +08:00
Yiyang Huang
4598f52057 fix: set root password never expire 
Signed-off-by: Yiyang Huang <huangyiyang@caicloud.io>
 2020-04-16 00:15:28 +08:00
He Weiwei
355c16943c chore(clair): bump up clair adapter version to 1.0.2 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-04-15 14:07:46 +00:00
He Weiwei
385aaac00d
Merge pull request #11620 from heww/fix-issue-11524 
feat(scanner): make Clair and Trivy as reserved name for scanners
 2020-04-15 15:21:35 +08:00
He Weiwei
f5487479dd feat(scanner): make Clair and Trivy as reserved name for scanners 
Closes #11524

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-04-15 02:26:02 +00:00
Qian Deng
95d7c9382b
Merge pull request #11592 from ninjadq/min_version_tls_to_12 
Min version tls to 12
 2020-04-14 18:12:55 +08:00
wang yan
ff2a6c7a01 add warning to registry binary name 
Fixes #11606

As we DO NOT want to user to execute GC in the container, rename it and append the warning message.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-04-14 15:16:50 +08:00
DQ
75f78b64b2 Set registry tls version to 1.2 
when internal tls enabled set min version of registry to 1.2

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-13 18:13:30 +08:00
jwangyangls
e28b5811f7
Merge pull request #11176 from jwangyangls/change-helm-version 
Separate swagger to get v2.0 swagger and chart swagger
 2020-04-10 17:12:00 +08:00
Yogi_Wang
33ed4fb67e Separate swagger to get v2.0 swagger and chart swagger 
1. Partial helm api version number clear
2. Separate swagger to get v2.0 swagger and chart swagger
3. router add chart swagger

Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-04-10 16:25:30 +08:00
DQ
e907cbe2b6 Fix health check for jobservice and regctl 
need cert when mTLS is enabled

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-09 20:35:46 +08:00
DQ
08ff622310 Remove lines not needed 
volume already defined above

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-09 20:06:51 +08:00
Ziming Zhang
572ebef685 feat(cicd) parameterize docker base image and external url 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-04-08 00:21:47 +08:00
DQ
6ae1b1dc97 Add missiong entrypoint file for trivy-adapter 
Signed-off-by: DQ <dengq@vmware.com>
 2020-04-07 10:39:07 +00:00
He Weiwei
3f567514b5
Merge pull request #11468 from wy65701436/remove-count-quota-code 
remove the chart handling in quota
 2020-04-07 16:51:07 +08:00
Daniel Jiang
5bcd015d6f
Merge pull request #11469 from ninjadq/clean_up_migrator 
Remove migrator flags in script
 2020-04-07 16:37:24 +08:00
Daniel Jiang
e064bd4c01
Merge pull request #11428 from ninjadq/fix_container_unhealth 
Fix container unhealth
 2020-04-07 15:57:00 +08:00
wang yan
a96d2f3746 remove the chart handling in quota 
1, remove the chartmuseum controller
2, doesn't handle chartrepo url in v2 middleware

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-04-07 15:26:34 +08:00
DQ
1ae50b8d66 Remove migrator flags in script 
Because migrator tool removed

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-07 14:57:10 +08:00
DQ
4a836ea975 Fix health check url 
health check url should depend on internal https

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-07 03:35:52 +00:00
wang yan
44825e819e deprecate quota count on artifact 
Fixes #11241

1, remove count quota from quota manager
2, remove count in DB scheme
3, remove UI relates on quota
4, update UT, API test and UI UT.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-04-06 16:56:11 +08:00
DQ
cdb675bf3d Add proxy cert file to jobservice when https enabled 
jobservice may request via absolute path of url to harbor

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-04 17:44:34 +00:00
DQ
23ed189ed4 Add SAN to gencert script 
add localhost and 127.0.0.1 to SAN

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-04 17:44:34 +00:00
He Weiwei
77a8c3205f fix(prepare): not accpet items of false value in external_redis 
Item in yaml without value will be as None in python, which will make
the password of redis as `None` in `get_redis_configs`. This fix will
not accept items of `false value` in `external_redis` configurations.

Closes #11367

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-04-03 04:09:26 +00:00
Qian Deng
a702c32346
Merge pull request #11063 from ninjadq/fix_syslog_dir_in_tpl 
Fix: fix logrotate is dir issue
 2020-04-02 11:37:29 +08:00
Qian Deng
0319baabcb
Merge pull request #11381 from ninjadq/enhance_migrate_config 
Enhance migrate config
 2020-04-02 10:00:38 +08:00
DQ
dc271e1a87 Add packaging to pipenv 
Signed-off-by: DQ <dengq@vmware.com>
 2020-04-01 22:54:47 +08:00
DQ
d636f2ea5c Enhance help message 
Provide more info in help message
Add requried opition and they will show missing option if you are not provide them instead of Exception

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-01 17:02:59 +08:00
DQ
b2e1905e7a Enhance: Stop upgrade when input version less then 1.9.0 
The migration script should failure early when version is not supported

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-01 15:35:49 +08:00
Ziming Zhang
ae7834af0b feat(cicd) fix build base image 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-03-26 10:55:40 +08:00
Qian Deng
9e101b73a4
Merge pull request #11156 from ninjadq/migrate_config_to_harbor2 
Migrate config to harbor2
 2020-03-25 16:02:18 +08:00
DQ
85ec0e7820 Enhance: Refactor the migration structure 
1. Refactor structure of migrate file
2. fix some previous bugs

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-23 21:26:28 +08:00
DQ
444678fe07 Fix: module path raise exception when it is loop 
add test for loop

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-23 19:29:59 +08:00
Maosheng Ren
1dbec0c1d7
Fix a typo in the help message of install.sh (#11167) 
Signed-off-by: ren maosheng <stevenr@vmware.com>
 2020-03-23 10:30:37 +08:00
DQ
e8bb977ae1 Feat: Upgrade configs to harbor 2.0 
add migrate files for harbor 2.0

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-20 15:20:32 +08:00
DQ
1e0c9f7231 Feat: Add config migrator to prepare 
deprecated migrator container and move config migration to prepare

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-20 03:04:10 +08:00
Steven Zou
2859cd8b69
Merge pull request #11134 from danielpacak/feat/issue_11090/trivy_skip_update_flag 
feat(trivy): Configure Trivy to skip database updates
 2020-03-19 18:13:08 +08:00
Wenkai Yin(尹文开)
9ebcf95758
Merge pull request #11122 from ywk253100/200318_replication_task 
Increase the length the columns (src_resource, dst_resource)of replication_task
 2020-03-19 12:16:27 +08:00
DQ
f18a546429 Fix: return error when internal_tls_not_provided 
When iinternal_tls is empty, prepare should works as usual

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-19 10:37:58 +08:00
Daniel Pacak
7325105714 feat(trivy): Configure Trivy to skip database updates 
Resolves: #11090

Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
 2020-03-18 17:11:47 +01:00
DQ
6e8d44101f Enhance: User can generate cert by their own ca key pair 
User can put their ca key pair on internal cert dir and name them to `harbor_internal_ca.key` and `harbor_internal_ca.crt` we wil use them to generate other certs

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:10 +08:00
DQ
b93092e012 Add tls for trivy 
Add trivy tls cert files
Add tivey tls env and config
enhance gencert

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:10 +08:00
DQ
c954969bcd Add mTLS configs 
mTLS only enabled in jobservice and registryctl

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:10 +08:00
DQ
c5d73e6a0c Add switch to https 
use switch to make decision whether mTLS or server TLS

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:10 +08:00
DQ
454382149f TLS update for chart, clairadapter, registry 
Remove trustca in chartmuseum
Remove trustca in registry
Add tls in clair-adapter

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:10 +08:00
DQ
03e11c63c7 Fix docker file with secure tls change 
Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:10 +08:00
DQ
dcc6950af7 Feat: auto install ca in registry 
Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
DQ
b852605193 Feat: enable mtls in harbor replication 
Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
DQ
40e67f3b14 Feat: Enable mtls for registry 
Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
DQ
07a1d51693 Feat: enable tls in registryctlAdd tls related code in registryctl 
Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
DQ
da359f609f Feat: enable mtls in core 
add mtls related code in core

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
DQ
a4855cca36 Feat: update prepare to support tls 
update makefile
add model for prepare
update jinja template for prepare

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
Wang Yan
b4e941e961
drop table access log in migration (#11118) 
Use the audit log instead, the access log table should be dropped after migration

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-18 19:04:38 +08:00
Wenkai Yin
ac9658bc1e Increase the length the columns (src_resource, dst_resource)of replication_task 
Fixes #10786 by increaseing the length of src_resource and dst_resource to 256

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-18 17:05:32 +08:00
He Weiwei
7d20154db5
fix: remove old artifact model (#11112) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-18 14:20:06 +08:00
Daniel Pacak
9c13116963 chore(trivy): Allow configuring HTTP(S) proxy 
Resolves: #11032

Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
 2020-03-16 12:26:49 +01:00
Daniel Pacak
46fb43bc25 chore: Bump up Trivy adapter to v0.4.0 
Allows configuring SCANNER_TRIVY_GITHUB_TOKEN environment variable,
which is passed to trivy executable binary when it starts scanning
a given artifact.

This is to increase GitHub requests rate limit from 60 per hours
(for anonymous requests) to 5000 when Trivy download its
vulnerabilities database.

Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
 2020-03-16 09:53:16 +01:00
DQ
1eeea6b888 Fix: fix logrotate is dir issue 
Change it to bind command

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-13 14:58:45 +08:00
Wenkai Yin
a4a1913598 Repair the count usage during the upgrading 
As the count quota is against artifact rather than tag in 2.0, the count usage should be recalculated

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-13 13:59:48 +08:00
Ted Guan
4ac31c6d46
Add API for query supported event types and notify types; Return policy name in last trigger info; Remove project_id unique constraint in table notification_policy (#11029) 
Signed-off-by: guanxiatao <guanxiatao@corp.netease.com>
 2020-03-11 18:06:58 +08:00
Wenkai Yin(尹文开)
8452100148
Merge pull request #10942 from ywk253100/200305_reference 
Persistent the URLs and annotations of artifact references in database
 2020-03-11 16:20:18 +08:00
Wang Yan
bd7940217a
upgrade golang version to v1.13.8 (#11006) 
The vesrion contains two security bug fix - CVE-2020-0601, CVE-2020-7919

More details, see the golang milestone:

https://github.com/golang/go/issues?q=milestone%3AGo1.13.8+label%3ACherryPickApproved

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-11 12:20:06 +08:00
Ziming Zhang
695a2559be feat(cicd) use unified version as tag name, clean more 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-03-09 17:13:28 +08:00
Ziming Zhang
200c352c35 feat(cicd) use unified version as tag name 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-03-09 15:30:03 +08:00
Daniel Jiang
6d89553c4d
Merge pull request #10937 from reasonerjt/csrf-2.0 
Update CSRF mechanism
 2020-03-09 12:31:08 +08:00
Wenkai Yin(尹文开)
75eb7a8c5a
Merge pull request #10955 from wy65701436/migrate-access 
add sql for migrating access log
 2020-03-09 10:00:08 +08:00
Daniel Jiang
ae5ffce83a Update CSRF mechanism 
This commit replaces beego's CSRF mechanism with gorilla's csrf library.
The criteria for requests to skip the csrf check remain the same.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-03-09 01:15:54 +08:00
wang yan
288c7790d0 add sql for migrating access log 
1, loop each access log, change to resource/resource_type, and insert into audit log
2, loop each first push operation, change it to create repository and insert into audit log.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-06 12:06:12 +08:00
wang yan
2b0b7576b2 Fix gc issue on clean the artifact trash 
1, enable dao test for artifact trash
2, set default flush trash table to false
3, hanlder empty parameter in API call
4, add registry auth info into jobservice container

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-06 03:11:31 +08:00
stonezdj(Daojun Zhang)
49619e1907
Merge pull request #10939 from wy65701436/access-log-mgr 
add audit logs API
 2020-03-05 16:24:21 +08:00
wang yan
df237a5b17 add audit logs API 
1, add API entry for get audit logs
2. add audit log manager to hanlder CRUD

Use the new format of audit log to cover differernt resource, artifact/tag/repostory/project

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-05 11:40:51 +08:00
Wenkai Yin
76c04b0219 Persistent the URLs and annotations of artifact references in database 
Persistent the URLs and annotations of artifact references in database

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-05 10:54:45 +08:00
Will Sun
a5d9a3b65d
Merge pull request #10863 from AllForNothing/api-center 
Fix Api cennter
 2020-03-05 10:00:15 +08:00
Wenkai Yin(尹文开)
4fa4c4e74c
Merge pull request #10815 from cd1989/redis-idle-timeout 
Set redis idle timeout for core
 2020-03-03 14:10:22 +08:00
Wenkai Yin
4c9b59c904 Migrate artifact data in 2.0 
Abstract extra attributes and annotations for artifacts stored in database

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-28 18:09:02 +08:00
jwangyangls
572510c82d
Merge pull request #10755 from Snaacker/master 
Fix typo
 2020-02-28 11:59:51 +08:00
Daniel Jiang
1823c984f7
Merge branch 'master' into redis-idle-timeout 2020-02-27 22:01:22 +08:00
AllForNothing
d41c5496a2 Fix Api cennter 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-02-27 15:55:20 +08:00
Wenkai Yin
02c2647e1e Use the repository name of artifact model 
As we store the repository name in the artifact table, we can use it direclty in the code to reduce the database query

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-26 13:37:09 +08:00
stonezdj(Daojun Zhang)
a7e5873f46
Merge pull request #10821 from stonezdj/20200224_remove_notification 
Remove registry notification and change core health check url
 2020-02-25 13:34:37 +08:00
Ziming Zhang
94230b5e19 feat(cicd) fix some build problem 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-02-25 12:05:39 +08:00
stonezdj
6005101c95 Remove registry notification and change /api/ping 
Update config.yaml.jinja to remove notification
Change api/ping in core/Dockerfile

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2020-02-25 11:24:21 +08:00
Wang Yan
948d45604c Revise the GC job flow, 
1, set harbor to readonly
2, select the candidate artifacts from Harbor DB.
3, call registry API(--delete-untagged=false) to delete manifest bases on the results of #2
4, clean keys of redis DB of registry, clean artifact trash and untagged from DB.
5, roll back readonly.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-02-24 18:29:55 +08:00
Wenkai Yin
bd204464f3 Remove dead code 
Remove dead code

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-23 17:11:46 +08:00
dechen
e642a73280 Set redis idle timeout for core 
Signed-off-by: dechen <xxyydream@gmail.com>
 2020-02-23 12:31:56 +08:00
Wenkai Yin
9312b788dc Upgrade the artifact table 
Split the table artifact into artifact and tags, and populate related data

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-21 20:37:31 +08:00
Wang Yan
22021a988e
Merge pull request #10753 from wy65701436/artifact-trash 
add artifact trash manager
 2020-02-19 19:14:48 +08:00
Steven Zou
f1374737f6
Merge pull request #10694 from danielpacak/feature/install_with_trivy 
chore(install): Add --with-trivy arg to the installation script
 2020-02-19 16:27:57 +08:00