Upstream/harbor
1
0
Fork 0
mirror of https://github.com/goharbor/harbor.git synced 2024-12-02 15:14:09 +01:00
Code Issues Projects Releases Wiki Activity
9,139 Commits 53 Branches 317 Tags 335 MiB
e3cbfac0cc
Commit Graph

950 Commits

Author SHA1 Message Date
DQ
da359f609f Feat: enable mtls in core 
add mtls related code in core

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
DQ
a4855cca36 Feat: update prepare to support tls 
update makefile
add model for prepare
update jinja template for prepare

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
Wang Yan
b4e941e961
drop table access log in migration (#11118) 
Use the audit log instead, the access log table should be dropped after migration

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-18 19:04:38 +08:00
Wenkai Yin
ac9658bc1e Increase the length the columns (src_resource, dst_resource)of replication_task 
Fixes #10786 by increaseing the length of src_resource and dst_resource to 256

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-18 17:05:32 +08:00
He Weiwei
7d20154db5
fix: remove old artifact model (#11112) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-18 14:20:06 +08:00
Daniel Pacak
9c13116963 chore(trivy): Allow configuring HTTP(S) proxy 
Resolves: #11032

Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
 2020-03-16 12:26:49 +01:00
Daniel Pacak
46fb43bc25 chore: Bump up Trivy adapter to v0.4.0 
Allows configuring SCANNER_TRIVY_GITHUB_TOKEN environment variable,
which is passed to trivy executable binary when it starts scanning
a given artifact.

This is to increase GitHub requests rate limit from 60 per hours
(for anonymous requests) to 5000 when Trivy download its
vulnerabilities database.

Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
 2020-03-16 09:53:16 +01:00
DQ
1eeea6b888 Fix: fix logrotate is dir issue 
Change it to bind command

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-13 14:58:45 +08:00
Wenkai Yin
a4a1913598 Repair the count usage during the upgrading 
As the count quota is against artifact rather than tag in 2.0, the count usage should be recalculated

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-13 13:59:48 +08:00
Ted Guan
4ac31c6d46
Add API for query supported event types and notify types; Return policy name in last trigger info; Remove project_id unique constraint in table notification_policy (#11029) 
Signed-off-by: guanxiatao <guanxiatao@corp.netease.com>
 2020-03-11 18:06:58 +08:00
Wenkai Yin(尹文开)
8452100148
Merge pull request #10942 from ywk253100/200305_reference 
Persistent the URLs and annotations of artifact references in database
 2020-03-11 16:20:18 +08:00
Wang Yan
bd7940217a
upgrade golang version to v1.13.8 (#11006) 
The vesrion contains two security bug fix - CVE-2020-0601, CVE-2020-7919

More details, see the golang milestone:

https://github.com/golang/go/issues?q=milestone%3AGo1.13.8+label%3ACherryPickApproved

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-11 12:20:06 +08:00
Ziming Zhang
695a2559be feat(cicd) use unified version as tag name, clean more 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-03-09 17:13:28 +08:00
Ziming Zhang
200c352c35 feat(cicd) use unified version as tag name 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-03-09 15:30:03 +08:00
Daniel Jiang
6d89553c4d
Merge pull request #10937 from reasonerjt/csrf-2.0 
Update CSRF mechanism
 2020-03-09 12:31:08 +08:00
Wenkai Yin(尹文开)
75eb7a8c5a
Merge pull request #10955 from wy65701436/migrate-access 
add sql for migrating access log
 2020-03-09 10:00:08 +08:00
Daniel Jiang
ae5ffce83a Update CSRF mechanism 
This commit replaces beego's CSRF mechanism with gorilla's csrf library.
The criteria for requests to skip the csrf check remain the same.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-03-09 01:15:54 +08:00
wang yan
288c7790d0 add sql for migrating access log 
1, loop each access log, change to resource/resource_type, and insert into audit log
2, loop each first push operation, change it to create repository and insert into audit log.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-06 12:06:12 +08:00
wang yan
2b0b7576b2 Fix gc issue on clean the artifact trash 
1, enable dao test for artifact trash
2, set default flush trash table to false
3, hanlder empty parameter in API call
4, add registry auth info into jobservice container

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-06 03:11:31 +08:00
stonezdj(Daojun Zhang)
49619e1907
Merge pull request #10939 from wy65701436/access-log-mgr 
add audit logs API
 2020-03-05 16:24:21 +08:00
wang yan
df237a5b17 add audit logs API 
1, add API entry for get audit logs
2. add audit log manager to hanlder CRUD

Use the new format of audit log to cover differernt resource, artifact/tag/repostory/project

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-05 11:40:51 +08:00
Wenkai Yin
76c04b0219 Persistent the URLs and annotations of artifact references in database 
Persistent the URLs and annotations of artifact references in database

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-05 10:54:45 +08:00
Will Sun
a5d9a3b65d
Merge pull request #10863 from AllForNothing/api-center 
Fix Api cennter
 2020-03-05 10:00:15 +08:00
Wenkai Yin(尹文开)
4fa4c4e74c
Merge pull request #10815 from cd1989/redis-idle-timeout 
Set redis idle timeout for core
 2020-03-03 14:10:22 +08:00
Wenkai Yin
4c9b59c904 Migrate artifact data in 2.0 
Abstract extra attributes and annotations for artifacts stored in database

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-28 18:09:02 +08:00
jwangyangls
572510c82d
Merge pull request #10755 from Snaacker/master 
Fix typo
 2020-02-28 11:59:51 +08:00
Daniel Jiang
1823c984f7
Merge branch 'master' into redis-idle-timeout 2020-02-27 22:01:22 +08:00
AllForNothing
d41c5496a2 Fix Api cennter 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-02-27 15:55:20 +08:00
Wenkai Yin
02c2647e1e Use the repository name of artifact model 
As we store the repository name in the artifact table, we can use it direclty in the code to reduce the database query

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-26 13:37:09 +08:00
stonezdj(Daojun Zhang)
a7e5873f46
Merge pull request #10821 from stonezdj/20200224_remove_notification 
Remove registry notification and change core health check url
 2020-02-25 13:34:37 +08:00
Ziming Zhang
94230b5e19 feat(cicd) fix some build problem 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-02-25 12:05:39 +08:00
stonezdj
6005101c95 Remove registry notification and change /api/ping 
Update config.yaml.jinja to remove notification
Change api/ping in core/Dockerfile

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2020-02-25 11:24:21 +08:00
Wang Yan
948d45604c Revise the GC job flow, 
1, set harbor to readonly
2, select the candidate artifacts from Harbor DB.
3, call registry API(--delete-untagged=false) to delete manifest bases on the results of #2
4, clean keys of redis DB of registry, clean artifact trash and untagged from DB.
5, roll back readonly.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-02-24 18:29:55 +08:00
Wenkai Yin
bd204464f3 Remove dead code 
Remove dead code

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-23 17:11:46 +08:00
dechen
e642a73280 Set redis idle timeout for core 
Signed-off-by: dechen <xxyydream@gmail.com>
 2020-02-23 12:31:56 +08:00
Wenkai Yin
9312b788dc Upgrade the artifact table 
Split the table artifact into artifact and tags, and populate related data

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-21 20:37:31 +08:00
Wang Yan
22021a988e
Merge pull request #10753 from wy65701436/artifact-trash 
add artifact trash manager
 2020-02-19 19:14:48 +08:00
Steven Zou
f1374737f6
Merge pull request #10694 from danielpacak/feature/install_with_trivy 
chore(install): Add --with-trivy arg to the installation script
 2020-02-19 16:27:57 +08:00
Wang Yan
f160505686 add artifact trash manager 
1, move the deleted artifact into trash
2, disable GC to delete the untagged manifest

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-02-19 14:52:58 +08:00
Tran Huy
2aed96851e Fix typo 
Signed-off-by: Tran Huy <Tran.Huy@ericsson.com>
 2020-02-18 13:26:59 +01:00
Wenkai Yin(尹文开)
341cb88cba
Merge pull request #10727 from ywk253100/200214_bump_up_legacy_api_version 
Bump up legacy api version to v2.0
 2020-02-18 10:37:25 +08:00
Wenkai Yin(尹文开)
d7903fcf1b
Merge pull request #10682 from ywk253100/200126_label 
Support add/remove label to/from artifact
 2020-02-17 11:33:45 +08:00
Daniel Pacak
1b60bb255c refactor(Makefile): Add variables for download URLs 
Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
 2020-02-14 11:58:59 +01:00
Wenkai Yin
94787ea60d Bump up the version of legacy APIs to v2.0 
Bump up the version of legacy APIs to v2.0

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-14 13:16:30 +08:00
Daniel Pacak
70dda1387a chore: Configure Redis URL for Trivy adapter 
Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
 2020-02-13 17:57:02 +01:00
Daniel Pacak
4755439b75 chore: Build Trivy adapter from sources 
Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
 2020-02-13 15:32:57 +01:00
Wenkai Yin
93731eeb2e Support add/remove label to/from artifact 
This commit add supporting for adding/removing label to/from artifacts and populates labels when listing artifacts

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-13 10:46:23 +08:00
Daniel Pacak
a642667ffc chore(install): Add --with-trivy arg to the installation script 
Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
 2020-02-12 23:47:56 +01:00
Ziming Zhang
6047a8102a fix(feat) fix prepare version in installer 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-02-06 10:58:29 +08:00
Will Sun
acfcd2d175
Merge pull request #10489 from AllForNothing/postinstall 
Fix postinstall script in Docker
 2020-02-03 14:13:12 +08:00
Daniel Jiang
2064a1cd6d Switch to basic authentication for registry 
1. Add basic authorizer for registry which modify the request
to add basic authorization header to request based on configuration.
2. Set basic auth header for proxy when accessing registry
3. Switche the registry to use basic auth by default and use the basic
authorizer to access Harbor.
4. Make necessary change to test cases, particularly
"test_robot_account.py" and "docker_api.py", because the error is
changed after siwtched to basic auth from token auth.  #10604 is opened
to track the follow up work.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-01-31 21:46:47 +09:00
Wenkai Yin
7dc28bcff9 Add foreign key to avoid the concurrent issue 
Add foreign key to avoid the concurrent issue when operating the artifacts, tags and references

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-01-21 16:03:51 +08:00
sshijun
3175d5f646 Fix postinstall script in Docker 
Signed-off-by: sshijun <sshijun@vmware.com>
 2020-01-15 16:28:57 +08:00
Daniel Jiang
a087ba02e3 Populate basic auth information for registry 
This commit updates `prepare` and templates to populate the credential
for registry for basic authentication.

A temporary flag `registry_use_basic_auth` was added to avoid breakage.
It MUST be removed before the release.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-12-31 14:50:46 +08:00
Wenkai Yin(尹文开)
5f11ca4df6
Merge pull request #10248 from ywk253100/191213_controller 
Define the interface for artifact manager
 2019-12-24 17:44:18 +08:00
Wenkai Yin
ac605db5da Define the controller/manager interface for artifact and tag 
1. Define the controller/manager interface for artifact and tag
    2. Provide a null implementation for artifact manager

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-12-24 17:12:55 +08:00
Ziming
e32649adb4 enhance[cicd] introduce github action for CICD 
In order to replace travis.
Implement 5 CI jobs
- UTTEST
- APITEST_DB
- APITEST_LDAP
- OFFLINE
- UI_UT

Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2019-12-17 18:36:33 +08:00
Wenkai Yin
7b41c900a7 Create the models for OCI supporting 
This commits does some basic init work for supporting OCI:
1. Create the artifact and tag model
2. Create database tables

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-12-12 16:05:38 +08:00
Steven Zou
7bf9372f32 chore[api]:rename API folder to api 
- update swagger yaml file reference in `Makefile`
- update swagger yaml file reference in `README`
- update swagger yaml file reference in `docs/configure_swagger.md`
- update swagger yaml file reference in `make/photon/portal/Dockerfile`
- update swagger yaml file reference in `tests/swaggerchecker.sh`

Signed-off-by: Steven Zou <szou@vmware.com>
 2019-12-11 17:41:27 +08:00
Wang Yan
550d690997
Merge pull request #10135 from bitsf/upgrade_clair 
upgrade clair to v2.1.1
 2019-12-06 11:52:10 +08:00
Ziming
9cad403762 fix(build): npm install with special endpoint (#10168) 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: Iaaf33310a2621d58cdc3b9d3359607a961fef05e
 2019-12-06 11:45:48 +08:00
Wang Yan
2a63382236
Merge pull request #10047 from bitsf/makefile_clean 
optimize the makefile process
 2019-12-05 19:03:19 +08:00
He Weiwei
4ea5c41553
chore(scanner): upgrade clair scanner to 1.0.1 (#10147) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-12-05 17:52:37 +08:00
Ziming Zhang
332f88ec8c add make clean 
Change-Id: Ibe806972a19cd69bfd90be051cdc340c4d7c6afb
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2019-12-05 14:44:07 +08:00
Ziming Zhang
744ae62831 upgrade clair to v2.1.1 
Change-Id: Idb2ad0470a51666d75895d8c5e68d80a67e05276
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2019-12-05 14:27:26 +08:00
Wenkai Yin(尹文开)
d145f4baf4
Merge pull request #10034 from ywk253100/191128_clean 
Clean up admiral-related code
 2019-12-04 17:33:31 +08:00
stonezdj(Daojun Zhang)
339c1d4cab
Merge pull request #10088 from reasonerjt/authproxy-cert-setting 
Support pinning to authproxy server's cert
 2019-12-04 14:03:27 +08:00
Daniel Jiang
7bb71db478
Merge pull request #10003 from ninjadq/migrator_miss_component_no_proxy 
Add default domainname for no_proxy
 2019-12-03 10:50:32 +08:00
Daniel Jiang
902598fabd Support pinning to authproxy server's cert 
This commit add an attribute to configurations, whose value is the
certificate of authproxy server.  When this attribute is set Harbor will
pin to this cert when connecting authproxy.
This value will also be part of the response of systemInfo API.

This commit will be cherrypicked to 1.10 and 1.9 branch.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-12-03 07:31:26 +08:00
Qian Deng
e5f8c2d779
Merge pull request #10022 from ninjadq/fix_ca_bundle_path_join 
Fix ca bundle path join issue
 2019-12-02 11:31:23 +08:00
Wenkai Yin
dd2bc0ecef Clean up admiral-related code 
Clean up admiral-related code as it's useless

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-11-28 17:28:54 +08:00
DQ
79344887b9 Fix ca bundle path join issue 
CA bundle name start with '/' will break the os path join

Signed-off-by: DQ <dengq@vmware.com>
 2019-11-27 18:37:45 +08:00
Will Sun
a52b99e180
Merge branch 'master' into remove-lib 2019-11-27 17:44:30 +08:00
DQ
ed6438cf69 Add default domainname for no_proxy 
All internal service and known internal hostname shuold add to no_proxy by default

Signed-off-by: DQ <dengq@vmware.com>
 2019-11-27 15:10:42 +08:00
sshijun
c692f5c67e Move lib into src for better UI building 
Signed-off-by: sshijun <sshijun@vmware.com>
 2019-11-27 09:59:06 +08:00
wang yan
7b664f64f1 Bump up golang version to v1.13.4 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-11-26 19:18:45 +08:00
Wang Yan
60101c4ea1
Merge pull request #9964 from heww/fix-clair-updaters-disable 
fix(prepaire,clair): disable clair updaters when its interval is 0
 2019-11-22 13:38:34 +08:00
He Weiwei
b8308f41a0 fix(prepaire,clair): disable clair updaters when its interval is 0 
Closes #9961

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-11-22 03:31:20 +00:00
stonezdj
938168b8ad Failed to start harbor when proxy is set 
Fix #9614, all communication between internal components should bypass the proxy
Add chartmuseum, notary-server,clair-adapter to the no_proxy list in harbor.yml

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-11-22 11:22:51 +08:00
stonezdj(Daojun Zhang)
2b0ede5341
Merge pull request #9829 from reasonerjt/rm-k8s-install 
Remove scripts to deploy Harbor on k8s
 2019-11-19 15:53:49 +08:00
Wang Yan
eab974419c
Merge pull request #9825 from stonezdj/bug_9681 
Avoid to create duplicated immutable tag rules in the same project
 2019-11-18 17:26:22 +08:00
stonezdj
15898f2069 Avoid to create duplicated immutable tag rules in the same project 
Fix #9681, add constraint on immutable_tag_rule and catch the error

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-11-15 14:46:23 +08:00
Daniel Jiang
2fcd174e4b
Merge pull request #9828 from wy65701436/cii-docker-base 
add base images when to build harbor assets
 2019-11-15 14:24:11 +08:00
He Weiwei
fe69a5df99 build(scanner-adapter): bump up clair adapter to v1.0.1-rc2 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-11-13 02:35:21 +00:00
wang yan
47793e77e3 update base file name ane pass base version to build file 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-11-12 19:12:49 +08:00
Wang Yan
544cc98971 add base images when to build harbor assets 
* add base images when to build harbor assets

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-11-12 15:38:51 +08:00
Daniel Jiang
39a22d4470 Remove scripts to deploy Harbor on k8s 
This commit removes scripts have been deprecated for a while and users have to use
helm chart to deploy Harbor on top of k8s cluster.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-11-11 19:23:22 +08:00
Yogi_Wang
cddc1149f1 Modify the memory of nodejs used from 8192MB to 2048MB 
Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2019-11-11 17:05:42 +08:00
Wang Yan
6da183d576
Merge pull request #9800 from ninjadq/failure_earlier_of_ca_bundle_permission_check 
Failure earlier of ca bundle permission check
 2019-11-11 14:09:21 +08:00
Wang Yan
0b09bd14b1
Merge pull request #9756 from ninjadq/add_ignore_media_type 
Add ignore mediatypes for registry
 2019-11-08 18:34:13 +08:00
DQ
80c3e76b5a check the permission of ca bundle file 
CA bundle need check before use

Signed-off-by: DQ <dengq@vmware.com>
 2019-11-08 15:34:17 +08:00
DQ
7237067d63 Bump config version 
Bump version to 1.10

Signed-off-by: DQ <dengq@vmware.com>
 2019-11-07 17:06:20 +08:00
Daniel Jiang
06e4e124d8
Refine request handle process (#9760) 
* Refine request handle process

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-11-07 13:02:17 +08:00
DQ
45868107aa Add ignore mediatypes for registry 
Add these mediatypes to reduce the amount of registry event

Signed-off-by: DQ <dengq@vmware.com>
 2019-11-06 21:39:08 +08:00
Wang Yan
a9e8c6a430
Merge pull request #9738 from ninjadq/fix_install_script 
Fix install script
 2019-11-06 14:24:07 +08:00
Steven Zou
240f771006
Merge pull request #9733 from steven-zou/fix/scan_all_enhancements 
do improvements to the scan all job
 2019-11-05 16:22:50 +08:00
Steven Zou
ebc5d2482b do improvements to the scan all job 
- update scan all job to avoid sending too many HTTP requets
- update scan controller to support scan options
- update the db schema of the scan report to introduce requester
- introduce scan all metrics to report the overall progress of scan all job
- fix the status updating bug in scan report
- enhance the admin job status updats
- add duplicate checking before triggering generic admin job
- update the db scheme of admin job

fix #9705
fix #9722
fix #9670

Signed-off-by: Steven Zou <szou@vmware.com>
 2019-11-05 15:12:07 +08:00
Wang Yan
27cb25cc04
Merge pull request #9400 from ninjadq/inject_certs_to_non_root 
Inject certs to non root
 2019-11-05 14:49:08 +08:00
DQ
75c91273bc Fix install script 
Move load images to above

Signed-off-by: DQ <dengq@vmware.com>
 2019-11-05 11:22:30 +08:00
DQ
ece321a53a Change certs's owner to 10000 
Signed-off-by: DQ <dengq@vmware.com>
 2019-11-04 17:38:41 +08:00