Upstream/harbor
1
0
Fork 0
mirror of https://github.com/goharbor/harbor.git synced 2024-11-22 10:15:35 +01:00
Code Issues Projects Releases Wiki Activity
11,889 Commits 48 Branches 317 Tags 352 MiB
17c17ffd8e
Commit Graph

1339 Commits

Author SHA1 Message Date
Chenyu Zhang
5de1a62002
migrate: add db index on artifact repository name (#17035) 
Signed-off-by: chlins <chenyuzh@vmware.com>
 2022-06-20 16:01:04 +08:00
MinerYang
b8a71ac348
bump up golang version to v1.18.3 (#16957) 
Signed-off-by: yminer <yminer@vmware.com>

update ci/ut_install.sh

Signed-off-by: yminer <yminer@vmware.com>
 2022-06-08 17:56:14 +08:00
Chenyu Zhang
7ecd4a3f29
fix(prepare): validate expire hours when enable cache (#16949) (#16953) 
Signed-off-by: chlins <chenyuzh@vmware.com>
 2022-06-07 13:21:47 +08:00
Wilmar den Ouden
79600f3fee
feat: enabled Github GHCR as proxy cache (#16834) 
Signed-off-by: wilmardo <info@wilmardenouden.nl>
 2022-06-06 15:36:16 +08:00
Franz Nemeth
7c2e591c26
fix: add patch for registry layers larger than 10G with S3 backend (#16322) 
add patch for registry layers larger than 10G with S3 backend

Signed-off-by: Franz Nemeth <franz.nemeth@dynatrace.com>
 2022-06-02 16:51:29 +02:00
MinerYang
1db7d6375a
support docker compose v2 (#16919) 
Signed-off-by: yminer <yminer@vmmware.com>

Co-authored-by: yminer <yminer@vmmware.com>
 2022-05-30 15:19:01 +08:00
Mac Chaffee
ecc8c59101
Use exec in registryctl so signals are passed properly (#16642) 
Signed-off-by: Mac Chaffee <machaffe@renci.org>
 2022-05-25 14:11:31 +08:00
prahaladdarkin
27ec871185
System Artifact Manager database schema creation, model definitons, and tests (#16678) 
Closes:
https://github.com/goharbor/harbor/issues/16540
https://github.com/goharbor/harbor/issues/16541
https://github.com/goharbor/harbor/issues/16542

Signed-off-by: prahaladdarkin <prahaladd@vmware.com>
 2022-05-09 17:32:57 +08:00
Shengwen Yu
3e8bf9faed fix: registry/redis.patch & registry/builder 
Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
 2022-04-29 08:51:49 +08:00
MinerYang
615a4cbb52
install only htpasswd binary from photon package httpd (#16771) 
Signed-off-by: yminer <yminer@vmmware.com>

cpio -u

Signed-off-by: yminer <yminer@vmmware.com>

Co-authored-by: yminer <yminer@vmmware.com>
 2022-04-28 13:20:03 +08:00
chlins
8c223135e7 feat: add cache layer for artifact (#16739) 
Implement cache layer for resource artifact and define common
cache manager and workflow. Also add cache related options to
configuration yaml.

Signed-off-by: chlins <chenyuzh@vmware.com>
 2022-04-25 18:01:45 +08:00
chlins
f7d10474d1 migrations: correct project metadata public value 
Signed-off-by: chlins <chenyuzh@vmware.com>
 2022-03-28 15:17:20 +08:00
stonezdj
9d480a84d2 Create index on task with job_id 
Missing index with job_id, when query task with job_id, it cause a full table scan, caused performance issue
  Fixes #15271

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2022-03-02 20:51:36 +08:00
MinerYang
74d2670616
bump up Go version to 1.17.7 (#16415) 
Signed-off-by: yminer <yminer@vmmware.com>

update CONTRIBUTING.md 2.4 golang version to 1.17.7

Signed-off-by: yminer <yminer@vmmware.com>

Co-authored-by: yminer <yminer@vmmware.com>
 2022-02-25 17:23:18 +08:00
MinerYang
6f1f2c0395
update 2.5.0 migration script harbor.yml.jinja (#16354) 
Signed-off-by: yminer <yminer@vmmware.com>

Co-authored-by: yminer <yminer@vmmware.com>
 2022-02-11 17:30:06 +08:00
MinerYang
6c97d3f0c7
rollback to delete external redis username (#16353) 
Signed-off-by: yminer <yminer@vmmware.com>

Co-authored-by: yminer <yminer@vmmware.com>
 2022-02-11 17:28:10 +08:00
MinerYang
77b6a130f8
Add Migration Script for Harbor 2.5 (#16346) 
* add Harbor2.5 migration script

Signed-off-by: yminer <yminer@vmmware.com>

* update 2_5_0 migration harbor.yml.jinja

Signed-off-by: yminer <yminer@vmmware.com>

Co-authored-by: yminer <yminer@vmmware.com>
 2022-02-11 12:18:06 +08:00
Wang Yan
6c3f9a8366
bump up distribution version to v2.8 (#16338) 
Signed-off-by: Wang Yan <wangyan@vmware.com>
 2022-02-10 11:46:58 +08:00
MinerYang
1a2dd256b5
Add external redis username for AUTH (#16336) 
* add external redis username for AUTH

* update harbor.yml.tmpl

* add external redis username for AUTH

update harbor.yml.tmpl

Co-authored-by: yminer <yminer@vmmware.com>
 2022-02-09 22:06:28 +08:00
Shengwen YU
0bf5998f96
upgrade Chartmuseum to v0.14.0 (#16334) 
Signed-off-by: Shengwen Yu <yshengwen@vmware.com>

Co-authored-by: Shengwen Yu <yshengwen@vmware.com>
 2022-02-09 12:13:05 +08:00
Shengwen Yu
aacebcc8c4 fix: vulnerability_record & report_vulnerability_record table "integer out of range" error 
Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
 2022-02-09 10:19:46 +08:00
Yang Jiao
709a154724 Specify postgresql version to 13 
Signed-off-by: Yang Jiao <jiaoya@vmware.com>
 2022-01-27 06:46:15 +00:00
He Weiwei
8554b7d7c5
refactor: replace lib/pq with jackc/pgx (#16267) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2022-01-27 11:09:37 +08:00
He Weiwei
89fe7b0819
Move removing old version scan reports of trivy to 2.4.2 (#16261) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2022-01-24 14:05:24 +08:00
Shengwen Yu
46f97ecf6c feat: bump TRIVYVERSION to v0.22.0 and bump TRIVYADAPTERVERSION to v0.25.0 
Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
 2022-01-21 09:14:07 +08:00
Morlay
9ad68aa526
Updates goharbor to var IMAGENAMESPACE in Makefiles to make injectable (#15218) 
Signed-off-by: Morlay <morlay.null@gmail.com>
 2022-01-20 12:14:29 +08:00
MinerYang
6abc505060
Merge pull request #16239 from MinerYang/bump_up/go_v1.17.6 
bump up go version to v1.17.6
 2022-01-18 10:32:09 +08:00
yminer
9acf3d21ce bump up go version to v1.17.6 
Signed-off-by: yminer <yminer@vmmware.com>
 2022-01-17 17:45:22 +00:00
孙世军
8f77567589
Upgrade UI dependencies (#16233) 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2022-01-17 14:20:20 +08:00
He Weiwei
097efb201b
fix: Clean old version scan reports of trivy (#16235) 
From harbor 2.4, the trivy does not support scan report v1.0, we need to remove the old version scan reports from the DB.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2022-01-17 13:59:28 +08:00
Yang Jiao
b4de95941b Specify postgresql version to 13 
Signed-off-by: Yang Jiao <jiaoya@vmware.com>
 2022-01-12 08:55:42 +00:00
stonezdj
17d8b7b813 Add upload purge config to registry/config.yml 
Enable the uploadpurging by default
  Fixes #15641

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2022-01-04 11:15:51 +08:00
Qian Deng
69a194b2b4 Fix: using traditional PKCS#1 format RSA key 
The openssl 3.0.0 using newer `PKCS#8` format.
But it's not compatitable with harbor core
So using tradictional format instead

Signed-off-by: Qian Deng <dengq@vmware.com>
 2021-12-10 11:34:12 +08:00
Wang Yan
742e7ded00
add accessory dao service (#16045) 
Signed-off-by: wang yan <wangyan@vmware.com>
 2021-12-03 14:34:02 +08:00
Qian Deng
5fc4449450
Merge pull request #16029 from ninjadq/fix_chart_replication_issue 
Fix: chart replication dup files issue
 2021-12-03 13:47:42 +08:00
Qian Deng
593117a127 Fix: chart replication dup files issue 
this patch fixed #15522

Signed-off-by: Qian Deng <dengq@vmware.com>
 2021-11-17 11:24:43 +00:00
Wang Yan
a956758302
bump up go version to v1.17 (#15865) 
* bump up go version to v1.17

Signed-off-by: Wang Yan <wangyan@vmware.com>

* gofmt fail

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-10-25 17:28:29 +08:00
Rolf Ahrenberg
5f3972f86d
Add configurable timeout for Trivy scans (#15796) 
Signed-off-by: Rolf Ahrenberg <Rolf.Ahrenberg@saunalahti.fi>
 2021-10-22 14:36:12 +08:00
stonezdj(Daojun Zhang)
c814daa894
Update the jaeger settings in harbor.yaml.tmpl (#15841) 
agent default port to 6831

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2021-10-22 13:43:28 +08:00
Yurii Paneiko
be1e762b70
For some reason this script hangs on bzip2 presence checking (#15647) 
Steps to reproduce:

Clone repo from master branch
Run: make install COMPILETAG=compile_golangimage
Wait until script will check that bzip2 is installed

Signed-off-by: YuriiPaneiko <yurapaneyko@gmail.com>
 2021-10-14 20:11:14 +08:00
He Weiwei
b390112f5a
fix: convert severity from negligible to none before saving to db (#15791) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2021-10-14 16:02:38 +08:00
stonezdj(Daojun Zhang)
32023891eb
Merge pull request #15675 from stonezdj/21sep22_email_nullable 
Allow empty email attribute for ldap/oidc user
 2021-10-02 10:33:38 +08:00
stonezdj
06715af303 Allow empty email attribute for ldap/oidc user 
Define user.Email as sql.NullString to avoid unique constraint when email is empty in LDAP/OIDC
  Separate the common/models/User with the pkg/user/dao/User
  Fixes #10400
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2021-09-30 14:35:18 +08:00
Qian Deng
3c23926bdc Add validation for tracing 
* add  validation of tracing in validating process

Signed-off-by: Qian Deng <dengq@vmware.com>
 2021-09-27 13:10:55 +00:00
stonezdj(Daojun Zhang)
972fa0880b
Merge pull request #15505 from ninjadq/add_distributed_tracing 
Add distributed tracing
 2021-09-22 14:16:19 +08:00
Qian Deng
354a2bd80d Enhance the trace related code 
* Move request id to requestid middleware
* fix span pass to child ctx on orm
* fix typos
* remove unused code
* add operation name to Transaction

Signed-off-by: Qian Deng <dengq@vmware.com>
 2021-09-22 04:14:43 +00:00
Qian Deng
05bc946444
Merge pull request #14383 from XavierDuthil/use-exec-in-entrypoints 
Use exec in all components' entrypoints
 2021-09-22 10:49:21 +08:00
Qian Deng
bad913cf6d Refactor trace code 
* use lib trace helper function
* add gracefull shutdown
* Add commens for new added exposed function
* Add licence on top of new created files
* Update trace library
* Update configs
* Add attribute and namespance in config

Signed-off-by: Qian Deng <dengq@vmware.com>
 2021-09-18 10:58:52 +00:00
Qian Deng
6fec5b2873 Add trace to jobservice 
* Add trace init in main
* Add env template
* Add trace for router

Signed-off-by: Qian Deng <dengq@vmware.com>
 2021-09-18 10:58:52 +00:00
Qian Deng
14095fb10b Add trace to registryctl 
* Add trace init to main
* Add trace for http server
* Add trace for gc
* Add env template trace

Signed-off-by: Qian Deng <dengq@vmware.com>
 2021-09-18 10:58:52 +00:00
Qian Deng
a15983432c Add trace for core 
* Add trace related lib
* Add trace middleware for core
* add rid for middleware

Signed-off-by: Qian Deng <dengq@vmware.com>
 2021-09-18 10:58:52 +00:00
Qian Deng
b812a300be Add trace related configs 
* Update harbor config template
* Update python config parsing
* Update env template

Signed-off-by: Qian Deng <dengq@vmware.com>
 2021-09-18 10:58:52 +00:00
He Weiwei
ecc1a04c92
refactor: implement a lock free quota (#15399) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2021-09-17 11:26:37 +08:00
DQ
e4c1521efc Add migration script for harbor 2.4 
Signed-off-by: DQ <dengq@vmware.com>
 2021-09-16 09:31:39 +00:00
Ziming Zhang
98cef43ead limit replication bandwidth 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2021-09-14 11:23:28 +08:00
Wenkai Yin(尹文开)
3ce072016f
Merge pull request #15527 from sevendials/logrotate_fails_when_cwd_is_not_readable 
Logrotate fails when cwd is not accessible
 2021-09-13 16:08:23 +08:00
Wang Yan
dbcbc8bad0
bump up go to v1.16.7 (#15564) 
Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-09-09 17:58:00 +08:00
Wang Yan
93a078d225
deprecate dns search (#15557) 
For details, please refer to https://github.com/goharbor/harbor/issues/14146#issuecomment-793390718
and https://github.com/docker/for-linux/issues/1164.

If anyone encounter the issue mentioned by https://github.com/goharbor/harbor/issues/6031, add the dns_search: . to the releated container.

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-09-08 14:22:59 +08:00
Christopher Jenkins
0585b148c7 Logrotate fails when cwd is not accessible 
Logrotate is run with sudo as the syslog user by cron.hourly
The current working directory is `/root` which is inaccessible to the syslog
user so the logrotate command fails. Currently the following stderr is being
thrown away by the cron script:
```
error: cannot open current directory: Permission denied
```

Fixes #15468

Signed-off-by: Christopher Jenkins <christj@gmail.com>
 2021-09-01 15:28:30 -07:00
孙世军
67681b1d83
update node version for portal container (#15396) 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2021-08-10 09:14:46 +08:00
stonezdj(Daojun Zhang)
2fa530eefa
Merge pull request #15247 from stonezdj/21jun30_remove_auth_user 
Delete users under auth_mode other than db_auth
 2021-07-14 13:25:43 +08:00
stonezdj
c163bc8317 Delete users under auth_mode other than db_auth 
The following information should cleanup before delete user:
  Delete project member of this user.
  Delete oidc_user when auth_mode is oidc_auth.
  Fixes #8424
  It also removes the deleted user from project member and the deleted condition in the project member query for consistency
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2021-07-12 11:31:40 +08:00
Wang Yan
494d74d32d
bump up go version to 1.16 (#15286) 
Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-07-08 16:25:15 +08:00
Wang Yan
4017e995b7 roll back go for notary binary 
Fixes #14932

Harbor recompiles the notary v0.6.1 with go 1.15 from v2.2.0, which introduces an break change that leads to notary key not found after migration.

[Root cause]
Notary v0.6.1 consumed an old version dvsekhvalnov/jose2, which is not compatible with go 1.15.

[References]
https://github.com/dvsekhvalnov/jose2go/issues/26
https://github.com/golang/go/issues/41089

[Resolve]
To resolve this issue, we have to roll back go vesrion to v1.14 for notary v0.6.1 binary and keep it until upstream have a patch release to support go 1.15 or above.

[Break change]
If you pushed and signed image using Harbor v2.2.0 ~ v2.2.2 and created new repository key in notary, you will encouter the same issue after migrate to v2.2.3(or above) or v2.3.1(or above) because of the go version downgrade. We will have a FAQ to help you to resovle this particular scenario.

The influence path of the particular case:
Harbor v2.1.0(or lower) --> [v2.2.0 ~ v2.2.2] --> v2.2.3(or above)
Harbor v2.1.0(or lower) --> v2.3.0 --> v2.3.1(or above)

The non influence path of the paticular case:
Harbor v2.1.0(or lower) --> v2.2.3(or above)
Harbor v2.1.0(or lower) --> v2.3.1(or above)

[Fix in Version]
Harbor v2.2.3 or above
Harbor v2.3.1 or above

[Note]
If you're a heavy user of notary, avoid using v2.2.0, v2.2.1, v2.2.2 and v2.3.0, and use the fixed version for instead.

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-06-29 16:18:55 +08:00
danfengliu
c39345da96
Merge pull request #15148 from danfengliu/missing-db-base-login-in-build-base-workflow 
Fix issue of missing db base build process in build base workflow
 2021-06-16 16:39:56 +08:00
Will Sun
410c5bcd78
Undate Angular to the latest version (#15129) 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2021-06-16 16:39:10 +08:00
danfengliu
f367aad760 Fix issue of missing db base build process in build base workflow 
There is extra build step for db base image building since v2.3, so this
step should be added back.

Signed-off-by: danfengliu <danfengl@vmware.com>
 2021-06-16 15:22:32 +08:00
He Weiwei
72f1afe2cc
perf: configurations for perf improvement (#15142) 
Closes #15041

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2021-06-15 18:39:15 +08:00
danfengliu
254c4a3497
Merge pull request #15046 from danfengliu/refine-base-build-process-in-makefile 
Refind build base process in Makefile
 2021-06-09 11:21:48 +08:00
He Weiwei
28921e2997 fix: listen to 5443 when clean unexpected status of db 
Closes #15081

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2021-06-08 06:41:23 +00:00
DQ
68ac772726 Fix: Remove v6 format of harbor 
Signed-off-by: DQ <dengq@vmware.com>
 2021-06-07 16:26:53 +08:00
Qian Deng
0867a6bfd6
Merge pull request #15055 from ninjadq/health_check_url_2_dn_instead_of_ip 
Add IPv6 support
 2021-06-03 17:57:55 +08:00
DQ
1b6b47f860 Add IPv6 support 
* 127.0.0.1 to localhost
* listening net addr add ipv6 format

Signed-off-by: DQ <dengq@vmware.com>
 2021-06-03 09:04:49 +00:00
danfengliu
d66ea07c2f Refind build base process in Makefile 
Remove build base executable in Makefile by replacing it as an input parameter.
Add add more input parameters for controlling docker pull/push to make
build base process flexible for users.

Signed-off-by: danfengliu <danfengl@vmware.com>
 2021-06-03 14:48:34 +08:00
Wang Yan
eec9893918 set shm size of postgres 
Fixed #15034, as for postgres 13, the default shm size is 64MB, set to 1gb to avoid could not resize shared memory segment error.

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-06-03 11:05:33 +08:00
Wang Yan
ae06ac2fae
fix db issue on helm upgrade (#15028) 
fixes #15000

It needs to double confirm that old pg is stopped before migration

Signed-off-by: wang yan <wangyan@vmware.com>
 2021-06-01 15:59:41 +08:00
ChenYu Zhang
a038ba672f
perf: add 2.2.3 schema up sql to master (#15027) 
Signed-off-by: chlins <chlins.zhang@gmail.com>
 2021-06-01 14:12:25 +08:00
Wang Yan
66b8a8f8dd
add build arch parameter in Makefile (#14995) 
* add build arch parameter in Makefile

Add parameter BUILDARCH for make file. DB base builds pg96 for x86_64 only

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-06-01 10:38:05 +08:00
DQ
5d02acd043 Add upgrade script for harbor 2.3 
no new config item added. harbor.yml keep the same as last version

Signed-off-by: DQ <dengq@vmware.com>
 2021-05-28 20:30:12 +08:00
Wenkai Yin
c585f92034 Fix bug of migration sql for replication policy 
Fix bug of migration sql for replication policy

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2021-05-26 18:57:23 +08:00
Qian Deng
9ea8aade01
Upgrade prepare to consistent with photon 4 (#14698) 
* requires version to 3.9.1
* upgrade packages

Signed-off-by: DQ <dengq@vmware.com>
 2021-05-26 16:39:04 +08:00
Wang Yan
39bdd7b506
pg upgrade failure handling (#14934) 
To ensure the upgrade execution idempotence, it needs to clean the $PGDATANEW on pg_upgrade failure.
Otherwise, the upgrade will skip the upgrade process from the second time launch as the exist of $PGDATANEW.

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-05-21 13:53:39 +08:00
danfengliu
6c14e699b1
Merge pull request #14844 from danfengliu/fix-notary-trust-directory-issue-in-nightly 
Fix notary trust directory issue in nightly
 2021-05-20 21:56:27 +08:00
Wang Yan
73bd373a75
create index for audit (#14930) 
Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-05-20 19:50:13 +08:00
danfengliu
3ef4dc17fc Fix notary trust directory issue and add login for each base image in makefile 
1. Use root instead of ~ in notary parameter;
2. Fix tag immutability issue caused by GUI change;
3. Replace email domain name to harbor test;
4. Add login for each base image in makefile;
5. Add customize look test in nightly.

Signed-off-by: danfengliu <danfengl@vmware.com>
 2021-05-20 18:23:13 +08:00
Wang Yan
86185989cf
support pg upgrade (#14846) 
1, use the pg source and photon spec to build postgres 9.6
2, install 9.6 on the photon 4.0
3, then leverage pg_upgrade to handle the pg major version migration

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-05-20 16:25:50 +08:00
Wenkai Yin
dc059a9a8f Improve the performance of artifact related APIs 
Improve the performance of artifact related APIs by adding indexes and refactoring sql logic

Closes #13890 #14813 #14814

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2021-05-20 11:25:43 +08:00
Wang Yan
0fb520a33b bump up go to v1.15.12 
Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-05-14 15:36:23 +08:00
Qian Deng
17dd48e5a3
Merge pull request #14854 from ninjadq/upgrade_helm_chart 
Upgrade helm chart
 2021-05-12 19:33:03 +08:00
Wenkai Yin
c04f3a2aac Fix duplicate execution record issue 
When the core service cannot response the checkin request in time, duplicated execution records may be created, this commit introduces the revision column to make sure there is only one record for one schedule trigger

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2021-05-12 15:40:52 +08:00
DQ
04ba4a4033 Upgrade chartmuseum version 
from 1.12.0 to 1.13.1

Signed-off-by: DQ <dengq@vmware.com>
 2021-05-11 13:51:55 +00:00
He Weiwei
476732df89 fix: change art_size to bigint in migrations 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2021-05-11 04:01:02 +00:00
Wenkai Yin(尹文开)
9bbffa06db
Merge pull request #14662 from ywk253100/210413_dest_ns 
Refactor the replication policy destination namespace logic
 2021-04-29 11:18:43 +08:00
Wenkai Yin
710c80078b Refactor the replication policy destination namespace logic 
Support specifying what part of the repository will be replaced by the provided namespace

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2021-04-28 10:16:20 +08:00
Daniel Jiang
c701ce09fa
Merge pull request #14681 from bitsf/fix_typo_NOTARYURL 
Fixed typo in NOTARYURL variable name
 2021-04-21 17:38:01 +08:00
Wenkai Yin(尹文开)
0f6692f68f
Merge pull request #14682 from ninjadq/hostname_hardcode_to_localhost 
Fix: Use local host on db's healthcheck
 2021-04-19 21:02:31 +08:00
danfengliu
f0ebd17994 Add build base image step in build package git action workflow 
Build base image step should be in build package workflow, and local base images build by new step should be removed since images have been pushed to docker hub.

Signed-off-by: danfengliu <danfengl@vmware.com>
 2021-04-17 18:10:44 +08:00
Ziming Zhang
39f70287b4 Fixed typo in NOTARYURL variable name 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2021-04-17 15:55:02 +08:00
DQ
ffed6459c7 Fix: Use local host on db's healthcheck 
hostname -i will malfunction in some cases like the `nsswitch.conf` file does'nt exist

Signed-off-by: DQ <dengq@vmware.com>
 2021-04-16 18:37:24 +08:00
Pushkar Joglekar
3947c5faff Add --no-cache and --pull flag to image builds to ensure latest security fixes are pulled from base image 
Signed-off-by: Pushkar Joglekar <pjoglekar@vmware.com>
 2021-04-12 09:49:27 -07:00
Qian Deng
c5d12ce8ee
Merge pull request #14542 from ninjadq/add_task_info_in_exporter 
Add task info in exporter
 2021-04-07 18:17:26 +08:00
Alexis
06fa88cfb7 Fix typo 
Signed-off-by: Alexis <60alexis@gmail.com>
 2021-04-07 15:58:17 +08:00
Alexis
e33f7aa9dd Add redis port to 2.1.0 jinja template 
Signed-off-by: Alexis <60alexis@gmail.com>
 2021-04-07 15:58:17 +08:00
Alexis
7742aec4af Add port to 2.0.0 jinja file 
Signed-off-by: Alexis <60alexis@gmail.com>
 2021-04-07 15:58:17 +08:00
Alexis
d28845af51 Remove external_redis.port since not used since v1.10.0 
Signed-off-by: Alexis <60alexis@gmail.com>
 2021-04-07 15:58:17 +08:00
stonezdj(Daojun Zhang)
448f0b6e28
Merge pull request #14579 from stonezdj/21apr_add_docker_registry_proxy 
Support proxy cache for docker-registry type
 2021-04-07 10:59:24 +08:00
Steven Zou
e2148f9eea
Merge pull request #14514 from goharbor/dependabot/pip/make/photon/prepare/pyyaml-5.4 
Bump pyyaml from 4.2b1 to 5.4 in /make/photon/prepare
 2021-04-07 09:57:07 +08:00
Steven Zou
10711b7de1
Merge pull request #14482 from goharbor/dependabot/pip/make/photon/prepare/jinja2-2.11.3 
Bump jinja2 from 2.11.1 to 2.11.3 in /make/photon/prepare
 2021-04-07 09:56:23 +08:00
stonezdj
ccd9ee8c56 Support proxy cache for docker-registry type 
Add proxy cache for docker registry type
Fixes #14477, #14547
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2021-04-06 16:47:12 +08:00
Wang Yan
d03a29e531 bump up photon to 4.0 
Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-03-31 13:25:29 +08:00
DQ
7eebbeebdf Add jobservice task queue related task 
add jobservice metrics
add redis client

Signed-off-by: DQ <dengq@vmware.com>
 2021-03-30 10:29:36 +00:00
DQ
fbe9cd88f8 Enabled Prometheus for Jobservice 
* Add prom server on jobservice
* Enabeld configs in templates
* Enabeld jobservice metrics in nginx

Signed-off-by: DQ <dengq@vmware.com>
 2021-03-30 08:52:59 +00:00
dependabot[bot]
f20f4215c3
Bump pyyaml from 4.2b1 to 5.4 in /make/photon/prepare 
Bumps [pyyaml](https://github.com/yaml/pyyaml) from 4.2b1 to 5.4.
- [Release notes](https://github.com/yaml/pyyaml/releases)
- [Changelog](https://github.com/yaml/pyyaml/blob/master/CHANGES)
- [Commits](https://github.com/yaml/pyyaml/commits/5.4)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-03-25 22:46:56 +00:00
DQ
f5fcc7bd31 Add base image for exporter 
* Add base
* update Makefile

Signed-off-by: DQ <dengq@vmware.com>
 2021-03-25 16:35:57 +08:00
dependabot[bot]
0ec667c4d8
Bump jinja2 from 2.11.1 to 2.11.3 in /make/photon/prepare 
Bumps [jinja2](https://github.com/pallets/jinja) from 2.11.1 to 2.11.3.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/master/CHANGES.rst)
- [Commits](https://github.com/pallets/jinja/compare/2.11.1...2.11.3)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-03-20 00:43:21 +00:00
Wenkai Yin
8b1817be0f Fix the consume too much CPU issue 
1. Update execution status during the upgrade
2. Refine the execution sweeper

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2021-03-17 13:34:24 +08:00
Wenkai Yin
43df3bf8a4 Add upgrade sql file introduced in 2.1.4 
1. Add upgrade sql file introduced in 2.1.4
2. Minor improvement for task/execution to cover corner cases

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2021-03-15 16:48:51 +08:00
Xavier Duthil
280c8272f8
Use exec in all components' entrypoints 
Use the exec Bash command so that the final running application becomes
the container’s PID 1. This allows the application to receive any Unix
signals sent to the container, in accordance with
https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#entrypoint

Currently, SIGTERM signals sent by kubernetes are not passed to the
executed binary.

Signed-off-by: Xavier Duthil <xavier.duthil@corp.ovh.com>
 2021-03-05 15:00:25 +01:00
Wang Yan
3dfddfdf4e
patch upstream fix for io reader (#14356) 
Fixes #12850
This patch can fix the GC failure in the NFS v3 env, see https://github.com/distribution/distribution/pull/3309#issuecomment-783606968

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-03-04 15:33:09 +08:00
Daniel Jiang
387be3686a Refine the way to set X-Forwarded-Proto in nginx 
Refine the way to set the header so user won't need to comment it if
Harbor is sitting behind a reverse proxy.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2021-02-25 17:43:55 +08:00
danfengliu
9e3f0de12c
Merge pull request #14124 from danfengliu/reschedule-docker-login-policy-in-build-base-image-in-master 
Reschedule docker login policy in base image build process
 2021-02-23 10:10:59 +08:00
danfengliu
7d05c8e513 Reschedule docker login policy in base image build process 
Signed-off-by: danfengliu <danfengl@vmware.com>
 2021-02-22 10:05:25 +08:00
Josh Soref
dfe360040b Spelling 
* addition
* attribute
* auditing
* availability
* available
* bandwidth
* browser
* business
* cadence
* chartmuseum
* client
* column
* content
* demonstrate
* described
* endpoints
* facilitate
* github
* harbor
* information
* instance
* manual
* meaningful
* operation
* overridden
* password
* possible
* project
* refactor
* replication
* requires
* running
* scanned
* settings
* signup
* those
* unsigned
* vulnerability

--
Also removes trailing space from a filename

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
 2021-02-19 11:59:15 +08:00
DQ
307c5a8ed4 Fix metrics template for http mode 
the port shouldn't be hardcode

Signed-off-by: DQ <dengq@vmware.com>
 2021-02-05 18:44:28 +00:00
Ziming Zhang
ec83f49a1a fix(retention) migrate sql error 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2021-02-05 09:09:26 +00:00
Wang Yan
24ec772978
fix gc migration issue (#14174) 
For the upgrade path v1.10 - v2.1.* - v2.2.0, if user doesn't reset the GC schdule that was created in 1.10 in the v2.1,
the job parameters will keep empty in the database. The fix gives a default value for the schedule record.

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-02-05 12:25:24 +08:00
DQ
051b5f289d Add sen existed check for internal cert 
fali ealier when there is no san

Signed-off-by: DQ <dengq@vmware.com>
 2021-01-28 08:22:07 +00:00
Wenkai Yin(尹文开)
50a1e85095
Make sure the revision of execution isn't null during the upgrade (#14085) 
Make sure the revision of execution isn't null during the upgrade
Fixes #14075

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2021-01-27 10:10:36 +08:00
Qian Deng
f013d88efc
Merge pull request #14013 from ninjadq/upgrade_script_for_2_2_0 
Harbor upgrading for 2.2
 2021-01-22 18:10:24 +08:00
Qian Deng
045e1d9abe
Merge pull request #14040 from ninjadq/metric_improvement 
Metric improvement
 2021-01-22 17:13:57 +08:00
DQ
489f31d8fe Add upgrade scirpt for 2.2 
1. add metrics config item in config
2. upgrade version in template

Signed-off-by: DQ <dengq@vmware.com>
 2021-01-22 16:15:06 +08:00
Wang Yan
dba229d0df
build third party binaries in CI (#14019) 
Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-01-22 11:33:42 +08:00
DQ
92cf728371 Add custom cert for exporter 
* injecting custom certs related config to exporter

Signed-off-by: DQ <dengq@vmware.com>
 2021-01-20 10:52:34 +08:00
DQ
a61e9b0e2e Add san for notary upgrading 
if san not exists then remove that cert, prepare will regenerate one

Signed-off-by: DQ <dengq@vmware.com>
 2021-01-18 21:00:35 +08:00
Wenkai Yin
7c072e17a6 Fix the legacy scheduled job issue for GC/scan all 
Fix the legacy scheduled job issue for GC/scan all

Fixes #13968

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2021-01-15 22:02:36 +08:00
Daniel Jiang
1b64b9fdc2
Bump up the go-migrate (#13914) 
Bump it up to v4.11.0 to be consistent with harbor-core

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2021-01-11 19:08:17 +08:00
He Weiwei
4a326aa8b0 chore: delete records of scan_report 
The report in previous scan_report records not work well the
vulnerabilities stored in the schema table, so delete the scan_report
records.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2021-01-08 03:39:11 +00:00
Qian Deng
642d56041d
Add san for notary cert (#13928) 
Signed-off-by: DQ <dengq@vmware.com>
 2021-01-08 01:00:34 +08:00
stonezdj
6b8fb8431d Add quay registry to proxy cache 
Update env.jinja to add quay

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2021-01-06 17:22:57 +08:00
Ziming Zhang
8faa76a1b6 feat(retention) refactor task manager 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2021-01-05 12:08:03 +08:00
Wenkai Yin(尹文开)
19ad8ad68d
Merge pull request #13823 from reasonerjt/inst-cert-home-dir 
Replace tilde in install_cert.sh
 2020-12-25 10:25:51 +08:00
prahaladdarkin
a890b28e1e
Store vulnerability data from scanner into a relational format (#13616) 
feat: Store vulnerability report from scanner into a relational format

Convert vulnerability report JSON obtained  from scanner into a relational format describe in:https://github.com/goharbor/community/pull/145

Signed-off-by: prahaladdarkin <prahaladd@vmware.com>
 2020-12-25 08:47:46 +08:00
Wang Yan
7a8a8fa104
upgrade go version to v1.15.6 (#13836) 
Signed-off-by: wang yan <wangyan@vmware.com>
 2020-12-23 18:53:09 +08:00
He Weiwei
3831e82b20
refactor: remove code of admin job (#13819) 
Remove code of admin job as it's not needed by scan all/gc now.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-12-22 11:48:16 +08:00
Daniel Jiang
9d99dfa82b Replace tilde in install_cert.sh 
This commit fixes #13287 to remove the usage of tilde as the $HOME is not available in some
cases.  More details see #13287

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-12-21 20:39:34 +08:00
Qian Deng
31138f12b0
Merge pull request #13806 from ninjadq/fix_python_yaml_load 
Fix pythom yaml load to safe_load
 2020-12-21 16:04:12 +08:00
Qian Deng
9197471e70
Add Scan for internal tls (#13810) 
Signed-off-by: DQ <dengq@vmware.com>
 2020-12-21 15:23:11 +08:00
Wang Yan
9bc6f3cee4
fix robot account update issue (#13741) 
* fix robot account update issue

enable the update method to support both v1 & v2 robot update

Signed-off-by: Wang Yan <wangyan@vmware.com>

* resolve review comments

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2020-12-18 20:01:26 +08:00
Wang Yan
6bc1047013
migration admin job data (#13766) 
1, migrate gc and scan all schedule to schedule/task/exectuion
2, migrate gc history to task/execution

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2020-12-18 16:35:24 +08:00
Will Sun
4392a626f3
Merge pull request #13804 from AllForNothing/scan-all 
Fix robot account UI issues
 2020-12-18 15:48:26 +08:00