Upstream
/
harbor
mirror of https://github.com/goharbor/harbor.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
12,115 Commits 41 Branches 299 Tags 351 MiB
Commit Graph

1361 Commits

Author SHA1 Message Date
MinerYang 83bc24f6c1
Bump up golang version to v1.19 (#17588) 
Signed-off-by: yminer <yminer@vmware.com>

update
 2022-09-22 17:38:26 +08:00
Yang Jiao b44c9d01dd
Drop build postgresql 9.6 (#17458) 
Drop pg9.6, will not support upgrading from pg9.6 to pg13, if you use pg9.6 you can upgrade to 2.5 and upgrade to 2.7

Signed-off-by: Yang Jiao <jiaoya@vmware.com>
 2022-09-21 17:59:25 +08:00
MinerYang 306e780590
Add autoescape parameter for 2.5 & 2.6 jinja (#17439) 
Signed-off-by: yminer <yminer@vmware.com>

update
 2022-08-29 10:22:33 +08:00
Yang Jiao 1ea2ce6134
Update portal-base and nginx-base Dockerfile.base (#17442) 
Installing nginx 1.22 creates the nginx group and nginx user, so instead
of creating them again, modify them.

Signed-off-by: Yang Jiao <jiaoya@vmware.com>
 2022-08-24 14:48:09 +08:00
Shijun Sun 83bce02e61
Refactor swagger ui (#17428) 
Signed-off-by: AllForNothing <sshijun@vmware.com>

Signed-off-by: AllForNothing <sshijun@vmware.com>
 2022-08-23 12:03:45 +08:00
Soumik Majumder 861ca553df
Add autoescape parameter to jinja environments (#15770) 
Signed-off-by: Soumik Majumder <soumikm@vmware.com>

Signed-off-by: Soumik Majumder <soumikm@vmware.com>
 2022-08-23 11:02:16 +08:00
MinerYang a272158023
add deprecation msg for chartmsuem and notary (#17398) 
Signed-off-by: yminer <yminer@vmware.com>

adjust msg

adjust typo

adjust
 2022-08-17 09:30:15 +08:00
MinerYang bd102fbf7d
_version:2.6.0 correction in migration yml jinja (#17328) 
Signed-off-by: yminer <yminer@vmware.com>
 2022-08-05 16:54:16 +08:00
Chenyu Zhang 41105b5191
docs: add helpful description for cache layer configurations (#17317) 
docs: add helpful description for cache layer configurations in harbor.yml.tmpl

Signed-off-by: chlins <chenyuzh@vmware.com>
 2022-08-04 15:14:45 +08:00
Chenyu Zhang 70a95a9696
fix: reduce the high db cpu usage for tag retention (#17296) 
1. Add two indexes to database migrations.
2. Skip refresh quota in middleware for requests from jobservice.
3. Refresh quota by self in the end of tag retention job.

Closes: #14708

Signed-off-by: chlins <chenyuzh@vmware.com>
 2022-08-04 10:08:02 +08:00
MinerYang 0fbff49eb0
bumpup golang to v1.18.5 (#17307) 
Signed-off-by: yminer <yminer@vmware.com>
 2022-08-03 17:44:03 +08:00
MinerYang bd1d441b01
bumpup golang version to v1.18.4 (#17257) 
Signed-off-by: yminer <yminer@vmware.com>
 2022-07-29 09:43:01 +08:00
Wang Yan ef8b8f0be7
resolve the cve export volumn permission (#17157) 
See the right uid and gid for the scandata_exports in the prepare

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2022-07-12 14:52:18 +08:00
prahaladdarkin 130452111b
Vulnerability scan data export functionality (#15998) 
Vulnerability Scan Data (CVE) Export Functionality
Proposal - goharbor/community#174
Closes - https://github.com/goharbor/harbor/issues/17150
Changes:
* CVE Data export to CSV with filtering support.
* Implement CSV data export job for creating CSVs
* APIs to trigger CSV export job executions

Signed-off-by: prahaladdarkin <prahaladd@vmware.com>
 2022-07-11 16:35:04 +08:00
Chenyu Zhang 41d10d571c
fix: repair execution status when it inconsistent (#17128) 
Add migrations sql to repair the execution status when it does not
consistent with task status.

Closes: #17114

Signed-off-by: chlins <chenyuzh@vmware.com>
 2022-07-06 17:02:03 +08:00
Chenyu Zhang 1a1ce634cc
Fix the process of cache layer (#17010) 
fix: fix cache layer issues (#16995,#16997,#16996,#17038)

1. Load config and initialize cache layer in jobservice(for GC)
2. Cache artifact by digest the key should contains repository name
3. Repository cache cleanup error when update
4. Skip save cache when request ctx in transaction

Signed-off-by: chlins <chenyuzh@vmware.com>
 2022-07-06 16:11:53 +08:00
MinerYang 8ba6a2bede
Add migration script for Harbor 2.6 (#17116) 
add migration script for Harbor 2.6

Signed-off-by: yminer <yminer@vmware.com>
 2022-07-06 14:08:24 +08:00
stonezdj(Daojun Zhang) 8115484f32
Create index for performance (#17022) 
Add index to avoid full table scan
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2022-07-04 14:38:05 +08:00
Yang Jiao 77d28105bc
Upgrade pipenv to 2022.1.8 (#17093) 
Signed-off-by: Yang Jiao <jiaoya@vmware.com>
 2022-06-28 23:40:48 +08:00
MinerYang b11970ac3c
update install htpasswd in prepare dockerfile (#17056) 
update install htpasswd in prepare dockerfle

Signed-off-by: yminer <yminer@vmware.com>
 2022-06-23 16:31:37 +08:00
Chenyu Zhang 0cf036e73a
migrate: add db index on artifact repository name (#17053) 
Signed-off-by: chlins <chenyuzh@vmware.com>
 2022-06-22 14:34:40 +08:00
MinerYang 541fbe784f
update support for docker compose v2 (#17039) 
updte supportt for docker compose v2

Signed-off-by: yminer <yminer@vmware.com>

adjust install.sh

Signed-off-by: yminer <yminer@vmware.com>
 2022-06-21 16:55:07 +08:00
Chenyu Zhang 5de1a62002
migrate: add db index on artifact repository name (#17035) 
Signed-off-by: chlins <chenyuzh@vmware.com>
 2022-06-20 16:01:04 +08:00
MinerYang b8a71ac348
bump up golang version to v1.18.3 (#16957) 
Signed-off-by: yminer <yminer@vmware.com>

update ci/ut_install.sh

Signed-off-by: yminer <yminer@vmware.com>
 2022-06-08 17:56:14 +08:00
Chenyu Zhang 7ecd4a3f29
fix(prepare): validate expire hours when enable cache (#16949) (#16953) 
Signed-off-by: chlins <chenyuzh@vmware.com>
 2022-06-07 13:21:47 +08:00
Wilmar den Ouden 79600f3fee
feat: enabled Github GHCR as proxy cache (#16834) 
Signed-off-by: wilmardo <info@wilmardenouden.nl>
 2022-06-06 15:36:16 +08:00
Franz Nemeth 7c2e591c26
fix: add patch for registry layers larger than 10G with S3 backend (#16322) 
add patch for registry layers larger than 10G with S3 backend

Signed-off-by: Franz Nemeth <franz.nemeth@dynatrace.com>
 2022-06-02 16:51:29 +02:00
MinerYang 1db7d6375a
support docker compose v2 (#16919) 
Signed-off-by: yminer <yminer@vmmware.com>

Co-authored-by: yminer <yminer@vmmware.com>
 2022-05-30 15:19:01 +08:00
Mac Chaffee ecc8c59101
Use exec in registryctl so signals are passed properly (#16642) 
Signed-off-by: Mac Chaffee <machaffe@renci.org>
 2022-05-25 14:11:31 +08:00
prahaladdarkin 27ec871185
System Artifact Manager database schema creation, model definitons, and tests (#16678) 
Closes:
https://github.com/goharbor/harbor/issues/16540
https://github.com/goharbor/harbor/issues/16541
https://github.com/goharbor/harbor/issues/16542

Signed-off-by: prahaladdarkin <prahaladd@vmware.com>
 2022-05-09 17:32:57 +08:00
Shengwen Yu 3e8bf9faed fix: registry/redis.patch & registry/builder 
Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
 2022-04-29 08:51:49 +08:00
MinerYang 615a4cbb52
install only htpasswd binary from photon package httpd (#16771) 
Signed-off-by: yminer <yminer@vmmware.com>

cpio -u

Signed-off-by: yminer <yminer@vmmware.com>

Co-authored-by: yminer <yminer@vmmware.com>
 2022-04-28 13:20:03 +08:00
chlins 8c223135e7 feat: add cache layer for artifact (#16739) 
Implement cache layer for resource artifact and define common
cache manager and workflow. Also add cache related options to
configuration yaml.

Signed-off-by: chlins <chenyuzh@vmware.com>
 2022-04-25 18:01:45 +08:00
chlins f7d10474d1 migrations: correct project metadata public value 
Signed-off-by: chlins <chenyuzh@vmware.com>
 2022-03-28 15:17:20 +08:00
stonezdj 9d480a84d2 Create index on task with job_id 
Missing index with job_id, when query task with job_id, it cause a full table scan, caused performance issue
  Fixes #15271

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2022-03-02 20:51:36 +08:00
MinerYang 74d2670616
bump up Go version to 1.17.7 (#16415) 
Signed-off-by: yminer <yminer@vmmware.com>

update CONTRIBUTING.md 2.4 golang version to 1.17.7

Signed-off-by: yminer <yminer@vmmware.com>

Co-authored-by: yminer <yminer@vmmware.com>
 2022-02-25 17:23:18 +08:00
MinerYang 6f1f2c0395
update 2.5.0 migration script harbor.yml.jinja (#16354) 
Signed-off-by: yminer <yminer@vmmware.com>

Co-authored-by: yminer <yminer@vmmware.com>
 2022-02-11 17:30:06 +08:00
MinerYang 6c97d3f0c7
rollback to delete external redis username (#16353) 
Signed-off-by: yminer <yminer@vmmware.com>

Co-authored-by: yminer <yminer@vmmware.com>
 2022-02-11 17:28:10 +08:00
MinerYang 77b6a130f8
Add Migration Script for Harbor 2.5 (#16346) 
* add Harbor2.5 migration script

Signed-off-by: yminer <yminer@vmmware.com>

* update 2_5_0 migration harbor.yml.jinja

Signed-off-by: yminer <yminer@vmmware.com>

Co-authored-by: yminer <yminer@vmmware.com>
 2022-02-11 12:18:06 +08:00
Wang Yan 6c3f9a8366
bump up distribution version to v2.8 (#16338) 
Signed-off-by: Wang Yan <wangyan@vmware.com>
 2022-02-10 11:46:58 +08:00
MinerYang 1a2dd256b5
Add external redis username for AUTH (#16336) 
* add external redis username for AUTH

* update harbor.yml.tmpl

* add external redis username for AUTH

update harbor.yml.tmpl

Co-authored-by: yminer <yminer@vmmware.com>
 2022-02-09 22:06:28 +08:00
Shengwen YU 0bf5998f96
upgrade Chartmuseum to v0.14.0 (#16334) 
Signed-off-by: Shengwen Yu <yshengwen@vmware.com>

Co-authored-by: Shengwen Yu <yshengwen@vmware.com>
 2022-02-09 12:13:05 +08:00
Shengwen Yu aacebcc8c4 fix: vulnerability_record & report_vulnerability_record table "integer out of range" error 
Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
 2022-02-09 10:19:46 +08:00
Yang Jiao 709a154724 Specify postgresql version to 13 
Signed-off-by: Yang Jiao <jiaoya@vmware.com>
 2022-01-27 06:46:15 +00:00
He Weiwei 8554b7d7c5
refactor: replace lib/pq with jackc/pgx (#16267) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2022-01-27 11:09:37 +08:00
He Weiwei 89fe7b0819
Move removing old version scan reports of trivy to 2.4.2 (#16261) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2022-01-24 14:05:24 +08:00
Shengwen Yu 46f97ecf6c feat: bump TRIVYVERSION to v0.22.0 and bump TRIVYADAPTERVERSION to v0.25.0 
Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
 2022-01-21 09:14:07 +08:00
Morlay 9ad68aa526
Updates goharbor to var IMAGENAMESPACE in Makefiles to make injectable (#15218) 
Signed-off-by: Morlay <morlay.null@gmail.com>
 2022-01-20 12:14:29 +08:00
MinerYang 6abc505060
Merge pull request #16239 from MinerYang/bump_up/go_v1.17.6 
bump up go version to v1.17.6
 2022-01-18 10:32:09 +08:00
yminer 9acf3d21ce bump up go version to v1.17.6 
Signed-off-by: yminer <yminer@vmmware.com>
 2022-01-17 17:45:22 +00:00
孙世军 8f77567589
Upgrade UI dependencies (#16233) 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2022-01-17 14:20:20 +08:00
He Weiwei 097efb201b
fix: Clean old version scan reports of trivy (#16235) 
From harbor 2.4, the trivy does not support scan report v1.0, we need to remove the old version scan reports from the DB.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2022-01-17 13:59:28 +08:00
Yang Jiao b4de95941b Specify postgresql version to 13 
Signed-off-by: Yang Jiao <jiaoya@vmware.com>
 2022-01-12 08:55:42 +00:00
stonezdj 17d8b7b813 Add upload purge config to registry/config.yml 
Enable the uploadpurging by default
  Fixes #15641

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2022-01-04 11:15:51 +08:00
Qian Deng 69a194b2b4 Fix: using traditional `PKCS#1` format RSA key 
The openssl 3.0.0 using newer `PKCS#8` format.
But it's not compatitable with harbor core
So using tradictional format instead

Signed-off-by: Qian Deng <dengq@vmware.com>
 2021-12-10 11:34:12 +08:00
Wang Yan 742e7ded00
add accessory dao service (#16045) 
Signed-off-by: wang yan <wangyan@vmware.com>
 2021-12-03 14:34:02 +08:00
Qian Deng 5fc4449450
Merge pull request #16029 from ninjadq/fix_chart_replication_issue 
Fix: chart replication dup files issue
 2021-12-03 13:47:42 +08:00
Qian Deng 593117a127 Fix: chart replication dup files issue 
this patch fixed #15522

Signed-off-by: Qian Deng <dengq@vmware.com>
 2021-11-17 11:24:43 +00:00
Wang Yan a956758302
bump up go version to v1.17 (#15865) 
* bump up go version to v1.17

Signed-off-by: Wang Yan <wangyan@vmware.com>

* gofmt fail

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-10-25 17:28:29 +08:00
Rolf Ahrenberg 5f3972f86d
Add configurable timeout for Trivy scans (#15796) 
Signed-off-by: Rolf Ahrenberg <Rolf.Ahrenberg@saunalahti.fi>
 2021-10-22 14:36:12 +08:00
stonezdj(Daojun Zhang) c814daa894
Update the jaeger settings in harbor.yaml.tmpl (#15841) 
agent default port to 6831

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2021-10-22 13:43:28 +08:00
Yurii Paneiko be1e762b70
For some reason this script hangs on bzip2 presence checking (#15647) 
Steps to reproduce:

Clone repo from master branch
Run: make install COMPILETAG=compile_golangimage
Wait until script will check that bzip2 is installed

Signed-off-by: YuriiPaneiko <yurapaneyko@gmail.com>
 2021-10-14 20:11:14 +08:00
He Weiwei b390112f5a
fix: convert severity from negligible to none before saving to db (#15791) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2021-10-14 16:02:38 +08:00
stonezdj(Daojun Zhang) 32023891eb
Merge pull request #15675 from stonezdj/21sep22_email_nullable 
Allow empty email attribute for ldap/oidc user
 2021-10-02 10:33:38 +08:00
stonezdj 06715af303 Allow empty email attribute for ldap/oidc user 
Define user.Email as sql.NullString to avoid unique constraint when email is empty in LDAP/OIDC
  Separate the common/models/User with the pkg/user/dao/User
  Fixes #10400
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2021-09-30 14:35:18 +08:00
Qian Deng 3c23926bdc Add validation for tracing 
* add  validation of tracing in validating process

Signed-off-by: Qian Deng <dengq@vmware.com>
 2021-09-27 13:10:55 +00:00
stonezdj(Daojun Zhang) 972fa0880b
Merge pull request #15505 from ninjadq/add_distributed_tracing 
Add distributed tracing
 2021-09-22 14:16:19 +08:00
Qian Deng 354a2bd80d Enhance the trace related code 
* Move request id to requestid middleware
* fix span pass to child ctx on orm
* fix typos
* remove unused code
* add operation name to Transaction

Signed-off-by: Qian Deng <dengq@vmware.com>
 2021-09-22 04:14:43 +00:00
Qian Deng 05bc946444
Merge pull request #14383 from XavierDuthil/use-exec-in-entrypoints 
Use exec in all components' entrypoints
 2021-09-22 10:49:21 +08:00
Qian Deng bad913cf6d Refactor trace code 
* use lib trace helper function
* add gracefull shutdown
* Add commens for new added exposed function
* Add licence on top of new created files
* Update trace library
* Update configs
* Add attribute and namespance in config

Signed-off-by: Qian Deng <dengq@vmware.com>
 2021-09-18 10:58:52 +00:00
Qian Deng 6fec5b2873 Add trace to jobservice 
* Add trace init in main
* Add env template
* Add trace for router

Signed-off-by: Qian Deng <dengq@vmware.com>
 2021-09-18 10:58:52 +00:00
Qian Deng 14095fb10b Add trace to registryctl 
* Add trace init to main
* Add trace for http server
* Add trace for gc
* Add env template trace

Signed-off-by: Qian Deng <dengq@vmware.com>
 2021-09-18 10:58:52 +00:00
Qian Deng a15983432c Add trace for core 
* Add trace related lib
* Add trace middleware for core
* add rid for middleware

Signed-off-by: Qian Deng <dengq@vmware.com>
 2021-09-18 10:58:52 +00:00
Qian Deng b812a300be Add trace related configs 
* Update harbor config template
* Update python config parsing
* Update env template

Signed-off-by: Qian Deng <dengq@vmware.com>
 2021-09-18 10:58:52 +00:00
He Weiwei ecc1a04c92
refactor: implement a lock free quota (#15399) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2021-09-17 11:26:37 +08:00
DQ e4c1521efc Add migration script for harbor 2.4 
Signed-off-by: DQ <dengq@vmware.com>
 2021-09-16 09:31:39 +00:00
Ziming Zhang 98cef43ead limit replication bandwidth 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2021-09-14 11:23:28 +08:00
Wenkai Yin(尹文开) 3ce072016f
Merge pull request #15527 from sevendials/logrotate_fails_when_cwd_is_not_readable 
Logrotate fails when cwd is not accessible
 2021-09-13 16:08:23 +08:00
Wang Yan dbcbc8bad0
bump up go to v1.16.7 (#15564) 
Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-09-09 17:58:00 +08:00
Wang Yan 93a078d225
deprecate dns search (#15557) 
For details, please refer to https://github.com/goharbor/harbor/issues/14146#issuecomment-793390718
and https://github.com/docker/for-linux/issues/1164.

If anyone encounter the issue mentioned by https://github.com/goharbor/harbor/issues/6031, add the dns_search: . to the releated container.

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-09-08 14:22:59 +08:00
Christopher Jenkins 0585b148c7 Logrotate fails when cwd is not accessible 
Logrotate is run with sudo as the syslog user by cron.hourly
The current working directory is `/root` which is inaccessible to the syslog
user so the logrotate command fails. Currently the following stderr is being
thrown away by the cron script:
```
error: cannot open current directory: Permission denied
```

Fixes #15468

Signed-off-by: Christopher Jenkins <christj@gmail.com>
 2021-09-01 15:28:30 -07:00
孙世军 67681b1d83
update node version for portal container (#15396) 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2021-08-10 09:14:46 +08:00
stonezdj(Daojun Zhang) 2fa530eefa
Merge pull request #15247 from stonezdj/21jun30_remove_auth_user 
Delete users under auth_mode other than db_auth
 2021-07-14 13:25:43 +08:00
stonezdj c163bc8317 Delete users under auth_mode other than db_auth 
The following information should cleanup before delete user:
  Delete project member of this user.
  Delete oidc_user when auth_mode is oidc_auth.
  Fixes #8424
  It also removes the deleted user from project member and the deleted condition in the project member query for consistency
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2021-07-12 11:31:40 +08:00
Wang Yan 494d74d32d
bump up go version to 1.16 (#15286) 
Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-07-08 16:25:15 +08:00
Wang Yan 4017e995b7 roll back go for notary binary 
Fixes #14932

Harbor recompiles the notary v0.6.1 with go 1.15 from v2.2.0, which introduces an break change that leads to notary key not found after migration.

[Root cause]
Notary v0.6.1 consumed an old version dvsekhvalnov/jose2, which is not compatible with go 1.15.

[References]
https://github.com/dvsekhvalnov/jose2go/issues/26
https://github.com/golang/go/issues/41089

[Resolve]
To resolve this issue, we have to roll back go vesrion to v1.14 for notary v0.6.1 binary and keep it until upstream have a patch release to support go 1.15 or above.

[Break change]
If you pushed and signed image using Harbor v2.2.0 ~ v2.2.2 and created new repository key in notary, you will encouter the same issue after migrate to v2.2.3(or above) or v2.3.1(or above) because of the go version downgrade. We will have a FAQ to help you to resovle this particular scenario.

The influence path of the particular case:
Harbor v2.1.0(or lower) --> [v2.2.0 ~ v2.2.2] --> v2.2.3(or above)
Harbor v2.1.0(or lower) --> v2.3.0 --> v2.3.1(or above)

The non influence path of the paticular case:
Harbor v2.1.0(or lower) --> v2.2.3(or above)
Harbor v2.1.0(or lower) --> v2.3.1(or above)

[Fix in Version]
Harbor v2.2.3 or above
Harbor v2.3.1 or above

[Note]
If you're a heavy user of notary, avoid using v2.2.0, v2.2.1, v2.2.2 and v2.3.0, and use the fixed version for instead.

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-06-29 16:18:55 +08:00
danfengliu c39345da96
Merge pull request #15148 from danfengliu/missing-db-base-login-in-build-base-workflow 
Fix issue of missing db base build process in build base workflow
 2021-06-16 16:39:56 +08:00
Will Sun 410c5bcd78
Undate Angular to the latest version (#15129) 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2021-06-16 16:39:10 +08:00
danfengliu f367aad760 Fix issue of missing db base build process in build base workflow 
There is extra build step for db base image building since v2.3, so this
step should be added back.

Signed-off-by: danfengliu <danfengl@vmware.com>
 2021-06-16 15:22:32 +08:00
He Weiwei 72f1afe2cc
perf: configurations for perf improvement (#15142) 
Closes #15041

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2021-06-15 18:39:15 +08:00
danfengliu 254c4a3497
Merge pull request #15046 from danfengliu/refine-base-build-process-in-makefile 
Refind build base process in Makefile
 2021-06-09 11:21:48 +08:00
He Weiwei 28921e2997 fix: listen to 5443 when clean unexpected status of db 
Closes #15081

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2021-06-08 06:41:23 +00:00
DQ 68ac772726 Fix: Remove v6 format of harbor 
Signed-off-by: DQ <dengq@vmware.com>
 2021-06-07 16:26:53 +08:00
Qian Deng 0867a6bfd6
Merge pull request #15055 from ninjadq/health_check_url_2_dn_instead_of_ip 
Add IPv6 support
 2021-06-03 17:57:55 +08:00
DQ 1b6b47f860 Add IPv6 support 
* 127.0.0.1 to localhost
* listening net addr add ipv6 format

Signed-off-by: DQ <dengq@vmware.com>
 2021-06-03 09:04:49 +00:00
danfengliu d66ea07c2f Refind build base process in Makefile 
Remove build base executable in Makefile by replacing it as an input parameter.
Add add more input parameters for controlling docker pull/push to make
build base process flexible for users.

Signed-off-by: danfengliu <danfengl@vmware.com>
 2021-06-03 14:48:34 +08:00
Wang Yan eec9893918 set shm size of postgres 
Fixed #15034, as for postgres 13, the default shm size is 64MB, set to 1gb to avoid could not resize shared memory segment error.

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-06-03 11:05:33 +08:00
Wang Yan ae06ac2fae
fix db issue on helm upgrade (#15028) 
fixes #15000

It needs to double confirm that old pg is stopped before migration

Signed-off-by: wang yan <wangyan@vmware.com>
 2021-06-01 15:59:41 +08:00
ChenYu Zhang a038ba672f
perf: add 2.2.3 schema up sql to master (#15027) 
Signed-off-by: chlins <chlins.zhang@gmail.com>
 2021-06-01 14:12:25 +08:00
Wang Yan 66b8a8f8dd
add build arch parameter in Makefile (#14995) 
* add build arch parameter in Makefile

Add parameter BUILDARCH for make file. DB base builds pg96 for x86_64 only

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-06-01 10:38:05 +08:00
DQ 5d02acd043 Add upgrade script for harbor 2.3 
no new config item added. harbor.yml keep the same as last version

Signed-off-by: DQ <dengq@vmware.com>
 2021-05-28 20:30:12 +08:00
Wenkai Yin c585f92034 Fix bug of migration sql for replication policy 
Fix bug of migration sql for replication policy

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2021-05-26 18:57:23 +08:00
Qian Deng 9ea8aade01
Upgrade prepare to consistent with photon 4 (#14698) 
* requires version to 3.9.1
* upgrade packages

Signed-off-by: DQ <dengq@vmware.com>
 2021-05-26 16:39:04 +08:00
Wang Yan 39bdd7b506
pg upgrade failure handling (#14934) 
To ensure the upgrade execution idempotence, it needs to clean the $PGDATANEW on pg_upgrade failure.
Otherwise, the upgrade will skip the upgrade process from the second time launch as the exist of $PGDATANEW.

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-05-21 13:53:39 +08:00
danfengliu 6c14e699b1
Merge pull request #14844 from danfengliu/fix-notary-trust-directory-issue-in-nightly 
Fix notary trust directory issue in nightly
 2021-05-20 21:56:27 +08:00
Wang Yan 73bd373a75
create index for audit (#14930) 
Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-05-20 19:50:13 +08:00
danfengliu 3ef4dc17fc Fix notary trust directory issue and add login for each base image in makefile 
1. Use root instead of ~ in notary parameter;
2. Fix tag immutability issue caused by GUI change;
3. Replace email domain name to harbor test;
4. Add login for each base image in makefile;
5. Add customize look test in nightly.

Signed-off-by: danfengliu <danfengl@vmware.com>
 2021-05-20 18:23:13 +08:00
Wang Yan 86185989cf
support pg upgrade (#14846) 
1, use the pg source and photon spec to build postgres 9.6
2, install 9.6 on the photon 4.0
3, then leverage pg_upgrade to handle the pg major version migration

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-05-20 16:25:50 +08:00
Wenkai Yin dc059a9a8f Improve the performance of artifact related APIs 
Improve the performance of artifact related APIs by adding indexes and refactoring sql logic

Closes #13890 #14813 #14814

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2021-05-20 11:25:43 +08:00
Wang Yan 0fb520a33b bump up go to v1.15.12 
Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-05-14 15:36:23 +08:00
Qian Deng 17dd48e5a3
Merge pull request #14854 from ninjadq/upgrade_helm_chart 
Upgrade helm chart
 2021-05-12 19:33:03 +08:00
Wenkai Yin c04f3a2aac Fix duplicate execution record issue 
When the core service cannot response the checkin request in time, duplicated execution records may be created, this commit introduces the revision column to make sure there is only one record for one schedule trigger

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2021-05-12 15:40:52 +08:00
DQ 04ba4a4033 Upgrade chartmuseum version 
from 1.12.0 to 1.13.1

Signed-off-by: DQ <dengq@vmware.com>
 2021-05-11 13:51:55 +00:00
He Weiwei 476732df89 fix: change art_size to bigint in migrations 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2021-05-11 04:01:02 +00:00
Wenkai Yin(尹文开) 9bbffa06db
Merge pull request #14662 from ywk253100/210413_dest_ns 
Refactor the replication policy destination namespace logic
 2021-04-29 11:18:43 +08:00
Wenkai Yin 710c80078b Refactor the replication policy destination namespace logic 
Support specifying what part of the repository will be replaced by the provided namespace

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2021-04-28 10:16:20 +08:00
Daniel Jiang c701ce09fa
Merge pull request #14681 from bitsf/fix_typo_NOTARYURL 
Fixed typo in NOTARYURL variable name
 2021-04-21 17:38:01 +08:00
Wenkai Yin(尹文开) 0f6692f68f
Merge pull request #14682 from ninjadq/hostname_hardcode_to_localhost 
Fix: Use local host on db's healthcheck
 2021-04-19 21:02:31 +08:00
danfengliu f0ebd17994 Add build base image step in build package git action workflow 
Build base image step should be in build package workflow, and local base images build by new step should be removed since images have been pushed to docker hub.

Signed-off-by: danfengliu <danfengl@vmware.com>
 2021-04-17 18:10:44 +08:00
Ziming Zhang 39f70287b4 Fixed typo in NOTARYURL variable name 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2021-04-17 15:55:02 +08:00
DQ ffed6459c7 Fix: Use local host on db's healthcheck 
hostname -i will malfunction in some cases like the `nsswitch.conf` file does'nt exist

Signed-off-by: DQ <dengq@vmware.com>
 2021-04-16 18:37:24 +08:00
Pushkar Joglekar 3947c5faff Add --no-cache and --pull flag to image builds to ensure latest security fixes are pulled from base image 
Signed-off-by: Pushkar Joglekar <pjoglekar@vmware.com>
 2021-04-12 09:49:27 -07:00
Qian Deng c5d12ce8ee
Merge pull request #14542 from ninjadq/add_task_info_in_exporter 
Add task info in exporter
 2021-04-07 18:17:26 +08:00
Alexis 06fa88cfb7 Fix typo 
Signed-off-by: Alexis <60alexis@gmail.com>
 2021-04-07 15:58:17 +08:00
Alexis e33f7aa9dd Add redis port to 2.1.0 jinja template 
Signed-off-by: Alexis <60alexis@gmail.com>
 2021-04-07 15:58:17 +08:00
Alexis 7742aec4af Add port to 2.0.0 jinja file 
Signed-off-by: Alexis <60alexis@gmail.com>
 2021-04-07 15:58:17 +08:00
Alexis d28845af51 Remove external_redis.port since not used since v1.10.0 
Signed-off-by: Alexis <60alexis@gmail.com>
 2021-04-07 15:58:17 +08:00
stonezdj(Daojun Zhang) 448f0b6e28
Merge pull request #14579 from stonezdj/21apr_add_docker_registry_proxy 
Support proxy cache for docker-registry type
 2021-04-07 10:59:24 +08:00
Steven Zou e2148f9eea
Merge pull request #14514 from goharbor/dependabot/pip/make/photon/prepare/pyyaml-5.4 
Bump pyyaml from 4.2b1 to 5.4 in /make/photon/prepare
 2021-04-07 09:57:07 +08:00
Steven Zou 10711b7de1
Merge pull request #14482 from goharbor/dependabot/pip/make/photon/prepare/jinja2-2.11.3 
Bump jinja2 from 2.11.1 to 2.11.3 in /make/photon/prepare
 2021-04-07 09:56:23 +08:00
stonezdj ccd9ee8c56 Support proxy cache for docker-registry type 
Add proxy cache for docker registry type
Fixes #14477, #14547
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2021-04-06 16:47:12 +08:00
Wang Yan d03a29e531 bump up photon to 4.0 
Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-03-31 13:25:29 +08:00
DQ 7eebbeebdf Add jobservice task queue related task 
add jobservice metrics
add redis client

Signed-off-by: DQ <dengq@vmware.com>
 2021-03-30 10:29:36 +00:00
DQ fbe9cd88f8 Enabled Prometheus for Jobservice 
* Add prom server on jobservice
* Enabeld configs in templates
* Enabeld jobservice metrics in nginx

Signed-off-by: DQ <dengq@vmware.com>
 2021-03-30 08:52:59 +00:00
dependabot[bot] f20f4215c3
Bump pyyaml from 4.2b1 to 5.4 in /make/photon/prepare 
Bumps [pyyaml](https://github.com/yaml/pyyaml) from 4.2b1 to 5.4.
- [Release notes](https://github.com/yaml/pyyaml/releases)
- [Changelog](https://github.com/yaml/pyyaml/blob/master/CHANGES)
- [Commits](https://github.com/yaml/pyyaml/commits/5.4)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-03-25 22:46:56 +00:00
DQ f5fcc7bd31 Add base image for exporter 
* Add base
* update Makefile

Signed-off-by: DQ <dengq@vmware.com>
 2021-03-25 16:35:57 +08:00
dependabot[bot] 0ec667c4d8
Bump jinja2 from 2.11.1 to 2.11.3 in /make/photon/prepare 
Bumps [jinja2](https://github.com/pallets/jinja) from 2.11.1 to 2.11.3.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/master/CHANGES.rst)
- [Commits](https://github.com/pallets/jinja/compare/2.11.1...2.11.3)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-03-20 00:43:21 +00:00
Wenkai Yin 8b1817be0f Fix the consume too much CPU issue 
1. Update execution status during the upgrade
2. Refine the execution sweeper

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2021-03-17 13:34:24 +08:00
Wenkai Yin 43df3bf8a4 Add upgrade sql file introduced in 2.1.4 
1. Add upgrade sql file introduced in 2.1.4
2. Minor improvement for task/execution to cover corner cases

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2021-03-15 16:48:51 +08:00
Xavier Duthil 280c8272f8
Use exec in all components' entrypoints 
Use the exec Bash command so that the final running application becomes
the container’s PID 1. This allows the application to receive any Unix
signals sent to the container, in accordance with
https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#entrypoint

Currently, SIGTERM signals sent by kubernetes are not passed to the
executed binary.

Signed-off-by: Xavier Duthil <xavier.duthil@corp.ovh.com>
 2021-03-05 15:00:25 +01:00
Wang Yan 3dfddfdf4e
patch upstream fix for io reader (#14356) 
Fixes #12850
This patch can fix the GC failure in the NFS v3 env, see https://github.com/distribution/distribution/pull/3309#issuecomment-783606968

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-03-04 15:33:09 +08:00
Daniel Jiang 387be3686a Refine the way to set X-Forwarded-Proto in nginx 
Refine the way to set the header so user won't need to comment it if
Harbor is sitting behind a reverse proxy.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2021-02-25 17:43:55 +08:00
danfengliu 9e3f0de12c
Merge pull request #14124 from danfengliu/reschedule-docker-login-policy-in-build-base-image-in-master 
Reschedule docker login policy in base image build process
 2021-02-23 10:10:59 +08:00
danfengliu 7d05c8e513 Reschedule docker login policy in base image build process 
Signed-off-by: danfengliu <danfengl@vmware.com>
 2021-02-22 10:05:25 +08:00
Josh Soref dfe360040b Spelling 
* addition
* attribute
* auditing
* availability
* available
* bandwidth
* browser
* business
* cadence
* chartmuseum
* client
* column
* content
* demonstrate
* described
* endpoints
* facilitate
* github
* harbor
* information
* instance
* manual
* meaningful
* operation
* overridden
* password
* possible
* project
* refactor
* replication
* requires
* running
* scanned
* settings
* signup
* those
* unsigned
* vulnerability

--
Also removes trailing space from a filename

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
 2021-02-19 11:59:15 +08:00
DQ 307c5a8ed4 Fix metrics template for http mode 
the port shouldn't be hardcode

Signed-off-by: DQ <dengq@vmware.com>
 2021-02-05 18:44:28 +00:00
Ziming Zhang ec83f49a1a fix(retention) migrate sql error 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2021-02-05 09:09:26 +00:00
Wang Yan 24ec772978
fix gc migration issue (#14174) 
For the upgrade path v1.10 - v2.1.* - v2.2.0, if user doesn't reset the GC schdule that was created in 1.10 in the v2.1,
the job parameters will keep empty in the database. The fix gives a default value for the schedule record.

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-02-05 12:25:24 +08:00
DQ 051b5f289d Add sen existed check for internal cert 
fali ealier when there is no san

Signed-off-by: DQ <dengq@vmware.com>
 2021-01-28 08:22:07 +00:00
Wenkai Yin(尹文开) 50a1e85095
Make sure the revision of execution isn't null during the upgrade (#14085) 
Make sure the revision of execution isn't null during the upgrade
Fixes #14075

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2021-01-27 10:10:36 +08:00