Upstream/harbor
1
0
Fork 0
mirror of https://github.com/goharbor/harbor.git synced 2024-11-28 21:25:55 +01:00
Code Issues Projects Releases Wiki Activity
12,217 Commits 53 Branches 317 Tags 334 MiB
66ccf302f3
Commit Graph

1373 Commits

Author SHA1 Message Date
asta1992
6e51dc0161
Fix: Small typos in harbor.yml.tmpl (#18009) 
Signed-off-by: Andreas Stalder <andreas.stalder@terreactive.ch>
 2022-12-21 09:56:12 +08:00
MinerYang
d168f080ed
bump up golang to 1.19.4 for main (#17997) 
bump up golang to 1.19.4 in main

Signed-off-by: yminer <yminer@vmware.com>

Signed-off-by: yminer <yminer@vmware.com>
 2022-12-15 13:55:30 +08:00
Wang Yan
74dfc905ec
remove the redundant data from table artifact_blob (#17937) 
use sql in the migration process to delete all the useless data of table artifact_blob

Signed-off-by: Wang Yan <wangyan@vmware.com>

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2022-12-13 15:32:57 +08:00
Jianwei Guo
cb11540a14
add parameters for PostgreSQL (#16641) 
Signed-off-by: sayaoailun <guojianwei007@126.com>
 2022-11-30 19:08:08 +08:00
MinerYang
9ae35dbc50
add migration script for 2.7 (#17838) 
add migration script for 2.7
Signed-off-by: yminer <yminer@vmware.com>
 2022-11-23 17:49:55 +08:00
stonezdj(Daojun Zhang)
57aff88eb5
Add REST API to list job queue status, pause/resume job queue and list schedulers (#17707) 
Add queue manager and redis client
  Update scheduler to add count and list

Signed-off-by: stonezdj <daojunz@vmware.com>

Signed-off-by: stonezdj <daojunz@vmware.com>
 2022-11-17 20:12:07 +08:00
MinerYang
f20a2e433f
bump up golang to 1.19.3 (#17751) 
Signed-off-by: yminer <yminer@vmware.com>

Signed-off-by: yminer <yminer@vmware.com>
 2022-11-04 16:17:11 +08:00
erismaster
ec7717e23b
Add JFrog Artifactory As Supported Proxy-Cache Registry Source (#17738) 
Add JFrog Artifactory As Supported Proxy-Cache

Enables the support of JFrog Artifactory as a source for proxy-cache. I
have tested this with a local build and was able to proxy images just
fine.

Signed-off-by: Derrik Campau <dcampau@vmware.com>

Signed-off-by: Derrik Campau <dcampau@vmware.com>
 2022-11-04 12:50:34 +08:00
Chlins Zhang
e81067badb
fix: cherry pick 262 sql for migrations (#17735) 
Signed-off-by: chlins <chenyuzh@vmware.com>
 2022-11-03 10:05:46 +08:00
Chlins Zhang
c330b8c63a
feat: introduce the copy by chunk for replication (#17602) 
1. Add sql migration to alter replication policy table
2. Implement the PullBlobChunk and PushBlobChunk for the underlying v2 registry client
3. Update image transfer logic to support copy by chunk
4. Update the replication policy API handler

Signed-off-by: chlins <chenyuzh@vmware.com>
 2022-11-01 11:19:17 +08:00
MinerYang
6b8cdc03c1
bump up golang to 1.19.2 (#17694) 
Signed-off-by: yminer <yminer@vmware.com>

Signed-off-by: yminer <yminer@vmware.com>
 2022-10-26 20:28:19 +08:00
Shengwen YU
0acfbdc7a1
feat: bump TRIVYVERSION to v0.32.1 and bump TRIVYADAPTERVERSION to v0.30.2 (#17681) 
Signed-off-by: Shengwen Yu <yshengwen@vmware.com>

Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
 2022-10-25 14:26:53 +08:00
MinerYang
83bc24f6c1
Bump up golang version to v1.19 (#17588) 
Signed-off-by: yminer <yminer@vmware.com>

update
 2022-09-22 17:38:26 +08:00
Yang Jiao
b44c9d01dd
Drop build postgresql 9.6 (#17458) 
Drop pg9.6, will not support upgrading from pg9.6 to pg13, if you use pg9.6 you can upgrade to 2.5 and upgrade to 2.7

Signed-off-by: Yang Jiao <jiaoya@vmware.com>
 2022-09-21 17:59:25 +08:00
MinerYang
306e780590
Add autoescape parameter for 2.5 & 2.6 jinja (#17439) 
Signed-off-by: yminer <yminer@vmware.com>

update
 2022-08-29 10:22:33 +08:00
Yang Jiao
1ea2ce6134
Update portal-base and nginx-base Dockerfile.base (#17442) 
Installing nginx 1.22 creates the nginx group and nginx user, so instead
of creating them again, modify them.

Signed-off-by: Yang Jiao <jiaoya@vmware.com>
 2022-08-24 14:48:09 +08:00
Shijun Sun
83bce02e61
Refactor swagger ui (#17428) 
Signed-off-by: AllForNothing <sshijun@vmware.com>

Signed-off-by: AllForNothing <sshijun@vmware.com>
 2022-08-23 12:03:45 +08:00
Soumik Majumder
861ca553df
Add autoescape parameter to jinja environments (#15770) 
Signed-off-by: Soumik Majumder <soumikm@vmware.com>

Signed-off-by: Soumik Majumder <soumikm@vmware.com>
 2022-08-23 11:02:16 +08:00
MinerYang
a272158023
add deprecation msg for chartmsuem and notary (#17398) 
Signed-off-by: yminer <yminer@vmware.com>

adjust msg

adjust typo

adjust
 2022-08-17 09:30:15 +08:00
MinerYang
bd102fbf7d
_version:2.6.0 correction in migration yml jinja (#17328) 
Signed-off-by: yminer <yminer@vmware.com>
 2022-08-05 16:54:16 +08:00
Chenyu Zhang
41105b5191
docs: add helpful description for cache layer configurations (#17317) 
docs: add helpful description for cache layer configurations in harbor.yml.tmpl

Signed-off-by: chlins <chenyuzh@vmware.com>
 2022-08-04 15:14:45 +08:00
Chenyu Zhang
70a95a9696
fix: reduce the high db cpu usage for tag retention (#17296) 
1. Add two indexes to database migrations.
2. Skip refresh quota in middleware for requests from jobservice.
3. Refresh quota by self in the end of tag retention job.

Closes: #14708

Signed-off-by: chlins <chenyuzh@vmware.com>
 2022-08-04 10:08:02 +08:00
MinerYang
0fbff49eb0
bumpup golang to v1.18.5 (#17307) 
Signed-off-by: yminer <yminer@vmware.com>
 2022-08-03 17:44:03 +08:00
MinerYang
bd1d441b01
bumpup golang version to v1.18.4 (#17257) 
Signed-off-by: yminer <yminer@vmware.com>
 2022-07-29 09:43:01 +08:00
Wang Yan
ef8b8f0be7
resolve the cve export volumn permission (#17157) 
See the right uid and gid for the scandata_exports in the prepare

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2022-07-12 14:52:18 +08:00
prahaladdarkin
130452111b
Vulnerability scan data export functionality (#15998) 
Vulnerability Scan Data (CVE) Export Functionality
Proposal - goharbor/community#174
Closes - https://github.com/goharbor/harbor/issues/17150
Changes:
* CVE Data export to CSV with filtering support.
* Implement CSV data export job for creating CSVs
* APIs to trigger CSV export job executions

Signed-off-by: prahaladdarkin <prahaladd@vmware.com>
 2022-07-11 16:35:04 +08:00
Chenyu Zhang
41d10d571c
fix: repair execution status when it inconsistent (#17128) 
Add migrations sql to repair the execution status when it does not
consistent with task status.

Closes: #17114

Signed-off-by: chlins <chenyuzh@vmware.com>
 2022-07-06 17:02:03 +08:00
Chenyu Zhang
1a1ce634cc
Fix the process of cache layer (#17010) 
fix: fix cache layer issues (#16995,#16997,#16996,#17038)

1. Load config and initialize cache layer in jobservice(for GC)
2. Cache artifact by digest the key should contains repository name
3. Repository cache cleanup error when update
4. Skip save cache when request ctx in transaction

Signed-off-by: chlins <chenyuzh@vmware.com>
 2022-07-06 16:11:53 +08:00
MinerYang
8ba6a2bede
Add migration script for Harbor 2.6 (#17116) 
add migration script for Harbor 2.6

Signed-off-by: yminer <yminer@vmware.com>
 2022-07-06 14:08:24 +08:00
stonezdj(Daojun Zhang)
8115484f32
Create index for performance (#17022) 
Add index to avoid full table scan
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2022-07-04 14:38:05 +08:00
Yang Jiao
77d28105bc
Upgrade pipenv to 2022.1.8 (#17093) 
Signed-off-by: Yang Jiao <jiaoya@vmware.com>
 2022-06-28 23:40:48 +08:00
MinerYang
b11970ac3c
update install htpasswd in prepare dockerfile (#17056) 
update install htpasswd in prepare dockerfle

Signed-off-by: yminer <yminer@vmware.com>
 2022-06-23 16:31:37 +08:00
Chenyu Zhang
0cf036e73a
migrate: add db index on artifact repository name (#17053) 
Signed-off-by: chlins <chenyuzh@vmware.com>
 2022-06-22 14:34:40 +08:00
MinerYang
541fbe784f
update support for docker compose v2 (#17039) 
updte supportt for docker compose v2

Signed-off-by: yminer <yminer@vmware.com>

adjust install.sh

Signed-off-by: yminer <yminer@vmware.com>
 2022-06-21 16:55:07 +08:00
Chenyu Zhang
5de1a62002
migrate: add db index on artifact repository name (#17035) 
Signed-off-by: chlins <chenyuzh@vmware.com>
 2022-06-20 16:01:04 +08:00
MinerYang
b8a71ac348
bump up golang version to v1.18.3 (#16957) 
Signed-off-by: yminer <yminer@vmware.com>

update ci/ut_install.sh

Signed-off-by: yminer <yminer@vmware.com>
 2022-06-08 17:56:14 +08:00
Chenyu Zhang
7ecd4a3f29
fix(prepare): validate expire hours when enable cache (#16949) (#16953) 
Signed-off-by: chlins <chenyuzh@vmware.com>
 2022-06-07 13:21:47 +08:00
Wilmar den Ouden
79600f3fee
feat: enabled Github GHCR as proxy cache (#16834) 
Signed-off-by: wilmardo <info@wilmardenouden.nl>
 2022-06-06 15:36:16 +08:00
Franz Nemeth
7c2e591c26
fix: add patch for registry layers larger than 10G with S3 backend (#16322) 
add patch for registry layers larger than 10G with S3 backend

Signed-off-by: Franz Nemeth <franz.nemeth@dynatrace.com>
 2022-06-02 16:51:29 +02:00
MinerYang
1db7d6375a
support docker compose v2 (#16919) 
Signed-off-by: yminer <yminer@vmmware.com>

Co-authored-by: yminer <yminer@vmmware.com>
 2022-05-30 15:19:01 +08:00
Mac Chaffee
ecc8c59101
Use exec in registryctl so signals are passed properly (#16642) 
Signed-off-by: Mac Chaffee <machaffe@renci.org>
 2022-05-25 14:11:31 +08:00
prahaladdarkin
27ec871185
System Artifact Manager database schema creation, model definitons, and tests (#16678) 
Closes:
https://github.com/goharbor/harbor/issues/16540
https://github.com/goharbor/harbor/issues/16541
https://github.com/goharbor/harbor/issues/16542

Signed-off-by: prahaladdarkin <prahaladd@vmware.com>
 2022-05-09 17:32:57 +08:00
Shengwen Yu
3e8bf9faed fix: registry/redis.patch & registry/builder 
Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
 2022-04-29 08:51:49 +08:00
MinerYang
615a4cbb52
install only htpasswd binary from photon package httpd (#16771) 
Signed-off-by: yminer <yminer@vmmware.com>

cpio -u

Signed-off-by: yminer <yminer@vmmware.com>

Co-authored-by: yminer <yminer@vmmware.com>
 2022-04-28 13:20:03 +08:00
chlins
8c223135e7 feat: add cache layer for artifact (#16739) 
Implement cache layer for resource artifact and define common
cache manager and workflow. Also add cache related options to
configuration yaml.

Signed-off-by: chlins <chenyuzh@vmware.com>
 2022-04-25 18:01:45 +08:00
chlins
f7d10474d1 migrations: correct project metadata public value 
Signed-off-by: chlins <chenyuzh@vmware.com>
 2022-03-28 15:17:20 +08:00
stonezdj
9d480a84d2 Create index on task with job_id 
Missing index with job_id, when query task with job_id, it cause a full table scan, caused performance issue
  Fixes #15271

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2022-03-02 20:51:36 +08:00
MinerYang
74d2670616
bump up Go version to 1.17.7 (#16415) 
Signed-off-by: yminer <yminer@vmmware.com>

update CONTRIBUTING.md 2.4 golang version to 1.17.7

Signed-off-by: yminer <yminer@vmmware.com>

Co-authored-by: yminer <yminer@vmmware.com>
 2022-02-25 17:23:18 +08:00
MinerYang
6f1f2c0395
update 2.5.0 migration script harbor.yml.jinja (#16354) 
Signed-off-by: yminer <yminer@vmmware.com>

Co-authored-by: yminer <yminer@vmmware.com>
 2022-02-11 17:30:06 +08:00
MinerYang
6c97d3f0c7
rollback to delete external redis username (#16353) 
Signed-off-by: yminer <yminer@vmmware.com>

Co-authored-by: yminer <yminer@vmmware.com>
 2022-02-11 17:28:10 +08:00
MinerYang
77b6a130f8
Add Migration Script for Harbor 2.5 (#16346) 
* add Harbor2.5 migration script

Signed-off-by: yminer <yminer@vmmware.com>

* update 2_5_0 migration harbor.yml.jinja

Signed-off-by: yminer <yminer@vmmware.com>

Co-authored-by: yminer <yminer@vmmware.com>
 2022-02-11 12:18:06 +08:00
Wang Yan
6c3f9a8366
bump up distribution version to v2.8 (#16338) 
Signed-off-by: Wang Yan <wangyan@vmware.com>
 2022-02-10 11:46:58 +08:00
MinerYang
1a2dd256b5
Add external redis username for AUTH (#16336) 
* add external redis username for AUTH

* update harbor.yml.tmpl

* add external redis username for AUTH

update harbor.yml.tmpl

Co-authored-by: yminer <yminer@vmmware.com>
 2022-02-09 22:06:28 +08:00
Shengwen YU
0bf5998f96
upgrade Chartmuseum to v0.14.0 (#16334) 
Signed-off-by: Shengwen Yu <yshengwen@vmware.com>

Co-authored-by: Shengwen Yu <yshengwen@vmware.com>
 2022-02-09 12:13:05 +08:00
Shengwen Yu
aacebcc8c4 fix: vulnerability_record & report_vulnerability_record table "integer out of range" error 
Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
 2022-02-09 10:19:46 +08:00
Yang Jiao
709a154724 Specify postgresql version to 13 
Signed-off-by: Yang Jiao <jiaoya@vmware.com>
 2022-01-27 06:46:15 +00:00
He Weiwei
8554b7d7c5
refactor: replace lib/pq with jackc/pgx (#16267) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2022-01-27 11:09:37 +08:00
He Weiwei
89fe7b0819
Move removing old version scan reports of trivy to 2.4.2 (#16261) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2022-01-24 14:05:24 +08:00
Shengwen Yu
46f97ecf6c feat: bump TRIVYVERSION to v0.22.0 and bump TRIVYADAPTERVERSION to v0.25.0 
Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
 2022-01-21 09:14:07 +08:00
Morlay
9ad68aa526
Updates goharbor to var IMAGENAMESPACE in Makefiles to make injectable (#15218) 
Signed-off-by: Morlay <morlay.null@gmail.com>
 2022-01-20 12:14:29 +08:00
MinerYang
6abc505060
Merge pull request #16239 from MinerYang/bump_up/go_v1.17.6 
bump up go version to v1.17.6
 2022-01-18 10:32:09 +08:00
yminer
9acf3d21ce bump up go version to v1.17.6 
Signed-off-by: yminer <yminer@vmmware.com>
 2022-01-17 17:45:22 +00:00
孙世军
8f77567589
Upgrade UI dependencies (#16233) 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2022-01-17 14:20:20 +08:00
He Weiwei
097efb201b
fix: Clean old version scan reports of trivy (#16235) 
From harbor 2.4, the trivy does not support scan report v1.0, we need to remove the old version scan reports from the DB.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2022-01-17 13:59:28 +08:00
Yang Jiao
b4de95941b Specify postgresql version to 13 
Signed-off-by: Yang Jiao <jiaoya@vmware.com>
 2022-01-12 08:55:42 +00:00
stonezdj
17d8b7b813 Add upload purge config to registry/config.yml 
Enable the uploadpurging by default
  Fixes #15641

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2022-01-04 11:15:51 +08:00
Qian Deng
69a194b2b4 Fix: using traditional PKCS#1 format RSA key 
The openssl 3.0.0 using newer `PKCS#8` format.
But it's not compatitable with harbor core
So using tradictional format instead

Signed-off-by: Qian Deng <dengq@vmware.com>
 2021-12-10 11:34:12 +08:00
Wang Yan
742e7ded00
add accessory dao service (#16045) 
Signed-off-by: wang yan <wangyan@vmware.com>
 2021-12-03 14:34:02 +08:00
Qian Deng
5fc4449450
Merge pull request #16029 from ninjadq/fix_chart_replication_issue 
Fix: chart replication dup files issue
 2021-12-03 13:47:42 +08:00
Qian Deng
593117a127 Fix: chart replication dup files issue 
this patch fixed #15522

Signed-off-by: Qian Deng <dengq@vmware.com>
 2021-11-17 11:24:43 +00:00
Wang Yan
a956758302
bump up go version to v1.17 (#15865) 
* bump up go version to v1.17

Signed-off-by: Wang Yan <wangyan@vmware.com>

* gofmt fail

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-10-25 17:28:29 +08:00
Rolf Ahrenberg
5f3972f86d
Add configurable timeout for Trivy scans (#15796) 
Signed-off-by: Rolf Ahrenberg <Rolf.Ahrenberg@saunalahti.fi>
 2021-10-22 14:36:12 +08:00
stonezdj(Daojun Zhang)
c814daa894
Update the jaeger settings in harbor.yaml.tmpl (#15841) 
agent default port to 6831

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2021-10-22 13:43:28 +08:00
Yurii Paneiko
be1e762b70
For some reason this script hangs on bzip2 presence checking (#15647) 
Steps to reproduce:

Clone repo from master branch
Run: make install COMPILETAG=compile_golangimage
Wait until script will check that bzip2 is installed

Signed-off-by: YuriiPaneiko <yurapaneyko@gmail.com>
 2021-10-14 20:11:14 +08:00
He Weiwei
b390112f5a
fix: convert severity from negligible to none before saving to db (#15791) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2021-10-14 16:02:38 +08:00
stonezdj(Daojun Zhang)
32023891eb
Merge pull request #15675 from stonezdj/21sep22_email_nullable 
Allow empty email attribute for ldap/oidc user
 2021-10-02 10:33:38 +08:00
stonezdj
06715af303 Allow empty email attribute for ldap/oidc user 
Define user.Email as sql.NullString to avoid unique constraint when email is empty in LDAP/OIDC
  Separate the common/models/User with the pkg/user/dao/User
  Fixes #10400
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2021-09-30 14:35:18 +08:00
Qian Deng
3c23926bdc Add validation for tracing 
* add  validation of tracing in validating process

Signed-off-by: Qian Deng <dengq@vmware.com>
 2021-09-27 13:10:55 +00:00
stonezdj(Daojun Zhang)
972fa0880b
Merge pull request #15505 from ninjadq/add_distributed_tracing 
Add distributed tracing
 2021-09-22 14:16:19 +08:00
Qian Deng
354a2bd80d Enhance the trace related code 
* Move request id to requestid middleware
* fix span pass to child ctx on orm
* fix typos
* remove unused code
* add operation name to Transaction

Signed-off-by: Qian Deng <dengq@vmware.com>
 2021-09-22 04:14:43 +00:00
Qian Deng
05bc946444
Merge pull request #14383 from XavierDuthil/use-exec-in-entrypoints 
Use exec in all components' entrypoints
 2021-09-22 10:49:21 +08:00
Qian Deng
bad913cf6d Refactor trace code 
* use lib trace helper function
* add gracefull shutdown
* Add commens for new added exposed function
* Add licence on top of new created files
* Update trace library
* Update configs
* Add attribute and namespance in config

Signed-off-by: Qian Deng <dengq@vmware.com>
 2021-09-18 10:58:52 +00:00
Qian Deng
6fec5b2873 Add trace to jobservice 
* Add trace init in main
* Add env template
* Add trace for router

Signed-off-by: Qian Deng <dengq@vmware.com>
 2021-09-18 10:58:52 +00:00
Qian Deng
14095fb10b Add trace to registryctl 
* Add trace init to main
* Add trace for http server
* Add trace for gc
* Add env template trace

Signed-off-by: Qian Deng <dengq@vmware.com>
 2021-09-18 10:58:52 +00:00
Qian Deng
a15983432c Add trace for core 
* Add trace related lib
* Add trace middleware for core
* add rid for middleware

Signed-off-by: Qian Deng <dengq@vmware.com>
 2021-09-18 10:58:52 +00:00
Qian Deng
b812a300be Add trace related configs 
* Update harbor config template
* Update python config parsing
* Update env template

Signed-off-by: Qian Deng <dengq@vmware.com>
 2021-09-18 10:58:52 +00:00
He Weiwei
ecc1a04c92
refactor: implement a lock free quota (#15399) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2021-09-17 11:26:37 +08:00
DQ
e4c1521efc Add migration script for harbor 2.4 
Signed-off-by: DQ <dengq@vmware.com>
 2021-09-16 09:31:39 +00:00
Ziming Zhang
98cef43ead limit replication bandwidth 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2021-09-14 11:23:28 +08:00
Wenkai Yin(尹文开)
3ce072016f
Merge pull request #15527 from sevendials/logrotate_fails_when_cwd_is_not_readable 
Logrotate fails when cwd is not accessible
 2021-09-13 16:08:23 +08:00
Wang Yan
dbcbc8bad0
bump up go to v1.16.7 (#15564) 
Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-09-09 17:58:00 +08:00
Wang Yan
93a078d225
deprecate dns search (#15557) 
For details, please refer to https://github.com/goharbor/harbor/issues/14146#issuecomment-793390718
and https://github.com/docker/for-linux/issues/1164.

If anyone encounter the issue mentioned by https://github.com/goharbor/harbor/issues/6031, add the dns_search: . to the releated container.

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-09-08 14:22:59 +08:00
Christopher Jenkins
0585b148c7 Logrotate fails when cwd is not accessible 
Logrotate is run with sudo as the syslog user by cron.hourly
The current working directory is `/root` which is inaccessible to the syslog
user so the logrotate command fails. Currently the following stderr is being
thrown away by the cron script:
```
error: cannot open current directory: Permission denied
```

Fixes #15468

Signed-off-by: Christopher Jenkins <christj@gmail.com>
 2021-09-01 15:28:30 -07:00
孙世军
67681b1d83
update node version for portal container (#15396) 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2021-08-10 09:14:46 +08:00
stonezdj(Daojun Zhang)
2fa530eefa
Merge pull request #15247 from stonezdj/21jun30_remove_auth_user 
Delete users under auth_mode other than db_auth
 2021-07-14 13:25:43 +08:00
stonezdj
c163bc8317 Delete users under auth_mode other than db_auth 
The following information should cleanup before delete user:
  Delete project member of this user.
  Delete oidc_user when auth_mode is oidc_auth.
  Fixes #8424
  It also removes the deleted user from project member and the deleted condition in the project member query for consistency
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2021-07-12 11:31:40 +08:00
Wang Yan
494d74d32d
bump up go version to 1.16 (#15286) 
Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-07-08 16:25:15 +08:00
Wang Yan
4017e995b7 roll back go for notary binary 
Fixes #14932

Harbor recompiles the notary v0.6.1 with go 1.15 from v2.2.0, which introduces an break change that leads to notary key not found after migration.

[Root cause]
Notary v0.6.1 consumed an old version dvsekhvalnov/jose2, which is not compatible with go 1.15.

[References]
https://github.com/dvsekhvalnov/jose2go/issues/26
https://github.com/golang/go/issues/41089

[Resolve]
To resolve this issue, we have to roll back go vesrion to v1.14 for notary v0.6.1 binary and keep it until upstream have a patch release to support go 1.15 or above.

[Break change]
If you pushed and signed image using Harbor v2.2.0 ~ v2.2.2 and created new repository key in notary, you will encouter the same issue after migrate to v2.2.3(or above) or v2.3.1(or above) because of the go version downgrade. We will have a FAQ to help you to resovle this particular scenario.

The influence path of the particular case:
Harbor v2.1.0(or lower) --> [v2.2.0 ~ v2.2.2] --> v2.2.3(or above)
Harbor v2.1.0(or lower) --> v2.3.0 --> v2.3.1(or above)

The non influence path of the paticular case:
Harbor v2.1.0(or lower) --> v2.2.3(or above)
Harbor v2.1.0(or lower) --> v2.3.1(or above)

[Fix in Version]
Harbor v2.2.3 or above
Harbor v2.3.1 or above

[Note]
If you're a heavy user of notary, avoid using v2.2.0, v2.2.1, v2.2.2 and v2.3.0, and use the fixed version for instead.

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-06-29 16:18:55 +08:00
danfengliu
c39345da96
Merge pull request #15148 from danfengliu/missing-db-base-login-in-build-base-workflow 
Fix issue of missing db base build process in build base workflow
 2021-06-16 16:39:56 +08:00
Will Sun
410c5bcd78
Undate Angular to the latest version (#15129) 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2021-06-16 16:39:10 +08:00
danfengliu
f367aad760 Fix issue of missing db base build process in build base workflow 
There is extra build step for db base image building since v2.3, so this
step should be added back.

Signed-off-by: danfengliu <danfengl@vmware.com>
 2021-06-16 15:22:32 +08:00
He Weiwei
72f1afe2cc
perf: configurations for perf improvement (#15142) 
Closes #15041

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2021-06-15 18:39:15 +08:00
danfengliu
254c4a3497
Merge pull request #15046 from danfengliu/refine-base-build-process-in-makefile 
Refind build base process in Makefile
 2021-06-09 11:21:48 +08:00
He Weiwei
28921e2997 fix: listen to 5443 when clean unexpected status of db 
Closes #15081

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2021-06-08 06:41:23 +00:00
DQ
68ac772726 Fix: Remove v6 format of harbor 
Signed-off-by: DQ <dengq@vmware.com>
 2021-06-07 16:26:53 +08:00
Qian Deng
0867a6bfd6
Merge pull request #15055 from ninjadq/health_check_url_2_dn_instead_of_ip 
Add IPv6 support
 2021-06-03 17:57:55 +08:00
DQ
1b6b47f860 Add IPv6 support 
* 127.0.0.1 to localhost
* listening net addr add ipv6 format

Signed-off-by: DQ <dengq@vmware.com>
 2021-06-03 09:04:49 +00:00
danfengliu
d66ea07c2f Refind build base process in Makefile 
Remove build base executable in Makefile by replacing it as an input parameter.
Add add more input parameters for controlling docker pull/push to make
build base process flexible for users.

Signed-off-by: danfengliu <danfengl@vmware.com>
 2021-06-03 14:48:34 +08:00
Wang Yan
eec9893918 set shm size of postgres 
Fixed #15034, as for postgres 13, the default shm size is 64MB, set to 1gb to avoid could not resize shared memory segment error.

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-06-03 11:05:33 +08:00
Wang Yan
ae06ac2fae
fix db issue on helm upgrade (#15028) 
fixes #15000

It needs to double confirm that old pg is stopped before migration

Signed-off-by: wang yan <wangyan@vmware.com>
 2021-06-01 15:59:41 +08:00
ChenYu Zhang
a038ba672f
perf: add 2.2.3 schema up sql to master (#15027) 
Signed-off-by: chlins <chlins.zhang@gmail.com>
 2021-06-01 14:12:25 +08:00
Wang Yan
66b8a8f8dd
add build arch parameter in Makefile (#14995) 
* add build arch parameter in Makefile

Add parameter BUILDARCH for make file. DB base builds pg96 for x86_64 only

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-06-01 10:38:05 +08:00
DQ
5d02acd043 Add upgrade script for harbor 2.3 
no new config item added. harbor.yml keep the same as last version

Signed-off-by: DQ <dengq@vmware.com>
 2021-05-28 20:30:12 +08:00
Wenkai Yin
c585f92034 Fix bug of migration sql for replication policy 
Fix bug of migration sql for replication policy

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2021-05-26 18:57:23 +08:00
Qian Deng
9ea8aade01
Upgrade prepare to consistent with photon 4 (#14698) 
* requires version to 3.9.1
* upgrade packages

Signed-off-by: DQ <dengq@vmware.com>
 2021-05-26 16:39:04 +08:00
Wang Yan
39bdd7b506
pg upgrade failure handling (#14934) 
To ensure the upgrade execution idempotence, it needs to clean the $PGDATANEW on pg_upgrade failure.
Otherwise, the upgrade will skip the upgrade process from the second time launch as the exist of $PGDATANEW.

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-05-21 13:53:39 +08:00
danfengliu
6c14e699b1
Merge pull request #14844 from danfengliu/fix-notary-trust-directory-issue-in-nightly 
Fix notary trust directory issue in nightly
 2021-05-20 21:56:27 +08:00
Wang Yan
73bd373a75
create index for audit (#14930) 
Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-05-20 19:50:13 +08:00
danfengliu
3ef4dc17fc Fix notary trust directory issue and add login for each base image in makefile 
1. Use root instead of ~ in notary parameter;
2. Fix tag immutability issue caused by GUI change;
3. Replace email domain name to harbor test;
4. Add login for each base image in makefile;
5. Add customize look test in nightly.

Signed-off-by: danfengliu <danfengl@vmware.com>
 2021-05-20 18:23:13 +08:00
Wang Yan
86185989cf
support pg upgrade (#14846) 
1, use the pg source and photon spec to build postgres 9.6
2, install 9.6 on the photon 4.0
3, then leverage pg_upgrade to handle the pg major version migration

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-05-20 16:25:50 +08:00
Wenkai Yin
dc059a9a8f Improve the performance of artifact related APIs 
Improve the performance of artifact related APIs by adding indexes and refactoring sql logic

Closes #13890 #14813 #14814

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2021-05-20 11:25:43 +08:00
Wang Yan
0fb520a33b bump up go to v1.15.12 
Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-05-14 15:36:23 +08:00
Qian Deng
17dd48e5a3
Merge pull request #14854 from ninjadq/upgrade_helm_chart 
Upgrade helm chart
 2021-05-12 19:33:03 +08:00
Wenkai Yin
c04f3a2aac Fix duplicate execution record issue 
When the core service cannot response the checkin request in time, duplicated execution records may be created, this commit introduces the revision column to make sure there is only one record for one schedule trigger

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2021-05-12 15:40:52 +08:00
DQ
04ba4a4033 Upgrade chartmuseum version 
from 1.12.0 to 1.13.1

Signed-off-by: DQ <dengq@vmware.com>
 2021-05-11 13:51:55 +00:00
He Weiwei
476732df89 fix: change art_size to bigint in migrations 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2021-05-11 04:01:02 +00:00
Wenkai Yin(尹文开)
9bbffa06db
Merge pull request #14662 from ywk253100/210413_dest_ns 
Refactor the replication policy destination namespace logic
 2021-04-29 11:18:43 +08:00
Wenkai Yin
710c80078b Refactor the replication policy destination namespace logic 
Support specifying what part of the repository will be replaced by the provided namespace

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2021-04-28 10:16:20 +08:00
Daniel Jiang
c701ce09fa
Merge pull request #14681 from bitsf/fix_typo_NOTARYURL 
Fixed typo in NOTARYURL variable name
 2021-04-21 17:38:01 +08:00
Wenkai Yin(尹文开)
0f6692f68f
Merge pull request #14682 from ninjadq/hostname_hardcode_to_localhost 
Fix: Use local host on db's healthcheck
 2021-04-19 21:02:31 +08:00
danfengliu
f0ebd17994 Add build base image step in build package git action workflow 
Build base image step should be in build package workflow, and local base images build by new step should be removed since images have been pushed to docker hub.

Signed-off-by: danfengliu <danfengl@vmware.com>
 2021-04-17 18:10:44 +08:00
Ziming Zhang
39f70287b4 Fixed typo in NOTARYURL variable name 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2021-04-17 15:55:02 +08:00
DQ
ffed6459c7 Fix: Use local host on db's healthcheck 
hostname -i will malfunction in some cases like the `nsswitch.conf` file does'nt exist

Signed-off-by: DQ <dengq@vmware.com>
 2021-04-16 18:37:24 +08:00
Pushkar Joglekar
3947c5faff Add --no-cache and --pull flag to image builds to ensure latest security fixes are pulled from base image 
Signed-off-by: Pushkar Joglekar <pjoglekar@vmware.com>
 2021-04-12 09:49:27 -07:00
Qian Deng
c5d12ce8ee
Merge pull request #14542 from ninjadq/add_task_info_in_exporter 
Add task info in exporter
 2021-04-07 18:17:26 +08:00
Alexis
06fa88cfb7 Fix typo 
Signed-off-by: Alexis <60alexis@gmail.com>
 2021-04-07 15:58:17 +08:00
Alexis
e33f7aa9dd Add redis port to 2.1.0 jinja template 
Signed-off-by: Alexis <60alexis@gmail.com>
 2021-04-07 15:58:17 +08:00
Alexis
7742aec4af Add port to 2.0.0 jinja file 
Signed-off-by: Alexis <60alexis@gmail.com>
 2021-04-07 15:58:17 +08:00
Alexis
d28845af51 Remove external_redis.port since not used since v1.10.0 
Signed-off-by: Alexis <60alexis@gmail.com>
 2021-04-07 15:58:17 +08:00
stonezdj(Daojun Zhang)
448f0b6e28
Merge pull request #14579 from stonezdj/21apr_add_docker_registry_proxy 
Support proxy cache for docker-registry type
 2021-04-07 10:59:24 +08:00
Steven Zou
e2148f9eea
Merge pull request #14514 from goharbor/dependabot/pip/make/photon/prepare/pyyaml-5.4 
Bump pyyaml from 4.2b1 to 5.4 in /make/photon/prepare
 2021-04-07 09:57:07 +08:00
Steven Zou
10711b7de1
Merge pull request #14482 from goharbor/dependabot/pip/make/photon/prepare/jinja2-2.11.3 
Bump jinja2 from 2.11.1 to 2.11.3 in /make/photon/prepare
 2021-04-07 09:56:23 +08:00
stonezdj
ccd9ee8c56 Support proxy cache for docker-registry type 
Add proxy cache for docker registry type
Fixes #14477, #14547
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2021-04-06 16:47:12 +08:00
Wang Yan
d03a29e531 bump up photon to 4.0 
Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-03-31 13:25:29 +08:00
DQ
7eebbeebdf Add jobservice task queue related task 
add jobservice metrics
add redis client

Signed-off-by: DQ <dengq@vmware.com>
 2021-03-30 10:29:36 +00:00
DQ
fbe9cd88f8 Enabled Prometheus for Jobservice 
* Add prom server on jobservice
* Enabeld configs in templates
* Enabeld jobservice metrics in nginx

Signed-off-by: DQ <dengq@vmware.com>
 2021-03-30 08:52:59 +00:00
dependabot[bot]
f20f4215c3
Bump pyyaml from 4.2b1 to 5.4 in /make/photon/prepare 
Bumps [pyyaml](https://github.com/yaml/pyyaml) from 4.2b1 to 5.4.
- [Release notes](https://github.com/yaml/pyyaml/releases)
- [Changelog](https://github.com/yaml/pyyaml/blob/master/CHANGES)
- [Commits](https://github.com/yaml/pyyaml/commits/5.4)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-03-25 22:46:56 +00:00
DQ
f5fcc7bd31 Add base image for exporter 
* Add base
* update Makefile

Signed-off-by: DQ <dengq@vmware.com>
 2021-03-25 16:35:57 +08:00
dependabot[bot]
0ec667c4d8
Bump jinja2 from 2.11.1 to 2.11.3 in /make/photon/prepare 
Bumps [jinja2](https://github.com/pallets/jinja) from 2.11.1 to 2.11.3.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/master/CHANGES.rst)
- [Commits](https://github.com/pallets/jinja/compare/2.11.1...2.11.3)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-03-20 00:43:21 +00:00
Wenkai Yin
8b1817be0f Fix the consume too much CPU issue 
1. Update execution status during the upgrade
2. Refine the execution sweeper

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2021-03-17 13:34:24 +08:00