Commit Graph

4883 Commits

Author SHA1 Message Date
Wenkai Yin
3d7fd070c7 Refeactor replication policy APIs
Refeactor replication policy APIs

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-03-17 10:57:31 +08:00
Wang Yan
ad47007438 fix catalog duplicate result
Add distinct to the sql to avoid duplicate records be returned

Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-03-16 12:37:33 +08:00
Daniel Jiang
5ab879a670
Merge pull request #14447 from ywk253100/210315_retention_exec
Add upgrade sql file introduced in 2.1.4
2021-03-16 11:14:45 +08:00
Wenkai Yin
43df3bf8a4 Add upgrade sql file introduced in 2.1.4
1. Add upgrade sql file introduced in 2.1.4
2. Minor improvement for task/execution to cover corner cases

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-03-15 16:48:51 +08:00
Will Sun
f03ca3b087
Modify usage of LDAP ping API on UI (#14440)
Signed-off-by: AllForNothing <sshijun@vmware.com>
2021-03-15 15:56:54 +08:00
Will Sun
fb3b1311d8
Support adding label to multiple artifacts (#14443)
Signed-off-by: AllForNothing <sshijun@vmware.com>
2021-03-15 15:56:34 +08:00
Will Sun
68d7c91596
Add server driven sorting (#14426)
Signed-off-by: AllForNothing <sshijun@vmware.com>
2021-03-15 10:07:31 +08:00
He Weiwei
85f9a49bc8
feat: add sorts support for the projects listing API (#14425)
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-03-12 15:04:08 +08:00
Daniel Jiang
0d4992a41e
API for system CVE allowlist to new model (#14412)
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-03-12 10:23:48 +08:00
Wenkai Yin(尹文开)
4ef93565f3
Merge pull request #14369 from ywk253100/210303_sort
Introduce "sort" in query to provide a general solution for sorting
2021-03-11 09:28:34 +08:00
Wenkai Yin
506d1ad465 Introduce "sort" in query to provide a general solution for sorting
Introduce "sort" in query to provide a general solution for sorting

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-03-11 08:25:49 +08:00
stonezdj
5a35b7a9c4 Move ldap API to new program model
Fix some issue with the LDAP connection test

Signed-off-by: stonezdj <stonezdj@gmail.com>
2021-03-10 16:26:45 +08:00
Wang Yan
85254ccc22
refactor immutable rule (#14371)
Migrate immutable realted APIs to v2 swagger

Signed-off-by: wang yan <wangyan@vmware.com>
2021-03-08 17:10:12 +08:00
Will Sun
f0dd9e5aeb
Fix label color issue (#14387)
Signed-off-by: AllForNothing <sshijun@vmware.com>
2021-03-08 16:10:19 +08:00
Daniel Jiang
4c47b1777e
Merge pull request #14370 from reasonerjt/oidc-ping-refact
Switch API to ping OIDC endpoint to new model
2021-03-05 11:49:30 +08:00
Wenkai Yin(尹文开)
4c2e698af8
Merge pull request #14341 from heww/refactor-scanner-apis
refactor: generate scanner APIs by go-swagger
2021-03-04 17:48:04 +08:00
Daniel Jiang
e96c1cbced Switch API to ping OIDC endpoint to new model
This commit updates the API POST /api/v2.0/system/oidc/ping to new
programming model, in which the code will be generated by go-swagger.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-03-04 15:44:11 +08:00
Will Sun
afd29280ab
Improve stop button for replication (#14346)
Signed-off-by: AllForNothing <sshijun@vmware.com>
2021-03-04 12:04:52 +08:00
Will Sun
5cae2d5d9a
Improve double star validator on UI (#14345)
Signed-off-by: AllForNothing <sshijun@vmware.com>
2021-03-04 11:57:43 +08:00
He Weiwei
ef0bdf6954 refactor: generate scanner APIs by go-swagger
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-03-03 04:23:36 +00:00
Daniel Jiang
ef72c76e0e Add "*" to the claim set in the token for /v2 apis
The "*" is used by notary server for permission checking:
84287fd8df/server/server.go (L200)
Hence, we need to add this into the JWT token such that actions like key
rotation can be executed.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-03-02 19:36:04 +08:00
Will Sun
3ecd60b84b
Add harbor datetime pipe (#14331)
Signed-off-by: AllForNothing <sshijun@vmware.com>
2021-03-01 15:45:21 +08:00
He Weiwei
9161a3fbdf
refactor: use ormer from the ctx for scanner ctl mgr and dao (#14313)
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-03-01 12:02:40 +08:00
Wang Yan
5c2d43ea38 add sorting for list blobs
Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-02-26 11:52:56 +08:00
Wenkai Yin(尹文开)
fba6dd799e
Merge pull request #14300 from heww/quota-apis
refactor: generate quota APIs by go-swagger
2021-02-26 10:49:52 +08:00
He Weiwei
4b033c266a refactor: generate quota APIs by go-swagger
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-02-25 08:19:55 +00:00
Will Sun
1955b57701
Remove forced ng-check for registries component (#14302)
Signed-off-by: AllForNothing <sshijun@vmware.com>
2021-02-25 15:35:30 +08:00
Daniel Jiang
a4a995327b
Merge pull request #14129 from piotrekfilip/Add-support-for-http-proxy-in-oidc-insecureTransport
Add support for http proxy in oidc insecureTransport
2021-02-24 18:51:30 +08:00
He Weiwei
7cfd372af2
fix: use clone query in loop of artifact.Iterator func (#14283)
Closes #14251

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-02-23 12:23:36 +08:00
Will Sun
617ebb4045
Fix redirection issue for registries (#14270)
Signed-off-by: sshijun <sshijun@vmware.com>
2021-02-22 10:59:42 +08:00
Josh Soref
dfe360040b Spelling
* addition
* attribute
* auditing
* availability
* available
* bandwidth
* browser
* business
* cadence
* chartmuseum
* client
* column
* content
* demonstrate
* described
* endpoints
* facilitate
* github
* harbor
* information
* instance
* manual
* meaningful
* operation
* overridden
* password
* possible
* project
* refactor
* replication
* requires
* running
* scanned
* settings
* signup
* those
* unsigned
* vulnerability

--
Also removes trailing space from a filename

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
2021-02-19 11:59:15 +08:00
Will Sun
74b6bfe731
Modify cron regex for UI (#14254)
Signed-off-by: sshijun <sshijun@vmware.com>
2021-02-18 15:09:10 +08:00
Will Sun
2d84b62fe2
Refactor UI routing and remove redundant code (#14166)
Signed-off-by: AllForNothing <sshijun@vmware.com>
2021-02-18 09:12:23 +08:00
Daniel Jiang
51c8375425
Merge pull request #14220 from wy65701436/fix-codeql-quota
fix quota dao sql injection
2021-02-10 10:34:54 +08:00
Wang Yan
496a178eb3 fix quota dao sql injection
Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-02-10 03:55:59 +08:00
Wenkai Yin
fe0216ce94 Fix sql issue for querying tasks
Fix sql issue for querying tasks

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-02-09 22:15:06 +08:00
Wang Yan
21d35f9702
update blob list query (#14195)
* update blob list query

Deprecate blob list parameters, and use the query for instead.

Signed-off-by: wang yan <wangyan@vmware.com>

* update per review comments

Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-02-09 17:08:26 +08:00
Daniel Jiang
7231679373
Use the Email address from DB instead of request (#14212)
This commit updates the controller for sending reset pwd Email,
to make it use the Email from DB query result.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-02-09 17:04:15 +08:00
He Weiwei
a0561da5ce
test: fix the TestRetryUntil (#14210)
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-02-09 13:25:22 +08:00
Daniel Jiang
0b2ef922c0
Escape the service and email passed via request (#14186)
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-02-08 14:34:52 +08:00
Steven Zou
8725fa3af5
Merge pull request #14198 from steven-zou/fix/issue_#14079
fix(jobservice):wrong depth of job logging
2021-02-08 11:38:46 +08:00
Steven Zou
d61e560890 fix(jobservice):wrong depth of job logging
- use separate std logger for job, not shared with jobservice std logger
- merge and remove useless functions

Signed-off-by: Steven Zou <szou@vmware.com>

fix #14079
2021-02-07 18:07:02 +08:00
piotrekfilip
db47cf7f46
Add support for http proxy in oidc insecureTransport
Signed-off-by: piotrekfilip <43957913+piotrekfilip@users.noreply.github.com>
2021-02-06 12:42:38 +01:00
Wenkai Yin
cc3e240d56 Recover the panic of the process of replication adapter and fix bug of gitlab adapter
Recover the panic of the process of replication adapter
Fix bug of gitlab adapter
Fixes #14153

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-02-05 20:21:10 +08:00
Ziming Zhang
ec83f49a1a fix(retention) migrate sql error
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2021-02-05 09:09:26 +00:00
He Weiwei
44ba7de738
fix: wrap report vuls record creating in transaction (#14176)
Make the creating of the ReportVulnerabilityRecord in transaction to
avoid parallel problem

Closes #14171

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-02-05 12:15:52 +08:00
He Weiwei
de97b900cf
fix: remove default execution sweeper count (#14168)
1. Remove the default execution sweeper count for execution vendor.
2. Set the execution sweeper count for gc, preheat, replication,
retention to 50.
3. Disable sweep for the executions of the scan job.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-02-05 12:15:15 +08:00
Wenkai Yin(尹文开)
99bc251a13
Merge pull request #14164 from heww/update-report-improve
fix: 404 if report not found when updating report
2021-02-04 16:30:18 +08:00
Wang Yan
885a268268
rename permission of robot account (#14150)
* rename permission of robot account

fixes #14060
Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-02-04 15:32:44 +08:00
He Weiwei
060a282e71 fix: 404 if report not found when updating report
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-02-04 07:10:58 +00:00
Wenkai Yin(尹文开)
c45adabf5d
Merge pull request #14155 from ywk253100/210203_sweeper
Ignore the not found error in some cases
2021-02-04 14:02:46 +08:00
Will Sun
e51b98b8dd
Fix UI issues with label 2.2.0 (#14151)
Signed-off-by: AllForNothing <sshijun@vmware.com>
2021-02-04 10:32:27 +08:00
Wenkai Yin
027c80ea58 Ignore the not found error in some cases
Ignore the not found error in some cases
Fixes #14154

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-02-03 20:26:50 +08:00
Wang Yan
3c9ec4c778 update adapter log level to debug
Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-02-03 10:58:05 +08:00
Steven Zou
4737a394ee
Merge pull request #13947 from shaobo322/fix_improper_spell
fix improper spelling
2021-02-01 16:50:02 +08:00
Wenkai Yin(尹文开)
120d88a0dd
Merge pull request #14116 from reasonerjt/reduce-oidc-warning
Reduce warning logs in OIDC middleware
2021-02-01 09:55:41 +08:00
Wang Yan
1b85c67f63
fix catalog performance issue (#14120)
Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-01-31 21:31:21 +08:00
Qian Deng
a211b0c9d7
Merge pull request #14126 from ninjadq/fix_unknown_metrics
Fix: unkonw metrics issue
2021-01-29 18:59:35 +08:00
DQ
15ad870262 Fix: unkonw metrics issue
Signed-off-by: DQ <dengq@vmware.com>
2021-01-29 18:07:06 +08:00
DQ
eb5e31a442 Chagne quota usage metric name
from `harbor_project_usage_byte` to `harbor_project_quota_usage_byte`

Signed-off-by: DQ <dengq@vmware.com>
2021-01-28 18:10:59 +08:00
Daniel Jiang
2dd499bacf Reduce warning logs in OIDC middleware
If the request does not have bearer token in the header, do not decode
the empty string.
Fixes #12261

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-01-28 18:08:28 +08:00
He Weiwei
3ba0d5a590 fix: make a new ctx when start scan all in async
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-01-28 09:45:17 +00:00
Qian Deng
ebe05919be
Merge pull request #14078 from ninjadq/fix_self_registration_metric
Fix metric label related things
2021-01-28 17:27:53 +08:00
Qian Deng
6f3774ac8b
Merge pull request #14084 from ninjadq/support_multiple_artifact_type_in_metric
Support multiple artifact type in metric
2021-01-28 17:27:16 +08:00
Daniel Jiang
ea76594469 Improve the way config store transforms a value to string
This commit provide a better way to transform the value to string when
they are loaded from the driver.
Fixes #14074
However the way the config driver loaded config values and configstore
stores it back and forth seems repetitive and should be optimized.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-01-27 08:41:49 +00:00
Will Sun
fbfc943e8f
Fix UI bugs found in round 2 testing (#14098)
Signed-off-by: AllForNothing <sshijun@vmware.com>
2021-01-27 15:38:17 +08:00
DQ
a7241c1eb6 Fix: Support multiple type for artifact metrics
artiact became a dict

Signed-off-by: DQ <dengq@vmware.com>
2021-01-26 09:05:57 +00:00
DQ
66fd4a9728 change label project id to project name
project_name is more accurate

Signed-off-by: DQ <dengq@vmware.com>
2021-01-26 16:15:04 +08:00
Qian Deng
9574f8c3c6
Merge pull request #14056 from ninjadq/reduce_the_number_of_metrics_in_core
Aggregate  metrics
2021-01-26 10:42:53 +08:00
DQ
23a02bd5a7 Fix sel registration in metric label
Original type is bool, should convert it to string

Signed-off-by: DQ <dengq@vmware.com>
2021-01-25 18:53:10 +08:00
DQ
28ae77e5c6 Aggregate metrics
1. Add operation id in ctx in baseapi before prepare
2. add operation id for registry proxy request
3. use url for other request

Signed-off-by: DQ <dengq@vmware.com>
2021-01-25 09:59:10 +08:00
He Weiwei
ff9b515630 feat: add health checker for trivy when it's enabled
Closes #14055

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-01-23 05:45:03 +00:00
Daniel Jiang
d3601e5a92
Merge pull request #14058 from reasonerjt/fix-oidc-helper-npe
Fix a potential nil pointer issue
2021-01-22 20:57:09 +08:00
He Weiwei
e0f7778027
fix: ignore tag in the scan webhook when it's empty (#14057)
Closes #13464

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-01-22 20:08:36 +08:00
Daniel Jiang
5ea43abc67 Fix a potential nil pointer issue
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-01-22 19:02:45 +08:00
Wang Yan
8e7a18dc80
fix robot v1 api update issue (#14050)
For the v1 api, it will not update the description of a robot account.

Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-01-22 18:50:16 +08:00
Sven Haardiek
b2fe254974
Username from /userinfo (#14038)
This patch enabled Harbor to receive the username from the /userinfo endpoint
instead of only from the ID Token.

Closes #14037

Signed-off-by: Sven Haardiek <sven@haardiek.de>
2021-01-22 18:48:53 +08:00
Qian Deng
045e1d9abe
Merge pull request #14040 from ninjadq/metric_improvement
Metric improvement
2021-01-22 17:13:57 +08:00
Will Sun
51a541f1e2
Fix UI issue for success rate calculation (#14052)
Signed-off-by: AllForNothing <sshijun@vmware.com>
2021-01-22 16:44:21 +08:00
He Weiwei
50b40445f4
feat: add BeforePrepare for operation of swagger API (#14048)
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-01-22 10:44:25 +08:00
Will Sun
19a72cf350
Improve add scanner and add robot page (#14042)
Signed-off-by: AllForNothing <sshijun@vmware.com>
2021-01-21 10:41:57 +08:00
Will Sun
9a897af67e
Improve vulnerability list page (#14031)
Signed-off-by: AllForNothing <sshijun@vmware.com>
2021-01-21 10:41:09 +08:00
Wenkai Yin(尹文开)
63831dfd08
Merge pull request #14027 from ywk253100/210116_status
Don't ignore the NotFoundErr when handling the status hook of tasks to avoid the status out of sync
2021-01-20 17:07:18 +08:00
Wenkai Yin
e55c7d05ff Don't ignore the NotFoundErr when handling the status hook of tasks to avoid the status out of sync
Don't ignore the NotFoundErr when handling the status hook of tasks to avoid the status out of sync
Fixes #14016

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-01-20 14:53:50 +08:00
He Weiwei
6a16d9a914
fix: correct Authorize of basic and berer authorizer (#14036)
Closes #13734

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-01-20 14:33:28 +08:00
DQ
c59b437970 Remove empty label in harbor info
some infomation can not get with guest user. So remove them

Signed-off-by: DQ <dengq@vmware.com>
2021-01-20 10:53:55 +08:00
sluetze
cc0e2bdd73
updated german translation for 2.2. (#13801)
Signed-off-by: sluetze <13255307+sluetze@users.noreply.github.com>
2021-01-19 17:12:36 +08:00
He Weiwei
59a0e321ed fix: always remove the robot after scan job finished
1. Register task status change function for the scan job triggered by
scan all.
2. Always to delete the robot account for the scan job after the job is
finished because the job does not retry again when it's failed.

Closes #14000

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-01-18 07:37:09 +00:00
Steven Zou
42559479e6
Merge pull request #14023 from ywk253100/210115_scheduled
Fix the legacy scheduled job issue for GC/scan all
2021-01-18 14:01:01 +08:00
Wenkai Yin(尹文开)
c3b986cbcd
Merge pull request #14012 from ywk253100/210113_replication
Query executions with both vendor type and ID when sweep the execution records
2021-01-18 11:15:31 +08:00
Wenkai Yin
7c072e17a6 Fix the legacy scheduled job issue for GC/scan all
Fix the legacy scheduled job issue for GC/scan all

Fixes #13968

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-01-15 22:02:36 +08:00
Wenkai Yin
38e0f102bf Query executions with both vendor type and ID when sweep the execution records
Query executions with both vendor type and ID when sweep the execution records

Fixes #13996

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-01-15 20:23:55 +08:00
Will Sun
b7c5fc0562
Modify i18n text for push permission (#14015)
Signed-off-by: AllForNothing <sshijun@vmware.com>
2021-01-15 15:26:48 +08:00
stonezdj(Daojun Zhang)
f31f861e90
Merge pull request #14014 from wy65701436/fixes-13985
fix robot list issue
2021-01-15 14:44:35 +08:00
Will Sun
cb040d23d1
Fix developer permission issue for robot account (#14005)
Signed-off-by: AllForNothing <sshijun@vmware.com>
2021-01-15 12:39:40 +08:00
Will Sun
ff3abf047a
Fix UI issues found in testing round 1 (#14002)
Signed-off-by: AllForNothing <sshijun@vmware.com>
2021-01-15 12:37:40 +08:00
Wang Yan
df8e7c45e3 fix robot list issue
fixes #13985
Add sorting for listing the robot account

Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-01-15 11:56:53 +08:00
He Weiwei
74d055b26b
fix: correct completed in the metrics of the scan all (#14003)
Correct the completed count in the metrics of the scan all to be the sum
of success, error and stopped count.

Closes #14001

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-01-15 10:26:34 +08:00
Wenkai Yin(尹文开)
17652c06a2
Merge pull request #14011 from reasonerjt/tokenreview-rm-header
Remove raw token from header in token review reuqest
2021-01-14 18:30:13 +08:00
Daniel Jiang
6ae94d3790 Remove raw token from header in token review reuqest
The server to handle token-review may have a limitation for the size of
the header.  When the token is huge the token review may fail.
This commit remove the necessary header to harden the flow.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-01-14 17:36:55 +08:00