Upstream/harbor
1
0
Fork 0
mirror of https://github.com/goharbor/harbor.git synced 2024-12-02 07:03:35 +01:00
Code Issues Projects Releases Wiki Activity
10,857 Commits 53 Branches 317 Tags 335 MiB
dc8477bd7a
Commit Graph

5148 Commits

Author SHA1 Message Date
Daniel Jiang
7231679373
Use the Email address from DB instead of request (#14212) 
This commit updates the controller for sending reset pwd Email,
to make it use the Email from DB query result.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2021-02-09 17:04:15 +08:00
He Weiwei
a0561da5ce
test: fix the TestRetryUntil (#14210) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2021-02-09 13:25:22 +08:00
Daniel Jiang
0b2ef922c0
Escape the service and email passed via request (#14186) 
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2021-02-08 14:34:52 +08:00
Steven Zou
8725fa3af5
Merge pull request #14198 from steven-zou/fix/issue_#14079 
fix(jobservice):wrong depth of job logging
 2021-02-08 11:38:46 +08:00
Steven Zou
d61e560890 fix(jobservice):wrong depth of job logging 
- use separate std logger for job, not shared with jobservice std logger
- merge and remove useless functions

Signed-off-by: Steven Zou <szou@vmware.com>

fix #14079
 2021-02-07 18:07:02 +08:00
piotrekfilip
db47cf7f46
Add support for http proxy in oidc insecureTransport 
Signed-off-by: piotrekfilip <43957913+piotrekfilip@users.noreply.github.com>
 2021-02-06 12:42:38 +01:00
Wenkai Yin
cc3e240d56 Recover the panic of the process of replication adapter and fix bug of gitlab adapter 
Recover the panic of the process of replication adapter
Fix bug of gitlab adapter
Fixes #14153

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2021-02-05 20:21:10 +08:00
Ziming Zhang
ec83f49a1a fix(retention) migrate sql error 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2021-02-05 09:09:26 +00:00
He Weiwei
44ba7de738
fix: wrap report vuls record creating in transaction (#14176) 
Make the creating of the ReportVulnerabilityRecord in transaction to
avoid parallel problem

Closes #14171

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2021-02-05 12:15:52 +08:00
He Weiwei
de97b900cf
fix: remove default execution sweeper count (#14168) 
1. Remove the default execution sweeper count for execution vendor.
2. Set the execution sweeper count for gc, preheat, replication,
retention to 50.
3. Disable sweep for the executions of the scan job.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2021-02-05 12:15:15 +08:00
Wenkai Yin(尹文开)
99bc251a13
Merge pull request #14164 from heww/update-report-improve 
fix: 404 if report not found when updating report
 2021-02-04 16:30:18 +08:00
Wang Yan
885a268268
rename permission of robot account (#14150) 
* rename permission of robot account

fixes #14060
Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-02-04 15:32:44 +08:00
He Weiwei
060a282e71 fix: 404 if report not found when updating report 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2021-02-04 07:10:58 +00:00
Wenkai Yin(尹文开)
c45adabf5d
Merge pull request #14155 from ywk253100/210203_sweeper 
Ignore the not found error in some cases
 2021-02-04 14:02:46 +08:00
Will Sun
e51b98b8dd
Fix UI issues with label 2.2.0 (#14151) 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2021-02-04 10:32:27 +08:00
Wenkai Yin
027c80ea58 Ignore the not found error in some cases 
Ignore the not found error in some cases
Fixes #14154

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2021-02-03 20:26:50 +08:00
Wang Yan
3c9ec4c778 update adapter log level to debug 
Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-02-03 10:58:05 +08:00
Steven Zou
4737a394ee
Merge pull request #13947 from shaobo322/fix_improper_spell 
fix improper spelling
 2021-02-01 16:50:02 +08:00
Wenkai Yin(尹文开)
120d88a0dd
Merge pull request #14116 from reasonerjt/reduce-oidc-warning 
Reduce warning logs in OIDC middleware
 2021-02-01 09:55:41 +08:00
Wang Yan
1b85c67f63
fix catalog performance issue (#14120) 
Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-01-31 21:31:21 +08:00
Qian Deng
a211b0c9d7
Merge pull request #14126 from ninjadq/fix_unknown_metrics 
Fix: unkonw metrics issue
 2021-01-29 18:59:35 +08:00
DQ
15ad870262 Fix: unkonw metrics issue 
Signed-off-by: DQ <dengq@vmware.com>
 2021-01-29 18:07:06 +08:00
DQ
eb5e31a442 Chagne quota usage metric name 
from `harbor_project_usage_byte` to `harbor_project_quota_usage_byte`

Signed-off-by: DQ <dengq@vmware.com>
 2021-01-28 18:10:59 +08:00
Daniel Jiang
2dd499bacf Reduce warning logs in OIDC middleware 
If the request does not have bearer token in the header, do not decode
the empty string.
Fixes #12261

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2021-01-28 18:08:28 +08:00
He Weiwei
3ba0d5a590 fix: make a new ctx when start scan all in async 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2021-01-28 09:45:17 +00:00
Qian Deng
ebe05919be
Merge pull request #14078 from ninjadq/fix_self_registration_metric 
Fix metric label related things
 2021-01-28 17:27:53 +08:00
Qian Deng
6f3774ac8b
Merge pull request #14084 from ninjadq/support_multiple_artifact_type_in_metric 
Support multiple artifact type in metric
 2021-01-28 17:27:16 +08:00
Daniel Jiang
ea76594469 Improve the way config store transforms a value to string 
This commit provide a better way to transform the value to string when
they are loaded from the driver.
Fixes #14074
However the way the config driver loaded config values and configstore
stores it back and forth seems repetitive and should be optimized.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2021-01-27 08:41:49 +00:00
Will Sun
fbfc943e8f
Fix UI bugs found in round 2 testing (#14098) 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2021-01-27 15:38:17 +08:00
Josh Soref
d45ce5cbf1 Return instead of crashing when New() fails 
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
 2021-01-26 20:41:18 -05:00
Josh Soref
5be895cb39 Check return from token.DefaultTokenOptions() 
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
 2021-01-26 20:39:55 -05:00
DQ
a7241c1eb6 Fix: Support multiple type for artifact metrics 
artiact became a dict

Signed-off-by: DQ <dengq@vmware.com>
 2021-01-26 09:05:57 +00:00
DQ
66fd4a9728 change label project id to project name 
project_name is more accurate

Signed-off-by: DQ <dengq@vmware.com>
 2021-01-26 16:15:04 +08:00
Qian Deng
9574f8c3c6
Merge pull request #14056 from ninjadq/reduce_the_number_of_metrics_in_core 
Aggregate  metrics
 2021-01-26 10:42:53 +08:00
DQ
23a02bd5a7 Fix sel registration in metric label 
Original type is bool, should convert it to string

Signed-off-by: DQ <dengq@vmware.com>
 2021-01-25 18:53:10 +08:00
DQ
28ae77e5c6 Aggregate metrics 
1. Add operation id in ctx in baseapi before prepare
2. add operation id for registry proxy request
3. use url for other request

Signed-off-by: DQ <dengq@vmware.com>
 2021-01-25 09:59:10 +08:00
He Weiwei
ff9b515630 feat: add health checker for trivy when it's enabled 
Closes #14055

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2021-01-23 05:45:03 +00:00
Daniel Jiang
d3601e5a92
Merge pull request #14058 from reasonerjt/fix-oidc-helper-npe 
Fix a potential nil pointer issue
 2021-01-22 20:57:09 +08:00
He Weiwei
e0f7778027
fix: ignore tag in the scan webhook when it's empty (#14057) 
Closes #13464

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2021-01-22 20:08:36 +08:00
Daniel Jiang
5ea43abc67 Fix a potential nil pointer issue 
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2021-01-22 19:02:45 +08:00
Wang Yan
8e7a18dc80
fix robot v1 api update issue (#14050) 
For the v1 api, it will not update the description of a robot account.

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-01-22 18:50:16 +08:00
Sven Haardiek
b2fe254974
Username from /userinfo (#14038) 
This patch enabled Harbor to receive the username from the /userinfo endpoint
instead of only from the ID Token.

Closes #14037

Signed-off-by: Sven Haardiek <sven@haardiek.de>
 2021-01-22 18:48:53 +08:00
Qian Deng
045e1d9abe
Merge pull request #14040 from ninjadq/metric_improvement 
Metric improvement
 2021-01-22 17:13:57 +08:00
Will Sun
51a541f1e2
Fix UI issue for success rate calculation (#14052) 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2021-01-22 16:44:21 +08:00
He Weiwei
50b40445f4
feat: add BeforePrepare for operation of swagger API (#14048) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2021-01-22 10:44:25 +08:00
Will Sun
19a72cf350
Improve add scanner and add robot page (#14042) 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2021-01-21 10:41:57 +08:00
Will Sun
9a897af67e
Improve vulnerability list page (#14031) 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2021-01-21 10:41:09 +08:00
Wenkai Yin(尹文开)
63831dfd08
Merge pull request #14027 from ywk253100/210116_status 
Don't ignore the NotFoundErr when handling the status hook of tasks to avoid the status out of sync
 2021-01-20 17:07:18 +08:00
Wenkai Yin
e55c7d05ff Don't ignore the NotFoundErr when handling the status hook of tasks to avoid the status out of sync 
Don't ignore the NotFoundErr when handling the status hook of tasks to avoid the status out of sync
Fixes #14016

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2021-01-20 14:53:50 +08:00
He Weiwei
6a16d9a914
fix: correct Authorize of basic and berer authorizer (#14036) 
Closes #13734

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2021-01-20 14:33:28 +08:00
DQ
c59b437970 Remove empty label in harbor info 
some infomation can not get with guest user. So remove them

Signed-off-by: DQ <dengq@vmware.com>
 2021-01-20 10:53:55 +08:00
sluetze
cc0e2bdd73
updated german translation for 2.2. (#13801) 
Signed-off-by: sluetze <13255307+sluetze@users.noreply.github.com>
 2021-01-19 17:12:36 +08:00
He Weiwei
59a0e321ed fix: always remove the robot after scan job finished 
1. Register task status change function for the scan job triggered by
scan all.
2. Always to delete the robot account for the scan job after the job is
finished because the job does not retry again when it's failed.

Closes #14000

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2021-01-18 07:37:09 +00:00
Steven Zou
42559479e6
Merge pull request #14023 from ywk253100/210115_scheduled 
Fix the legacy scheduled job issue for GC/scan all
 2021-01-18 14:01:01 +08:00
Wenkai Yin(尹文开)
c3b986cbcd
Merge pull request #14012 from ywk253100/210113_replication 
Query executions with both vendor type and ID when sweep the execution records
 2021-01-18 11:15:31 +08:00
Wenkai Yin
7c072e17a6 Fix the legacy scheduled job issue for GC/scan all 
Fix the legacy scheduled job issue for GC/scan all

Fixes #13968

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2021-01-15 22:02:36 +08:00
Wenkai Yin
38e0f102bf Query executions with both vendor type and ID when sweep the execution records 
Query executions with both vendor type and ID when sweep the execution records

Fixes #13996

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2021-01-15 20:23:55 +08:00
Will Sun
b7c5fc0562
Modify i18n text for push permission (#14015) 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2021-01-15 15:26:48 +08:00
stonezdj(Daojun Zhang)
f31f861e90
Merge pull request #14014 from wy65701436/fixes-13985 
fix robot list issue
 2021-01-15 14:44:35 +08:00
Will Sun
cb040d23d1
Fix developer permission issue for robot account (#14005) 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2021-01-15 12:39:40 +08:00
Will Sun
ff3abf047a
Fix UI issues found in testing round 1 (#14002) 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2021-01-15 12:37:40 +08:00
Wang Yan
df8e7c45e3 fix robot list issue 
fixes #13985
Add sorting for listing the robot account

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-01-15 11:56:53 +08:00
He Weiwei
74d055b26b
fix: correct completed in the metrics of the scan all (#14003) 
Correct the completed count in the metrics of the scan all to be the sum
of success, error and stopped count.

Closes #14001

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2021-01-15 10:26:34 +08:00
Wenkai Yin(尹文开)
17652c06a2
Merge pull request #14011 from reasonerjt/tokenreview-rm-header 
Remove raw token from header in token review reuqest
 2021-01-14 18:30:13 +08:00
Daniel Jiang
6ae94d3790 Remove raw token from header in token review reuqest 
The server to handle token-review may have a limitation for the size of
the header.  When the token is huge the token review may fail.
This commit remove the necessary header to harden the flow.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2021-01-14 17:36:55 +08:00
stonezdj(Daojun Zhang)
2b068c3f86
Merge pull request #13989 from wy65701436/fixes-robot-issues 
fix robot issues
 2021-01-14 15:54:46 +08:00
Ziming Zhang
a7a1149454 fix(retention) execution status for zero repo 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2021-01-14 06:37:47 +00:00
Wang Yan
77347c54cf fix robot issues 
fixes #13980
fixes #13981

1, add the robot prefix to the audit log
2, add duration maximum checking

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-01-13 18:06:43 +08:00
Wang Yan
dedb86384b
do not add prefix for the legacy robots (#13966) 
Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-01-12 16:43:54 +08:00
Will Sun
7abe2ad9a3
Fix tag-retention nightly fuilure (#13965) 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2021-01-12 16:42:37 +08:00
Wang Yan
2d4456c630
refractor project rbac (#13924) 
As the system rbac introduced, move the code of project rbac into project directory

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-01-11 11:27:26 +08:00
Wang Yan
64e61c0d11
fix robot cannot update expirsat (#13944) 
Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-01-11 11:26:05 +08:00
Ziming Zhang
be58c1d3ff fix(retention) swagger test case 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2021-01-11 03:21:12 +00:00
Daniel Jiang
efa63d905a
Update the reg-exp to match v2/catalog api (#13941) 
A more strict check is applied such that all requests to
/v2/_catalog/...  will be verified.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2021-01-09 13:17:51 +08:00
shaobo
fba0ff4d83 fix improper spelling 
Signed-off-by: shaobo <shaobo@caicloud.io>
 2021-01-08 16:25:51 +08:00
Wenkai Yin
7b7b5e4977 Check the stop error during the replication process 
Check the stop error during the replication process

Fixes #13912

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2021-01-08 16:14:05 +08:00
Ziming Zhang
39fb500318 feat(retention) refactor to use go swagger api 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2021-01-08 07:09:28 +00:00
Will Sun
f566748c77
Fix nightly case by add to scan button (#13937) 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2021-01-08 15:01:26 +08:00
Steven Zou
1f79ce1181
Merge pull request #13554 from shaobo322/reform_compare_method 
reform job status compare method
 2021-01-08 12:49:39 +08:00
He Weiwei
755c6490f9
feat: remove duplicate CVE in scan report and summary (#13918) 
1. Remove the duplicate CVE records in the report/summary for the image
index.
2. Add scanner field in the scan overview for the API.

Closes #13913

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2021-01-08 11:00:43 +08:00
Wenkai Yin(尹文开)
4580aeff3b
Merge pull request #13910 from mmpei/official-feature-artifact-hub-replicate 
Support artifact hub replication using new API
 2021-01-08 10:15:55 +08:00
He Weiwei
9402077695
feat: save summary for the scan all execution (#13931) 
Compute the summary info for the scan all and save it to the extra attrs
of the execution.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2021-01-08 10:10:31 +08:00
Will Sun
4ea881564e
Modify UI to match scanner upgrading (#13915) 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2021-01-07 17:16:52 +08:00
Wang Yan
0cf43d766c
enable system resource access (#13826) 
1, introduce & define the system resources.
2, replace the IsSysAdmin judge method.
3, give the robot the system access capability.

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-01-07 15:45:04 +08:00
Steven Zou
be98748ca7
Merge pull request #13896 from ywk253100/210105_replication 
Fix replication filter bug
 2021-01-07 13:45:46 +08:00
Wenkai Yin(尹文开)
3408f0b577
Merge pull request #13909 from ywk253100/210106_task 
Provide a mechanism to sweep the execution/task records in task manager
 2021-01-07 13:44:21 +08:00
Will Sun
bd585481c7
Fix tag retention ui loading issue (#13905) 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2021-01-07 10:20:12 +08:00
peimingming
28714f8b70 Support artifact hub replication using new API 
Signed-off-by: peimingming <peimingming@corp.netease.com>
 2021-01-06 20:46:04 +08:00
He Weiwei
ed31cf9417
feat: return scan report and summary by header (#13898) 
Add X-Accept-Vulnerabilities header to the list/get artifact and get
artifact vulnerability addition APIs, and these APIs will traverse the
mime types in this header and return the first report and summary found
from the mime type.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2021-01-06 17:54:36 +08:00
Wenkai Yin
fb871dbbe8 Provide a mechanism to sweep the execution/task records in task manager 
Provide a mechanism to sweep the execution/task records in task manager
Fixes #13888

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2021-01-06 16:41:46 +08:00
He Weiwei
e33ff932d1
test: fix the data race in the test of scan controller (#13899) 
Closes #13876

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2021-01-06 16:17:47 +08:00
stonezdj(Daojun Zhang)
b748852ee8
Merge pull request #13805 from stonezdj/201218_add_content_type_length 
Add content type and length in header
 2021-01-06 15:02:34 +08:00
Will Sun
a1a5ef5e20
support quay registry as proxy cache (#13902) 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2021-01-06 14:15:15 +08:00
stonezdj
1d50be31aa Refactor Manifest cache process 
Separate manifest, manifest list and image index.

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2021-01-06 14:01:02 +08:00
Daniel Jiang
06e993ff76
Remove scanner-pull from system admin's permission (#13901) 
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2021-01-06 12:35:58 +08:00
Wenkai Yin
d474750e9f Fix replication filter bug 
This commit fixes the bug of replication filter, see #13593 for more detail
Fixes #13593

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2021-01-06 09:36:31 +08:00
Will Sun
839c36c876
Fix bugs for robot account UI (#13894) 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2021-01-05 22:28:56 +08:00
Ziming Zhang
8faa76a1b6 feat(retention) refactor task manager 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2021-01-05 12:08:03 +08:00
shaobo
434758b71e reform job status compare method 
Signed-off-by: shaobo <shaobo@caicloud.io>
 2021-01-04 21:34:53 +08:00
Daniel Jiang
d0152cb446
Merge pull request #13872 from reasonerjt/token-scope 
Refine the token scope generation
 2021-01-04 11:16:59 +08:00
Wang Yan
0271efd3f7
enable visible when to list/create robot (#13840) 
1, enable the visible attribute when to create/list robots
2, rename package name from robot2 to robot

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-01-04 10:24:31 +08:00
Daniel Jiang
eb75123638 Refine the token scope generation 
This commit directly maps the actoin permission in security context to
the scope generated by the token service in harbor-core.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2021-01-03 23:12:04 +08:00
prahaladdarkin
a890b28e1e
Store vulnerability data from scanner into a relational format (#13616) 
feat: Store vulnerability report from scanner into a relational format

Convert vulnerability report JSON obtained  from scanner into a relational format describe in:https://github.com/goharbor/community/pull/145

Signed-off-by: prahaladdarkin <prahaladd@vmware.com>
 2020-12-25 08:47:46 +08:00
Wang Yan
47841a04b9
fix quota cannot be updated after gc (#13844) 
Signed-off-by: Wang Yan <wangyan@vmware.com>
 2020-12-24 20:10:57 +08:00
Will Sun
75da08303a
Improve scan all page 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-12-24 17:36:03 +08:00
Wenkai Yin
923a538570 Provide the function to update extra attributes in the task manager 
Provide the function to update extra attributes in the task manager

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-12-24 15:52:37 +08:00
He Weiwei
7b4c4b76e7
feat: add trigger to the metrics of the scan all job (#13838) 
Add the trigger to the metrics of the scan all job so that the customer
can know who trigger the latest scan all job.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-12-24 14:30:43 +08:00
Wenkai Yin(尹文开)
d32a243508
Use the same transaction to remove the schedule to avoid dead lock in database (#13820) 
Use the same transaction to remove the schedule to avoid dead lock in database
Fixes #13808

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-12-23 15:37:51 +08:00
Will Sun
28734d7ac4
Fix a save button bug on system setting page (#13830) 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-12-23 13:00:45 +08:00
Daniel Jiang
c660727877
Merge pull request #13800 from reasonerjt/authproxy-redirect 
Add handler to handle redirect via authproxy
 2020-12-23 03:00:18 +08:00
stonezdj(Daojun Zhang)
bc0b6b43ed
Merge pull request #13791 from reasonerjt/oidc-redirect-extra-parm 
Add extra parms when forming redirect URI for OIDC
 2020-12-22 21:45:53 +08:00
Daniel Jiang
7321e3547d Add handler to handle redirect via authproxy 
This commit add a handler to handle the request to
"/c/authproxy/redirect".  Harbor is configured to authenticate against
an authproxy, if a request with query string `?token=xxxx`
is sent to this URI, the handler will do tokenreview according to the
setting of authproxy and simulate a `login` workflow based on the result
of token review.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-12-22 18:59:17 +08:00
He Weiwei
3831e82b20
refactor: remove code of admin job (#13819) 
Remove code of admin job as it's not needed by scan all/gc now.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-12-22 11:48:16 +08:00
He Weiwei
8fa03e3739
refactor: remove code of scan all job (#13821) 
Remove code of scan all job as it's implemented by execution now.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-12-22 11:39:18 +08:00
stonezdj
aa3002e7a5 Add content type and length in header 
Fixes #13740
Update ManifestExist to return Descriptor instead of digest
For docker 20.10 or containerd, it HEAD the manifest before pull, then
it GET the manifest with digest, add logic to handle this scenario and
correlate the tag between the digest in proxy cache

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2020-12-21 20:21:28 +08:00
Wenkai Yin(尹文开)
53c8ad8228
Merge pull request #13813 from heww/scan-all-apis 
refactor(api): move scan all apis to go-swagger
 2020-12-21 16:40:09 +08:00
fanjiankong
1b93a9f4b0 Remove some TCR UT Case 
Signed-off-by: fanjiankong <fanjiankong@tencent.com>
 2020-12-21 16:24:46 +08:00
AllForNothing
b749ba4e54 Fix filter bug for replication tasks page 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-12-21 15:11:35 +08:00
He Weiwei
ce6ed3eeb7 refactor(api): move scan all apis to go-swagger 
Move scan all APIs from beego to go-swagger.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-12-21 02:17:02 +00:00
Wang Yan
9bc6f3cee4
fix robot account update issue (#13741) 
* fix robot account update issue

enable the update method to support both v1 & v2 robot update

Signed-off-by: Wang Yan <wangyan@vmware.com>

* resolve review comments

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2020-12-18 20:01:26 +08:00
Wang Yan
6bc1047013
migration admin job data (#13766) 
1, migrate gc and scan all schedule to schedule/task/exectuion
2, migrate gc history to task/execution

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2020-12-18 16:35:24 +08:00
Will Sun
4392a626f3
Merge pull request #13804 from AllForNothing/scan-all 
Fix robot account UI issues
 2020-12-18 15:48:26 +08:00
Wang Yan
e9797d7c16
fix gc log not found issue (#13790) 
It needs to use the execution ID to get task firstly and then use the required task id to query GC log

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2020-12-18 14:50:50 +08:00
AllForNothing
b20cc474b3 Fix robot account UI issues 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-12-18 14:11:08 +08:00
stonezdj
3334defd92 Revert "fix issue" 
This reverts commit e6f80259
This reverts commit 918fe125

Signed-off-by: stonezdj <stonezdj@gmail.com>

Revert "Add content type and length in header"

This reverts commit ca379111

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2020-12-18 12:35:39 +08:00
stonezdj(Daojun Zhang)
05f327cfbd
Merge pull request #13751 from stonezdj/201211_add_content_type_length 
Add content type and length in header
 2020-12-17 18:19:38 +08:00
Wang Yan
dfe678457f
fixes robot account list permission issue (#13792) 
fixes #13786, do not add the removed project into the permission list of a system level robot account.

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2020-12-17 17:50:14 +08:00
He Weiwei
792dcc4ac3
fix(scan): returns 400 when artifact not support by scanner (#13785) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-12-17 16:58:49 +08:00
stonezdj
e6f80259f7 fix issue 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2020-12-17 16:33:17 +08:00
stonezdj
918fe125b7 fix issue 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2020-12-17 16:17:56 +08:00
stonezdj
ca37911113 Add content type and length in header 
Fixes #13740
Update ManifestExist to return Descriptor instead of digest

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2020-12-17 15:42:49 +08:00
He Weiwei
18b850782e fix: fix errors detected by codeql 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-12-17 06:15:34 +00:00
Daniel Jiang
c1c55d0cee Add extra parms when forming redirect URI for OIDC 
Fixes #13092

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-12-16 19:41:13 +08:00
Daniel Jiang
3b04d2f8f5
Escape the values to contains operator in dao packages (#13774) 
fixes #13018

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-12-16 14:19:20 +08:00
Will Sun
488d802a2b
Merge pull request #13768 from AllForNothing/permission-list 
Add new permissions to robot account
 2020-12-16 13:30:24 +08:00
Wang Yan
01eb60c36b
add permission checking for gc operation (#13756) 
Signed-off-by: Wang Yan <wangyan@vmware.com>
 2020-12-15 23:27:12 +08:00
AllForNothing
055572df3d Add new permissions to robot account 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-12-15 18:34:37 +08:00
stonezdj(Daojun Zhang)
1eb0287ecb
Merge pull request #13709 from stonezdj/201209_dockerhub_limit2 
Cache manifest list for proxy cache
 2020-12-15 14:03:39 +08:00
Will Sun
6849cd03a9
Merge pull request #13719 from sluetze/13568-typofix 
fixed small typos in variable names
 2020-12-15 13:47:30 +08:00
stonezdj
670a94835b Cache manifest list for proxy cache 
Fixes #13566: Quota of dockerhub is still used in v2.1.1 after the image is cached
Cache manifest list in redis cache.
Trade off between efficiency and data integrity, it might cause the proxy cache return the full content of a manifest list instead of the actual manifest list saved in the Harbor storage, which is a part of the manifest list. but this change doesn't break any /v2/ API, just caches full manifest list.

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2020-12-15 11:30:30 +08:00
He Weiwei
10e4350360
test(cache): increase sleep time to fix ut (#13761) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-12-14 19:22:42 +08:00
Wenkai Yin(尹文开)
fd900889c1
Merge pull request #13729 from ywk253100/201210_fix 
Tiny fixes for task manager
 2020-12-14 18:41:46 +08:00
Wenkai Yin(尹文开)
c6bfa04661
Merge pull request #13759 from bitsf/fix_tencenttcr_getAdapterInfo 
fix_tencenttcr_getAdapterInfo
 2020-12-14 17:30:22 +08:00
Wenkai Yin
69808f033e Tiny fixes for task manager 
1. Add update time for execution
2. Add unique constraint for schedule to avoid dup records when updating policies
3. Format replication log
4. Keep the webhook handler for legacy replication jobs to avoid jobservice resending the status change request

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-12-14 17:26:32 +08:00
Will Sun
fff1bbe8d9
Merge pull request #13757 from AllForNothing/new-auth-mode 
Add admin groups parameter to http auth
 2020-12-14 17:02:24 +08:00
AllForNothing
11b706adeb Add admin groups parameter to http auth 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-12-14 16:02:34 +08:00
Ziming Zhang
d8b5746df1 fix_tencenttcr_getAdapterInfo 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-12-14 16:02:19 +08:00
He Weiwei
af24a073dc
feat(api): support project name in the path of apis (#13744) 
Support project name in the path of projects and robotsV1 APIs.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-12-14 15:48:52 +08:00
Will Sun
949379f7bc
Merge pull request #13755 from AllForNothing/new-registry 
Improve regitries UI to cover new registries
 2020-12-14 14:45:23 +08:00
AllForNothing
8de8122119 Improve regitries UI to cover new registries 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-12-14 14:07:36 +08:00
He Weiwei
08580f9fec
refactor(scan): refactor scan/scan all job to task manager (#13684) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-12-14 13:34:35 +08:00
Will Sun
90f9bea965
Merge pull request #13752 from AllForNothing/improve-robot 
Improve refresh secret page for robot account UI
 2020-12-14 11:13:10 +08:00
Wenkai Yin(尹文开)
6569016d35
Merge pull request #13139 from wy65701436/migrate-gc 
Migrate gc to task manager
 2020-12-14 10:43:44 +08:00
Wang Yan
39e1a4f2b4 add extra attributes in the schedule table 
Signed-off-by: Wang Yan <wangyan@vmware.com>
 2020-12-14 02:28:52 +08:00
Wenkai Yin(尹文开)
43104ab0b9
Merge pull request #13724 from reasonerjt/http-auth-admin-grp 
Support admin group in http authproxy
 2020-12-11 13:06:26 +08:00
Daniel Jiang
4ff1c99ea0
Merge pull request #13539 from ninjadq/add_exporter_for_harbor 
Add exporter for harbor
 2020-12-11 12:52:03 +08:00
Daniel Jiang
fc94da54c1
Merge pull request #13728 from heww/fix-ut-for-get-reserved-resources 
test(quota): increase sleep time to fix fail of test
 2020-12-11 12:49:01 +08:00
Wenkai Yin(尹文开)
a4f21b1d85
Merge pull request #13694 from kofj/ghcr-provider 
GHCR Provider
 2020-12-11 09:41:10 +08:00
He Weiwei
2e09e54b38 test(quota): increase sleep time to fix fail of test 
Increase the sleep time to fix the fail in TestGetReservedResources of
quota

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-12-10 11:23:47 +00:00
wang yan
1bb79d402d update code per review comments 
Signed-off-by: wang yan <wangyan@vmware.com>
 2020-12-10 16:08:52 +08:00
Daniel Jiang
60e3668d43 Support admin group in http authproxy 
This commit adds admin_groups into the configuration of http_auth
settings, it's a string in the form of "group1, group2".  If the token
review result shows the user is in one of the groups in the setting he
will have the administrator role in Harbor.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-12-10 15:57:15 +08:00
Wang Yan
dba5522d0b Migrate to task manager (#129) 
1, remove the gc to new programming model
2, move api define to harbor v2 swagger
3, leverage task & execution manager to manage gc job schedule, trigger and log.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-12-10 14:00:33 +08:00
sluetze
5584e55070 fixed small typos in variable names 
Signed-off-by: sluetze <13255307+sluetze@users.noreply.github.com>
 2020-12-09 15:24:22 +01:00
DQ
85bf88c74e Add test case exporter 
- cache ut
- cli ut
- project collector ut
- system collector ut

Signed-off-by: DQ <dengq@vmware.com>
 2020-12-09 21:47:29 +08:00
DQ
d95f22448c Add cache for exporter 
Add timed cache for exporter
default cache time is 30s, cleanup job run every 4 hours

Signed-off-by: DQ <dengq@vmware.com>
 2020-12-09 21:22:40 +08:00
DQ
ade69e20ef Fix typo 
Signed-off-by: DQ <dengq@vmware.com>
 2020-12-09 20:42:21 +08:00
DQ
e6b53593ee Add exporter to harbor 
- add main file
- add pkg file
- update go.mod
- update vendor

Signed-off-by: DQ <dengq@vmware.com>
 2020-12-09 20:42:14 +08:00
Steven Zou
c12664122c fix(api):update stop job api 
- update StopJob() of basic worker
- update UnSchedule() of basic scheduler
- update the policy store to get more data

fix #13599 , fix #13597

Signed-off-by: Steven Zou <szou@vmware.com>
 2020-12-09 20:42:01 +08:00
Steven Zou
4acb708938
Merge pull request #13705 from steven-zou/fix/js_api 
fix(api):update stop job api
 2020-12-09 13:41:58 +08:00
Will Sun
13900df810
Merge pull request #13710 from AllForNothing/ut-case 
Fix UI UT failure
 2020-12-09 12:13:34 +08:00
AllForNothing
2f2e85577d Fix UI UT failure 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-12-09 11:33:55 +08:00
fanjiankong
2898117411 GHCR Provider 
Signed-off-by: fanjiankong <fanjiankong@tencent.com>
 2020-12-09 09:54:04 +08:00
Steven Zou
85783f3c0c fix(api):update stop job api 
- update StopJob() of basic worker
- update UnSchedule() of basic scheduler
- update the policy store to get more data

fix #13599 , fix #13597

Signed-off-by: Steven Zou <szou@vmware.com>
 2020-12-08 23:15:05 +08:00
He Weiwei
e92674a42a
feat: add cache library and enable it in config manager (#13525) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-12-08 17:40:03 +08:00
疯魔慕薇
9f8a743da9 Tecent TCR Provider 
1. Docker image registry.
2. Helm chart registry.

Signed-off-by: 疯魔慕薇 <kfanjian@gmail.com>
Signed-off-by: fanjiankong <fanjiankong@tencent.com>
 2020-12-08 15:44:18 +08:00
AllForNothing
7cddb5845f Improve refresh secret page for robot account UI 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-12-08 13:42:32 +08:00
stonezdj(Daojun Zhang)
ec2f251d63
Merge pull request #13685 from wy65701436/robot-name-conflict 
fix robot name conflicate issue
 2020-12-07 11:18:49 +08:00
Wenkai Yin(尹文开)
65b6ae08bb
Merge pull request #13618 from mmpei/official-feature-artifact-hub 
Support replicate from artifact hub
 2020-12-07 10:47:05 +08:00
Daniel Jiang
fef5317aef
Merge pull request #13382 from flaviodsr/fix_core_init 
Fix deadlock on harbor-core initialization
 2020-12-04 19:51:56 +08:00
Will Sun
0840a10752
Merge pull request #13615 from AllForNothing/too-much-tag 
Add limitation to artifact tags
 2020-12-04 15:47:37 +08:00
AllForNothing
6e52fbdf71 Add system robot accout UI 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-12-04 14:19:36 +08:00
Wang Yan
e3a353d8ae fix robot name conflicate issue 
add project name into project level robot account name

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2020-12-04 12:40:59 +08:00
Wang Yan
d2fa2e6b84
update robot secret (#13654) 
* update robot secret

1, use SHA256 to generate and validate robot secret instread of symmetric encryption.
2, update the patch input object

Signed-off-by: Wang Yan <wangyan@vmware.com>

* update robot secret

1, use SHA256 to generate and validate robot secret instread of symmetric encryption.
2, update the patch input object

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2020-12-03 18:13:06 +08:00
Daniel Jiang
73c3f7c7ea Fix the pattern to match v2 catalog URI 
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-12-03 12:53:04 +08:00
Wenkai Yin(尹文开)
ddb29f2243
Set timezone as UTC for database connection (#13661) 
Set timezone as UTC for database connection

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-12-03 08:55:48 +08:00
Wenkai Yin(尹文开)
7e289e67ca
Merge pull request #13666 from reasonerjt/bump-up-goldap 
Bump up go-ldap to v3.2.4
 2020-12-02 18:15:00 +08:00
Daniel Jiang
34d776b062 Bump up go-ldap to v3.2.4 
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-12-02 15:28:54 +08:00
Will Sun
dbec54573c
Merge pull request #13511 from sluetze/12807-i18n-de-de 
[i18n] added support for German
 2020-12-02 13:42:52 +08:00
Qian Deng
688dc3af70
Merge pull request #13652 from ninjadq/fix_metric_inc_with_huge_num 
Fix instrumenting handler nest issue
 2020-12-01 20:23:41 +08:00
Daniel Jiang
0edbc0db75
Merge pull request #13641 from reasonerjt/v2-auth-host 
Consider the default port when comparing the hosts
 2020-12-01 18:55:34 +08:00
Wang Yan
732e9a21cd
updates on robot accounts (#13623) 
* updates on robot accounts

1, add patch method to refresh secret of a robot
2, fix robot account update issue
3, add editable attribute to handle the version 1 robot account
4, add duration for robot account
5, hide secret for get/list robot account

Signed-off-by: wang yan <wangyan@vmware.com>

* update code per review comments

1, change expirate creation func to AddDate().
2, remove the scanner duration specification, use the default value.

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2020-12-01 18:31:34 +08:00
DQ
72ff33f80b Fix instrumenting handler nest issue 
remove nested handler for instrumenting

Signed-off-by: DQ <dengq@vmware.com>
 2020-12-01 17:12:19 +08:00
Steven Zou
2b65f7eb4b
Merge pull request #13534 from bitsf/fix_redis_sentinel_chart 
fix(redis) redis sentinel for chart cache
 2020-12-01 16:56:24 +08:00
Daniel Jiang
413dad98a8 Consider the default port when comparing the hosts 
This commit cover the cases when the port is set in one of the Host of
request or the core URL, to make sure the comparison works as expected
when the default port (80, 443) is added in only one of them.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-12-01 16:48:28 +08:00
AllForNothing
b0ec990ce6 Fix null pointer exception for auth guard 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-11-30 22:55:11 +08:00
Wang Yan
8cbfa6f382
add with details option for list project (#13634) 
default is true and doesn't break any existing api, and when to set it to false, the api only return the basic project infor
without meta, CVE settings and etc of the project.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-11-30 16:13:07 +08:00
Qian Deng
b80b1a7abf
Merge pull request #13617 from ninjadq/remove_clair_in_harbor_code 
Remove clair code in harbor
 2020-11-30 15:28:02 +08:00
Daniel Jiang
db8ce49133
Rework systeminfo API. (#13606) 
This commit rework the systeminfo API under new programming model.
Also fixes #9149

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-11-30 14:15:18 +08:00
sluetze
33fe276114 added support for german 
Signed-off-by: sluetze <13255307+sluetze@users.noreply.github.com>
 2020-11-27 16:43:31 +01:00
DQ
590212b485 Remove clair related code 
- clair code in harbor core
- clair code in frontend
- clair code in robotcase

Signed-off-by: DQ <dengq@vmware.com>
 2020-11-27 14:01:04 +08:00
stonezdj(Daojun Zhang)
dec12308a1
Merge pull request #13621 from stonezdj/201127_fail_to_add_ldap_group 
Lowercase the LDAP DN in UnderBaseDN
 2020-11-27 11:45:07 +08:00
stonezdj(Daojun Zhang)
5a34f4e8fa
Merge pull request #13548 from wy65701436/robot2-swagger-api-dev 
add robot account 2 api handler
 2020-11-27 11:32:24 +08:00
Ziming Zhang
e44d58ee41 fix(redis) redis sentinel for chart cache 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-11-27 11:08:38 +08:00
stonezdj
ca245d3545 Lowercase the LDAP DN in UnderBaseDN 
Fixes #13362: Unable to add LDAP group with different letter case in DN

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2020-11-27 10:30:19 +08:00
peimingming
186109162b Support replicate from artifact hub 
Signed-off-by: peimingming <peimingming@corp.netease.com>
 2020-11-26 21:41:24 +08:00
Daniel Jiang
fd54a568d0
Merge pull request #13500 from thechristschn/fix-searchgroupbyname 
Fix api ldap group search by name
 2020-11-26 16:38:13 +08:00
AllForNothing
4390b2f259 Add limitation to artifact tags 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-11-26 14:39:33 +08:00
Wang Yan
02846194e0 parent 8e61a3ea31 
author Wang Yan <wangyan@vmware.com> 1605849192 +0800
committer Wang Yan <wangyan@vmware.com> 1606361046 +0800

update code per review comments

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-11-26 14:10:12 +08:00
Wang Yan
8e61a3ea31 add robot account 2 api handler 
Signed-off-by: Wang Yan <wangyan@vmware.com>
 2020-11-26 11:10:05 +08:00
stonezdj(Daojun Zhang)
90c5505430
Merge pull request #13591 from heww/fix-issue-13387 
fix(quota): ignore the fail when getting reference of quota
 2020-11-25 15:09:22 +08:00
He Weiwei
eb38180483 fix(quota): ignore the fail when getting reference of quota 
1. Clean the dirty data in quota/quota_usage.
2. Ignore the fail when getting the reference of quota.

Closes #13387

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-11-24 14:50:38 +00:00
Will Sun
039722c518
Merge pull request #13585 from AllForNothing/proxy 
Add new supported registries for proxy cache on UI
 2020-11-24 16:45:07 +08:00
Will Sun
fc4a6fdd7b
Merge pull request #13496 from AllForNothing/rep-cron 
Improve adding replication rule page
 2020-11-24 16:16:37 +08:00
AllForNothing
69b5722cef Add new supported registries for proxy cache on UI 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-11-24 15:56:18 +08:00
Wenkai Yin(尹文开)
fe8b628f0c
Merge pull request #13437 from ywk253100/200929_replication 
Refactor the replication execution
 2020-11-24 10:38:22 +08:00
Steven Zou
35a0fc7afd
Merge pull request #13330 from chlins/fix/scanner-apikey-authorization 
fix: fix scanner apikey type match error
 2020-11-23 17:06:35 +08:00
Wenkai Yin
294385c34d Refactor the replication execution 
1. Use the task manager to manage the underlying execution/task
2. Use the pkg/scheduler to schedule the periodical job
3. Apply the new program model
4. Migration the old data into the new data model

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-11-23 14:24:10 +08:00
Daniel Jiang
3536a5dfac
Merge pull request #13452 from ninjadq/fix_status_code_issue_on_metrics 
Fix status code in metrics issue
 2020-11-23 10:37:47 +08:00
Daniel Jiang
00c941a413
Merge pull request #13557 from heww/fix-issue-13556 
fix: remove auto re-readable body in BeforeRequest
 2020-11-23 01:08:52 +08:00
He Weiwei
f733f95300 fix: remove auto re-readable body in BeforeRequest 
Remove nop close body in the BeforeRequest helper function.
Middleware must make the request body re-readable itself when it wants
read the body in the middleware.

Closes #13556

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-11-20 01:41:12 +00:00
Wang Yan
04c4354df9
add robot account version 2 controller (#13472) 
the controller is for the enhanced robot account

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2020-11-19 15:39:45 +08:00
Wenkai Yin(尹文开)
def782b6f8
Don't escape the name when doing the exact matching for tag (#13533) 
Don't escape the name when doing the exact matching for tag
Fixes #13106

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-11-18 14:34:48 +08:00
DQ
a6edf51574 Fix status code in metrics issue 
Stauts code can't get from request. It shoud be recorded by a delegator

Signed-off-by: DQ <dengq@vmware.com>
 2020-11-17 19:53:06 +08:00
stonezdj(Daojun Zhang)
fb549b2d9e
Merge pull request #13444 from wy65701436/robot2-self-mgr 
add robot mgr
 2020-11-16 11:33:33 +08:00
Christian Baumann
4530e9feee Fix api ldap group search by name 
Signed-off-by: Christian Baumann <thechristschn@gmail.com>
 2020-11-12 23:01:02 +00:00
sshijun
3d526e220c Improve adding replication rule page 
Signed-off-by: sshijun <sshijun@vmware.com>
 2020-11-12 18:17:26 +08:00
He Weiwei
76f1afbe0d
refactor: remove core/promgr pkg (#13408) 
* refactor: remove core/promgr pkg

Remove `core/promgr` package and use `controller/project` instead of it.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-11-12 15:33:13 +08:00
Wang Yan
3550b5e5e9 add robot mgr 
the robot account manager to handle the CRUD

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-11-11 13:47:03 +08:00
stonezdj(Daojun Zhang)
ca8cb87790
Merge pull request #13439 from wy65701436/robot2-mgr 
add role permission manager for robot enhancement
 2020-11-11 10:35:10 +08:00
Will Sun
d32527dfd7
Merge pull request #13470 from AllForNothing/tooltp 
Fix tooltip bug for adding replication rule page
 2020-11-10 20:43:12 +08:00
Wang Yan
9723655378 update code per review comments 
1, rename table name to permission_policy
2, rename functions name

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2020-11-10 18:11:31 +08:00
Wang Yan
ec15e320bf add role permission manager for robot enhancement 
1, add two db tables of role permission and rbac policy
2, add manager of these two tables

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2020-11-10 16:49:29 +08:00
He Weiwei
ebc3443da9
Merge pull request #13474 from heww/fix-issue-11892 
fix: compute artifact size from db for schema1 manifest
 2020-11-10 16:20:39 +08:00
sshijun
bf99f6d5cd Fix tooltip bug for adding replication rule page 
Signed-off-by: sshijun <sshijun@vmware.com>
 2020-11-10 10:29:08 +08:00
Wenkai Yin
69ddb9c2a8 Downgrade the version of github.com/prometheus/client_golang to 1.7.1 to avoid the compile error in mac 
The version 1.8.0 of "github.com/prometheus/client_golang" bumps up lib "golang.org/x/sys" which referenced by "github.com/docker/docker@1.4.2". The changes of "golang.org/x/sys" causes the compile error in mac

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-11-10 08:49:03 +08:00
He Weiwei
9c8377909b fix: compute artifact size from db for schema1 manifest 
Closes #11892

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-11-09 12:32:07 +00:00
Will Sun
039733b200
Merge pull request #13435 from AllForNothing/robot-prefix 
[UI]add robot name prefix
 2020-11-05 10:36:53 +08:00
He Weiwei
955431257c
Merge pull request #13422 from heww/disable-db-acess-for-ping 
feat: skip middlewares require db for ping
 2020-11-05 08:19:49 +08:00
stonezdj(Daojun Zhang)
7c1ad11763
Merge pull request #13425 from wy65701436/robot-prefix 
add robot name prefix
 2020-11-04 11:48:42 +08:00
Steven Zou
af7000d277
Merge pull request #13417 from bitsf/fix_gitlab_replication_task_num 
fix(replication) gitlab replication task num not correct
 2020-11-04 11:33:57 +08:00
Will Sun
c42ffe0915
Merge pull request #13410 from AllForNothing/helm-hub 
Add deprecated mark to Helm Hub
 2020-11-04 11:03:45 +08:00
Wang Yan
62208dc3e4 fix code conflict 
Signed-off-by: Wang Yan <wangyan@vmware.com>
 2020-11-03 09:42:37 -08:00
AllForNothing
ca93eb2218 Add robot name prefix to system setting UI 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-11-03 16:34:36 +08:00
Will Sun
e0617be15d
Merge pull request #13419 from AllForNothing/improve-ui 
Improve UI according to the designer's commonts
 2020-11-03 16:11:33 +08:00
Wang Yan
5a22019e3d add robot name prefix 
The system admin can set the prefix in configuration UI.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-11-02 23:12:57 -08:00
AllForNothing
6452d38925 Add deprecated mark to Helm Hub 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-11-03 15:04:15 +08:00
DQ
ae80aa796e Upgrade Prometheus Client Version 
upgrade to v1.8.0

Signed-off-by: DQ <dengq@vmware.com>
 2020-11-03 14:33:11 +08:00
DQ
eb470501be Add metrics to Harbor Core 
1. Add configs in prepare
 2. Add models and config items in Core
 3. Encapdulate getting metric in commom package
 4. Add a middleware for global request to collect 3 metrics

Signed-off-by: DQ <dengq@vmware.com>
 2020-11-03 14:33:10 +08:00
Ziming Zhang
8c2c758677 fix(replication) gitlab replication task num not correct 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-11-03 10:55:46 +08:00
Wenkai Yin(尹文开)
99d818f4db
Merge pull request #13406 from mmpei/offical-replicate-chart-fix-20201031 
fix chart replication issue of uppercase letters
 2020-11-03 10:55:40 +08:00
He Weiwei
0c5aedb3d9 feat: skip middlewares require db for ping 
The ping endpoint will be blocked when DB conns reach the max open conns
of the sql.DB which will make ping request timeout,
so skip the middlewares which will require DB conn.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-11-03 01:58:59 +00:00
AllForNothing
58450c335f Improve UI according to the designer's commonts 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-11-02 17:17:35 +08:00
Wenkai Yin(尹文开)
dd3ffd4147
Merge pull request #13361 from bitsf/aws_adapter_namespace_check 
feature(replication) check namespace existing first for AWS adapter
 2020-11-02 16:56:50 +08:00
Will Sun
9a38dca1a6
Merge pull request #13401 from AllForNothing/label-selection 
Improve label selection for artifact list page
 2020-11-02 13:44:28 +08:00
peimingming
6394cc767e fix chart replication issue of uppercase letters 
Signed-off-by: peimingming <peimingming@corp.netease.com>
 2020-11-01 00:29:46 +08:00
疯魔慕薇
2bab4a10db
Merge pull request #13393 from chlins/fix/p2p-preheat-instance-and-policy-post-no-location 
fix(preheat): add location when create instance and policy
 2020-10-31 23:48:41 +08:00
AllForNothing
120e8dd3d1 Improve label selection for artifact list page 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-10-30 14:43:59 +08:00
Will Sun
a255f3e74d
Merge pull request #13376 from AllForNothing/p2p-css 
Fix a bug for editing p2p instance
 2020-10-29 19:57:51 +08:00
Will Sun
8fda6bbb83
Merge pull request #13386 from AllForNothing/align 
Fix css bug for adding p2p policy
 2020-10-29 19:57:14 +08:00
Greg
d1ee94bbc9
DTR replication support (#9512) 
Adding DTR replication support

Signed-off-by: Greg Sidelinger <gate@ilive4code.net>
 2020-10-29 18:54:44 +08:00
chlins
9887154a0a fix(preheat): add location when create instance and policy 
Signed-off-by: chlins <chlins.zhang@gmail.com>
 2020-10-29 18:16:59 +08:00
AllForNothing
1417be1ffc Fix css bug for adding p2p policy 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-10-29 09:25:35 +08:00
AllForNothing
cc1eb1de86 Fix a bug for editing p2p instance 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-10-29 09:14:14 +08:00
Will Sun
fc65f5bf69
Merge pull request #13375 from AllForNothing/rep-filter 
Fix filter bug for replication rule list
 2020-10-29 09:10:31 +08:00
Daniel Jiang
535728d11f
Merge pull request #13306 from heww/refactor-security-context 
refactor(security): use controller instead of promgr in security
 2020-10-29 02:39:59 +08:00
Flávio Ramalho
ef6414be3e
Fix deadlock on harbor-core initialization 
During the harbor core initialization if the database takes longer to
be ready there is a risk of deadlock when checking for the TCP connection
with the database.

The `TestTCPConn` function uses unbuffered channels to check when the
connection succeeds/timeouts. The timeout check is executed in parallel
with the connection check (this runs in a gorountine). The deadlock happens
when the goroutine execution takes longer than the function timeout
(hence setting `cancel <- 1`) and the DialTimeout call succeeds (hence
setting `success <- 1`). At this point both threads are waiting for the
channels values to be read.

This is reproducible mostly on slow systems where initializing the
database takes longer and finishes during the 5th time of the
`DialTimeout` call where it eventually exceeds the TestTCPConn timeout.

This fix sets the `success` and `cancel` channels as buffered
(non-blocking).

Signed-off-by: Flávio Ramalho <framalho@suse.com>
 2020-10-28 17:09:34 +01:00
AllForNothing
53dc8f0bcf Fix filter bug for replication rule list 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-10-28 12:29:33 +08:00
Will Sun
ec790d0afd
Merge pull request #13347 from AllForNothing/vul-list 
Improve vilnerability list UI
 2020-10-28 10:24:18 +08:00
Ziming Zhang
a7d906649c feature(replication) check namespace existing first for AWS adapter 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-10-26 16:40:13 +08:00
AllForNothing
5475738a28 Fix css and i18n bugs 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-10-26 16:09:13 +08:00
AllForNothing
64ddef2e1d Improve vilnerability list UI 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-10-23 15:03:33 +08:00
Daniel Jiang
9c1da3a405 Add more info in log message to help debug 
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-10-22 18:55:01 +08:00
Daniel Jiang
eadb65f988
Merge pull request #13312 from reasonerjt/oidc-admin-group 
Add admin group support to OIDC auth mode
 2020-10-22 18:30:10 +08:00
Daniel Jiang
649c9814e4 Address review comment by Yan 
Resolve review comment in PR #13312

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-10-22 16:53:37 +08:00
AllForNothing
ea494af06e Fix a css bug of Clarity page size selection 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-10-22 16:48:35 +08:00
chlins
21b56d241a fix: fix scanner apikey type match error 
Signed-off-by: chlins <chlins.zhang@gmail.com>
 2020-10-21 18:29:19 +08:00
danfengliu
8af343f27a
Merge pull request #13324 from ywk253100/201021_project_cherry_pick 
Return the location header when creating proxy cache project
 2020-10-21 17:07:15 +08:00
Wenkai Yin(尹文开)
84971fc77d
Merge pull request #13316 from stonezdj/201020_dup_group_filter 
Remove dup call to createGroupSearchFilter in searchGroup
 2020-10-21 14:53:45 +08:00
Wenkai Yin
3021558272 Return the location header when creating proxy cache project 
Fixes #13303. Return the location header when creating proxy cache project

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-10-21 14:35:48 +08:00
Will Sun
1e421748bc
Merge pull request #13302 from AllForNothing/rep-rule 
Add page size options to datagrid
 2020-10-21 11:14:44 +08:00
Daniel Jiang
7c57e984a8
Merge pull request #13313 from AllForNothing/oidc 
[UI]Add admin group support to OIDC auth mode
 2020-10-21 11:10:28 +08:00
stonezdj
ca7258617b Remove dup call to createGroupSearchFilter in searchGroup 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2020-10-20 19:07:33 +08:00
AllForNothing
e7c53dc28a Add oidc admin group to oidc setting ui 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-10-20 17:47:00 +08:00
Daniel Jiang
f4ff369ed0 Add admin group support to OIDC auth mode 
Add oidc_admin_group to configuration, and make sure a token with the
group name in group claim has the admin authority.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-10-20 15:38:58 +08:00
chlins
73981062a9 fix: fix quay adapter catalog api return error 
Signed-off-by: chlins <chlins.zhang@gmail.com>
 2020-10-20 14:24:25 +08:00
AllForNothing
7b32fdb9f9 Add page size options to datagrid 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-10-19 18:42:19 +08:00
He Weiwei
ea0fbbeace refactor(security): use controller instead of promgr in security 
Use `project.Controller` instead of `promgr.ProjectManager` in security
implementations because we will remove `promgr` package later.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-10-19 10:33:51 +00:00
Will Sun
69b6bfd50a
Merge pull request #13239 from AllForNothing/unlogged 
Allow un-logged users to visit public repos
 2020-10-19 11:22:43 +08:00
stonezdj(Daojun Zhang)
88005e5028
Merge pull request #13267 from stonezdj/201014_no_default_group_filter 
Refine ldap searchGroup function
 2020-10-19 09:58:04 +08:00
stonezdj(Daojun Zhang)
8156f74bf4
Merge pull request #13266 from stonezdj/201009_proxy_head_containerd 
Add middleware to proxy HEAD request for proxy cache project
 2020-10-16 20:26:01 +08:00
AllForNothing
59fe9830b5 Allow un-logged users to visit public repos 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-10-16 17:21:36 +08:00
Wenkai Yin(尹文开)
c491c4f513
Merge pull request #13257 from reasonerjt/jwt-refact 
Use pkg/token to generate JWT token
 2020-10-16 16:31:26 +08:00
stonezdj
20ef7d3219 Refine LDAP searchGroup function 
Search LDAP group with groupDN+filter, then match baseDN
Create a default filter when ldap group filter is empty
Fixes #13156

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2020-10-16 16:20:41 +08:00
stonezdj
0641b300f3 Add middleware to proxy HEAD request for proxy cache project 
Fixes: #13153, containerd need to head manifest before pull image
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2020-10-16 13:02:32 +08:00
Daniel Jiang
fb687aeef8 Use pkg/token to generate JWT token 
This commit refactors the approach to encode a token in handler of /service/token,
by reusing pkg/token to avoid inconsistency.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-10-15 16:16:44 +08:00
stonezdj(Daojun Zhang)
cdd0eee2d4
Merge pull request #13162 from stonezdj/200921_rating_limit 
Change the condition of LocalManifest
 2020-10-14 19:34:38 +08:00
stonezdj
ee4b16ccdb Change the condition of LocalManifest 
Compare the local digest and the remote digest when pull by tag
Use HEAD request (ManifestExist) instead of GET request (GetManifest) to avoid been throttled
For manifest list, it can avoid GET request because cached manifest list maybe different with the original manifest list
Make RemoteInterface public
Fixes #13112

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2020-10-14 15:15:48 +08:00
stonezdj
9264105934 Add pull time for proxy cache artifact in first pull 
Fixes #13152

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2020-10-14 13:35:42 +08:00
Daniel Jiang
c93858cdbc
Remove UT for helm hub adapter (#13255) 
The helm hub adapter will not work due to the shift to artifact hub and
the API has changed. more details see #13244
Remove the UT for helm hub adapter to unblock CI.
Later we need to rework the adapter to replicate from artifact hub.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-10-14 10:22:03 +08:00
Wenkai Yin(尹文开)
4d78fd4e4e
Merge pull request #13126 from Thoro/fix-10913-oidc-error-after-restart 
fix #10913: initialize oidc provider before calling Load
 2020-10-14 10:13:52 +08:00
Thomas Rosenstein
874b0b1c0c fix #10913: initialize provider before calling Load 
Signed-off-by: Thomas Rosenstein <thomas.rosenstein@creamfinance.com>
 2020-09-30 17:25:29 +02:00
stonezdj(Daojun Zhang)
5293c8ff4b
Merge pull request #13053 from reasonerjt/fix-ldap-admingroup 
Revert "Store User ID in session instead of the whole user model (#12…
 2020-09-30 20:18:30 +08:00
AllForNothing
e999a15b02 Handle 504 error from backend 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-09-28 18:09:05 +08:00
AllForNothing
8d9a18b539 Update UI package to latest version 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-09-25 09:46:05 +08:00
Will Sun
18f41bad88
Merge pull request #13085 from AllForNothing/replication 
Add disable/enable function to replication rules
 2020-09-24 17:35:51 +08:00
Wenkai Yin(尹文开)
8b9727f53f
Support store the cron type in the schedule (#13097) 
There is requirement that show the cron type(daily, weekly, etc.) on the UI, this commit adds the support for storing the cron type in the schedule model

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-09-24 16:48:56 +08:00
Wang Yan
1af4c5e7cb
bump up go-digest version (#12992) 
Signed-off-by: wang yan <wangyan@vmware.com>
 2020-09-24 16:47:29 +08:00
Wenkai Yin(尹文开)
a62f05893f
Merge pull request #13129 from ywk253100/200917_task_mgr 
Tiny improvement for the task manager
 2020-09-24 16:46:44 +08:00
Wenkai Yin
1a4106a996 Tiny improvement for the task manager 
Add a new method "StopAndWait" which stops the execution and wait until the execution stopped or get an error

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-09-23 15:24:26 +08:00
stonezdj(Daojun Zhang)
20d4e86e0d
Merge pull request #13109 from Thoro/fix-encrypt-error-message 
Add error log in case encryption on config save fails
 2020-09-23 14:03:20 +08:00
Wenkai Yin(尹文开)
59f9ef7e5c
Abstract more info into the extra attributes for images (#13014) 
1. Abstract the "config" property(which contains labels) of config layer into the extra attributes for images
2. Try to get the author information from the "maintainer" label

fixes 12066
fixes 12734

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-09-23 10:42:47 +08:00
Thomas Rosenstein
452a0c9c45 Add error log in case encryption on config save fails 
Signed-off-by: Thomas Rosenstein <thomas.rosenstein@creamfinance.com>
 2020-09-22 10:09:47 +02:00
AllForNothing
074f70158e Improve i18n for tag deletion 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-09-18 09:33:24 +08:00
AllForNothing
abdbd79ed4 Add disable/enable function to replication rules 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-09-17 10:29:31 +08:00
AllForNothing
2d1c7ac76b Disable endpont input fot docker-hub and helm-hub 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-09-16 14:56:31 +08:00
Wenkai Yin(尹文开)
5d22644136
Merge pull request #13040 from bitsf/replication_ecr_auth_role 
feature(replication) enable role based auth for AWS ECR
 2020-09-15 19:13:05 +08:00
Ziming Zhang
699518a22e feature(replication) enable role based auth for AWS ECR 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-09-15 12:46:28 +08:00
Daniel Jiang
df106cf243 Revert "Store User ID in session instead of the whole user model (#12984)" 
This reverts commit 6fc0c9d75a.

Because this erases the AdminRoleInAuth attribute in user model as it is
not stored in DB and it will break the admin group of LDAP.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-09-15 10:05:31 +08:00
AllForNothing
60c2350088 Improve notary cmd on UI 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-09-14 18:39:22 +08:00
Will Sun
49c80f954c
Merge pull request #13025 from AllForNothing/uppercase 
Change EXECUTE button to uppercase
 2020-09-11 11:01:03 +08:00
Daniel Jiang
8846011571
Merge pull request #13017 from chlins/fix/webhook-verify-cert 
fix(webhook): fix the verify cert logic of webhook endpoint
 2020-09-11 02:18:57 +08:00
chlins
685d9277c4 fix(webhook): fix the verify cert logic of webhook endpoint 
Signed-off-by: chlins <chlins.zhang@gmail.com>
 2020-09-10 21:01:25 +08:00
Daniel Jiang
354eaac195
Escape the query string in list user (#13013) 
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-09-10 20:17:53 +08:00
AllForNothing
01d4539a99 Change EXECUTE button to uppercase 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-09-10 11:09:13 +08:00
stonezdj(Daojun Zhang)
97b9cc2d5e
Merge pull request #12997 from stonezdj/200902_ldap_filter_fail 
Add ldap filter syntax validation when create search filter
 2020-09-09 16:21:01 +08:00
stonezdj
b9752f3112 Add ldap filter syntax validation when create search filter 
Correct ldap search filter is enclosed with '(' and ')'
Search ldap group with the ldap group base DN instead of group DN
Fixes #12613 LDAP Group Filter and Group Base DN have no affect

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2020-09-09 10:07:07 +08:00
Daniel Jiang
513c48d47c
Merge pull request #12936 from wy65701436/fix-swagger-dep 
remove the dependency on swagger models
 2020-09-08 18:14:42 +08:00
Will Sun
5586fe86bf
Merge pull request #12988 from AllForNothing/test-4 
Fix UI issues found on testing day
 2020-09-08 01:12:10 +08:00
He Weiwei
6d50988c8b fix(project): change to use user id to query projects of member 
We know the user id when query projects by member, so use the user id
as entity_id directly in project_member, no need to join harbor_user
table.

Closes #12968

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-09-07 06:35:26 +00:00
AllForNothing
3fa3e14bb6 Fix UI issues found on testing day 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-09-07 12:50:48 +08:00
Daniel Jiang
6fc0c9d75a
Store User ID in session instead of the whole user model (#12984) 
This commit makes a change so that the user id will be stored in sessoin
after user login instead of user model to avoid data inconsistency when
user model changes.

Fixes #12934

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-09-07 11:43:37 +08:00
stonezdj(Daojun Zhang)
4267570e99
Merge pull request #12960 from heww/ignore-enable-content-trust-for-proxy-cache 
feat(project): ignore enable_content_trust for proxy project
 2020-09-07 11:25:53 +08:00
Daniel Jiang
2f7c8c2abd Check the tag in isArtifactSigned func 
This commit ensures that when CLI is pulling a tag, the content trust middleware check the data in notary to ensure the particular tag is signed, not only the digest.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-09-05 17:00:55 +08:00
Chlins Zhang
0465ad733f
Merge pull request #12916 from chlins/fix/project-deletion-needs-delete-preheat-policy 
fix(p2p): delete all preheat policies before delete project
 2020-09-03 17:32:20 +08:00
Daniel Jiang
66c3fa9eec
Return basic auth challenge for /v2/_catalog (#12956) 
Fixes #12192

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-09-03 11:31:57 +08:00
He Weiwei
f4938ca966
fix(scanner): move RemoveImmutableScanners before EnsureScanners (#12957) 
RemoveImmutableScanners may delete the default scanner, so move it
before the EnsureScanners.

Closes #12938

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-09-03 11:11:16 +08:00
He Weiwei
41c839af88 feat(project): ignore enable_content_trust for proxy project 
Ignore enable_content_trust metadata for proxy cache project, see
https://github.com/goharbor/harbor/issues/12940 to get more info

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-09-02 15:24:45 +00:00
Wang Yan
262f22f5ef
fix gc log issue (#12943) 
1, Do not log redis url, just log the user input from UI.
2, Format the artifact trash items.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-09-02 17:11:05 +08:00
chlins
e01de8b201 fix(p2p): delete all preheat policies before delete project 
Signed-off-by: chlins <chlins.zhang@gmail.com>
 2020-09-02 16:39:51 +08:00
Wenkai Yin
7816ff5e85 retry several times when failed to copy blob during the replication 
Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-09-02 16:07:20 +08:00
wang yan
116d295462 remove the dependency on swagger models 
1, remove the pkg dependency on v2.0/server/models
2, remove the controller dependency on v2.0/server/models

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-09-01 17:43:44 +08:00
Steven Zou
d4108e3fac fix(preheat):handle fail case of preheat in job 
Signed-off-by: Steven Zou <szou@vmware.com>
 2020-08-28 17:55:36 +08:00
Wenkai Yin(尹文开)
560eba7bd1
Merge pull request #12894 from stonezdj/200827_dockerhub_offline 
Fallback to local repo on errors when proxy to remote repo
 2020-08-28 15:14:21 +08:00
Wenkai Yin(尹文开)
3abe8b8fab
Merge pull request #12456 from julienvey/fix-sql-typo 
Fix typo in sql log
 2020-08-28 10:04:27 +08:00
Wenkai Yin(尹文开)
bad704d38b
Merge pull request #12897 from reasonerjt/icon-refactor 
Fill in the icon of known artifacts in artifact controller
 2020-08-28 10:03:12 +08:00
Daniel Jiang
91e2779822 Fill in the icon of known artifacts in artifact controller 
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-08-28 01:33:26 +08:00
Ziming Zhang
bda66e9e1f Add aws China regions ECR support, both of cn-north-1 and cn-northwest-1 endpoints. 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-08-27 23:43:16 +08:00
stonezdj
6967d73476 Fallback to local repo on errors when proxy to remote repo 
When the remote repo is offline or network issue or credential issue, fall back to local repo.
 Fixes #12853

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2020-08-27 22:10:54 +08:00
Daniel Jiang
a651eb0949
Merge pull request #12883 from wy65701436/fixes-12254 
fix event log issue
 2020-08-27 16:52:24 +08:00
wang yan
b51aaac26e fix event log issue 
fixes #12554
Add string method for each event, and the detail can be logged in the core.log

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-08-26 18:27:21 +08:00
He Weiwei
815ab61bc6
Merge pull request #12881 from heww/operator-from-security-username 
refactor(event): change default operator to username of security
 2020-08-26 15:42:36 +08:00
Ziming Zhang
c3fde4e483 fix(replication) gcr deletion and tag deletion 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-08-26 15:18:05 +08:00
He Weiwei
793c5ca57b refactor(event): change default operator to username of security 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-08-26 05:52:02 +00:00
Steven Zou
87d18268e0
Merge pull request #12879 from kofj/fix_schedule_error_msg 
Fix schedule error msg.
 2020-08-26 11:04:10 +08:00
Steven Zou
79665ed997
Merge pull request #12875 from chlins/fix/disable-change-p2p-provider-vendor 
fix(p2p): disable change provider vendor type
 2020-08-26 10:59:50 +08:00
fanjiankong
bbad4db645 Fix schedule error msg. 
Signed-off-by: fanjiankong <fanjiankong@tencent.com>
 2020-08-25 23:41:15 +08:00
chlins
d8769a41cd fix(p2p): disable change provider vendor type 
Signed-off-by: chlins <chlins.zhang@gmail.com>
 2020-08-25 16:56:17 +08:00
AllForNothing
5ae55eff6c Fix UI issues found on round-2 testing day 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-08-25 13:10:45 +08:00
He Weiwei
2ab4fbefd6
Merge pull request #12862 from heww/fix-issue-12861 
fix(quota): change log to debug level when warning not found
 2020-08-24 16:37:58 +08:00
Wang Yan
ad47d2f444
fix upgrade issue (#12857) 
fixes #12849

1, gives a default value to blob status in the migration script, and use none to replace the empty string as
the StatusNone, that will more readable on debugging failure.

2, GC jobs marks all of blobs as StatusDelete in the mark phase, but if encounter any failure in the sweep phase,
GC job will quite and all of blobs are in StatusDelete. If user wants to execute the GC again, it will fail as the
StatusDelete cannot be marked as StatusDelete. So, add StatusDelete in the status map to make StatusDelete can be
marked as StatusDelete.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-08-24 16:08:15 +08:00
He Weiwei
356be80f56 fix(quota): change log to debug level when warning not found 
Closes #12861

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-08-24 07:55:46 +00:00
Daniel Jiang
c0602b5fb3
Merge pull request #12832 from ywk253100/200820_data 
Add id column to data_migration table
 2020-08-21 19:30:05 +08:00
Ted Guan
645dea36a6
Fix for duplicate webhook policy name (#12729) 
Signed-off-by: guanxiatao <guanxiatao@corp.netease.com>
 2020-08-20 18:02:13 +08:00
Wenkai Yin
975ef193dd Add id column to data_migration table 
Add id column to data_migration table and add logic to make sure there is only one data version record

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-08-20 17:43:15 +08:00
AllForNothing
b472907362 Change storage of csrf token from cookie to localstorage 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-08-20 16:22:25 +08:00
Daniel Jiang
05afb94b9b Store csrf token in the header of response instead of cookie 
The current approach will prevent the effectiveness of `Cache-Control`
header and gorilla's library add `Vary:Cookie` header in all responses.

We will set the token in a header of response so the response can be
cached when needed.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-08-20 15:46:02 +08:00
Will Sun
446ae4c173
Merge pull request #12817 from AllForNothing/test-2 
Fix issues with label target 2.1 and RC
 2020-08-20 15:13:27 +08:00
Wenkai Yin(尹文开)
d4f18139ef
Merge pull request #12618 from ywk253100/200729_tk_mgr 
Refresh the status of execution for every status changing of task
 2020-08-20 14:38:09 +08:00
AllForNothing
56083cd5c9 Fix issues with label target 2.1 and RC 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-08-19 16:46:28 +08:00
wang yan
a7b9495ba0 Merge branch 'master' of https://github.com/goharbor/harbor into fix-resource-order 2020-08-19 15:30:48 +08:00
He Weiwei
d894d4c8bc
Merge pull request #12799 from wy65701436/fixes-12795 
fix gc job submit issue
 2020-08-19 15:27:02 +08:00
Will Sun
c35a39d432
Merge pull request #12814 from AllForNothing/ui-ut 
Improve UI UT for artifact list page
 2020-08-19 15:08:19 +08:00
AllForNothing
2b441a7ee5 Improve UI UT for artifact list page 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-08-19 12:45:39 +08:00
wang yan
1cc73bd92a Merge branch 'master' of https://github.com/goharbor/harbor into fix-resource-order 2020-08-19 12:21:45 +08:00
wang yan
648b80bc34 udpate resource list order 
1, order label by creation time.
2, order webhook policy by creation time.
3, order replication policy by creation time.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-08-19 11:20:31 +08:00
Steven Zou
c1b6be6ac9
Merge pull request #12801 from heww/fix-issue-9471 
refactor(scan): remove duplicate CVESet types
 2020-08-19 10:37:10 +08:00
Ted Guan
eb317fb8cb
tag retention webhook support (#12749) 
Signed-off-by: guanxiatao <guanxiatao@corp.netease.com>
 2020-08-19 00:07:45 +08:00
stonezdj(Daojun Zhang)
f7e1d186e0
Merge pull request #12792 from ywk253100/200814_proxy_cache 
Return 403 when trying to push artifacts into the proxy cache project…
 2020-08-18 16:22:51 +08:00
stonezdj(Daojun Zhang)
716625a769
Merge pull request #12793 from ywk253100/200816_disable_tag 
Disable the tag creation for the artifact under a proxy cache project
 2020-08-18 16:15:33 +08:00
He Weiwei
ef37bd1afb refactor(scan): remove duplicate CVESet types 
Closes #9471

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-08-18 06:33:17 +00:00
wang yan
44f7bf6739 fix gc job submit issue 
fixes #12795
Return 409 on triggering a GC job when there is a job ongoing.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-08-18 14:31:54 +08:00
Wang Yan
da52e677e5
remove robot accounts when to delete a project (#12789) 
The robots associate with the project should be removed after the project is deleted.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-08-18 10:38:45 +08:00
Daniel Jiang
77281ca68b
Token service handling invalid resource (#12790) 
This commit updates the way token service handles invalid resource, for
example a resource without projectname.
It will clear the requested access instead of returning 500 error.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-08-18 10:37:20 +08:00
Will Sun
89b597a811
Merge pull request #12783 from AllForNothing/p2p-4 
Fix UI bugs found on testing day for target 2.1
 2020-08-18 10:29:52 +08:00
Will Sun
9671ff1f10
Merge pull request #12701 from AllForNothing/artifact-icon 
get artifact icon from backend API
 2020-08-18 10:29:15 +08:00
fanjiankong
09ba463cc7 Fix. 
Signed-off-by: fanjiankong <fanjiankong@tencent.com>
 2020-08-17 21:40:37 +08:00
He Weiwei
16d1613b10
Merge pull request #12786 from heww/fix-artifact-scan-permission 
fix(scan): fix the permission checking for artifact scanning
 2020-08-17 18:30:18 +08:00
He Weiwei
f659523f50 fix(scan): fix the permission checking for artifact scanning 
Closes #12778

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-08-17 09:55:38 +00:00
Wenkai Yin
0fd230c2d6 Refresh the status of execution for every status changing of task 
Refresh the status of execution for every status changing of task to support filtering executions by status directly

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-08-17 17:38:55 +08:00
AllForNothing
b664c3c235 Fix UI bugs found on testing day for target 2.1 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-08-17 15:22:18 +08:00
He Weiwei
929fc8b3f7
Merge pull request #12787 from chlins/fix/preheat-get-log-api 
fix(preheat): fix the swagger definition of preheat get log api
 2020-08-17 14:54:12 +08:00
Wenkai Yin(尹文开)
2de10700d8
Merge pull request #12762 from monofone/10712-remove-forgot-password 
remove reset password dialog from sign-in component
 2020-08-17 14:24:36 +08:00
chlins
4f95c4d067 fix(preheat): fix the swagger definition of preheat get log api 
Signed-off-by: chlins <chlins.zhang@gmail.com>
 2020-08-17 12:41:02 +08:00
AllForNothing
e6541672d5 get artifact icon from backend API 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-08-17 10:21:47 +08:00
Wenkai Yin
e9ce631aa3 Disable the tag creation for the artifact under a proxy cache project 
Disable the tag creation for the artifact under a proxy cache project
Fixes #12713

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-08-16 22:02:30 +08:00
Wenkai Yin
a73742c0a7 Return 403 when trying to push artifacts into the proxy cache project to avoid the retrying in the docker client 
Return 403 when trying to push artifacts into the proxy cache project to avoid the retrying in the docker client
fixes #12731

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-08-16 21:22:22 +08:00
He Weiwei
f309896f2f refactor(api): generate project apis by go-swagger 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-08-15 16:10:57 +00:00
Wenkai Yin
b1ddb5e2cc Implement the icon API to get the icon of artifact 
Implement the icon API to get the icon of artifact

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-08-15 08:40:38 +08:00
stonezdj(Daojun Zhang)
205f4f6695
Merge pull request #12768 from stonezdj/200813_bug12741 
Use orm.Context instead request context in background go routing
 2020-08-14 21:15:15 +08:00
stonezdj
ff937e90d3 Use orm.Context instead request context in background go routing 
Fixes #12741,
Avoid use ctx info in event handler because it is background go routine

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2020-08-14 15:25:06 +08:00