Commit Graph

10077 Commits

Author SHA1 Message Date
stonezdj
670a94835b Cache manifest list for proxy cache
Fixes #13566: Quota of dockerhub is still used in v2.1.1 after the image is cached
Cache manifest list in redis cache.
Trade off between efficiency and data integrity, it might cause the proxy cache return the full content of a manifest list instead of the actual manifest list saved in the Harbor storage, which is a part of the manifest list. but this change doesn't break any /v2/ API, just caches full manifest list.

Signed-off-by: stonezdj <stonezdj@gmail.com>
2020-12-15 11:30:30 +08:00
He Weiwei
e92674a42a
feat: add cache library and enable it in config manager (#13525)
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-12-08 17:40:03 +08:00
疯魔慕薇
9f8a743da9 Tecent TCR Provider
1. Docker image registry.
2. Helm chart registry.

Signed-off-by: 疯魔慕薇 <kfanjian@gmail.com>
Signed-off-by: fanjiankong <fanjiankong@tencent.com>
2020-12-08 15:44:18 +08:00
stonezdj(Daojun Zhang)
ec2f251d63
Merge pull request #13685 from wy65701436/robot-name-conflict
fix robot name conflicate issue
2020-12-07 11:18:49 +08:00
Wenkai Yin(尹文开)
65b6ae08bb
Merge pull request #13618 from mmpei/official-feature-artifact-hub
Support replicate from artifact hub
2020-12-07 10:47:05 +08:00
Daniel Jiang
fef5317aef
Merge pull request #13382 from flaviodsr/fix_core_init
Fix deadlock on harbor-core initialization
2020-12-04 19:51:56 +08:00
danfengliu
56a35437b4
Change image source in e2e pytest (#13640)
Change source of most of test image samples from docker-hub to local building ones, so it will cost less docker-hub pull requests.
And some of cases like push cnab, they have to use docker-hub, but image samples in cnab test will cost 17 quotas, in this PR, we
replace those samples, now cnab case will cost 6 quotas.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-12-04 02:28:29 -08:00
danfengliu
ff50e2363d
Move proxy cache e2e test into a new CI job (#13690)
CI job timeout is 60 minutes, but proxy cache case timeout is 20 minutes, once
proxy cache case is timeout, it will reach CI job timeout, then cause CI job terminated
by timeout without running rest of test steps, expecially uploading useful harbor logs,
so move this test into a new clean job for saving time and debugging.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-12-04 02:27:59 -08:00
Will Sun
0840a10752
Merge pull request #13615 from AllForNothing/too-much-tag
Add limitation to artifact tags
2020-12-04 15:47:37 +08:00
Will Sun
cf9b73d9a0
Merge pull request #13662 from AllForNothing/robot
Add system robot account UI
2020-12-04 15:37:00 +08:00
AllForNothing
6e52fbdf71 Add system robot accout UI
Signed-off-by: AllForNothing <sshijun@vmware.com>
2020-12-04 14:19:36 +08:00
Wang Yan
e3a353d8ae fix robot name conflicate issue
add project name into project level robot account name

Signed-off-by: Wang Yan <wangyan@vmware.com>
2020-12-04 12:40:59 +08:00
Wang Yan
d2fa2e6b84
update robot secret (#13654)
* update robot secret

1, use SHA256 to generate and validate robot secret instread of symmetric encryption.
2, update the patch input object

Signed-off-by: Wang Yan <wangyan@vmware.com>

* update robot secret

1, use SHA256 to generate and validate robot secret instread of symmetric encryption.
2, update the patch input object

Signed-off-by: Wang Yan <wangyan@vmware.com>
2020-12-03 18:13:06 +08:00
Daniel Jiang
878ef7c205
Merge pull request #13675 from reasonerjt/fix-v2catalog-uri
Fix the pattern to match v2 catalog URI
2020-12-03 14:15:26 +08:00
Daniel Jiang
73c3f7c7ea Fix the pattern to match v2 catalog URI
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-12-03 12:53:04 +08:00
Wenkai Yin(尹文开)
ddb29f2243
Set timezone as UTC for database connection (#13661)
Set timezone as UTC for database connection

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-12-03 08:55:48 +08:00
Wenkai Yin(尹文开)
7e289e67ca
Merge pull request #13666 from reasonerjt/bump-up-goldap
Bump up go-ldap to v3.2.4
2020-12-02 18:15:00 +08:00
Daniel Jiang
34d776b062 Bump up go-ldap to v3.2.4
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-12-02 15:28:54 +08:00
Will Sun
dbec54573c
Merge pull request #13511 from sluetze/12807-i18n-de-de
[i18n] added support for German
2020-12-02 13:42:52 +08:00
danfengliu
792a3c8c27
Merge pull request #13625 from danfengliu/fix-scan-all-py-test-script
Fix scan all python test script issue of missing test step
2020-12-02 09:17:10 +08:00
Qian Deng
688dc3af70
Merge pull request #13652 from ninjadq/fix_metric_inc_with_huge_num
Fix instrumenting handler nest issue
2020-12-01 20:23:41 +08:00
Daniel Jiang
0edbc0db75
Merge pull request #13641 from reasonerjt/v2-auth-host
Consider the default port when comparing the hosts
2020-12-01 18:55:34 +08:00
Wang Yan
732e9a21cd
updates on robot accounts (#13623)
* updates on robot accounts

1, add patch method to refresh secret of a robot
2, fix robot account update issue
3, add editable attribute to handle the version 1 robot account
4, add duration for robot account
5, hide secret for get/list robot account

Signed-off-by: wang yan <wangyan@vmware.com>

* update code per review comments

1, change expirate creation func to AddDate().
2, remove the scanner duration specification, use the default value.

Signed-off-by: Wang Yan <wangyan@vmware.com>
2020-12-01 18:31:34 +08:00
DQ
72ff33f80b Fix instrumenting handler nest issue
remove nested handler for instrumenting

Signed-off-by: DQ <dengq@vmware.com>
2020-12-01 17:12:19 +08:00
Steven Zou
2b65f7eb4b
Merge pull request #13534 from bitsf/fix_redis_sentinel_chart
fix(redis) redis sentinel for chart cache
2020-12-01 16:56:24 +08:00
Daniel Jiang
413dad98a8 Consider the default port when comparing the hosts
This commit cover the cases when the port is set in one of the Host of
request or the core URL, to make sure the comparison works as expected
when the default port (80, 443) is added in only one of them.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-12-01 16:48:28 +08:00
Will Sun
fe2ecf42ea
Merge pull request #13643 from AllForNothing/quick-fixing
Fix null pointer exception for auth guard
2020-12-01 13:34:05 +08:00
AllForNothing
b0ec990ce6 Fix null pointer exception for auth guard
Signed-off-by: AllForNothing <sshijun@vmware.com>
2020-11-30 22:55:11 +08:00
KeisukeYamashita
f24d349ede Remove deprecated set-env
Signed-off-by: KeisukeYamashita <19yamashita15@gmail.com>
2020-11-30 16:51:55 +08:00
Wang Yan
8cbfa6f382
add with details option for list project (#13634)
default is true and doesn't break any existing api, and when to set it to false, the api only return the basic project infor
without meta, CVE settings and etc of the project.

Signed-off-by: wang yan <wangyan@vmware.com>
2020-11-30 16:13:07 +08:00
Qian Deng
b80b1a7abf
Merge pull request #13617 from ninjadq/remove_clair_in_harbor_code
Remove clair code in harbor
2020-11-30 15:28:02 +08:00
Daniel Jiang
db8ce49133
Rework systeminfo API. (#13606)
This commit rework the systeminfo API under new programming model.
Also fixes #9149

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-11-30 14:15:18 +08:00
DQ
907904f480 Add DB Migration code for clair cleanning
- Delete clair scanner if exist
- Delete report is it is scanned by clair
- Set Trivy to Default if it exist and not default scanner

Signed-off-by: DQ <dengq@vmware.com>
2020-11-29 16:19:02 +08:00
sluetze
33fe276114 added support for german
Signed-off-by: sluetze <13255307+sluetze@users.noreply.github.com>
2020-11-27 16:43:31 +01:00
danfengliu
b36a4cf3be Fix scan all python test script issue of missing test step
1. In scan all py-test, missing a important test step for checkin scan all status,
2. which indicate the ending of scan all job.
In swagger, ongoig field is missing in stats struct.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-11-27 20:27:37 +08:00
DQ
590212b485 Remove clair related code
- clair code in harbor core
- clair code in frontend
- clair code in robotcase

Signed-off-by: DQ <dengq@vmware.com>
2020-11-27 14:01:04 +08:00
stonezdj(Daojun Zhang)
dec12308a1
Merge pull request #13621 from stonezdj/201127_fail_to_add_ldap_group
Lowercase the LDAP DN in UnderBaseDN
2020-11-27 11:45:07 +08:00
stonezdj(Daojun Zhang)
5a34f4e8fa
Merge pull request #13548 from wy65701436/robot2-swagger-api-dev
add robot account 2 api handler
2020-11-27 11:32:24 +08:00
Ziming Zhang
e44d58ee41 fix(redis) redis sentinel for chart cache
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2020-11-27 11:08:38 +08:00
stonezdj
ca245d3545 Lowercase the LDAP DN in UnderBaseDN
Fixes #13362: Unable to add LDAP group with different letter case in DN

Signed-off-by: stonezdj <stonezdj@gmail.com>
2020-11-27 10:30:19 +08:00
peimingming
186109162b Support replicate from artifact hub
Signed-off-by: peimingming <peimingming@corp.netease.com>
2020-11-26 21:41:24 +08:00
Daniel Jiang
fd54a568d0
Merge pull request #13500 from thechristschn/fix-searchgroupbyname
Fix api ldap group search by name
2020-11-26 16:38:13 +08:00
AllForNothing
4390b2f259 Add limitation to artifact tags
Signed-off-by: AllForNothing <sshijun@vmware.com>
2020-11-26 14:39:33 +08:00
Wang Yan
02846194e0 parent 8e61a3ea31
author Wang Yan <wangyan@vmware.com> 1605849192 +0800
committer Wang Yan <wangyan@vmware.com> 1606361046 +0800

update code per review comments

Signed-off-by: wang yan <wangyan@vmware.com>
2020-11-26 14:10:12 +08:00
stonezdj(Daojun Zhang)
be4e6a5985
Merge pull request #13537 from stonezdj/201118_add_more_registry_type
Add more registry type to proxy cache
2020-11-26 11:16:16 +08:00
Wang Yan
8e61a3ea31 add robot account 2 api handler
Signed-off-by: Wang Yan <wangyan@vmware.com>
2020-11-26 11:10:05 +08:00
Ziming Zhang
d55f55aeb9 fix(chartmuseum) compatible s3 cache fail
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2020-11-25 17:00:16 +08:00
stonezdj(Daojun Zhang)
90c5505430
Merge pull request #13591 from heww/fix-issue-13387
fix(quota): ignore the fail when getting reference of quota
2020-11-25 15:09:22 +08:00
He Weiwei
eb38180483 fix(quota): ignore the fail when getting reference of quota
1. Clean the dirty data in quota/quota_usage.
2. Ignore the fail when getting the reference of quota.

Closes #13387

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-11-24 14:50:38 +00:00
Will Sun
039722c518
Merge pull request #13585 from AllForNothing/proxy
Add new supported registries for proxy cache on UI
2020-11-24 16:45:07 +08:00