Commit Graph

10804 Commits

Author SHA1 Message Date
stonezdj
fe830ac4d2 Change configurations in swagger.yaml make it more readable
Update the genyaml to generate the configurations type
Add toCfgMap in config handler

Signed-off-by: stonezdj <stonezdj@gmail.com>
2021-05-24 19:13:55 +08:00
Wenkai Yin(尹文开)
544fa806fa
Set null list when or list is nil in query (#14941)
Set null list when or list is nil in query

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-05-24 16:22:39 +08:00
stonezdj
d7d26c0966 Fall back to local registry when upstream registry is not working
Fixes #14822
When upstream registry not working, but status might stay healthy because the health check interval is 5 minutes, if a pull request comes before registry status turns to unhealthy, the proxy cache middleware might proxy the request to the upstream registry and get a 401 error and this 401 error might translate to a http 500 error to the client eventually.

To solve this issue, it fall back all error to local registry when proxying manifest except the NotFoundError from the local registry.

Signed-off-by: stonezdj <stonezdj@gmail.com>
2021-05-24 14:29:05 +08:00
fanjiankong
306e519dbe Fix tencentcr adapter.
1. Limit API qps for the adapter.
2. Allow set qps via env.
3. Fix Tencnet SDK pagenation.
4. Fix resource filter.

Signed-off-by: fanjiankong <fanjiankong@tencent.com>
2021-05-24 10:33:49 +08:00
He Weiwei
a1ac9b98df
fix: delete robots by project id using raw sql (#14951)
QuerySeter.Delete will call two queies in the orm of beego. First it
will query all primary keys of the model by the QuerySeter, and then the
orm will delete the models using the IN statement of these primary keys.
This will be failed in the postgres when the records more than 65535.
This commit changes to use the raw query to delete the robots of the
project.

Closes #14678

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-05-24 10:31:48 +08:00
yunkunrao
fa8a0d09d6 Fix dragonfly preheat error.
Signed-off-by: yunkunrao <yunkunrao@gmail.com>
2021-05-23 17:50:54 +08:00
Will Sun
bea27a7da1
Improve scan all page (#14940)
Signed-off-by: AllForNothing <sshijun@vmware.com>
2021-05-21 17:55:57 +08:00
Wang Yan
39bdd7b506
pg upgrade failure handling (#14934)
To ensure the upgrade execution idempotence, it needs to clean the $PGDATANEW on pg_upgrade failure.
Otherwise, the upgrade will skip the upgrade process from the second time launch as the exist of $PGDATANEW.

Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-05-21 13:53:39 +08:00
danfengliu
6c14e699b1
Merge pull request #14844 from danfengliu/fix-notary-trust-directory-issue-in-nightly
Fix notary trust directory issue in nightly
2021-05-20 21:56:27 +08:00
Wang Yan
73bd373a75
create index for audit (#14930)
Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-05-20 19:50:13 +08:00
Daniel Jiang
dee6064e60
Merge pull request #14926 from reasonerjt/rm-common-dao-oidc-user-onboard
Remove the onboard and update funcs for OIDC user from common/dao
2021-05-20 19:34:59 +08:00
danfengliu
3ef4dc17fc Fix notary trust directory issue and add login for each base image in makefile
1. Use root instead of ~ in notary parameter;
2. Fix tag immutability issue caused by GUI change;
3. Replace email domain name to harbor test;
4. Add login for each base image in makefile;
5. Add customize look test in nightly.

Signed-off-by: danfengliu <danfengl@vmware.com>
2021-05-20 18:23:13 +08:00
Wang Yan
86185989cf
support pg upgrade (#14846)
1, use the pg source and photon spec to build postgres 9.6
2, install 9.6 on the photon 4.0
3, then leverage pg_upgrade to handle the pg major version migration

Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-05-20 16:25:50 +08:00
Daniel Jiang
66766a8f69 Remove the onboard and update funcs for OIDC user from common/dao
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-05-20 16:23:07 +08:00
ChenYu Zhang
a50f782beb
fix: change sql to improve the performance of catalog api (#14820)
Signed-off-by: chlins <chlins.zhang@gmail.com>
2021-05-20 15:52:53 +08:00
Daniel Jiang
8600b06975
Merge pull request #14925 from ywk253100/210520_perf
Improve the performance of artifact related APIs
2021-05-20 14:10:45 +08:00
Daniel Jiang
5be0c6d35a
Merge pull request #14911 from reasonerjt/fix-14386
Use system configuration resource for permission checking
2021-05-20 14:05:06 +08:00
Wenkai Yin(尹文开)
9afeb90646
Merge pull request #14923 from reasonerjt/fix-14862
Fail with 401 for GET .../users/current/permissions
2021-05-20 14:02:32 +08:00
Wenkai Yin(尹文开)
0fe551274d
Merge pull request #14921 from ywk253100/210519_db
Clean up tech debt codes
2021-05-20 13:44:10 +08:00
Wenkai Yin(尹文开)
36d76d8704
Merge pull request #14912 from ywk253100/210518_bug
Truncate the string when deletinng the resources when the length exceeds the limit
2021-05-20 12:09:36 +08:00
Wenkai Yin
998e392bb4 Clean up tech debt codes
Clean up tech debt codes

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-05-20 11:47:46 +08:00
Wenkai Yin
dc059a9a8f Improve the performance of artifact related APIs
Improve the performance of artifact related APIs by adding indexes and refactoring sql logic

Closes #13890 #14813 #14814

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-05-20 11:25:43 +08:00
Wenkai Yin(尹文开)
08ed886936
Merge pull request #14914 from ywk253100/210518_registry
Update the field name of registry model
2021-05-19 17:20:12 +08:00
Daniel Jiang
93fe178dc3 Fail with 401 for GET .../users/current/permissions
fixes #14862

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-05-19 16:28:39 +08:00
Will Sun
1896df2cfb
Improve css style (#14909)
Signed-off-by: AllForNothing <sshijun@vmware.com>
2021-05-19 12:49:38 +08:00
Daniel Jiang
294ee52d7a
Merge pull request #14900 from reasonerjt/rm-common-dao-getuser-onboard
Remove GetUser and Onboard from common/dao
2021-05-19 11:21:00 +08:00
He Weiwei
c6bd7b2ec2
ci: lint the swagger file (#14916)
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-05-19 10:36:09 +08:00
Daniel Jiang
4492e47e89
Merge pull request #14910 from reasonerjt/fix-oidc-callback-nil-pointer
Check user in security context before getting the ID
2021-05-18 23:35:46 +08:00
Wenkai Yin
315f490d3c Update the field name of registry model
Update the field name of registry mode

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-05-18 17:26:20 +08:00
Wenkai Yin
5ebc8b4fdd Truncate the string when deletinng the resources when the length exceeds the limit
Truncate the string when deletinng the resources when the length exceeds the limit

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-05-18 17:17:41 +08:00
Daniel Jiang
c41d75fb31 Use system configuration resource for permission checking
This commit uses system configuration resource for permission check
against API to ping OIDC and update systen CVE allowlist.
Fixes #14386

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-05-18 16:06:58 +08:00
Daniel Jiang
6ebcc95582 Fix docker version in CI.yml
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-05-18 15:29:44 +08:00
Daniel Jiang
952644e23f Remove GetUser and Onboard from common/dao
Replaced by funcs in src/pkg/user and src/controller/user

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-05-18 15:28:41 +08:00
Daniel Jiang
ebe5733f27 Check user in security context before getting the ID
This commit fix a potential nil pointer issue when trying to read User
ID from security context

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-05-18 14:54:58 +08:00
He Weiwei
0c315d8aee
refactor: remove allowlist in GetSummary of scan controller (#14836)
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-05-18 14:01:59 +08:00
Vadim Bauer
1a3335edc5
Provide information about missing RBAC permissions in debug mode. (#14906)
Signed-off-by: Vadim Bauer <vb@container-registry.com>

Co-authored-by: Vadim Bauer <vb@container-registry.com>
2021-05-18 10:02:34 +08:00
Wenkai Yin(尹文开)
aaff28f6ff
Merge pull request #14843 from ninjadq/fix_webhook_parse_chartname
Fix webhook parse chartname panic
2021-05-18 09:30:22 +08:00
Will Sun
0a8ff4c1f9
Update project summary page (#14874)
Signed-off-by: AllForNothing <sshijun@vmware.com>
2021-05-17 10:39:33 +08:00
Wenkai Yin(尹文开)
e553cbe795
Merge pull request #14887 from wy65701436/golang-15
bump up go to v1.15.12
2021-05-17 10:32:19 +08:00
He Weiwei
73045c55ce
fix: disable x-omitempty for repo_count and chart_count (#14895)
Closes #13664

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-05-17 10:22:51 +08:00
Will Sun
ba68f16b8d
Support searching quota by project name (#14881)
Signed-off-by: AllForNothing <sshijun@vmware.com>
2021-05-17 09:46:24 +08:00
He Weiwei
7fb1bc538c
refactor: remove unused core/api/models pkg (#14882)
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-05-14 17:15:44 +08:00
Wang Yan
0fb520a33b bump up go to v1.15.12
Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-05-14 15:36:23 +08:00
He Weiwei
6f3607cebd
perf: cache the metadata of the scanner (#14879)
1. Cache the metadata of scanner 30s.
2. Change the scanner client request timeout to 5s.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-05-14 14:27:37 +08:00
Qian Deng
a19f6f8748
Add grafana example for metrics (#14504)
add json file for grafana

Signed-off-by: DQ <dengq@vmware.com>
2021-05-14 14:19:27 +08:00
Ziming
1665dbcbc8
fix(retention) add delete retention API (#14747)
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2021-05-14 14:15:42 +08:00
Daniel Jiang
35c9a98272
Merge pull request #14875 from heww/fix-scan-all
fix: using new ctx to scan artifact when scanAll
2021-05-14 12:56:10 +08:00
He Weiwei
eed91dab45
fix: requires authn for apis of project scanner (#14880)
Closes #14863

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-05-14 11:27:23 +08:00
Daniel Pacak
7e6235bd3b
chore(trivy): Bump up Trivy scanner adapter to v0.19.0 (#14797)
Trivy scanner adpater v0.19.0 comes with Trivy v0.17
which adds support for Java JAR/WAR/EAR archives and
Go binaries scanning.

The release notes are published on
https://github.com/aquasecurity/trivy/releases/tag/v0.17.0

Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
2021-05-14 10:30:17 +08:00
He Weiwei
f800d531ec fix: using new ctx to scan artifact when scanAll
DB transaction failure may break the loop query of the artifacts, the
result is that not all artifacts are scanned in one scan all job. Using
a new DB connection to call the Scan method of the controller to avoid
this problem.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-05-13 12:33:10 +00:00