Commit Graph

265 Commits

Author SHA1 Message Date
danfengliu
f367aad760 Fix issue of missing db base build process in build base workflow
There is extra build step for db base image building since v2.3, so this
step should be added back.

Signed-off-by: danfengliu <danfengl@vmware.com>
2021-06-16 15:22:32 +08:00
danfengliu
d66ea07c2f Refind build base process in Makefile
Remove build base executable in Makefile by replacing it as an input parameter.
Add add more input parameters for controlling docker pull/push to make
build base process flexible for users.

Signed-off-by: danfengliu <danfengl@vmware.com>
2021-06-03 14:48:34 +08:00
Wang Yan
66b8a8f8dd
add build arch parameter in Makefile (#14995)
* add build arch parameter in Makefile

Add parameter BUILDARCH for make file. DB base builds pg96 for x86_64 only

Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-06-01 10:38:05 +08:00
danfengliu
344e51d2c6 Fix build base workflow issues
Signed-off-by: danfengliu <danfengl@vmware.com>
2021-05-25 18:13:15 +08:00
danfengliu
6c14e699b1
Merge pull request #14844 from danfengliu/fix-notary-trust-directory-issue-in-nightly
Fix notary trust directory issue in nightly
2021-05-20 21:56:27 +08:00
danfengliu
3ef4dc17fc Fix notary trust directory issue and add login for each base image in makefile
1. Use root instead of ~ in notary parameter;
2. Fix tag immutability issue caused by GUI change;
3. Replace email domain name to harbor test;
4. Add login for each base image in makefile;
5. Add customize look test in nightly.

Signed-off-by: danfengliu <danfengl@vmware.com>
2021-05-20 18:23:13 +08:00
Wang Yan
86185989cf
support pg upgrade (#14846)
1, use the pg source and photon spec to build postgres 9.6
2, install 9.6 on the photon 4.0
3, then leverage pg_upgrade to handle the pg major version migration

Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-05-20 16:25:50 +08:00
He Weiwei
c6bd7b2ec2
ci: lint the swagger file (#14916)
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-05-19 10:36:09 +08:00
Wang Yan
0fb520a33b bump up go to v1.15.12
Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-05-14 15:36:23 +08:00
Daniel Pacak
7e6235bd3b
chore(trivy): Bump up Trivy scanner adapter to v0.19.0 (#14797)
Trivy scanner adpater v0.19.0 comes with Trivy v0.17
which adds support for Java JAR/WAR/EAR archives and
Go binaries scanning.

The release notes are published on
https://github.com/aquasecurity/trivy/releases/tag/v0.17.0

Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
2021-05-14 10:30:17 +08:00
DQ
04ba4a4033 Upgrade chartmuseum version
from 1.12.0 to 1.13.1

Signed-off-by: DQ <dengq@vmware.com>
2021-05-11 13:51:55 +00:00
danfengliu
81b26418c1
Fix build base related issues (#14733)
1. Add build base image step in build package git action workflow;
2. Add build base step to UT test in CI, base image used by UI test should be built before building harbor image in the same runtime;
3. In build package workflow, trigger build base image step in condition of changing both in
Dockerfile.base and VERSION;
4. Add tag for setup nightly test.

Signed-off-by: danfengliu <danfengl@vmware.com>
2021-04-23 11:55:30 +08:00
Daniel Jiang
c701ce09fa
Merge pull request #14681 from bitsf/fix_typo_NOTARYURL
Fixed typo in NOTARYURL variable name
2021-04-21 17:38:01 +08:00
He Weiwei
c58ccdfb8c
feat: bump up go-swagger to v0.25.0 (#14703)
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-04-21 11:07:10 +08:00
Ziming Zhang
39f70287b4 Fixed typo in NOTARYURL variable name
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2021-04-17 15:55:02 +08:00
He Weiwei
dc37c83e11
refactor: use singular as the tag for user APIs (#14654)
Use singular as the tag for user APIs to align with other APIs.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-04-15 10:52:48 +08:00
DQ
f5fcc7bd31 Add base image for exporter
* Add base
* update Makefile

Signed-off-by: DQ <dengq@vmware.com>
2021-03-25 16:35:57 +08:00
danfengliu
9e3f0de12c
Merge pull request #14124 from danfengliu/reschedule-docker-login-policy-in-build-base-image-in-master
Reschedule docker login policy in base image build process
2021-02-23 10:10:59 +08:00
danfengliu
7d05c8e513 Reschedule docker login policy in base image build process
Signed-off-by: danfengliu <danfengl@vmware.com>
2021-02-22 10:05:25 +08:00
Josh Soref
dfe360040b Spelling
* addition
* attribute
* auditing
* availability
* available
* bandwidth
* browser
* business
* cadence
* chartmuseum
* client
* column
* content
* demonstrate
* described
* endpoints
* facilitate
* github
* harbor
* information
* instance
* manual
* meaningful
* operation
* overridden
* password
* possible
* project
* refactor
* replication
* requires
* running
* scanned
* settings
* signup
* those
* unsigned
* vulnerability

--
Also removes trailing space from a filename

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
2021-02-19 11:59:15 +08:00
Daniel Pacak
202342cf0c
chore: Bump up Trivy scanner adapter from v0.17.0 to v0.18.0 (#14240)
Trivy adapter version v0.18.0 ships with Trivy v0.16.0.
It also changes the way we return links to upstream
vulnerability references. Instead of returning multiple
URLs Trivy returns the primary URL to Aqua Vulnerability
Database (e.g. https://avd.aquasec.com/nvd/cve-2020-10688/)
with up to date status and remediation guide.

Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
2021-02-18 14:40:07 +08:00
Qian Deng
a8e4b09b39
Add exporter to offline and online (#14022)
Signed-off-by: DQ <dengq@vmware.com>
2021-01-20 14:49:06 +08:00
Daniel Jiang
1b64b9fdc2
Bump up the go-migrate (#13914)
Bump it up to v4.11.0 to be consistent with harbor-core

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-01-11 19:08:17 +08:00
Daniel Pacak
bd46af691c
chore(trivy): Bump up Trivy scanner adapter to v0.17.0 (#13639)
The adapter implements Pluggable Scanners API v1.1
and ships with Trivy v0.14.0.

There's also a tiny change in the way Trivy settings
are displayed in the scanner metadata response, i.e.
instead of com.github.aquasecurity.trivy.debugMode
it prints env.SCANNER_TRIVY_DEBUG_MODE. It makes it
explicit which env is use to set this parameter.

Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
2020-12-25 15:34:00 +08:00
Wang Yan
7a8a8fa104
upgrade go version to v1.15.6 (#13836)
Signed-off-by: wang yan <wangyan@vmware.com>
2020-12-23 18:53:09 +08:00
He Weiwei
ce6ed3eeb7 refactor(api): move scan all apis to go-swagger
Move scan all APIs from beego to go-swagger.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-12-21 02:17:02 +00:00
He Weiwei
164acad24f
build: add cleanbaseimage target in Makefile (#13777)
Add cleanbaseimage target in Makefile, and append it to the dependencies
of the cleanall target.

Closes #13602

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-12-17 16:18:12 +08:00
DQ
dc0047c48c Add build script for exporter
- Add dockerfile
- update makefile

Signed-off-by: DQ <dengq@vmware.com>
2020-12-09 20:42:21 +08:00
DQ
8a584aff89 Clean up clair and clair-adapter in build scripts
1. Makefles
  2. Dockerfiles
  3. Installation script
  4. harbor.yml template

Signed-off-by: DQ <dengq@vmware.com>
2020-11-10 11:39:18 +08:00
Ángel Barrera Sánchez
283f9989e9 FIX: Update clair version to fix NVD deprecated links
Signed-off-by: Ángel Barrera Sánchez <angel@sighup.io>
2020-10-14 08:07:57 +02:00
Daniel Pacak
224cfec4f7
chore(trivy): Bump up trivy adapter to v0.14.1 (#12840)
This is the maintenance release to recompile the trivy
adapter service with Go 1.14.7 and pull Trivy v0.9.2.

Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
2020-08-21 10:40:06 +08:00
danfengliu
143062fe20
Merge pull request #12788 from danfengliu/upgrade-robotframework-in-git-action
upgrade robotframework in git action
2020-08-20 20:08:36 +08:00
danfengliu
3f5bd9ae0a Re-script ldap API script
1. upgrade robotframework in git action
2. Re-script ldap API script:test_assign_role_to_ldap_group.py

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-08-20 11:24:51 +00:00
He Weiwei
b749d6544e chore(clair): bump up clair adapter to v1.1.1
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-08-20 09:19:56 +00:00
Dirk Mueller
08a4d8efd2
Update to golang 1.14.7 (#12809)
We should use a golang that isn't having security issues.

This includes:
* go1.14.6 (released 2020/07/16) includes fixes to the go command, the
compiler, the linker, vet, and the database/sql, encoding/json,
net/http, reflect, and testing packages. See the Go 1.14.6 milestone on
our issue tracker for details.

* go1.14.7 (released 2020/08/06) includes security fixes to the
encoding/binary package. See the Go 1.14.7 milestone on our issue
tracker for details (CVE-2020-16845)

Signed-off-by: Dirk Mueller <dirk@dmllr.de>
Signed-off-by: Dirk Mueller <dmueller@suse.com>
2020-08-20 15:38:35 +08:00
Wang Yan
60427e7187
build base image in CI (#12750)
In git action, use the local build base images instead of pulling from docker hub.

Signed-off-by: wang yan <wangyan@vmware.com>
2020-08-18 14:11:52 +08:00
He Weiwei
f309896f2f refactor(api): generate project apis by go-swagger
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-08-15 16:10:57 +00:00
He Weiwei
31d2d9ee9f chore(clair): bump up clair adapter to v1.1.0
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-08-10 18:45:52 +08:00
Daniel Jiang
4f94f59d2a Provide a standalone migrator to migrate DB schema.
Fixes #11885
This part will not by default be packaged into release.
A README.md will be added in another commit.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-08-06 18:57:55 +08:00
He Weiwei
aa55fcfce7 chore(mocks): add make targets to generate and check mocks
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-07-22 01:59:32 +00:00
Ziming Zhang
8857e89e40 feature(redis) support redis sentinel
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2020-07-19 21:19:03 +08:00
Wang Yan
bad8f026fc
upgrade golang to v1.14.5 (#12489)
Signed-off-by: wang yan <wangyan@vmware.com>
2020-07-16 16:20:54 +08:00
Daniel Pacak
4b558baaf8 chore(trivy): Bump up Trivy adapter to v0.12.0
The new version of the adapter service improves the Redis connection pool
management. In the previous versions a new connection pool was created for
each scan job, which might negatively impact the performance and resources
utilisation.

There is also a bug fix in Trivy v0.9.1 to properly handle the debug mode.

Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
2020-06-29 10:14:28 +02:00
Max Rosin
34d5591b1b Fix DOCKERIMASES and SWAAGER_IMAGE_BUILD_CMD typos in Makefiles
Fix #12259

Signed-off-by: Max Rosin <git@hackrid.de>
2020-06-16 12:18:55 +02:00
Wang Yan
976a812d21
bump up clair v2.1.4 (#12245)
Signed-off-by: wang yan <wangyan@vmware.com>
2020-06-16 12:51:44 +08:00
Daniel Pacak
dfcee80ae5 fix(trivy): Bump up Trivy adapter to v0.11.0
This commit bumps up Trivy to resolve the following issues reported
in the aquasecurity/harbor-scanner-trivy repository:

- https://github.com/aquasecurity/harbor-scanner-trivy/issues/114
- https://github.com/aquasecurity/harbor-scanner-trivy/issues/108

Note that this adapter vendors in Trivy v0.9.0 which has changed
the algorithm for qualifying severities. Previous versions of Trivy
preferred NVD scores, whereas this version will use vendor score
whenever it's possible.

We believe it's more suitable approach for qualifying severities.
Even though this change might impact vulnerability summaries in
some cases, the total number of vulnerabilities should stay the
same.

Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
2020-06-05 10:48:16 +02:00
danfengliu
5da22bc2fc
Merge pull request #12100 from danfengliu/add-replication-schedule-test-1
Add replication schedule test
2020-06-03 16:55:18 +08:00
Daniel Pacak
f5d482854b
fix(trivy): Handle gracefully scratch and slim images (#11983)
This commit bumps up Trivy to 0.7.0 and Trivy adapter service to 0.10.0
in order to handle scratch and slim images, for which we cannot detect
the underlying operating system.

Resolves: #11964

Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
2020-06-02 10:51:14 +02:00
danfengliu
ceaa0a57b3 Add replication schedule test
1. Add replication schedule test;
2. Add some sleep after project creation before push image to avoid push image v2 error.
   It fixed issue# 12094: Scan image vmware/photon:1.0 return unsupported occasionally.
3. Add some sleep in test_user_view_logs.py, can't get operation logs sometimes.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-06-02 15:51:10 +08:00
DQ
f7ffd991cc Enhance: Upgrade chartmuseum version
Upgrade chartmuseum version 0.12.0

Signed-off-by: DQ <dengq@vmware.com>
2020-05-26 15:59:58 +08:00