Ziming Zhang
39f70287b4
Fixed typo in NOTARYURL variable name
...
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2021-04-17 15:55:02 +08:00
DQ
ffed6459c7
Fix: Use local host on db's healthcheck
...
hostname -i will malfunction in some cases like the `nsswitch.conf` file does'nt exist
Signed-off-by: DQ <dengq@vmware.com>
2021-04-16 18:37:24 +08:00
Pushkar Joglekar
3947c5faff
Add --no-cache and --pull flag to image builds to ensure latest security fixes are pulled from base image
...
Signed-off-by: Pushkar Joglekar <pjoglekar@vmware.com>
2021-04-12 09:49:27 -07:00
Qian Deng
c5d12ce8ee
Merge pull request #14542 from ninjadq/add_task_info_in_exporter
...
Add task info in exporter
2021-04-07 18:17:26 +08:00
Alexis
06fa88cfb7
Fix typo
...
Signed-off-by: Alexis <60alexis@gmail.com>
2021-04-07 15:58:17 +08:00
Alexis
e33f7aa9dd
Add redis port to 2.1.0 jinja template
...
Signed-off-by: Alexis <60alexis@gmail.com>
2021-04-07 15:58:17 +08:00
Alexis
7742aec4af
Add port to 2.0.0 jinja file
...
Signed-off-by: Alexis <60alexis@gmail.com>
2021-04-07 15:58:17 +08:00
Alexis
d28845af51
Remove external_redis.port since not used since v1.10.0
...
Signed-off-by: Alexis <60alexis@gmail.com>
2021-04-07 15:58:17 +08:00
stonezdj(Daojun Zhang)
448f0b6e28
Merge pull request #14579 from stonezdj/21apr_add_docker_registry_proxy
...
Support proxy cache for docker-registry type
2021-04-07 10:59:24 +08:00
Steven Zou
e2148f9eea
Merge pull request #14514 from goharbor/dependabot/pip/make/photon/prepare/pyyaml-5.4
...
Bump pyyaml from 4.2b1 to 5.4 in /make/photon/prepare
2021-04-07 09:57:07 +08:00
Steven Zou
10711b7de1
Merge pull request #14482 from goharbor/dependabot/pip/make/photon/prepare/jinja2-2.11.3
...
Bump jinja2 from 2.11.1 to 2.11.3 in /make/photon/prepare
2021-04-07 09:56:23 +08:00
stonezdj
ccd9ee8c56
Support proxy cache for docker-registry type
...
Add proxy cache for docker registry type
Fixes #14477 , #14547
Signed-off-by: stonezdj <stonezdj@gmail.com>
2021-04-06 16:47:12 +08:00
Wang Yan
d03a29e531
bump up photon to 4.0
...
Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-03-31 13:25:29 +08:00
DQ
7eebbeebdf
Add jobservice task queue related task
...
add jobservice metrics
add redis client
Signed-off-by: DQ <dengq@vmware.com>
2021-03-30 10:29:36 +00:00
DQ
fbe9cd88f8
Enabled Prometheus for Jobservice
...
* Add prom server on jobservice
* Enabeld configs in templates
* Enabeld jobservice metrics in nginx
Signed-off-by: DQ <dengq@vmware.com>
2021-03-30 08:52:59 +00:00
dependabot[bot]
f20f4215c3
Bump pyyaml from 4.2b1 to 5.4 in /make/photon/prepare
...
Bumps [pyyaml](https://github.com/yaml/pyyaml ) from 4.2b1 to 5.4.
- [Release notes](https://github.com/yaml/pyyaml/releases )
- [Changelog](https://github.com/yaml/pyyaml/blob/master/CHANGES )
- [Commits](https://github.com/yaml/pyyaml/commits/5.4 )
Signed-off-by: dependabot[bot] <support@github.com>
2021-03-25 22:46:56 +00:00
DQ
f5fcc7bd31
Add base image for exporter
...
* Add base
* update Makefile
Signed-off-by: DQ <dengq@vmware.com>
2021-03-25 16:35:57 +08:00
dependabot[bot]
0ec667c4d8
Bump jinja2 from 2.11.1 to 2.11.3 in /make/photon/prepare
...
Bumps [jinja2](https://github.com/pallets/jinja ) from 2.11.1 to 2.11.3.
- [Release notes](https://github.com/pallets/jinja/releases )
- [Changelog](https://github.com/pallets/jinja/blob/master/CHANGES.rst )
- [Commits](https://github.com/pallets/jinja/compare/2.11.1...2.11.3 )
Signed-off-by: dependabot[bot] <support@github.com>
2021-03-20 00:43:21 +00:00
Wenkai Yin
8b1817be0f
Fix the consume too much CPU issue
...
1. Update execution status during the upgrade
2. Refine the execution sweeper
Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-03-17 13:34:24 +08:00
Wenkai Yin
43df3bf8a4
Add upgrade sql file introduced in 2.1.4
...
1. Add upgrade sql file introduced in 2.1.4
2. Minor improvement for task/execution to cover corner cases
Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-03-15 16:48:51 +08:00
Wang Yan
3dfddfdf4e
patch upstream fix for io reader ( #14356 )
...
Fixes #12850
This patch can fix the GC failure in the NFS v3 env, see https://github.com/distribution/distribution/pull/3309#issuecomment-783606968
Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-03-04 15:33:09 +08:00
Daniel Jiang
387be3686a
Refine the way to set X-Forwarded-Proto in nginx
...
Refine the way to set the header so user won't need to comment it if
Harbor is sitting behind a reverse proxy.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-02-25 17:43:55 +08:00
danfengliu
9e3f0de12c
Merge pull request #14124 from danfengliu/reschedule-docker-login-policy-in-build-base-image-in-master
...
Reschedule docker login policy in base image build process
2021-02-23 10:10:59 +08:00
danfengliu
7d05c8e513
Reschedule docker login policy in base image build process
...
Signed-off-by: danfengliu <danfengl@vmware.com>
2021-02-22 10:05:25 +08:00
Josh Soref
dfe360040b
Spelling
...
* addition
* attribute
* auditing
* availability
* available
* bandwidth
* browser
* business
* cadence
* chartmuseum
* client
* column
* content
* demonstrate
* described
* endpoints
* facilitate
* github
* harbor
* information
* instance
* manual
* meaningful
* operation
* overridden
* password
* possible
* project
* refactor
* replication
* requires
* running
* scanned
* settings
* signup
* those
* unsigned
* vulnerability
--
Also removes trailing space from a filename
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
2021-02-19 11:59:15 +08:00
DQ
307c5a8ed4
Fix metrics template for http mode
...
the port shouldn't be hardcode
Signed-off-by: DQ <dengq@vmware.com>
2021-02-05 18:44:28 +00:00
Ziming Zhang
ec83f49a1a
fix(retention) migrate sql error
...
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2021-02-05 09:09:26 +00:00
Wang Yan
24ec772978
fix gc migration issue ( #14174 )
...
For the upgrade path v1.10 - v2.1.* - v2.2.0, if user doesn't reset the GC schdule that was created in 1.10 in the v2.1,
the job parameters will keep empty in the database. The fix gives a default value for the schedule record.
Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-02-05 12:25:24 +08:00
DQ
051b5f289d
Add sen existed check for internal cert
...
fali ealier when there is no san
Signed-off-by: DQ <dengq@vmware.com>
2021-01-28 08:22:07 +00:00
Wenkai Yin(尹文开)
50a1e85095
Make sure the revision of execution isn't null during the upgrade ( #14085 )
...
Make sure the revision of execution isn't null during the upgrade
Fixes #14075
Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-01-27 10:10:36 +08:00
Qian Deng
f013d88efc
Merge pull request #14013 from ninjadq/upgrade_script_for_2_2_0
...
Harbor upgrading for 2.2
2021-01-22 18:10:24 +08:00
Qian Deng
045e1d9abe
Merge pull request #14040 from ninjadq/metric_improvement
...
Metric improvement
2021-01-22 17:13:57 +08:00
DQ
489f31d8fe
Add upgrade scirpt for 2.2
...
1. add metrics config item in config
2. upgrade version in template
Signed-off-by: DQ <dengq@vmware.com>
2021-01-22 16:15:06 +08:00
Wang Yan
dba229d0df
build third party binaries in CI ( #14019 )
...
Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-01-22 11:33:42 +08:00
DQ
92cf728371
Add custom cert for exporter
...
* injecting custom certs related config to exporter
Signed-off-by: DQ <dengq@vmware.com>
2021-01-20 10:52:34 +08:00
DQ
a61e9b0e2e
Add san for notary upgrading
...
if san not exists then remove that cert, prepare will regenerate one
Signed-off-by: DQ <dengq@vmware.com>
2021-01-18 21:00:35 +08:00
Wenkai Yin
7c072e17a6
Fix the legacy scheduled job issue for GC/scan all
...
Fix the legacy scheduled job issue for GC/scan all
Fixes #13968
Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-01-15 22:02:36 +08:00
Daniel Jiang
1b64b9fdc2
Bump up the go-migrate ( #13914 )
...
Bump it up to v4.11.0 to be consistent with harbor-core
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-01-11 19:08:17 +08:00
He Weiwei
4a326aa8b0
chore: delete records of scan_report
...
The report in previous scan_report records not work well the
vulnerabilities stored in the schema table, so delete the scan_report
records.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-01-08 03:39:11 +00:00
Qian Deng
642d56041d
Add san for notary cert ( #13928 )
...
Signed-off-by: DQ <dengq@vmware.com>
2021-01-08 01:00:34 +08:00
stonezdj
6b8fb8431d
Add quay registry to proxy cache
...
Update env.jinja to add quay
Signed-off-by: stonezdj <stonezdj@gmail.com>
2021-01-06 17:22:57 +08:00
Ziming Zhang
8faa76a1b6
feat(retention) refactor task manager
...
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2021-01-05 12:08:03 +08:00
Wenkai Yin(尹文开)
19ad8ad68d
Merge pull request #13823 from reasonerjt/inst-cert-home-dir
...
Replace tilde in install_cert.sh
2020-12-25 10:25:51 +08:00
prahaladdarkin
a890b28e1e
Store vulnerability data from scanner into a relational format ( #13616 )
...
feat: Store vulnerability report from scanner into a relational format
Convert vulnerability report JSON obtained from scanner into a relational format describe in:https://github.com/goharbor/community/pull/145
Signed-off-by: prahaladdarkin <prahaladd@vmware.com>
2020-12-25 08:47:46 +08:00
Wang Yan
7a8a8fa104
upgrade go version to v1.15.6 ( #13836 )
...
Signed-off-by: wang yan <wangyan@vmware.com>
2020-12-23 18:53:09 +08:00
He Weiwei
3831e82b20
refactor: remove code of admin job ( #13819 )
...
Remove code of admin job as it's not needed by scan all/gc now.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-12-22 11:48:16 +08:00
Daniel Jiang
9d99dfa82b
Replace tilde in install_cert.sh
...
This commit fixes #13287 to remove the usage of tilde as the $HOME is not available in some
cases. More details see #13287
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-12-21 20:39:34 +08:00
Qian Deng
31138f12b0
Merge pull request #13806 from ninjadq/fix_python_yaml_load
...
Fix pythom yaml load to safe_load
2020-12-21 16:04:12 +08:00
Qian Deng
9197471e70
Add Scan for internal tls ( #13810 )
...
Signed-off-by: DQ <dengq@vmware.com>
2020-12-21 15:23:11 +08:00
Wang Yan
9bc6f3cee4
fix robot account update issue ( #13741 )
...
* fix robot account update issue
enable the update method to support both v1 & v2 robot update
Signed-off-by: Wang Yan <wangyan@vmware.com>
* resolve review comments
Signed-off-by: Wang Yan <wangyan@vmware.com>
2020-12-18 20:01:26 +08:00
Wang Yan
6bc1047013
migration admin job data ( #13766 )
...
1, migrate gc and scan all schedule to schedule/task/exectuion
2, migrate gc history to task/execution
Signed-off-by: Wang Yan <wangyan@vmware.com>
2020-12-18 16:35:24 +08:00
Will Sun
4392a626f3
Merge pull request #13804 from AllForNothing/scan-all
...
Fix robot account UI issues
2020-12-18 15:48:26 +08:00
Qian Deng
64fcfeaa2f
Merge pull request #13754 from ninjadq/fix_loglevel_parsing_for_registry
...
Fix log level issue in registry
2020-12-18 14:47:15 +08:00
AllForNothing
b20cc474b3
Fix robot account UI issues
...
Signed-off-by: AllForNothing <sshijun@vmware.com>
2020-12-18 14:11:08 +08:00
DQ
234b29e170
Fix pythom yaml load to safe_load
...
Signed-off-by: DQ <dengq@vmware.com>
2020-12-16 14:59:06 +08:00
Daniel Jiang
b0c8cadad7
Add default CVE allowlist to project library ( #13770 )
...
fixes #12700
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-12-16 14:20:56 +08:00
Wenkai Yin
69808f033e
Tiny fixes for task manager
...
1. Add update time for execution
2. Add unique constraint for schedule to avoid dup records when updating policies
3. Format replication log
4. Keep the webhook handler for legacy replication jobs to avoid jobservice resending the status change request
Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-12-14 17:26:32 +08:00
He Weiwei
08580f9fec
refactor(scan): refactor scan/scan all job to task manager ( #13684 )
...
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-12-14 13:34:35 +08:00
DQ
19e8527cc1
Fix log level issue in registry
...
1. fix level issue in registry.jinja
2. add log level to registryctl
Signed-off-by: DQ <dengq@vmware.com>
2020-12-14 11:52:42 +08:00
Wenkai Yin(尹文开)
6569016d35
Merge pull request #13139 from wy65701436/migrate-gc
...
Migrate gc to task manager
2020-12-14 10:43:44 +08:00
Wang Yan
39e1a4f2b4
add extra attributes in the schedule table
...
Signed-off-by: Wang Yan <wangyan@vmware.com>
2020-12-14 02:28:52 +08:00
wang yan
1bb79d402d
update code per review comments
...
Signed-off-by: wang yan <wangyan@vmware.com>
2020-12-10 16:08:52 +08:00
Wang Yan
dba5522d0b
Migrate to task manager ( #129 )
...
1, remove the gc to new programming model
2, move api define to harbor v2 swagger
3, leverage task & execution manager to manage gc job schedule, trigger and log.
Signed-off-by: wang yan <wangyan@vmware.com>
2020-12-10 14:00:33 +08:00
DQ
d95f22448c
Add cache for exporter
...
Add timed cache for exporter
default cache time is 30s, cleanup job run every 4 hours
Signed-off-by: DQ <dengq@vmware.com>
2020-12-09 21:22:40 +08:00
DQ
f0db193895
Add prepare file for exporter
...
prepare env for exporter
Signed-off-by: DQ <dengq@vmware.com>
2020-12-09 21:22:13 +08:00
DQ
dc0047c48c
Add build script for exporter
...
- Add dockerfile
- update makefile
Signed-off-by: DQ <dengq@vmware.com>
2020-12-09 20:42:21 +08:00
Wang Yan
d2fa2e6b84
update robot secret ( #13654 )
...
* update robot secret
1, use SHA256 to generate and validate robot secret instread of symmetric encryption.
2, update the patch input object
Signed-off-by: Wang Yan <wangyan@vmware.com>
* update robot secret
1, use SHA256 to generate and validate robot secret instread of symmetric encryption.
2, update the patch input object
Signed-off-by: Wang Yan <wangyan@vmware.com>
2020-12-03 18:13:06 +08:00
Wang Yan
732e9a21cd
updates on robot accounts ( #13623 )
...
* updates on robot accounts
1, add patch method to refresh secret of a robot
2, fix robot account update issue
3, add editable attribute to handle the version 1 robot account
4, add duration for robot account
5, hide secret for get/list robot account
Signed-off-by: wang yan <wangyan@vmware.com>
* update code per review comments
1, change expirate creation func to AddDate().
2, remove the scanner duration specification, use the default value.
Signed-off-by: Wang Yan <wangyan@vmware.com>
2020-12-01 18:31:34 +08:00
DQ
907904f480
Add DB Migration code for clair cleanning
...
- Delete clair scanner if exist
- Delete report is it is scanned by clair
- Set Trivy to Default if it exist and not default scanner
Signed-off-by: DQ <dengq@vmware.com>
2020-11-29 16:19:02 +08:00
DQ
590212b485
Remove clair related code
...
- clair code in harbor core
- clair code in frontend
- clair code in robotcase
Signed-off-by: DQ <dengq@vmware.com>
2020-11-27 14:01:04 +08:00
stonezdj(Daojun Zhang)
be4e6a5985
Merge pull request #13537 from stonezdj/201118_add_more_registry_type
...
Add more registry type to proxy cache
2020-11-26 11:16:16 +08:00
Ziming Zhang
d55f55aeb9
fix(chartmuseum) compatible s3 cache fail
...
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2020-11-25 17:00:16 +08:00
He Weiwei
eb38180483
fix(quota): ignore the fail when getting reference of quota
...
1. Clean the dirty data in quota/quota_usage.
2. Ignore the fail when getting the reference of quota.
Closes #13387
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-11-24 14:50:38 +00:00
Wenkai Yin(尹文开)
fe8b628f0c
Merge pull request #13437 from ywk253100/200929_replication
...
Refactor the replication execution
2020-11-24 10:38:22 +08:00
Wenkai Yin
294385c34d
Refactor the replication execution
...
1. Use the task manager to manage the underlying execution/task
2. Use the pkg/scheduler to schedule the periodical job
3. Apply the new program model
4. Migration the old data into the new data model
Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-11-23 14:24:10 +08:00
stonezdj
e667121a34
Add more registry type to proxy cache
...
Includes: azure-acr, aws-ecr, google-gcr
Signed-off-by: stonezdj <stonezdj@gmail.com>
2020-11-18 10:38:07 +08:00
Will Sun
eca3de3489
Merge pull request #13494 from dirkmueller/lock_json_include
...
Include package.json/package-lock.json in portal image
2020-11-16 16:38:02 +08:00
stonezdj(Daojun Zhang)
fb549b2d9e
Merge pull request #13444 from wy65701436/robot2-self-mgr
...
add robot mgr
2020-11-16 11:33:33 +08:00
He Weiwei
83c07d6680
fix: ensure the role_id of role is correct ( #13476 )
...
Closes #13317
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-11-12 15:34:04 +08:00
Dirk Mueller
12adc63a48
Include package.json/package-lock.json in portal image
...
This allows Trivy and other vulnerability scanners to correctly
determine the embedded dependencies in minified harbor-portal image.
Also simplify build process by reducing the number of layers in the
final stage container image
Signed-off-by: Dirk Mueller <dirk@dmllr.de>
2020-11-11 21:21:28 +01:00
Wang Yan
3550b5e5e9
add robot mgr
...
the robot account manager to handle the CRUD
Signed-off-by: wang yan <wangyan@vmware.com>
2020-11-11 13:47:03 +08:00
Wang Yan
9723655378
update code per review comments
...
1, rename table name to permission_policy
2, rename functions name
Signed-off-by: Wang Yan <wangyan@vmware.com>
2020-11-10 18:11:31 +08:00
Wang Yan
ec15e320bf
add role permission manager for robot enhancement
...
1, add two db tables of role permission and rbac policy
2, add manager of these two tables
Signed-off-by: Wang Yan <wangyan@vmware.com>
2020-11-10 16:49:29 +08:00
He Weiwei
ebc3443da9
Merge pull request #13474 from heww/fix-issue-11892
...
fix: compute artifact size from db for schema1 manifest
2020-11-10 16:20:39 +08:00
DQ
c10a6325d8
Add deprecated msg for clair
...
Signed-off-by: DQ <dengq@vmware.com>
2020-11-10 11:39:18 +08:00
DQ
0c9faea294
Clean up Clair in prepare script
...
Signed-off-by: DQ <dengq@vmware.com>
2020-11-10 11:39:18 +08:00
DQ
8a584aff89
Clean up clair and clair-adapter in build scripts
...
1. Makefles
2. Dockerfiles
3. Installation script
4. harbor.yml template
Signed-off-by: DQ <dengq@vmware.com>
2020-11-10 11:39:18 +08:00
He Weiwei
9c8377909b
fix: compute artifact size from db for schema1 manifest
...
Closes #11892
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-11-09 12:32:07 +00:00
DQ
9152521b11
Fix: log container password expire
...
move chage command to base image
Signed-off-by: DQ <dengq@vmware.com>
2020-11-09 18:29:41 +08:00
DQ
eb470501be
Add metrics to Harbor Core
...
1. Add configs in prepare
2. Add models and config items in Core
3. Encapdulate getting metric in commom package
4. Add a middleware for global request to collect 3 metrics
Signed-off-by: DQ <dengq@vmware.com>
2020-11-03 14:33:10 +08:00
Daniel Jiang
fb687aeef8
Use pkg/token to generate JWT token
...
This commit refactors the approach to encode a token in handler of /service/token,
by reusing pkg/token to avoid inconsistency.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-10-15 16:16:44 +08:00
DQ
184e89365b
Fix internal tls config upgrade issue
...
internal tls config upgrade is not included in template, this pr is to add it.
Signed-off-by: DQ <dengq@vmware.com>
2020-09-25 09:54:31 +08:00
Wenkai Yin(尹文开)
8b9727f53f
Support store the cron type in the schedule ( #13097 )
...
There is requirement that show the cron type(daily, weekly, etc.) on the UI, this commit adds the support for storing the cron type in the schedule model
Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-09-24 16:48:56 +08:00
DQ
17f3bfccb4
Fix trivy setting in upgrading script
...
Signed-off-by: DQ <dengq@vmware.com>
2020-09-08 18:15:57 +08:00
Daniel Jiang
1b8bec3994
Merge pull request #12896 from wy65701436/fixes-12889
...
fix migration issue
2020-08-28 14:16:21 +08:00
He Weiwei
687043c298
Merge pull request #12880 from stefannica/use-exit-in-db-entrypoint
...
Use exec in harbor database entrypoint
2020-08-28 10:09:58 +08:00
Daniel Jiang
91e2779822
Fill in the icon of known artifacts in artifact controller
...
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-08-28 01:33:26 +08:00
wang yan
84094e7a5d
fix migration issue
...
fixes #12889
Before the migration script to fix the nativate repo_id issue, is has to remove the duplicate tags
from the tag table, which may caused by user in v2.0.2 to retag & repush the missing image.
Signed-off-by: wang yan <wangyan@vmware.com>
2020-08-27 19:04:39 +08:00
Daniel Jiang
7b42defb9a
Make the 2.1.0 migration SQL script idempotent
...
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-08-26 16:50:25 +08:00
Daniel Jiang
37e0aa0798
Merge pull request #12873 from wy65701436/fixes-12827
...
fix db migration issue
2020-08-26 14:42:24 +08:00