Qian Deng
17dd48e5a3
Merge pull request #14854 from ninjadq/upgrade_helm_chart
...
Upgrade helm chart
2021-05-12 19:33:03 +08:00
Wenkai Yin
c04f3a2aac
Fix duplicate execution record issue
...
When the core service cannot response the checkin request in time, duplicated execution records may be created, this commit introduces the revision column to make sure there is only one record for one schedule trigger
Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-05-12 15:40:52 +08:00
DQ
04ba4a4033
Upgrade chartmuseum version
...
from 1.12.0 to 1.13.1
Signed-off-by: DQ <dengq@vmware.com>
2021-05-11 13:51:55 +00:00
He Weiwei
476732df89
fix: change art_size to bigint in migrations
...
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-05-11 04:01:02 +00:00
Wenkai Yin(尹文开)
9bbffa06db
Merge pull request #14662 from ywk253100/210413_dest_ns
...
Refactor the replication policy destination namespace logic
2021-04-29 11:18:43 +08:00
Wenkai Yin
710c80078b
Refactor the replication policy destination namespace logic
...
Support specifying what part of the repository will be replaced by the provided namespace
Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-04-28 10:16:20 +08:00
Daniel Jiang
c701ce09fa
Merge pull request #14681 from bitsf/fix_typo_NOTARYURL
...
Fixed typo in NOTARYURL variable name
2021-04-21 17:38:01 +08:00
Wenkai Yin(尹文开)
0f6692f68f
Merge pull request #14682 from ninjadq/hostname_hardcode_to_localhost
...
Fix: Use local host on db's healthcheck
2021-04-19 21:02:31 +08:00
danfengliu
f0ebd17994
Add build base image step in build package git action workflow
...
Build base image step should be in build package workflow, and local base images build by new step should be removed since images have been pushed to docker hub.
Signed-off-by: danfengliu <danfengl@vmware.com>
2021-04-17 18:10:44 +08:00
Ziming Zhang
39f70287b4
Fixed typo in NOTARYURL variable name
...
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2021-04-17 15:55:02 +08:00
DQ
ffed6459c7
Fix: Use local host on db's healthcheck
...
hostname -i will malfunction in some cases like the `nsswitch.conf` file does'nt exist
Signed-off-by: DQ <dengq@vmware.com>
2021-04-16 18:37:24 +08:00
Pushkar Joglekar
3947c5faff
Add --no-cache and --pull flag to image builds to ensure latest security fixes are pulled from base image
...
Signed-off-by: Pushkar Joglekar <pjoglekar@vmware.com>
2021-04-12 09:49:27 -07:00
Qian Deng
c5d12ce8ee
Merge pull request #14542 from ninjadq/add_task_info_in_exporter
...
Add task info in exporter
2021-04-07 18:17:26 +08:00
Alexis
06fa88cfb7
Fix typo
...
Signed-off-by: Alexis <60alexis@gmail.com>
2021-04-07 15:58:17 +08:00
Alexis
e33f7aa9dd
Add redis port to 2.1.0 jinja template
...
Signed-off-by: Alexis <60alexis@gmail.com>
2021-04-07 15:58:17 +08:00
Alexis
7742aec4af
Add port to 2.0.0 jinja file
...
Signed-off-by: Alexis <60alexis@gmail.com>
2021-04-07 15:58:17 +08:00
Alexis
d28845af51
Remove external_redis.port since not used since v1.10.0
...
Signed-off-by: Alexis <60alexis@gmail.com>
2021-04-07 15:58:17 +08:00
stonezdj(Daojun Zhang)
448f0b6e28
Merge pull request #14579 from stonezdj/21apr_add_docker_registry_proxy
...
Support proxy cache for docker-registry type
2021-04-07 10:59:24 +08:00
Steven Zou
e2148f9eea
Merge pull request #14514 from goharbor/dependabot/pip/make/photon/prepare/pyyaml-5.4
...
Bump pyyaml from 4.2b1 to 5.4 in /make/photon/prepare
2021-04-07 09:57:07 +08:00
Steven Zou
10711b7de1
Merge pull request #14482 from goharbor/dependabot/pip/make/photon/prepare/jinja2-2.11.3
...
Bump jinja2 from 2.11.1 to 2.11.3 in /make/photon/prepare
2021-04-07 09:56:23 +08:00
stonezdj
ccd9ee8c56
Support proxy cache for docker-registry type
...
Add proxy cache for docker registry type
Fixes #14477 , #14547
Signed-off-by: stonezdj <stonezdj@gmail.com>
2021-04-06 16:47:12 +08:00
Wang Yan
d03a29e531
bump up photon to 4.0
...
Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-03-31 13:25:29 +08:00
DQ
7eebbeebdf
Add jobservice task queue related task
...
add jobservice metrics
add redis client
Signed-off-by: DQ <dengq@vmware.com>
2021-03-30 10:29:36 +00:00
DQ
fbe9cd88f8
Enabled Prometheus for Jobservice
...
* Add prom server on jobservice
* Enabeld configs in templates
* Enabeld jobservice metrics in nginx
Signed-off-by: DQ <dengq@vmware.com>
2021-03-30 08:52:59 +00:00
dependabot[bot]
f20f4215c3
Bump pyyaml from 4.2b1 to 5.4 in /make/photon/prepare
...
Bumps [pyyaml](https://github.com/yaml/pyyaml ) from 4.2b1 to 5.4.
- [Release notes](https://github.com/yaml/pyyaml/releases )
- [Changelog](https://github.com/yaml/pyyaml/blob/master/CHANGES )
- [Commits](https://github.com/yaml/pyyaml/commits/5.4 )
Signed-off-by: dependabot[bot] <support@github.com>
2021-03-25 22:46:56 +00:00
DQ
f5fcc7bd31
Add base image for exporter
...
* Add base
* update Makefile
Signed-off-by: DQ <dengq@vmware.com>
2021-03-25 16:35:57 +08:00
dependabot[bot]
0ec667c4d8
Bump jinja2 from 2.11.1 to 2.11.3 in /make/photon/prepare
...
Bumps [jinja2](https://github.com/pallets/jinja ) from 2.11.1 to 2.11.3.
- [Release notes](https://github.com/pallets/jinja/releases )
- [Changelog](https://github.com/pallets/jinja/blob/master/CHANGES.rst )
- [Commits](https://github.com/pallets/jinja/compare/2.11.1...2.11.3 )
Signed-off-by: dependabot[bot] <support@github.com>
2021-03-20 00:43:21 +00:00
Wenkai Yin
8b1817be0f
Fix the consume too much CPU issue
...
1. Update execution status during the upgrade
2. Refine the execution sweeper
Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-03-17 13:34:24 +08:00
Wenkai Yin
43df3bf8a4
Add upgrade sql file introduced in 2.1.4
...
1. Add upgrade sql file introduced in 2.1.4
2. Minor improvement for task/execution to cover corner cases
Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-03-15 16:48:51 +08:00
Xavier Duthil
280c8272f8
Use exec in all components' entrypoints
...
Use the exec Bash command so that the final running application becomes
the container’s PID 1. This allows the application to receive any Unix
signals sent to the container, in accordance with
https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#entrypoint
Currently, SIGTERM signals sent by kubernetes are not passed to the
executed binary.
Signed-off-by: Xavier Duthil <xavier.duthil@corp.ovh.com>
2021-03-05 15:00:25 +01:00
Wang Yan
3dfddfdf4e
patch upstream fix for io reader ( #14356 )
...
Fixes #12850
This patch can fix the GC failure in the NFS v3 env, see https://github.com/distribution/distribution/pull/3309#issuecomment-783606968
Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-03-04 15:33:09 +08:00
Daniel Jiang
387be3686a
Refine the way to set X-Forwarded-Proto in nginx
...
Refine the way to set the header so user won't need to comment it if
Harbor is sitting behind a reverse proxy.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-02-25 17:43:55 +08:00
danfengliu
9e3f0de12c
Merge pull request #14124 from danfengliu/reschedule-docker-login-policy-in-build-base-image-in-master
...
Reschedule docker login policy in base image build process
2021-02-23 10:10:59 +08:00
danfengliu
7d05c8e513
Reschedule docker login policy in base image build process
...
Signed-off-by: danfengliu <danfengl@vmware.com>
2021-02-22 10:05:25 +08:00
Josh Soref
dfe360040b
Spelling
...
* addition
* attribute
* auditing
* availability
* available
* bandwidth
* browser
* business
* cadence
* chartmuseum
* client
* column
* content
* demonstrate
* described
* endpoints
* facilitate
* github
* harbor
* information
* instance
* manual
* meaningful
* operation
* overridden
* password
* possible
* project
* refactor
* replication
* requires
* running
* scanned
* settings
* signup
* those
* unsigned
* vulnerability
--
Also removes trailing space from a filename
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
2021-02-19 11:59:15 +08:00
DQ
307c5a8ed4
Fix metrics template for http mode
...
the port shouldn't be hardcode
Signed-off-by: DQ <dengq@vmware.com>
2021-02-05 18:44:28 +00:00
Ziming Zhang
ec83f49a1a
fix(retention) migrate sql error
...
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2021-02-05 09:09:26 +00:00
Wang Yan
24ec772978
fix gc migration issue ( #14174 )
...
For the upgrade path v1.10 - v2.1.* - v2.2.0, if user doesn't reset the GC schdule that was created in 1.10 in the v2.1,
the job parameters will keep empty in the database. The fix gives a default value for the schedule record.
Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-02-05 12:25:24 +08:00
DQ
051b5f289d
Add sen existed check for internal cert
...
fali ealier when there is no san
Signed-off-by: DQ <dengq@vmware.com>
2021-01-28 08:22:07 +00:00
Wenkai Yin(尹文开)
50a1e85095
Make sure the revision of execution isn't null during the upgrade ( #14085 )
...
Make sure the revision of execution isn't null during the upgrade
Fixes #14075
Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-01-27 10:10:36 +08:00
Qian Deng
f013d88efc
Merge pull request #14013 from ninjadq/upgrade_script_for_2_2_0
...
Harbor upgrading for 2.2
2021-01-22 18:10:24 +08:00
Qian Deng
045e1d9abe
Merge pull request #14040 from ninjadq/metric_improvement
...
Metric improvement
2021-01-22 17:13:57 +08:00
DQ
489f31d8fe
Add upgrade scirpt for 2.2
...
1. add metrics config item in config
2. upgrade version in template
Signed-off-by: DQ <dengq@vmware.com>
2021-01-22 16:15:06 +08:00
Wang Yan
dba229d0df
build third party binaries in CI ( #14019 )
...
Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-01-22 11:33:42 +08:00
DQ
92cf728371
Add custom cert for exporter
...
* injecting custom certs related config to exporter
Signed-off-by: DQ <dengq@vmware.com>
2021-01-20 10:52:34 +08:00
DQ
a61e9b0e2e
Add san for notary upgrading
...
if san not exists then remove that cert, prepare will regenerate one
Signed-off-by: DQ <dengq@vmware.com>
2021-01-18 21:00:35 +08:00
Wenkai Yin
7c072e17a6
Fix the legacy scheduled job issue for GC/scan all
...
Fix the legacy scheduled job issue for GC/scan all
Fixes #13968
Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-01-15 22:02:36 +08:00
Daniel Jiang
1b64b9fdc2
Bump up the go-migrate ( #13914 )
...
Bump it up to v4.11.0 to be consistent with harbor-core
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-01-11 19:08:17 +08:00
He Weiwei
4a326aa8b0
chore: delete records of scan_report
...
The report in previous scan_report records not work well the
vulnerabilities stored in the schema table, so delete the scan_report
records.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-01-08 03:39:11 +00:00
Qian Deng
642d56041d
Add san for notary cert ( #13928 )
...
Signed-off-by: DQ <dengq@vmware.com>
2021-01-08 01:00:34 +08:00
stonezdj
6b8fb8431d
Add quay registry to proxy cache
...
Update env.jinja to add quay
Signed-off-by: stonezdj <stonezdj@gmail.com>
2021-01-06 17:22:57 +08:00
Ziming Zhang
8faa76a1b6
feat(retention) refactor task manager
...
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2021-01-05 12:08:03 +08:00
Wenkai Yin(尹文开)
19ad8ad68d
Merge pull request #13823 from reasonerjt/inst-cert-home-dir
...
Replace tilde in install_cert.sh
2020-12-25 10:25:51 +08:00
prahaladdarkin
a890b28e1e
Store vulnerability data from scanner into a relational format ( #13616 )
...
feat: Store vulnerability report from scanner into a relational format
Convert vulnerability report JSON obtained from scanner into a relational format describe in:https://github.com/goharbor/community/pull/145
Signed-off-by: prahaladdarkin <prahaladd@vmware.com>
2020-12-25 08:47:46 +08:00
Wang Yan
7a8a8fa104
upgrade go version to v1.15.6 ( #13836 )
...
Signed-off-by: wang yan <wangyan@vmware.com>
2020-12-23 18:53:09 +08:00
He Weiwei
3831e82b20
refactor: remove code of admin job ( #13819 )
...
Remove code of admin job as it's not needed by scan all/gc now.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-12-22 11:48:16 +08:00
Daniel Jiang
9d99dfa82b
Replace tilde in install_cert.sh
...
This commit fixes #13287 to remove the usage of tilde as the $HOME is not available in some
cases. More details see #13287
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-12-21 20:39:34 +08:00
Qian Deng
31138f12b0
Merge pull request #13806 from ninjadq/fix_python_yaml_load
...
Fix pythom yaml load to safe_load
2020-12-21 16:04:12 +08:00
Qian Deng
9197471e70
Add Scan for internal tls ( #13810 )
...
Signed-off-by: DQ <dengq@vmware.com>
2020-12-21 15:23:11 +08:00
Wang Yan
9bc6f3cee4
fix robot account update issue ( #13741 )
...
* fix robot account update issue
enable the update method to support both v1 & v2 robot update
Signed-off-by: Wang Yan <wangyan@vmware.com>
* resolve review comments
Signed-off-by: Wang Yan <wangyan@vmware.com>
2020-12-18 20:01:26 +08:00
Wang Yan
6bc1047013
migration admin job data ( #13766 )
...
1, migrate gc and scan all schedule to schedule/task/exectuion
2, migrate gc history to task/execution
Signed-off-by: Wang Yan <wangyan@vmware.com>
2020-12-18 16:35:24 +08:00
Will Sun
4392a626f3
Merge pull request #13804 from AllForNothing/scan-all
...
Fix robot account UI issues
2020-12-18 15:48:26 +08:00
Qian Deng
64fcfeaa2f
Merge pull request #13754 from ninjadq/fix_loglevel_parsing_for_registry
...
Fix log level issue in registry
2020-12-18 14:47:15 +08:00
AllForNothing
b20cc474b3
Fix robot account UI issues
...
Signed-off-by: AllForNothing <sshijun@vmware.com>
2020-12-18 14:11:08 +08:00
DQ
234b29e170
Fix pythom yaml load to safe_load
...
Signed-off-by: DQ <dengq@vmware.com>
2020-12-16 14:59:06 +08:00
Daniel Jiang
b0c8cadad7
Add default CVE allowlist to project library ( #13770 )
...
fixes #12700
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-12-16 14:20:56 +08:00
Wenkai Yin
69808f033e
Tiny fixes for task manager
...
1. Add update time for execution
2. Add unique constraint for schedule to avoid dup records when updating policies
3. Format replication log
4. Keep the webhook handler for legacy replication jobs to avoid jobservice resending the status change request
Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-12-14 17:26:32 +08:00
He Weiwei
08580f9fec
refactor(scan): refactor scan/scan all job to task manager ( #13684 )
...
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-12-14 13:34:35 +08:00
DQ
19e8527cc1
Fix log level issue in registry
...
1. fix level issue in registry.jinja
2. add log level to registryctl
Signed-off-by: DQ <dengq@vmware.com>
2020-12-14 11:52:42 +08:00
Wenkai Yin(尹文开)
6569016d35
Merge pull request #13139 from wy65701436/migrate-gc
...
Migrate gc to task manager
2020-12-14 10:43:44 +08:00
Wang Yan
39e1a4f2b4
add extra attributes in the schedule table
...
Signed-off-by: Wang Yan <wangyan@vmware.com>
2020-12-14 02:28:52 +08:00
wang yan
1bb79d402d
update code per review comments
...
Signed-off-by: wang yan <wangyan@vmware.com>
2020-12-10 16:08:52 +08:00
Wang Yan
dba5522d0b
Migrate to task manager ( #129 )
...
1, remove the gc to new programming model
2, move api define to harbor v2 swagger
3, leverage task & execution manager to manage gc job schedule, trigger and log.
Signed-off-by: wang yan <wangyan@vmware.com>
2020-12-10 14:00:33 +08:00
DQ
d95f22448c
Add cache for exporter
...
Add timed cache for exporter
default cache time is 30s, cleanup job run every 4 hours
Signed-off-by: DQ <dengq@vmware.com>
2020-12-09 21:22:40 +08:00
DQ
f0db193895
Add prepare file for exporter
...
prepare env for exporter
Signed-off-by: DQ <dengq@vmware.com>
2020-12-09 21:22:13 +08:00
DQ
dc0047c48c
Add build script for exporter
...
- Add dockerfile
- update makefile
Signed-off-by: DQ <dengq@vmware.com>
2020-12-09 20:42:21 +08:00
Wang Yan
d2fa2e6b84
update robot secret ( #13654 )
...
* update robot secret
1, use SHA256 to generate and validate robot secret instread of symmetric encryption.
2, update the patch input object
Signed-off-by: Wang Yan <wangyan@vmware.com>
* update robot secret
1, use SHA256 to generate and validate robot secret instread of symmetric encryption.
2, update the patch input object
Signed-off-by: Wang Yan <wangyan@vmware.com>
2020-12-03 18:13:06 +08:00
Wang Yan
732e9a21cd
updates on robot accounts ( #13623 )
...
* updates on robot accounts
1, add patch method to refresh secret of a robot
2, fix robot account update issue
3, add editable attribute to handle the version 1 robot account
4, add duration for robot account
5, hide secret for get/list robot account
Signed-off-by: wang yan <wangyan@vmware.com>
* update code per review comments
1, change expirate creation func to AddDate().
2, remove the scanner duration specification, use the default value.
Signed-off-by: Wang Yan <wangyan@vmware.com>
2020-12-01 18:31:34 +08:00
DQ
907904f480
Add DB Migration code for clair cleanning
...
- Delete clair scanner if exist
- Delete report is it is scanned by clair
- Set Trivy to Default if it exist and not default scanner
Signed-off-by: DQ <dengq@vmware.com>
2020-11-29 16:19:02 +08:00
DQ
590212b485
Remove clair related code
...
- clair code in harbor core
- clair code in frontend
- clair code in robotcase
Signed-off-by: DQ <dengq@vmware.com>
2020-11-27 14:01:04 +08:00
stonezdj(Daojun Zhang)
be4e6a5985
Merge pull request #13537 from stonezdj/201118_add_more_registry_type
...
Add more registry type to proxy cache
2020-11-26 11:16:16 +08:00
Ziming Zhang
d55f55aeb9
fix(chartmuseum) compatible s3 cache fail
...
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2020-11-25 17:00:16 +08:00
He Weiwei
eb38180483
fix(quota): ignore the fail when getting reference of quota
...
1. Clean the dirty data in quota/quota_usage.
2. Ignore the fail when getting the reference of quota.
Closes #13387
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-11-24 14:50:38 +00:00
Wenkai Yin(尹文开)
fe8b628f0c
Merge pull request #13437 from ywk253100/200929_replication
...
Refactor the replication execution
2020-11-24 10:38:22 +08:00
Wenkai Yin
294385c34d
Refactor the replication execution
...
1. Use the task manager to manage the underlying execution/task
2. Use the pkg/scheduler to schedule the periodical job
3. Apply the new program model
4. Migration the old data into the new data model
Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-11-23 14:24:10 +08:00
stonezdj
e667121a34
Add more registry type to proxy cache
...
Includes: azure-acr, aws-ecr, google-gcr
Signed-off-by: stonezdj <stonezdj@gmail.com>
2020-11-18 10:38:07 +08:00
Will Sun
eca3de3489
Merge pull request #13494 from dirkmueller/lock_json_include
...
Include package.json/package-lock.json in portal image
2020-11-16 16:38:02 +08:00
stonezdj(Daojun Zhang)
fb549b2d9e
Merge pull request #13444 from wy65701436/robot2-self-mgr
...
add robot mgr
2020-11-16 11:33:33 +08:00
He Weiwei
83c07d6680
fix: ensure the role_id of role is correct ( #13476 )
...
Closes #13317
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-11-12 15:34:04 +08:00
Dirk Mueller
12adc63a48
Include package.json/package-lock.json in portal image
...
This allows Trivy and other vulnerability scanners to correctly
determine the embedded dependencies in minified harbor-portal image.
Also simplify build process by reducing the number of layers in the
final stage container image
Signed-off-by: Dirk Mueller <dirk@dmllr.de>
2020-11-11 21:21:28 +01:00
Wang Yan
3550b5e5e9
add robot mgr
...
the robot account manager to handle the CRUD
Signed-off-by: wang yan <wangyan@vmware.com>
2020-11-11 13:47:03 +08:00
Wang Yan
9723655378
update code per review comments
...
1, rename table name to permission_policy
2, rename functions name
Signed-off-by: Wang Yan <wangyan@vmware.com>
2020-11-10 18:11:31 +08:00
Wang Yan
ec15e320bf
add role permission manager for robot enhancement
...
1, add two db tables of role permission and rbac policy
2, add manager of these two tables
Signed-off-by: Wang Yan <wangyan@vmware.com>
2020-11-10 16:49:29 +08:00
He Weiwei
ebc3443da9
Merge pull request #13474 from heww/fix-issue-11892
...
fix: compute artifact size from db for schema1 manifest
2020-11-10 16:20:39 +08:00
DQ
c10a6325d8
Add deprecated msg for clair
...
Signed-off-by: DQ <dengq@vmware.com>
2020-11-10 11:39:18 +08:00
DQ
0c9faea294
Clean up Clair in prepare script
...
Signed-off-by: DQ <dengq@vmware.com>
2020-11-10 11:39:18 +08:00
DQ
8a584aff89
Clean up clair and clair-adapter in build scripts
...
1. Makefles
2. Dockerfiles
3. Installation script
4. harbor.yml template
Signed-off-by: DQ <dengq@vmware.com>
2020-11-10 11:39:18 +08:00
He Weiwei
9c8377909b
fix: compute artifact size from db for schema1 manifest
...
Closes #11892
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-11-09 12:32:07 +00:00
DQ
9152521b11
Fix: log container password expire
...
move chage command to base image
Signed-off-by: DQ <dengq@vmware.com>
2020-11-09 18:29:41 +08:00
DQ
eb470501be
Add metrics to Harbor Core
...
1. Add configs in prepare
2. Add models and config items in Core
3. Encapdulate getting metric in commom package
4. Add a middleware for global request to collect 3 metrics
Signed-off-by: DQ <dengq@vmware.com>
2020-11-03 14:33:10 +08:00
Daniel Jiang
fb687aeef8
Use pkg/token to generate JWT token
...
This commit refactors the approach to encode a token in handler of /service/token,
by reusing pkg/token to avoid inconsistency.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-10-15 16:16:44 +08:00
DQ
184e89365b
Fix internal tls config upgrade issue
...
internal tls config upgrade is not included in template, this pr is to add it.
Signed-off-by: DQ <dengq@vmware.com>
2020-09-25 09:54:31 +08:00
Wenkai Yin(尹文开)
8b9727f53f
Support store the cron type in the schedule ( #13097 )
...
There is requirement that show the cron type(daily, weekly, etc.) on the UI, this commit adds the support for storing the cron type in the schedule model
Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-09-24 16:48:56 +08:00
DQ
17f3bfccb4
Fix trivy setting in upgrading script
...
Signed-off-by: DQ <dengq@vmware.com>
2020-09-08 18:15:57 +08:00
Daniel Jiang
1b8bec3994
Merge pull request #12896 from wy65701436/fixes-12889
...
fix migration issue
2020-08-28 14:16:21 +08:00
He Weiwei
687043c298
Merge pull request #12880 from stefannica/use-exit-in-db-entrypoint
...
Use exec in harbor database entrypoint
2020-08-28 10:09:58 +08:00
Daniel Jiang
91e2779822
Fill in the icon of known artifacts in artifact controller
...
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-08-28 01:33:26 +08:00
wang yan
84094e7a5d
fix migration issue
...
fixes #12889
Before the migration script to fix the nativate repo_id issue, is has to remove the duplicate tags
from the tag table, which may caused by user in v2.0.2 to retag & repush the missing image.
Signed-off-by: wang yan <wangyan@vmware.com>
2020-08-27 19:04:39 +08:00
Daniel Jiang
7b42defb9a
Make the 2.1.0 migration SQL script idempotent
...
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-08-26 16:50:25 +08:00
Daniel Jiang
37e0aa0798
Merge pull request #12873 from wy65701436/fixes-12827
...
fix db migration issue
2020-08-26 14:42:24 +08:00
wang yan
9822e5bb15
fix db migration issue
...
fixes #12827
After user migrates Harbor from v2.0.2, user got 404 when to pull specific images, and no work after push the same images again.
Fix:
1, If the issue is caused by missing repository data, this fix can revert the missing repository data and all things should be fine.
2, If the issue is caused by missing blob data, this fix can revert the missing repository data and still left the media type of artifact
as 'UNKNOWN', which leads the meta data and build history of the image cannot be shown in UI. User can delete and push the image again to
resolve it.
Signed-off-by: wang yan <wangyan@vmware.com>
2020-08-26 12:02:20 +08:00
Ziming Zhang
ff19dd499c
fix(jobservice) redis sentinel failover hang
...
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2020-08-26 10:42:44 +08:00
Stefan Nica
1c768d0bf1
Use exec in harbor database entrypoint
...
The harbor-db pod takes a long time to terminate. Using an `exec`
command in the entrypoint ensures that Unix signals reach the
postgres process [1].
[1] https://docs.docker.com/engine/reference/builder/#exec-form-entrypoint-example
Signed-off-by: Stefan Nica <snica@suse.com>
2020-08-25 20:24:52 +02:00
Wang Yan
ad47d2f444
fix upgrade issue ( #12857 )
...
fixes #12849
1, gives a default value to blob status in the migration script, and use none to replace the empty string as
the StatusNone, that will more readable on debugging failure.
2, GC jobs marks all of blobs as StatusDelete in the mark phase, but if encounter any failure in the sweep phase,
GC job will quite and all of blobs are in StatusDelete. If user wants to execute the GC again, it will fail as the
StatusDelete cannot be marked as StatusDelete. So, add StatusDelete in the status map to make StatusDelete can be
marked as StatusDelete.
Signed-off-by: wang yan <wangyan@vmware.com>
2020-08-24 16:08:15 +08:00
Daniel Jiang
c0602b5fb3
Merge pull request #12832 from ywk253100/200820_data
...
Add id column to data_migration table
2020-08-21 19:30:05 +08:00
Daniel Jiang
4f812f7926
Merge pull request #12811 from ninjadq/fix_portal_health_check
...
Fix schema of the portal health check
2020-08-21 13:44:47 +08:00
Ted Guan
645dea36a6
Fix for duplicate webhook policy name ( #12729 )
...
Signed-off-by: guanxiatao <guanxiatao@corp.netease.com>
2020-08-20 18:02:13 +08:00
Wenkai Yin
975ef193dd
Add id column to data_migration table
...
Add id column to data_migration table and add logic to make sure there is only one data version record
Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-08-20 17:43:15 +08:00
Dirk Mueller
08a4d8efd2
Update to golang 1.14.7 ( #12809 )
...
We should use a golang that isn't having security issues.
This includes:
* go1.14.6 (released 2020/07/16) includes fixes to the go command, the
compiler, the linker, vet, and the database/sql, encoding/json,
net/http, reflect, and testing packages. See the Go 1.14.6 milestone on
our issue tracker for details.
* go1.14.7 (released 2020/08/06) includes security fixes to the
encoding/binary package. See the Go 1.14.7 milestone on our issue
tracker for details (CVE-2020-16845)
Signed-off-by: Dirk Mueller <dirk@dmllr.de>
Signed-off-by: Dirk Mueller <dmueller@suse.com>
2020-08-20 15:38:35 +08:00
DQ
e9323ca268
Fix schema of the portal health check
...
it should be https
Signed-off-by: DQ <dengq@vmware.com>
2020-08-19 15:58:51 +08:00
Wenkai Yin
0fd230c2d6
Refresh the status of execution for every status changing of task
...
Refresh the status of execution for every status changing of task to support filtering executions by status directly
Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-08-17 17:38:55 +08:00
Wenkai Yin
b1ddb5e2cc
Implement the icon API to get the icon of artifact
...
Implement the icon API to get the icon of artifact
Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-08-15 08:40:38 +08:00
Wenkai Yin
cca1dcca51
Use a separated database table to store the data version
...
Use a separated database table to store the data version.
Fixes #12747
Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-08-14 11:38:13 +08:00
Qian Deng
5dbbfa76d3
Merge pull request #12766 from ninjadq/add_log_dependency_to_trivy
...
Add log denpendency ti trivy
2020-08-13 18:23:09 +08:00
Qian Deng
85fa6654ec
Fix: Add privileged for prepare command ( #12689 )
...
Mount `/` dir in container require privilege
And this change will make `z` label useless. So remove them
Signed-off-by: DQ <dengq@vmware.com>
2020-08-13 14:55:42 +08:00
Qian Deng
78d4b54ddc
Merge pull request #12765 from ninjadq/fix_trivy_append_in_2_1_0_config
...
Fix: append trivy every time when run migrate
2020-08-13 14:47:54 +08:00
DQ
a251e90507
Add log denpendency ti trivy
...
To void trivy can not start issue
Signed-off-by: DQ <dengq@vmware.com>
2020-08-13 11:35:21 +08:00
DQ
7ba498be5b
Fix: append trivy every time run migrate
...
Signed-off-by: DQ <dengq@vmware.com>
2020-08-11 17:43:25 +08:00
Yiyang Huang
b98dc97fbd
feat: enhanced default processor
...
Signed-off-by: Yiyang Huang <huangyiyang.huangyy@bytedance.com>
2020-08-11 01:31:02 +08:00
He Weiwei
8f036c765a
chore(images): install shadow package in base images
...
The latest `photon:2.0` does not include `groupadd` and `useradd`
we need to install `shadow` package which includes these commands.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-08-10 10:23:48 +00:00
Wenkai Yin(尹文开)
d599cd98bf
Merge pull request #10455 from chlins/fix/quay-replication-adapter-refactor
...
fix(replication): refactor quay adapter to fix authorization and supp…
2020-08-10 16:37:19 +08:00
Wenkai Yin(尹文开)
e8f9fb63c0
Merge pull request #12674 from reasonerjt/standalone-db-migrator
...
Provide a standalone migrator to migrate DB schema.
2020-08-10 15:11:52 +08:00
Daniel Pacak
9397dff093
docs: Explain how to use Trivy in offline mode ( #12102 )
...
Resolves : #11985
Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
2020-08-10 08:48:04 +02:00
chlins
b765cfe0ce
fix(replication): refactor quay adapter to fix authorization and support quay.io and enterprise quay ( #10317 )
...
Signed-off-by: chlins <chlins.zhang@gmail.com>
2020-08-08 13:17:01 +08:00
Tianon Gravi
4752cac051
Remove unused "sudo" package from most images
...
Notably missing is the "log" image, which still uses sudo.
Signed-off-by: Tianon Gravi <tianon@infosiftr.com>
2020-08-06 12:44:06 -07:00
Daniel Jiang
4f94f59d2a
Provide a standalone migrator to migrate DB schema.
...
Fixes #11885
This part will not by default be packaged into release.
A README.md will be added in another commit.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-08-06 18:57:55 +08:00
Qian Deng
26dc5d1b15
Merge pull request #12557 from ninjadq/rm_expose_port
...
Remove expose port in dockerfiles
2020-08-06 14:29:51 +08:00
Wenkai Yin
d6288a43e8
Do some refine for the scheduler
...
1. Accept vendorType and vendorID when creating the schedule
2. Provide more methods in the scheduler interface to reduce the duplicated works of callers
3. Use a new ormer and transaction when creating the schedule
Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-08-05 17:43:18 +08:00
DQ
b015440074
Remove expose port in dockerfiles
...
The export is dynamical now because of introduce of internal TLS
Signed-off-by: DQ <dengq@vmware.com>
2020-08-05 10:42:46 +08:00
Qian Deng
fbef7fd088
Merge pull request #12651 from ninjadq/add_migration_2_1_0
...
Add migration 2.1.0
2020-08-03 15:59:28 +08:00
DQ
1e32792dc5
Add migration 2.1.0
...
db_max_open_comms should be 1000 if its value between 100 and 1000
Signed-off-by: DQ <dengq@vmware.com>
2020-08-03 15:17:41 +08:00
DQ
d3ab9d7c6b
Add internal tls configs for portal
...
add related file, config, command to enabled https for portal
Signed-off-by: DQ <dengq@vmware.com>
2020-07-31 12:10:47 +08:00
Qian Deng
a2112bfa40
Merge pull request #12539 from ninjadq/core_config_port
...
Fix: beego app config port hardcode
2020-07-27 17:21:18 +08:00
DQ
d7618a6274
Fix: beego app config port hardcode
...
the port should be flexible depend on the internal tls
Signed-off-by: DQ <dengq@vmware.com>
2020-07-27 15:35:43 +08:00
Steven Zou
ee35e1ecc6
Merge pull request #12507 from chlins/fix/preheat-update-instance
...
fix(preheat): fix preheat handler PingInstance and UpdateInstance
2020-07-20 17:45:24 +08:00
Steven Zou
46d7434d0b
Merge pull request #12473 from ywk253100/200706_scheduler
...
Refactor the scheduler with the task manager mechanism
2020-07-20 15:53:14 +08:00
chlins
78927af032
fix(preheat): fix preheat handler PingInstance and UpdateInstance
...
Signed-off-by: chlins <chlins.zhang@gmail.com>
2020-07-20 15:42:24 +08:00
Wenkai Yin
4dc4b6728c
Refactor the scheduler with the task manager mechanism
...
Refactor the scheduler with the task manager mechanism, this will reduce the duplicate code
Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-07-20 14:03:15 +08:00
Ziming Zhang
8857e89e40
feature(redis) support redis sentinel
...
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2020-07-19 21:19:03 +08:00
Wang Yan
bad8f026fc
upgrade golang to v1.14.5 ( #12489 )
...
Signed-off-by: wang yan <wangyan@vmware.com>
2020-07-16 16:20:54 +08:00