Commit Graph

605 Commits

Author SHA1 Message Date
慕薇疯魔
1039800fa9 1. Migration for policy manager.
2. Clear test database after run test.
3. UT for policy manager and dao.

Signed-off-by: 慕薇疯魔 <kfanjian@gmail.com>
2019-03-11 16:01:31 +08:00
Qian Deng
b0f158c4c8 Add migratior script
Add migrator template and script

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-03-08 16:46:14 +08:00
Qian Deng
7b7cb82f86 Enhance: refactor the format of harbor.yml
refactor the format of the harbor.yml configuration items

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-03-08 16:46:13 +08:00
Qian Deng
ab7c81dac6 Fix: the adminserver caused regression
Remove some code related to adminserver
Fix some issues by adminserver removeing

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-03-08 16:46:13 +08:00
Qian Deng
5f80fe7b8a Refacotr the prepare script base on the proposal https://github.com/goharbor/community/pull/22
Signed-off-by: Qian Deng <dengq@vmware.com>
2019-03-08 16:46:13 +08:00
cd1989
8732a20709 Rewrite registry manager with new interface
Signed-off-by: cd1989 <chende@caicloud.io>
2019-02-27 11:54:04 +08:00
Wenkai Yin
b530905b40 Update the upgrade sql to fix #6698
This commit updates the upgrade sql script to fix #6698: cannot recreate the same name replication policy after it is deleted

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-02-26 15:15:08 +08:00
wang yan
91aa67a541 Update expiration variable name to expiresat/tokenduration
Signed-off-by: wang yan <wangyan@vmware.com>
2019-02-25 11:55:42 +08:00
wang yan
36a778b482 Update expiration schema to bigint and default unit to minute
Signed-off-by: wang yan <wangyan@vmware.com>
2019-02-22 18:42:43 +08:00
wang yan
47a09b5891 add expiration of robot account
This commit is to make the expiration of robot account configurable

1, The expiration could be set by system admin in the configuation page or
by /api/config with robot_token_expiration=60, the default value is 30 days.
2, The expiration could be shown in the robot account infor both on UI and API.

Signed-off-by: wang yan <wangyan@vmware.com>
2019-02-22 18:42:34 +08:00
stonezdj
0cba36d79f Remove everything of adminserver
Signed-off-by: stonezdj <stonezdj@gmail.com>
2019-02-22 16:34:39 +08:00
He Weiwei
f19ad5c522 Disable validation for registry (#6993)
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-02-22 10:30:40 +08:00
Daniel Jiang
1832699e93 Bump up the migrate tool of notary
fixes #5863
The migrate binary that we include in notary is quite out dated.
Additionally it introduced a breaking change, more details see #5863

In this commit a go program was added to workaround this issue to ensure the
migration process works, and refined bootstrap scripts and make process accordingly.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-02-21 00:36:24 -08:00
Steven Zou
056ccbac41
Merge pull request #6931 from ninjadq/bump_chart_version
Enhhance: bump chartmuseum version to 0.8.1
2019-02-20 14:51:07 +08:00
Wenkai Yin
38d3c33ce4
Merge pull request #6729 from stonezdj/ref_admin_replace_backup
Refactor adminserver stage3
2019-02-19 13:52:46 +08:00
Wenkai Yin
696264bee9 Run logrotate as user 10000 to avoid issue #6895 (#6913)
This commit fixes issue #6895 by running logrotate with user 10000

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-02-19 13:18:45 +08:00
stonezdj
c9a8de9002 copy migration script to core container instead of mount volumn
Signed-off-by: stonezdj <stonezdj@gmail.com>
2019-02-18 14:06:19 +08:00
stonezdj
1ae5126bb4 Refactor adminserver stage 3: replace config api and change ut settings
Signed-off-by: stonezdj <stonezdj@gmail.com>
2019-02-18 14:06:19 +08:00
Qian Deng
4a4ebc2fba Enhhance: bump chartmuseum version to 0.8.1
bump the version of chartmuseum to 0.8.1

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-02-15 15:19:53 +08:00
Daniel Jiang
81639e2110
Merge pull request #6865 from wy65701436/remove-token
Remove the token attribute from robot table
2019-02-13 19:23:06 +08:00
Wenkai Yin
cd57f70f2f
Merge pull request #6901 from wy65701436/upgrade-registry-270
Upgrade registry binary to v2.7.1
2019-02-13 19:01:37 +08:00
Yan
161f2127e2
Fix format of makefile (#6909)
Signed-off-by: wang yan <wangyan@vmware.com>
2019-02-13 10:54:32 +08:00
wang yan
c77b387c53 Upgrade registry binary to v2.7.0
Signed-off-by: wang yan <wangyan@vmware.com>
2019-02-13 10:24:08 +08:00
Yan
5412e581de
Add a flag judging on building migrator (#6905)
This commit is to add a flag judging when to build image of migator, which is not necessary.

Signed-off-by: wang yan <wangyan@vmware.com>
2019-02-12 20:33:42 +08:00
wang yan
5d6a28d73e Remove the token attribute for robot table
This commit is to remove the token attribute as harbor doesn't store the token in DB.

Signed-off-by: wang yan <wangyan@vmware.com>
2019-01-30 23:56:23 +08:00
He Weiwei
3f8e06a8bc Support master role for project member create and update apis (#6780)
* Support master role for project member create and update apis

Signed-off-by: He Weiwei <hweiwei@vmware.com>

* Fix description for role_id in swagger.yaml

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-01-23 14:56:23 +08:00
Daniel Jiang
a1d4bfd332
Merge pull request #6344 from reasonerjt/bump-up-golang
Bump up golang to 1.11.2
2019-01-11 16:15:59 +08:00
Daniel Jiang
5d59d6fab8 Bump up golang to 1.11.2
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-01-11 14:44:32 +08:00
Daniel Jiang
80af81154c
Merge pull request #6702 from wy65701436/robot-db-scheme
Add DB table for robot account
2019-01-10 14:25:58 +08:00
wang yan
db09f9f101 Update token length and upper case the sql key words
Signed-off-by: wang yan <wangyan@vmware.com>
2019-01-09 10:00:54 +08:00
wang yan
362a0638d0 Add DB table for robot account
This commit is to add DB scheme for robot account and update the db orm releated.

Signed-off-by: wang yan <wangyan@vmware.com>
2019-01-08 18:46:16 +08:00
Wenkai Yin
edcbb8f29f Update the upgrade sql to rename the duplicate records
Rename the "name" colume in table "replication_policy" and "replication_target" before adding the "UNIQUE" constraint to avoid the upgrade failure

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-01-04 14:55:33 +08:00
overdogwatch
a8d0ab1a21 Update Dockerfile
I guess that the purpose of this check is to verify that the container is listening on port 10514. Healthcheck default timeout is 30 sec. In places where the DNS resolver is not working properly, this check could take more than 30 sec, which leads to decide that the container health is unhealthy. I advise you to add to your check the option n, which prevents netstat trying to determine the symbolic host.

Signed-off-by: overdogwatch <overdogwatch@gmail.com>
2018-12-27 09:47:48 +02:00
Qian Deng
af7b61ebd5 Add customized config file for proxy
This is a solution for product that using harbor as proxy to add customized location

Signed-off-by: Qian Deng <dengq@vmware.com>
2018-12-26 13:35:07 +08:00
Meina Zhou
d45ccbbb29 add developer center in swagger ui way
Signed-off-by: Meina Zhou <meinaz@vmware.com>
2018-12-13 15:17:38 +08:00
Daniel Jiang
481bdb01a2
Merge pull request #6409 from ninjadq/upgrade_config_2_170
Feat: Upgrade harbor.cfg from 1.6.0 to 1.7.0
2018-12-04 01:05:59 -08:00
Qian Deng
7647e688fd Feat: Upgrade harbor.cfg from 1.6.0 to 1.7.0
Add script to upgrade harbor.cfg

Signed-off-by: Qian Deng <dengq@vmware.com>
2018-12-04 10:36:11 +08:00
Steven Zou
ec2ad4d0b8
Merge pull request #6093 from cd1989/replication-record-id
Add op uuid to image replication
2018-11-30 14:54:43 +08:00
Wenkai Yin
9d5cf57373 Check the existence of name when creating replication rule and fix bugs in testing library (#6381)
1. Fix #5102 by checking the existence of name when creating/editing replication rule
2. Add unique constraint to the name of replication policy and target
3. Fix bugs of testing library

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2018-11-30 13:32:20 +08:00
James Zabala
399c9c44d8
Merge pull request #6384 from SDBrett/photon-build
Corrected errors in photon build
2018-11-29 18:08:52 -05:00
Brett Johnson
1eb64e43ef added stage alias
resolve build error 'invalid from flag value 0: repository sha256 not found: does not exist or no pull access'

Signed-off-by: Brett Johnson <brett@sdbrett.com>
2018-11-29 16:02:26 +11:00
peimingming
c67fdc40f5 Support store job log in DB (#6144)
Signed-off-by: peimingming <peimingming@corp.netease.com>
2018-11-28 15:09:29 +08:00
Wenkai Yin
fefb955cfe Drop all capabilities when starting containers
Drop all capabilities when starting containers by modifying docker-compose files to avoid security issue

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2018-11-23 15:38:21 +08:00
stonezdj(Daojun Zhang)
ae007c2a49
Merge pull request #6247 from stonezdj/trust_cert2
Install custom cert for clair, registry, chartmuseum
2018-11-12 14:07:34 +08:00
Daniel Jiang
555c0cb181
Merge pull request #6220 from stonezdj/add_util_linux
Fix issue when query psql cli failed on more command not found
2018-11-09 17:39:47 +08:00
stonezdj
0a72f3729a Install custom cert for clair, registry, chartmuseum
Signed-off-by: stonezdj <stonezdj@gmail.com>
2018-11-09 15:03:03 +08:00
Daniel Jiang
dd9bd10e68
Merge pull request #6238 from ywk253100/181108_proxy
Replace ui with core for no_proxy in harbor.cfg
2018-11-09 14:19:13 +08:00
Steven Zou
b3c87673a6
Merge pull request #6215 from steven-zou/logger_framework_job_service
Build logger framework to support configurable loggers/sweepers/getters
2018-11-08 18:15:36 +08:00
Wenkai Yin
64cb507421 Replace ui with core for no_proxy in harbor.cfg
As we split UI into two components: portal and core, the value of no_proxy needs to be updated

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2018-11-08 16:10:26 +08:00
Steven Zou
093bf9fc8a Update the job service config yaml in the installation template
Signed-off-by: Steven Zou <szou@vmware.com>
2018-11-08 10:40:39 +08:00
stonezdj
8c37b0877a Fix issue when query psql cli failed on more command
Signed-off-by: stonezdj <stonezdj@gmail.com>
2018-11-06 13:09:20 +08:00
Daniel Jiang
1e9b34325c Make rendered files less visible
Some configuration files and env files contain sensitive information,
they should not be readable by any user by default.

This commit updates the `prepare` script to update the mask of the
rendered files.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-11-05 15:06:00 +08:00
James Zabala
b58ff42dff
Merge pull request #6184 from wy65701436/bump-up-clair
Build clair version into clair image
2018-11-02 17:04:25 -04:00
Daniel Jiang
ec01a97eb8 Clair image should accept parms
Update the entrypoint to allow the image accept other parms,
to help debug in the future.

If replace "$*" with "$@" only one parm will be passed to dumbinit

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-11-01 22:25:55 -07:00
wang yan
072127a70c Build clair version into clair image
This commit is to add the clair_version into the harbor images, then clair
will use it in the user-agent, and helpful for the debugging.

Signed-off-by: wang yan <wangyan@vmware.com>
2018-10-31 16:29:06 +08:00
Daniel Jiang
39b4d011c7 Not submit scan all job when core container starts
Fixes #6115

As for the change in migration sql file, in 1.7 we'll switch to
jobservice for scheduling "scan all" job.  To avoid inconsistency,
this item will be reset and user will need to configure the policy again.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-10-25 19:01:52 +08:00
Steven Zou
3b76a960e1
Merge pull request #6039 from stonezdj/refact_5996
Refactor capacity
2018-10-24 10:50:11 +08:00
Stéphane Albert
0d44e4f535 Send 308 status on redirect from http to https
Modify nginx configuration to use 308 instead of 301 on the http to
https redirect.
Fix problems with some clients on POST requests that are transformed to
GET on 301 redirect (per HTTP 1.1 standard).
See [RFC7538](https://tools.ietf.org/html/rfc7538).

Signed-off-by: Stéphane Albert <sheeprine@oh.its.fake.nullplace.com>
2018-10-23 20:25:35 +02:00
Christian Witts
e9c01255da
Fix install issue with default hostname commented
Update the `grep` filter to anchor at the start of the line
and allow for whitespace characters, in order to correctly
determine the hostname being set if the default is merely
commented out and the custom one added, instead of overridden.

Fixes #6117

Signed-off-by: Christian Witts <cwitts@gmail.com>
2018-10-23 14:55:27 +02:00
Daniel Jiang
6f4f941854 Fix permission issue in rsyslog container.
This commit fixes the permission issues introduced after migration to photon:2.0 base image.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-10-21 21:45:42 -07:00
陈德
1ffd9d8fba Add op uuid to image replication
Signed-off-by: 陈德 <chende@caicloud.io>
2018-10-21 23:55:57 +08:00
Daniel Jiang
3d09089a9c Rebuild Harbor images based on photon:2.0 (#6054)
Make necessary change to make things work with photon 2.0 docker image.
Remove distro-sync to mitigate the build issue and add `--pull` to docker build
command to make sure the latest photon:2.0 will be pulled during build process.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-10-18 11:03:28 +08:00
Daniel Jiang
3c577d588c
Merge pull request #6063 from wy65701436/fix-dns
Use docker official way to unset dns search
2018-10-17 17:08:00 +08:00
wang yan
f44ff2e4c3 Remove the env GODEBUG=netdns=cgo
This env is the workaroud of dns resolver on golang 1.7.3.
Remove it is bacause of harbor is using golang 1.9.2, the bug
has already been fixed.

Signed-off-by: wang yan <wangyan@vmware.com>
2018-10-17 16:11:25 +08:00
Yan
a5e7ac9164
Upgrade notary complile golang version to 1.9.4 (#6064)
This commit is to upgrade the golang version to 1.9.4, it because a
bug of golang 17.3 could introduce one dns resolver issue for harbor
mentioned by #6031.

The bug of golang is https://github.com/golang/go/issues/15419, it makes
harbor containers to lookup 'endpoint.' firstly which may cause network
issue.

Signed-off-by: wang yan <wangyan@vmware.com>
2018-10-17 16:04:14 +08:00
wang yan
bad68c5429 Use docker official way to unset dns search
According docker official document, use 'dns_search= .' in the docker
compose file if you don't wish to set the search domain.

https://docs.docker.com/v17.09/engine/userguide/networking/default_network/configure-dns/

Signed-off-by: wang yan <wangyan@vmware.com>
2018-10-17 14:27:29 +08:00
stonezdj
0278981523 Change admin server to core in jobservice
Signed-off-by: stonezdj <stonezdj@gmail.com>
2018-10-16 19:23:12 +08:00
Yan
08ae5f2f37
Limit dns search in harbor containers (#6057)
This commit is to set dns search to null in the harbor containers,
that means the dns search domains of docker host doesn't impact
the network IO in the containers.

If do not set this, Harbor notary-server and notary-signer are resolving
the "mysql" alias to the resolv.conf search path instead of to "mysql."
for the notary-db bridge IP, see #6031.

Signed-off-by: wang yan <wangyan@vmware.com>
2018-10-16 18:34:36 +08:00
stonezdj(Daojun Zhang)
b764033fc9
Merge pull request #6007 from stonezdj/refact_5998
Change admin server to core in jobservice
2018-10-15 17:52:24 +08:00
stonezdj
79bac7a64e Change admin server to core in jobservice
Signed-off-by: stonezdj <stonezdj@gmail.com>
2018-10-15 14:56:18 +08:00
Daniel Jiang
c8cb2f8481 Create shcema migration table in DB container
The migrate tool will try to create table schema_migration upon opening
the connection to DB.  This will cause error when there are multiple
instance of adminserver trying to access the migrator upon start.
This commit move the creation of the table during the initialization of
the DB container.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-10-12 17:17:31 +08:00
Daniel Jiang
31096a35af Run chown to job log directory
This commit revoke part of the change introduced in commit #1fc4142, by
calling chown to job log directory within the container when the job
service bootstraps.  The reason is we are seeing permission issue in
helm-chart deployment, and we want to reduce effort to handle the
permission on different deployment approaches.

There are some code in `prepare` script to change the ownership of the
JOB_LOG directory, it will be left for now to avoid regression in VIC
integration.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-09-22 12:05:37 +08:00
Qian Deng
88bb461314 Reactor backend api for authrization
1. Change backend api
2. Change frontend api
3. Change the proxy config file

Signed-off-by: Qian Deng <dengq@vmware.com>
2018-09-21 14:03:17 +08:00
Qian Deng
7873a0312a Rename harbor-ui to harbor-core
1. Update the nginx.conf
2. Update Makefile
3. Update docker-compose
4. Update image name
5. Rename folder ui to core
6. Change the harbor-ui's package name to core
7. Remove unused static file on harbor-core
8. Remove unused code for harbor-portal

Signed-off-by: Qian Deng <dengq@vmware.com>
2018-09-19 16:35:13 +08:00
Wenkai Yin
89893779fb Support configuring sslmode for the connection of database (#5861)
The sslmode of the connection with postgresql is hardcoded as "disable" currently, this commit expose it as an environment variable so that users can configure it

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2018-09-14 13:05:05 +08:00
Daniel Jiang
36ab8a5bf1 Add depdends_on to portal container
Fixes #5878

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-09-13 11:00:21 +08:00
Qian Deng
ac6c9d79ba Remove nodeclarity
Remove the nodeclarity container related code and ui_builder parameter no longer needed when build clarity

Signed-off-by: Qian Deng <dengq@vmware.com>
2018-09-08 20:33:21 +08:00
Qian Deng
097da4bb8d Fix typo
adminiserver to adminserver

Signed-off-by: Qian Deng <dengq@vmware.com>
2018-09-07 16:20:03 +08:00
Qian Deng
ba4762095f Update dockerfile of portal based on angular6 upgrade
1. Update entrypoint to reflect angular 6 upgrade

Signed-off-by: Qian Deng <dengq@vmware.com>
2018-09-07 15:06:15 +08:00
Qian Deng
db8d7b9c1c Merge branch 'seprate_harbor_portal_from_harbor_core' into angular6 2018-09-07 14:56:16 +08:00
Qian Deng
d797c50438 Fix trivial issues about rename ui_ng to portal
Update ui_ng to portal

Signed-off-by: Qian Deng <dengq@vmware.com>
2018-09-07 13:25:41 +08:00
Qian Deng
870653a5fb Update nginx config to redirect traffic to specific backend
1. Update nginx.conf file
2. Update photon makefile
3. Update global makefile

Signed-off-by: Qian Deng <dengq@vmware.com>
2018-09-07 13:21:27 +08:00
Qian Deng
dc21f3f5e2 Add container for harbor-portal
1. Add dockerfile for building harbor-portal
2. change the name from ui_ng to harbor-portal

Signed-off-by: Qian Deng <dengq@vmware.com>
2018-09-07 13:20:08 +08:00
Meina Zhou
a330d4e116 upgrade to angular 6
Signed-off-by: Meina Zhou <meinaz@vmware.com>
2018-09-07 11:30:00 +08:00
Yan
4eba01fc31 Clean make file unused code and unify docker build method (#59)
Signed-off-by: Yan <wangyan@vmware.com>
2018-09-04 17:18:15 +08:00
Daniel Jiang
768f165877
Merge pull request #5771 from wy65701436/deprecate-make-dev
Deprecate dockerfiles in make/dev
2018-09-04 12:58:52 +08:00
Daniel Jiang
823a9d11e9 Bump Clair to v2.0.5
The PR to fix the Alpine issue has been merged to Clair's release-2.0
branch, and released v2.0.5.
This commit updates Harbor to include that change and re-enable
Clair's updaters by default.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-09-02 12:27:04 +08:00
wang yan
9bb7902003 Deprecate dockerfiles in make/dev
Signed-off-by: wang yan <wangyan@vmware.com>
2018-08-31 15:26:26 +08:00
wang yan
f8b964d8cf Extend configuration length to 1024 in DB
Signed-off-by: wang yan <wangyan@vmware.com>
2018-08-31 13:42:24 +08:00
Steven Zou
3e241be34f
Merge pull request #5739 from steven-zou/fix_s3_storage_issue
Fix issues related with chart storage
2018-08-28 15:16:34 +08:00
Daniel Jiang
c4eaf25ed1
Merge pull request #5742 from reasonerjt/remove-compose-ha
Remove reference of docker-compsoe based HA
2018-08-28 15:09:22 +08:00
Steven Zou
43ecf62c25 Fix issues related with chart storage
- inject custom CA bundle into chart repo
- update prepare script to inject credentials

Signed-off-by: Steven Zou <szou@vmware.com>
2018-08-28 14:10:50 +08:00
Daniel Jiang
e1153eec0a Remove reference of docker-compsoe based HA
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-08-28 14:04:44 +08:00
wang yan
854f606f63 upgrade notary to latest release v0.6.1
Move the notary-server and notary signer into ./notary/release-${notaryversion} as this will not impact the
release branches, the binaries in ./notary are v0.5.1.

Signed-off-by: wang yan <wangyan@vmware.com>
2018-08-28 13:51:55 +08:00
James Zabala
141f6056e7
Merge pull request #5700 from xxxdreamer/test
add some comments to harbor.cfg
2018-08-24 17:18:28 -04:00
Wenkai Yin
2c5b06350e
Merge pull request #5716 from reasonerjt/update-go-import-path
Update import path in go code
2018-08-23 19:02:50 +08:00
Daniel Jiang
dcf4e2ee78 Update import path in go code
vmware -> goharbor

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-08-23 17:50:53 +08:00
unknown
9b94800216 To make the DCO bot happy^_^
Signed-off-by: unknown <cyz.dreamer@gmail.com>
2018-08-23 14:43:54 +08:00
unknown
7dd68c992f add some comments to harbor.cfg
Signed-off-by: unknown <cyz.dreamer@gmail.com>
2018-08-23 14:38:47 +08:00
Daniel Jiang
78bddd831a Temporarily disable updaters of clair
Set the updater interval to "0" to mitigate the impact of Apline URL
change that cause clair keep polling vuln data.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-08-23 14:00:37 +08:00