Commit Graph

847 Commits

Author SHA1 Message Date
Qian Deng
85fdf885e8
Merge pull request #7386 from ninjadq/fix_http_scheme_on_install_sh
Fix: grep https not work
2019-04-17 16:29:35 +08:00
Qian Deng
c1e676ad99 Add migration script from 1.7.0 to 1.8.0
Add jinja2 to migrator
Add template to migrator
Add config upgrading script

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-04-17 15:14:54 +08:00
Wenkai Yin
6e0d892963 Support creating project with service account
This commit introduces a solution to workaround the restriction of project creation API: only normal users can create projects

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-04-17 14:50:32 +08:00
Qian Deng
8a308a63e2 Set default mode to https
Default mode to https
The cert file value is set to previous version style

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-04-16 15:50:12 +08:00
Qian Deng
aad63e7ae5 Add compatibility config
Add compatibility config in registry.yml

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-04-16 15:44:47 +08:00
Qian Deng
9ddfd259d3 Fix bug when rendering port in proxy
rendering 443 when https enabled
rendering 4443 when notary enabled

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-04-16 11:20:13 +08:00
Qian Deng
0e8436263f Fix: grep https not work
Add space before right braket

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-04-15 14:32:36 +08:00
cd1989
27c1bdc5e2 Fix make prepare problem
Signed-off-by: cd1989 <chende@caicloud.io>
2019-04-10 19:48:13 +08:00
Daniel Jiang
0a2343f542 Support secret for docker CLI
As CLI does not support oauth flow, we'll use secret for help OIDC user
to authenticate via CLI.
Add column to store secret and token, and add code to support
verify/refresh token associates with secret.  Such that when the user is
removed from OIDC provider the secret will no longer work.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-04-10 19:38:11 +08:00
Daniel Jiang
a243c7f05a
Merge pull request #7330 from wy65701436/reg-271
Patch regsitry v2.7.1 with fix on distribution issue 2819
2019-04-10 09:45:56 +08:00
wang yan
469473b31c Patch regsitry v2.7.1 with fix on distribution issue 2819
This commit is to build a regsitry bases on v2.7.1 code and introduces
an fix on issue #2819, this is a P0 bug on v2.7.1 which causes GCS doesn't
work well on v2.7.1

For more details, refer to https://github.com/docker/distribution/pull/2821

Signed-off-by: wang yan <wangyan@vmware.com>
2019-04-09 18:42:29 +08:00
cd1989
92b04cffd5 Fix make prepare problem
Signed-off-by: cd1989 <chende@caicloud.io>
2019-04-09 17:02:09 +08:00
Wenkai Yin
580674f3da Merge remote-tracking branch 'upstream/master' into 190409_sync 2019-04-09 17:01:09 +08:00
Qian Deng
deba378842 Enhance: Refacotr Registry config file
1. Refactor registry configs
2. cp gcs keyfile is exist

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-04-09 14:40:41 +08:00
Qian Deng
74c4e243e3 Refator the host related config
1. Refactor host config
2. Refactor certiface config
3. Add port config
4. Add log info config

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-04-09 12:55:07 +08:00
Qian Deng
fef7702e9a Enhance: Refactor the config parse logic
Refactor the config parse logic

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-04-09 12:55:07 +08:00
Qian Deng
ac1b7bb1fb Enhance: remove the reload key item
Remove the reload_config item in config file

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-04-09 12:55:07 +08:00
Qian Deng
cd65b68ab3 Enhance: Refactor the format
Refactor the whole structure of harbor.yml

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-04-09 12:55:07 +08:00
Wenkai Yin
4ffa0c3da0 Upgrade the replication_job table
This commit migrates the replication_job table, add one execution record and one task record for each job

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-04-08 22:23:53 +08:00
Wenkai Yin
e8fe2aa60c Upgrade the registry and replication policy tables
Upgrade the registry and replication tables in database

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-04-05 13:25:00 +08:00
Wenkai Yin
b66b1f341e Merge remote-tracking branch 'upstream/master' into 190404_sync 2019-04-04 14:55:09 +08:00
Yan
0de5999f52 add the controller for ocdi onboard user
Signed-off-by: wang yan <wangyan@vmware.com>
2019-04-03 09:52:22 +08:00
Wenkai Yin
4484bca756 Fix replication related issues
1. Add operation property for tasks
2. Add trigger property for executions
3. Update the getting registry info API to allow passing 0 as ID to get the info of local Harbor registry

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-04-02 14:26:17 +08:00
Qian Deng
15c2c9048f Fix: clair env file should using empty string is not exist
This is quick fix, further fixs will in the config refactor PR

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-04-01 18:12:34 +08:00
Wenkai Yin
8c7b63bac2
Merge pull request #7248 from ywk253100/190326_event
Add event based trigger and scheduled trigger
2019-03-29 14:58:09 +08:00
Wenkai Yin
4f8e283e8e Add event based trigger and scheduled trigger
This commit implements the event based trigger and scheduled trigger in replilcation

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-03-29 13:48:34 +08:00
Qian Deng
cb846bd936 Fix: copy upstream file to nginx config file
Copy notary.upstream.conf to nginx config file

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-03-27 17:56:31 +08:00
Wenkai Yin
017bba8dc1 Merge remote-tracking branch 'upstream/master' into 190327_sync 2019-03-27 11:43:51 +08:00
wang yan
1ba1c5726a Upgrade node version to 10.15
To fix the issue https://lists.debian.org/debian-devel-announce/2019/03/msg00006.html,
it needs to upgrade node to 10.15, which has pitched the fix.

Signed-off-by: wang yan <wangyan@vmware.com>
2019-03-27 10:47:13 +08:00
Qian Deng
215149fc2f
Merge pull request #7206 from ninjadq/update_pyyaml_version
Fix: upgrade pyyaml version to 4.2b1
2019-03-26 15:11:42 +08:00
Qian Deng
df2425a02b Fix: upgrade pyyaml version to 4.2b1
Because previous version has security issue

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-03-22 19:05:16 +08:00
Qian Deng
e538a4c448 Fix: install.sh failure if $host env is setted (#7203)
Fix sed replace cmd

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-03-22 19:04:40 +08:00
Wenkai Yin
49cf50adb1 Merge remote-tracking branch 'upstream/master' into 190324_sync
Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-03-22 15:55:52 +08:00
Qian Deng
ba4764c61d
Merge pull request #6755 from ninjadq/refactor_prepare
Refactor the prepare script
2019-03-22 14:54:30 +08:00
Qian Deng
fcdab4d4af Fix: packaging offline in new prepare
This new prepare script now support offline packaging

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-03-22 13:56:15 +08:00
Meina Zhou
130e132f86 Merge branch 'master' into replication_ng
Signed-off-by: Meina Zhou <meinaz@vmware.com>
2019-03-21 14:16:33 +08:00
Qian Deng
0c84751a10 Enhance: Refactor the notary structure
1. Update notary template on docker-compose
2. automatic generate cert if not exist

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-03-18 10:20:42 +08:00
Qian Deng
93af296eeb Enhance: refactor the mount dirs and workflow of generate cert
mount a temp dir input for all input files and configs
generated secrets file stored in data volumns keys dir
certs file stored in data volumns nginx dir

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-03-15 15:51:11 +08:00
Che-Wei Lin
7aa00aee87 fix hostname command not found (#7045)
Signed-off-by: mycroftlin <mycroftlin@tencent.com>
2019-03-15 10:52:47 +08:00
peimingming
4efad287ce Add execution and hooks
Signed-off-by: peimingming <peimingming@corp.netease.com>
2019-03-13 09:35:01 +08:00
Wenkai Yin
53529b4b1b Merge the sql script into one file
Keep all the sql script for v1.8 in only one file

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-03-11 20:38:45 +08:00
Wenkai Yin
772367498f Merge remote-tracking branch 'upstream/master' into 190311_sync 2019-03-11 20:34:49 +08:00
慕薇疯魔
1039800fa9 1. Migration for policy manager.
2. Clear test database after run test.
3. UT for policy manager and dao.

Signed-off-by: 慕薇疯魔 <kfanjian@gmail.com>
2019-03-11 16:01:31 +08:00
Qian Deng
b0f158c4c8 Add migratior script
Add migrator template and script

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-03-08 16:46:14 +08:00
Qian Deng
7b7cb82f86 Enhance: refactor the format of harbor.yml
refactor the format of the harbor.yml configuration items

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-03-08 16:46:13 +08:00
Qian Deng
ab7c81dac6 Fix: the adminserver caused regression
Remove some code related to adminserver
Fix some issues by adminserver removeing

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-03-08 16:46:13 +08:00
Qian Deng
5f80fe7b8a Refacotr the prepare script base on the proposal https://github.com/goharbor/community/pull/22
Signed-off-by: Qian Deng <dengq@vmware.com>
2019-03-08 16:46:13 +08:00
cd1989
8732a20709 Rewrite registry manager with new interface
Signed-off-by: cd1989 <chende@caicloud.io>
2019-02-27 11:54:04 +08:00
Wenkai Yin
b530905b40 Update the upgrade sql to fix #6698
This commit updates the upgrade sql script to fix #6698: cannot recreate the same name replication policy after it is deleted

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-02-26 15:15:08 +08:00
wang yan
91aa67a541 Update expiration variable name to expiresat/tokenduration
Signed-off-by: wang yan <wangyan@vmware.com>
2019-02-25 11:55:42 +08:00
wang yan
36a778b482 Update expiration schema to bigint and default unit to minute
Signed-off-by: wang yan <wangyan@vmware.com>
2019-02-22 18:42:43 +08:00
wang yan
47a09b5891 add expiration of robot account
This commit is to make the expiration of robot account configurable

1, The expiration could be set by system admin in the configuation page or
by /api/config with robot_token_expiration=60, the default value is 30 days.
2, The expiration could be shown in the robot account infor both on UI and API.

Signed-off-by: wang yan <wangyan@vmware.com>
2019-02-22 18:42:34 +08:00
stonezdj
0cba36d79f Remove everything of adminserver
Signed-off-by: stonezdj <stonezdj@gmail.com>
2019-02-22 16:34:39 +08:00
He Weiwei
f19ad5c522 Disable validation for registry (#6993)
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-02-22 10:30:40 +08:00
Daniel Jiang
1832699e93 Bump up the migrate tool of notary
fixes #5863
The migrate binary that we include in notary is quite out dated.
Additionally it introduced a breaking change, more details see #5863

In this commit a go program was added to workaround this issue to ensure the
migration process works, and refined bootstrap scripts and make process accordingly.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-02-21 00:36:24 -08:00
Steven Zou
056ccbac41
Merge pull request #6931 from ninjadq/bump_chart_version
Enhhance: bump chartmuseum version to 0.8.1
2019-02-20 14:51:07 +08:00
Wenkai Yin
38d3c33ce4
Merge pull request #6729 from stonezdj/ref_admin_replace_backup
Refactor adminserver stage3
2019-02-19 13:52:46 +08:00
Wenkai Yin
696264bee9 Run logrotate as user 10000 to avoid issue #6895 (#6913)
This commit fixes issue #6895 by running logrotate with user 10000

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-02-19 13:18:45 +08:00
stonezdj
c9a8de9002 copy migration script to core container instead of mount volumn
Signed-off-by: stonezdj <stonezdj@gmail.com>
2019-02-18 14:06:19 +08:00
stonezdj
1ae5126bb4 Refactor adminserver stage 3: replace config api and change ut settings
Signed-off-by: stonezdj <stonezdj@gmail.com>
2019-02-18 14:06:19 +08:00
Qian Deng
4a4ebc2fba Enhhance: bump chartmuseum version to 0.8.1
bump the version of chartmuseum to 0.8.1

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-02-15 15:19:53 +08:00
Daniel Jiang
81639e2110
Merge pull request #6865 from wy65701436/remove-token
Remove the token attribute from robot table
2019-02-13 19:23:06 +08:00
Wenkai Yin
cd57f70f2f
Merge pull request #6901 from wy65701436/upgrade-registry-270
Upgrade registry binary to v2.7.1
2019-02-13 19:01:37 +08:00
Yan
161f2127e2
Fix format of makefile (#6909)
Signed-off-by: wang yan <wangyan@vmware.com>
2019-02-13 10:54:32 +08:00
wang yan
c77b387c53 Upgrade registry binary to v2.7.0
Signed-off-by: wang yan <wangyan@vmware.com>
2019-02-13 10:24:08 +08:00
Yan
5412e581de
Add a flag judging on building migrator (#6905)
This commit is to add a flag judging when to build image of migator, which is not necessary.

Signed-off-by: wang yan <wangyan@vmware.com>
2019-02-12 20:33:42 +08:00
wang yan
5d6a28d73e Remove the token attribute for robot table
This commit is to remove the token attribute as harbor doesn't store the token in DB.

Signed-off-by: wang yan <wangyan@vmware.com>
2019-01-30 23:56:23 +08:00
He Weiwei
3f8e06a8bc Support master role for project member create and update apis (#6780)
* Support master role for project member create and update apis

Signed-off-by: He Weiwei <hweiwei@vmware.com>

* Fix description for role_id in swagger.yaml

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-01-23 14:56:23 +08:00
Daniel Jiang
a1d4bfd332
Merge pull request #6344 from reasonerjt/bump-up-golang
Bump up golang to 1.11.2
2019-01-11 16:15:59 +08:00
Daniel Jiang
5d59d6fab8 Bump up golang to 1.11.2
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-01-11 14:44:32 +08:00
Daniel Jiang
80af81154c
Merge pull request #6702 from wy65701436/robot-db-scheme
Add DB table for robot account
2019-01-10 14:25:58 +08:00
wang yan
db09f9f101 Update token length and upper case the sql key words
Signed-off-by: wang yan <wangyan@vmware.com>
2019-01-09 10:00:54 +08:00
wang yan
362a0638d0 Add DB table for robot account
This commit is to add DB scheme for robot account and update the db orm releated.

Signed-off-by: wang yan <wangyan@vmware.com>
2019-01-08 18:46:16 +08:00
Wenkai Yin
edcbb8f29f Update the upgrade sql to rename the duplicate records
Rename the "name" colume in table "replication_policy" and "replication_target" before adding the "UNIQUE" constraint to avoid the upgrade failure

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-01-04 14:55:33 +08:00
overdogwatch
a8d0ab1a21 Update Dockerfile
I guess that the purpose of this check is to verify that the container is listening on port 10514. Healthcheck default timeout is 30 sec. In places where the DNS resolver is not working properly, this check could take more than 30 sec, which leads to decide that the container health is unhealthy. I advise you to add to your check the option n, which prevents netstat trying to determine the symbolic host.

Signed-off-by: overdogwatch <overdogwatch@gmail.com>
2018-12-27 09:47:48 +02:00
Qian Deng
af7b61ebd5 Add customized config file for proxy
This is a solution for product that using harbor as proxy to add customized location

Signed-off-by: Qian Deng <dengq@vmware.com>
2018-12-26 13:35:07 +08:00
Meina Zhou
d45ccbbb29 add developer center in swagger ui way
Signed-off-by: Meina Zhou <meinaz@vmware.com>
2018-12-13 15:17:38 +08:00
Daniel Jiang
481bdb01a2
Merge pull request #6409 from ninjadq/upgrade_config_2_170
Feat: Upgrade harbor.cfg from 1.6.0 to 1.7.0
2018-12-04 01:05:59 -08:00
Qian Deng
7647e688fd Feat: Upgrade harbor.cfg from 1.6.0 to 1.7.0
Add script to upgrade harbor.cfg

Signed-off-by: Qian Deng <dengq@vmware.com>
2018-12-04 10:36:11 +08:00
Steven Zou
ec2ad4d0b8
Merge pull request #6093 from cd1989/replication-record-id
Add op uuid to image replication
2018-11-30 14:54:43 +08:00
Wenkai Yin
9d5cf57373 Check the existence of name when creating replication rule and fix bugs in testing library (#6381)
1. Fix #5102 by checking the existence of name when creating/editing replication rule
2. Add unique constraint to the name of replication policy and target
3. Fix bugs of testing library

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2018-11-30 13:32:20 +08:00
James Zabala
399c9c44d8
Merge pull request #6384 from SDBrett/photon-build
Corrected errors in photon build
2018-11-29 18:08:52 -05:00
Brett Johnson
1eb64e43ef added stage alias
resolve build error 'invalid from flag value 0: repository sha256 not found: does not exist or no pull access'

Signed-off-by: Brett Johnson <brett@sdbrett.com>
2018-11-29 16:02:26 +11:00
peimingming
c67fdc40f5 Support store job log in DB (#6144)
Signed-off-by: peimingming <peimingming@corp.netease.com>
2018-11-28 15:09:29 +08:00
Wenkai Yin
fefb955cfe Drop all capabilities when starting containers
Drop all capabilities when starting containers by modifying docker-compose files to avoid security issue

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2018-11-23 15:38:21 +08:00
stonezdj(Daojun Zhang)
ae007c2a49
Merge pull request #6247 from stonezdj/trust_cert2
Install custom cert for clair, registry, chartmuseum
2018-11-12 14:07:34 +08:00
Daniel Jiang
555c0cb181
Merge pull request #6220 from stonezdj/add_util_linux
Fix issue when query psql cli failed on more command not found
2018-11-09 17:39:47 +08:00
stonezdj
0a72f3729a Install custom cert for clair, registry, chartmuseum
Signed-off-by: stonezdj <stonezdj@gmail.com>
2018-11-09 15:03:03 +08:00
Daniel Jiang
dd9bd10e68
Merge pull request #6238 from ywk253100/181108_proxy
Replace ui with core for no_proxy in harbor.cfg
2018-11-09 14:19:13 +08:00
Steven Zou
b3c87673a6
Merge pull request #6215 from steven-zou/logger_framework_job_service
Build logger framework to support configurable loggers/sweepers/getters
2018-11-08 18:15:36 +08:00
Wenkai Yin
64cb507421 Replace ui with core for no_proxy in harbor.cfg
As we split UI into two components: portal and core, the value of no_proxy needs to be updated

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2018-11-08 16:10:26 +08:00
Steven Zou
093bf9fc8a Update the job service config yaml in the installation template
Signed-off-by: Steven Zou <szou@vmware.com>
2018-11-08 10:40:39 +08:00
stonezdj
8c37b0877a Fix issue when query psql cli failed on more command
Signed-off-by: stonezdj <stonezdj@gmail.com>
2018-11-06 13:09:20 +08:00
Daniel Jiang
1e9b34325c Make rendered files less visible
Some configuration files and env files contain sensitive information,
they should not be readable by any user by default.

This commit updates the `prepare` script to update the mask of the
rendered files.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-11-05 15:06:00 +08:00
James Zabala
b58ff42dff
Merge pull request #6184 from wy65701436/bump-up-clair
Build clair version into clair image
2018-11-02 17:04:25 -04:00
Daniel Jiang
ec01a97eb8 Clair image should accept parms
Update the entrypoint to allow the image accept other parms,
to help debug in the future.

If replace "$*" with "$@" only one parm will be passed to dumbinit

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-11-01 22:25:55 -07:00
wang yan
072127a70c Build clair version into clair image
This commit is to add the clair_version into the harbor images, then clair
will use it in the user-agent, and helpful for the debugging.

Signed-off-by: wang yan <wangyan@vmware.com>
2018-10-31 16:29:06 +08:00
Daniel Jiang
39b4d011c7 Not submit scan all job when core container starts
Fixes #6115

As for the change in migration sql file, in 1.7 we'll switch to
jobservice for scheduling "scan all" job.  To avoid inconsistency,
this item will be reset and user will need to configure the policy again.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-10-25 19:01:52 +08:00
Steven Zou
3b76a960e1
Merge pull request #6039 from stonezdj/refact_5996
Refactor capacity
2018-10-24 10:50:11 +08:00
Stéphane Albert
0d44e4f535 Send 308 status on redirect from http to https
Modify nginx configuration to use 308 instead of 301 on the http to
https redirect.
Fix problems with some clients on POST requests that are transformed to
GET on 301 redirect (per HTTP 1.1 standard).
See [RFC7538](https://tools.ietf.org/html/rfc7538).

Signed-off-by: Stéphane Albert <sheeprine@oh.its.fake.nullplace.com>
2018-10-23 20:25:35 +02:00
Christian Witts
e9c01255da
Fix install issue with default hostname commented
Update the `grep` filter to anchor at the start of the line
and allow for whitespace characters, in order to correctly
determine the hostname being set if the default is merely
commented out and the custom one added, instead of overridden.

Fixes #6117

Signed-off-by: Christian Witts <cwitts@gmail.com>
2018-10-23 14:55:27 +02:00
Daniel Jiang
6f4f941854 Fix permission issue in rsyslog container.
This commit fixes the permission issues introduced after migration to photon:2.0 base image.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-10-21 21:45:42 -07:00
陈德
1ffd9d8fba Add op uuid to image replication
Signed-off-by: 陈德 <chende@caicloud.io>
2018-10-21 23:55:57 +08:00
Daniel Jiang
3d09089a9c Rebuild Harbor images based on photon:2.0 (#6054)
Make necessary change to make things work with photon 2.0 docker image.
Remove distro-sync to mitigate the build issue and add `--pull` to docker build
command to make sure the latest photon:2.0 will be pulled during build process.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-10-18 11:03:28 +08:00
Daniel Jiang
3c577d588c
Merge pull request #6063 from wy65701436/fix-dns
Use docker official way to unset dns search
2018-10-17 17:08:00 +08:00
wang yan
f44ff2e4c3 Remove the env GODEBUG=netdns=cgo
This env is the workaroud of dns resolver on golang 1.7.3.
Remove it is bacause of harbor is using golang 1.9.2, the bug
has already been fixed.

Signed-off-by: wang yan <wangyan@vmware.com>
2018-10-17 16:11:25 +08:00
Yan
a5e7ac9164
Upgrade notary complile golang version to 1.9.4 (#6064)
This commit is to upgrade the golang version to 1.9.4, it because a
bug of golang 17.3 could introduce one dns resolver issue for harbor
mentioned by #6031.

The bug of golang is https://github.com/golang/go/issues/15419, it makes
harbor containers to lookup 'endpoint.' firstly which may cause network
issue.

Signed-off-by: wang yan <wangyan@vmware.com>
2018-10-17 16:04:14 +08:00
wang yan
bad68c5429 Use docker official way to unset dns search
According docker official document, use 'dns_search= .' in the docker
compose file if you don't wish to set the search domain.

https://docs.docker.com/v17.09/engine/userguide/networking/default_network/configure-dns/

Signed-off-by: wang yan <wangyan@vmware.com>
2018-10-17 14:27:29 +08:00
stonezdj
0278981523 Change admin server to core in jobservice
Signed-off-by: stonezdj <stonezdj@gmail.com>
2018-10-16 19:23:12 +08:00
Yan
08ae5f2f37
Limit dns search in harbor containers (#6057)
This commit is to set dns search to null in the harbor containers,
that means the dns search domains of docker host doesn't impact
the network IO in the containers.

If do not set this, Harbor notary-server and notary-signer are resolving
the "mysql" alias to the resolv.conf search path instead of to "mysql."
for the notary-db bridge IP, see #6031.

Signed-off-by: wang yan <wangyan@vmware.com>
2018-10-16 18:34:36 +08:00
stonezdj(Daojun Zhang)
b764033fc9
Merge pull request #6007 from stonezdj/refact_5998
Change admin server to core in jobservice
2018-10-15 17:52:24 +08:00
stonezdj
79bac7a64e Change admin server to core in jobservice
Signed-off-by: stonezdj <stonezdj@gmail.com>
2018-10-15 14:56:18 +08:00
Daniel Jiang
c8cb2f8481 Create shcema migration table in DB container
The migrate tool will try to create table schema_migration upon opening
the connection to DB.  This will cause error when there are multiple
instance of adminserver trying to access the migrator upon start.
This commit move the creation of the table during the initialization of
the DB container.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-10-12 17:17:31 +08:00
Daniel Jiang
31096a35af Run chown to job log directory
This commit revoke part of the change introduced in commit #1fc4142, by
calling chown to job log directory within the container when the job
service bootstraps.  The reason is we are seeing permission issue in
helm-chart deployment, and we want to reduce effort to handle the
permission on different deployment approaches.

There are some code in `prepare` script to change the ownership of the
JOB_LOG directory, it will be left for now to avoid regression in VIC
integration.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-09-22 12:05:37 +08:00
Qian Deng
88bb461314 Reactor backend api for authrization
1. Change backend api
2. Change frontend api
3. Change the proxy config file

Signed-off-by: Qian Deng <dengq@vmware.com>
2018-09-21 14:03:17 +08:00
Qian Deng
7873a0312a Rename harbor-ui to harbor-core
1. Update the nginx.conf
2. Update Makefile
3. Update docker-compose
4. Update image name
5. Rename folder ui to core
6. Change the harbor-ui's package name to core
7. Remove unused static file on harbor-core
8. Remove unused code for harbor-portal

Signed-off-by: Qian Deng <dengq@vmware.com>
2018-09-19 16:35:13 +08:00
Wenkai Yin
89893779fb Support configuring sslmode for the connection of database (#5861)
The sslmode of the connection with postgresql is hardcoded as "disable" currently, this commit expose it as an environment variable so that users can configure it

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2018-09-14 13:05:05 +08:00
Daniel Jiang
36ab8a5bf1 Add depdends_on to portal container
Fixes #5878

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-09-13 11:00:21 +08:00
Qian Deng
ac6c9d79ba Remove nodeclarity
Remove the nodeclarity container related code and ui_builder parameter no longer needed when build clarity

Signed-off-by: Qian Deng <dengq@vmware.com>
2018-09-08 20:33:21 +08:00
Qian Deng
097da4bb8d Fix typo
adminiserver to adminserver

Signed-off-by: Qian Deng <dengq@vmware.com>
2018-09-07 16:20:03 +08:00
Qian Deng
ba4762095f Update dockerfile of portal based on angular6 upgrade
1. Update entrypoint to reflect angular 6 upgrade

Signed-off-by: Qian Deng <dengq@vmware.com>
2018-09-07 15:06:15 +08:00
Qian Deng
db8d7b9c1c Merge branch 'seprate_harbor_portal_from_harbor_core' into angular6 2018-09-07 14:56:16 +08:00
Qian Deng
d797c50438 Fix trivial issues about rename ui_ng to portal
Update ui_ng to portal

Signed-off-by: Qian Deng <dengq@vmware.com>
2018-09-07 13:25:41 +08:00
Qian Deng
870653a5fb Update nginx config to redirect traffic to specific backend
1. Update nginx.conf file
2. Update photon makefile
3. Update global makefile

Signed-off-by: Qian Deng <dengq@vmware.com>
2018-09-07 13:21:27 +08:00
Qian Deng
dc21f3f5e2 Add container for harbor-portal
1. Add dockerfile for building harbor-portal
2. change the name from ui_ng to harbor-portal

Signed-off-by: Qian Deng <dengq@vmware.com>
2018-09-07 13:20:08 +08:00
Meina Zhou
a330d4e116 upgrade to angular 6
Signed-off-by: Meina Zhou <meinaz@vmware.com>
2018-09-07 11:30:00 +08:00
Yan
4eba01fc31 Clean make file unused code and unify docker build method (#59)
Signed-off-by: Yan <wangyan@vmware.com>
2018-09-04 17:18:15 +08:00
Daniel Jiang
768f165877
Merge pull request #5771 from wy65701436/deprecate-make-dev
Deprecate dockerfiles in make/dev
2018-09-04 12:58:52 +08:00
Daniel Jiang
823a9d11e9 Bump Clair to v2.0.5
The PR to fix the Alpine issue has been merged to Clair's release-2.0
branch, and released v2.0.5.
This commit updates Harbor to include that change and re-enable
Clair's updaters by default.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-09-02 12:27:04 +08:00
wang yan
9bb7902003 Deprecate dockerfiles in make/dev
Signed-off-by: wang yan <wangyan@vmware.com>
2018-08-31 15:26:26 +08:00
wang yan
f8b964d8cf Extend configuration length to 1024 in DB
Signed-off-by: wang yan <wangyan@vmware.com>
2018-08-31 13:42:24 +08:00
Steven Zou
3e241be34f
Merge pull request #5739 from steven-zou/fix_s3_storage_issue
Fix issues related with chart storage
2018-08-28 15:16:34 +08:00
Daniel Jiang
c4eaf25ed1
Merge pull request #5742 from reasonerjt/remove-compose-ha
Remove reference of docker-compsoe based HA
2018-08-28 15:09:22 +08:00
Steven Zou
43ecf62c25 Fix issues related with chart storage
- inject custom CA bundle into chart repo
- update prepare script to inject credentials

Signed-off-by: Steven Zou <szou@vmware.com>
2018-08-28 14:10:50 +08:00
Daniel Jiang
e1153eec0a Remove reference of docker-compsoe based HA
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-08-28 14:04:44 +08:00
wang yan
854f606f63 upgrade notary to latest release v0.6.1
Move the notary-server and notary signer into ./notary/release-${notaryversion} as this will not impact the
release branches, the binaries in ./notary are v0.5.1.

Signed-off-by: wang yan <wangyan@vmware.com>
2018-08-28 13:51:55 +08:00
James Zabala
141f6056e7
Merge pull request #5700 from xxxdreamer/test
add some comments to harbor.cfg
2018-08-24 17:18:28 -04:00
Wenkai Yin
2c5b06350e
Merge pull request #5716 from reasonerjt/update-go-import-path
Update import path in go code
2018-08-23 19:02:50 +08:00
Daniel Jiang
dcf4e2ee78 Update import path in go code
vmware -> goharbor

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-08-23 17:50:53 +08:00
unknown
9b94800216 To make the DCO bot happy^_^
Signed-off-by: unknown <cyz.dreamer@gmail.com>
2018-08-23 14:43:54 +08:00
unknown
7dd68c992f add some comments to harbor.cfg
Signed-off-by: unknown <cyz.dreamer@gmail.com>
2018-08-23 14:38:47 +08:00
Daniel Jiang
78bddd831a Temporarily disable updaters of clair
Set the updater interval to "0" to mitigate the impact of Apline URL
change that cause clair keep polling vuln data.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-08-23 14:00:37 +08:00
Wenkai Yin
0673e7c0a9 Add VOLUME definition in Dockerfile of chart museum
The VOLUME definition in Dockerfile of chart museum will mount a volume automatically by docker if no specific volume is provided.

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2018-08-20 17:00:40 +08:00
Daniel Jiang
bda3878ab8 Update registry image to enable inject root cert
In some user's environment, there's local object storage hosted with
self-signed certificate.
Because registry process runs in a photon container, it has to trust
the certificate in the photon level such that the registry can access
the storage service.

This commit updates the registry image to append custom cert to the root
bundle when the container is started.  And make the customer cert
configurable in `harbor.cfg`

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-08-16 18:36:21 +08:00
wangyan
29d0d51403 Signed-off-by: wangyan <wangyan@vmware.com>
Add clean registry cache to gc job

To workaround the issue: https://github.com/docker/distribution/issues/2094
GC needs to clean cache before to call the docker reigstry api to delete blobs.
Otherwise, the following docker push will not be performed as docker registry
does not clean cache in GC, it thinks the image is still there, and the new
blobs will be uploaded.
2018-08-13 02:58:27 -07:00
wangyan
e2ff77c4cd Fix namespace when to build harbor images 2018-08-09 23:41:30 -07:00
wangyan
7713764aec Batch update docker image namespace to goharbor
This commit is to move all the images of harbor from vmware to goharbor
2018-08-09 23:24:21 -07:00
Steven Zou
79b2f01bab
Merge pull request #5569 from steven-zou/change_notes_in_installation_script
Update the usage message to include chart repository server related info
2018-08-09 15:42:34 +08:00
Yan
0ffa6e076c
Unify redis configuration for harbor components (#5564)
this commit is to specrate the redis_url into host,port,pwd and index for
different components, and make it possible to set external redis server.
2018-08-09 15:27:46 +08:00
Steven Zou
7fe16eba19 Update the usage message to include chart repository server related info 2018-08-09 14:55:31 +08:00
wangyan
063e44c486 Remove the tdnf error eater in docker files 2018-08-07 22:58:01 -07:00
Deng, Qian
37176c8fe5 Unlink harbor-ui after UI complling.
After UI compilling should unlink harbor-ui.
Because it create a link with root user in container that will cause permission issues next time you want to aceess this file.
2018-08-06 19:53:07 +08:00
Steven Zou
e0ed44cc13 Enable chart/prov files overwriting option
change 'ALLOW_OVERWRITE' from 'false' to 'true'
2018-08-01 15:20:38 +08:00
Deng, Qian
8feb49c64e Feature of helm chart UI
1. Add Charts list view
2. Add Charts card view
3. Add Chart version list view
4. Add chart version card view
5. Add Chart Detail Summary
6. Add Chart Detail Value
6. Add Chart Detail Deps
7. Update nodeclarity Dockerfile
8. Add markdown support
9. Add package-lock file to src
2018-08-01 13:20:06 +08:00
Daniel Jiang
bda0a92ea9
Merge pull request #5427 from ywk253100/180730_redirect
Remove the URL rewrite for docker registry request
2018-07-31 13:34:15 +08:00
wangyan
22411cf6b2 Fix pgsql creation column bug
Root cause: Use default 'now'::timestamp will not generate timestamp for each transaction,
PG will convert now to a timestamp as soon as the constant is parsed. To fix it, update it
to defult CURRENT_TIMESTAMP, thie setting is the same as default now(), which returns the
start time of current transaction because ther are fuction calls, hey will give the desired
behavior of defaulting to the time of row insertion.

Reference: https://www.postgresql.org/docs/9.6/static/functions-datetime.html#FUNCTIONS-DATETIME-CURRENT
PG version: 9.6.9
2018-07-30 04:58:44 -07:00
Wenkai Yin
7e6a13915b Remove the URL rewrite for docker registry request
This commit redirects the request to UI directory without URL rewrite
2018-07-30 17:24:15 +08:00
Daniel Jiang
46de1432f2 Enable cfg migrator to 1.6.0
In 1.6, there will be only one DB process in the default deployment.
The migrator will try to handle the setting by "guessing" whether Harbor
was pointed to external DB.

Verified 1.5->1.6 and 1.4->1.6 migration.
2018-07-27 17:11:45 +08:00
Wenkai Yin
c3106fc447
Merge pull request #5401 from reasonerjt/config-clair-interval
Enable configuring the interval of clair updaters
2018-07-26 18:18:16 +08:00
Daniel Jiang
733a89dea2 Enable configuring the interval of clair updaters
To mitigate the impact we saw in the updater issues in clair, this
commit enable configuring the interval, include disabling the updaters
of clair.
2018-07-26 16:27:23 +08:00
Daniel Jiang
ad0c0eba36 Add registry to default no_proxy hosts for Clair
When proxy is set for Clair, there may be issue when Clair pulls image
from the registryif the `no_proxy` attribute is not updated.  This
commit adds `registry` to the default setting.
2018-07-26 14:41:03 +08:00
Steven Zou
32f0ceade3 Modify the cfg option 'max_job_workers' from 50 to 10 to reduce the resource requirement 2018-07-23 13:36:46 +08:00
Steven Zou
bb380e6dbc
Merge pull request #5314 from steven-zou/chart_repo_supporting
Refactor chart API endpoints
2018-07-20 20:43:55 +08:00
Steven Zou
0227a1315a Keep the chart server related configurations in adminserver
append chart server related config options to the supporting list of adminserver
provide chart server related config access method in the API layer
update prepare script and ui env template file to enable cache driver config for chart server API
append flag info in the systeminfo API to indicate if chart server is deployed with Harbor
refactor the response rewriting logic to return structual error object
add api init method to initilizing objects required in API handlers
chage owner of the storage folder
update offline/online package scripts in Harbor-Util.robot
2018-07-20 19:40:33 +08:00
Yan
efdb57548f
add admin job api (#5344)
It supports Harbor admin to trigger job either manual or
schedule. The job will be populated to job service to execute. 
The api includes:
1. POST /api/system/gc
2, GET /api/system/gc/:id 
3, GET /api/system/gc/:id/log
4, PUT/GET/POST /api/system/gc/schedule
2018-07-20 19:22:37 +08:00
Daniel Jiang
6c664ee993 Update photon base images (#5346)
This commit update the base photon image from vmware/photon:1.0 to
photon:1.0, per suggestion by photon team.
2018-07-19 20:45:20 +08:00
Steven Zou
726d81803b Fix conflicts in Makefiles and prepare script files with upstream 2018-07-19 03:17:05 -07:00
Steven Zou
22ea7dd91f Update the related build scripts to package the chart repo server
add env file template for chart repo server in make/common/config/chartserver
update the Makefiles to support build chart repo server
add docker file and related build scripts for upstream chart server - chartmuseum
update prepare to support generating chart server related configs
add docker compose file for the chart server
add build/install command options to install with/without chart repo server
update install.sh to support chart repo server installation
2018-07-19 16:47:05 +08:00
Yan
d5b85a6748
Add the registry controller httpserver, it's responsible for controlling (#5265)
docker regsitry. This version has the API to call regsitry GC with jobservice
secret. Seprates it into a standalone container as do not want to invoke two
processes in one container.

It needs to mount the registry storage into this container in order to do GC,
and needs to copy the registry binary into it.
2018-07-16 16:50:28 +08:00
Wenkai Yin
8a92019e8e
Merge pull request #5310 from reasonerjt/adminserver-update-schema
Let adminserver initialise the DB schema.
2018-07-16 12:52:11 +08:00
Daniel Jiang
0d6ea995e1 Let adminserver initialise the DB schema.
This commit make update to remove the code from ui container to init the
DB schema.  As UI has dependency on admin server, so it's safe to assume
adminserver has to be ready first.  Regardless the setting of the config
store of admin server, it will try to access and intialize the schema of
database.
2018-07-13 17:32:17 +08:00
Daniel Jiang
bd92b165c8
Merge pull request #5309 from ywk253100/180713_redis
Fix permission deny issue when Redis starting
2018-07-13 15:33:51 +08:00
Wenkai Yin
e0f2a3d5ce Fix permission deny issue when Redis starting
This commit changes the ownership of directory that Redis uses to user redis when starting up
2018-07-13 14:20:20 +08:00
timchenxiaoyu
a912a55ac2 add sync registry env (#5294)
By default Harbor will call catalog API of registry and sync the result to DB, this becomes problematic when registry is configured to custom storage service, there maybe inconsistent result and the whole process may be very time consuming.
So in this commit a env var SYNC_REGISTRY is introduced if user want Harbor to sync the repo when it starts up, by default it's false.
2018-07-13 11:15:41 +08:00
Daniel Jiang
f7a29363ed
Merge pull request #5296 from reasonerjt/clair-bump-up-v2.0.4
Bump up clair to v2.0.4
2018-07-12 15:09:17 +08:00
Daniel Jiang
bc1969156e Bump up clair to v2.0.4
This commit bump up clair to v2.0.4.  The current build process is
download the binary from google storage, the update of the binary in
google storage is not reflected in this commit.
2018-07-12 13:59:51 +08:00
Steven Zou
0dfc273ee8
Merge pull request #5246 from kulong0105/master
make/docker-compose.tpl: fix wrong mount configuration(#5208)
2018-07-12 13:54:35 +08:00
wangyan
bba96b3669 Update docker registry cache from inmemory to redis.
It gives Harbor the capability to controll the cache of docker
registry, and the workaround for cache invalidation bug caused
by garbage collection, that is clean cache in GC job.

For more details, see Harbor issue #5078.
2018-07-09 02:32:07 -07:00
Daniel Jiang
3bb4e2c921
Merge pull request #5260 from halfa/master
Change empty_subj to fix #2920 openssl issue
2018-07-06 15:08:10 +08:00
stonezdj
62acdb14f3 Add settings to define admin with LDAP group DN 2018-07-05 14:46:44 +08:00
Yan
d366134fe8
Fix bug of packaging offline installer (#5245)
The init sql script name nad path was changed by PR #5197, this
commit is to update these and log the package command to console,
make it more easy to debug in future. Also remove the action to
pull migrator as it will built each time locally.
2018-07-04 20:03:44 +08:00
Daniel Jiang
c04d99b1ab Workaround the Clair issue in ubuntu updater
This commit is a temp fix to workaround coreos/clair#562
Recompiled the code at the tip of release-2.0 branch of clair and
updated Makefile.
Once clair provides a new release, we'll need to make update in
Makefiles and Dockerfiles again to consume it.
2018-07-04 17:28:47 +08:00
Yilong Ren
15d6145f5c make/docker-compose.tpl: fix wrong mount configuration(#5208) 2018-07-04 14:12:10 +08:00
Daniel Jiang
a161f2c95b
Merge pull request #4965 from jouve/reg_upstream
remove unused upstream
2018-07-03 16:29:08 +08:00
Steven Zou
6dfccc7dea
Merge pull request #5074 from ninjadq/ldap_search_ui
Add LDAP search UI
2018-07-03 15:30:18 +08:00
Deng, Qian
72dfdd552f Add ldap serach
1. Add group management
2. Add rewrite import user to member ui
3. Add import group to member
4. Add new items in configuration page
2018-07-03 14:00:59 +08:00
Daniel Jiang
cb0acbace4 Restrict the CPU usage of Clair (#5217)
This commit fixes #5072
Due to an issue in bzr, Clair container may consume a lot of CPU
resource while updating the vuln data.  This commit mitigates the impact
by setting the cpu_quota of clair container. (default value of
cpu_period is 100000 in v2 docker-compose template)
2018-07-03 11:23:56 +08:00
Deng, Qian
edbe2fe620 Update migrator to 1 6 0
1. Add new alembic_pg folder for postgres
2. Add migration file for 1.6.0
3. Update version to 1.6.0
4. update migrator dockerfile
2018-07-02 21:23:47 +08:00
Daniel Jiang
aef3213dfa
Merge pull request #5190 from stonezdj/reload_config
Fix issue that harbor tile can not save customized settings
2018-06-29 13:04:36 +08:00
Daniel Jiang
c9b1962b1e Initialise Harbor DB schema in Harbor UI/adminserver container
This commit fixes #5040, the harbor-db image will only contain empty
databases, and harbor ui container will use migrate tool to run initial
SQL scripts to do initialization.  This is helpful for the case to
configure Harbor against external DB or DBaaS like RDS for HA deployment
However, this change will results some confusion as there are two tables
to track schema versions have been using alembic for migration, for this
release we'll try to use alembic to mock a `migration` table during
upgrade so the migrator will be bypassed, in future we'll consider to
consolidate to the golang based migrator.
Another issue is that the UI and adminserver containers will access DB
after start up in different congurations, can't ensure the sequence, so
both of them will try to update the schema when started up.
2018-06-28 16:22:53 +08:00
stonezdj
72e9b22e10 Fix issue that harbor tile can not save customized settings 2018-06-28 16:20:10 +08:00
mricher
ee60eaec16
Change empty_subj to fix openssl issue 2018-06-27 16:50:26 +02:00
Daniel Jiang
cfc95c69e6 Fix failure of running prepare with python3
This commit fixes #5053.
It removes the usage of `string.strip` which will fail in python3.
2018-06-13 18:17:00 +08:00
Daniel Jiang
ccbd23d14e Change owner of the secret file in prepare script
The secretkey file will be loaded by adminserver which is run by non-root
user (uid:10000) previously the entrypoint script will run `chown` to a
lot files, and there's a breakage in upgrade when we skip running
`chown` inside container.
This commit will fix the issue during upgrade by changing the owner of
the secretkey file.
2018-06-08 16:43:16 +08:00
Wenkai Yin 79628
0c56493fb6 Soft delete label
Modify the deletion of label to soft deletion, in this way the names of deleted labels referenced by replication rules can be shown to users
2018-06-07 17:14:12 +08:00
Yan
6d800cabbd
enable migrator to support 1.5.0 migration from mysql to pgsql (#5029)
This commit is to enable data migrator to support migrates data
from mysql to pgsql, this is a specific step for user to upgrade
harbor across v1.5.0, as we have move harbor DB to pgsql from
1.5.0. It supports both harbor and notary db data migration,
and be split into two steps with dependency.

It also fix issue #4847, add build DB migrator in make process.
2018-06-01 14:58:43 +08:00
Daniel Jiang
9f13453d5f
chown only to the folder to store the config.json (#4978)
Narrow down the scope of `chown` in adminserver because the
/etc/adminserver/config/ is the location to store the config.json file.
And /etc/adminserver/key should be readonly.
2018-05-16 15:36:22 +08:00
Cyril Jouve
086ebbfe84 remove unused upstream 2018-05-15 14:11:28 +02:00
Deng, Qian
282a63f57f Fix legacy issues that html and css file are written on ts file.
Currently, our html and css files are written as string on .ts file. This pr is to solve the legacy issue.
2018-05-10 18:39:48 +08:00
Tan Jiang
5ff07cf619 Fix permission issue on VIC appliance
The job logs directory's permission is not changed by prepare script
because the everything is moved from /data to /storage/data on VIC
appliance.  This commit will make sure both cases the directory is
readable by user 10000:10000.
This PR also makes sure the config json of notary signer has 0644
permission.
2018-05-06 22:02:19 +08:00
Tan Jiang
21ec4808ec Collect log of redis
Previously the log file was set to a hard coded file, but given this
redis should run in container, the update is made to have the process
output log messages to standard output, and redirect it to syslog in
docker-compose template.
2018-04-30 18:16:11 +08:00
Wenkai Yin
fa8bbe821a Modify unique constraint of table harbor_label in pgsql (#4811)
Add unique constraint to column name, scope and project_id of table harbor_label to make creating same name labels under different projects valid
2018-04-27 08:01:20 -07:00
Yan
ae257433cc
Fully migrate harbor db to postgresql (#4689)
* Merge harbor db to postgres
2018-04-27 02:27:12 -07:00
Wenkai Yin
73babbf1ab Modify unique constraint of table harbor_label
Add unique constraint to column name, scope and project_id  of table harbor_label to make creating same name labels under different projects valid
2017-12-19 22:15:56 +08:00
Tan Jiang
1fc4142e1a Do not call chown to config files
This commit fixes a recently discovered issue on Kubernetes #4496
It make necessary to avoid calling `chown` to config files during the
bootstrap of the containers.
2018-04-20 13:44:21 +08:00
Steven Zou
9d13842a29 Fix the issue of missing copying the setting.json to container image (#4714)
add `cp ./src/setting.json ../ui/static/`
2018-04-18 17:43:24 +08:00
Steven Zou
43018dc755 Update the entrypoint script of clarity builder image to fix customized image copying issue 2018-04-17 19:10:43 +08:00
Steven Zou
ca8d3bdcc9
Merge pull request #4638 from vmware/use_redis_url_addr
Use redis URL address to replace host:port when connecting to redis server
2018-04-17 10:12:51 +08:00
Steven Zou
adc2f8f124 Use redis URL address to replace host:port when connecting to redis server
replace tcp host:port with
'redis://arbitrary_usrname:password@ipaddress:port/database_index'

update prepare to generate config yaml file of job service based on harbor.cfg

update harbor.cfg default values
2018-04-13 19:19:56 +08:00
yixingj
221a8b0892 Update HA tempalte
Update the template.
Update the installation guide.
2018-04-13 11:12:48 +08:00
Yan
946b4b4ad8
Update harbor default log level to info (#4639) 2018-04-12 19:04:44 +08:00
Daniel Jiang
ebc9d04479
Merge pull request #4624 from yixingjia/clairdbconfigurable
Add dbname paramter in dburl
2018-04-11 18:43:52 +08:00
yixingj
5b7f176c1d Add dbname paramter in dburl
add dbname to dburl
2018-04-11 17:38:45 +08:00
Steven Zou
a7c2e049cc Update the worker concurrency of job service from 10 to 50 to improve performance 2018-04-11 17:12:09 +08:00
Daniel Jiang
ba4c7f5731
Merge pull request #4514 from yixingjia/usemonitorapi
Use new health check api for docker files
2018-04-08 10:03:49 +08:00
Steven Zou
44808650be Merge branch 'master' into job_service 2018-04-03 16:28:26 +08:00
Daniel Jiang
53cea404fd
Merge pull request #4523 from ninjadq/migrator_1_5
Upgrade migrator to 1.5.0
2018-04-02 09:25:11 +08:00
Deng, Qian
93c96da18b Upgrade migrator to 1.5.0 2018-03-30 22:45:32 +08:00
Steven Zou
250360307b Modify docker compose file template and make file to enable new job service
Fix typo in Makefile under photon

Fix version tag issue of redis container

Assign container name for redis container

Update docker compose template to enable network for redis

Remove exposed ports of redis from compose yaml tpl
2018-03-30 16:52:55 +08:00
Steven Zou
d5a696d821 Merge branch 'master' into job_service 2018-03-30 11:23:20 +08:00
Steven Zou
afd3ffd63c
Merge pull request #4472 from yixingjia/redisserver
Add Redis docker file
2018-03-30 11:21:23 +08:00
Steven Zou
d1899c840d Merge branch 'master' into job_service 2018-03-29 23:25:20 +08:00
Jesse Hu
debcf7858a
Merge pull request #4521 from vmware/clair-http-proxy
Add http_proxy configuration for Clair service
2018-03-29 14:04:34 +08:00
yixingj
22f682c020 Use new health check api for docker files
ui
nginx
adminserver
postgresql
2018-03-29 13:00:28 +08:00
Jesse Hu
87c622141b Add http_proxy configuration for Clair service
Clair needs Internet access to download vulnerabilities data.

Fix issue https://github.com/vmware/harbor/issues/4272
2018-03-28 18:49:40 +08:00
Tan Jiang
b6df6cf169 Add indexes to job tables and bump up schema version. 2018-03-28 16:15:54 +08:00
Tan Jiang
41ce0891ab Trigger scan job from UI. 2018-03-26 18:07:21 +08:00
Tan Jiang
745b21abbc Merge remote-tracking branch 'upstream/master' into scan-job-migrate 2018-03-26 15:39:42 +08:00
stone
203b1b52bb
Merge pull request #4415 from stonezdj/user_group_and_project_member
Add DAO for user group and project member
2018-03-26 15:21:20 +08:00
Daniel Jiang
93568854d9
Merge pull request #4474 from reasonerjt/job_uuid
Update SQL script to add uuid to job tables.
2018-03-26 15:17:53 +08:00
stonezdj
49d960b060 Add DAO for project member and user group 2018-03-26 14:38:32 +08:00
yixingj
cb64ad96ff Make endpoint configurable
Move all the endpoint to harbor.cfg
2018-03-26 10:50:18 +08:00
Tan Jiang
381ecc3521 Merge with master 2018-03-26 10:37:17 +08:00
Tan Jiang
df69f7e410 Fix error in registry_sqlite.sql 2018-03-25 13:47:03 +08:00
Yan
cbcca015b0
add read only mode to stop docker push (#4433) 2018-03-23 03:16:08 -07:00
Jesse Hu
0b5e0aa041
Enhance registry docker file to make it work with NFS server (#4466) 2018-03-23 17:57:26 +08:00
yixingj
ee6a6af3c5 Add Redis docker file
Add the redis docker files
2018-03-23 17:46:43 +08:00
Tan Jiang
6f1c46624f Update SQL script to add uuid to job tables.
We have to add the uuid/id mapping as new job service will only store uuid.

Further work is in feature branch for now, commit this change to
accelerate migration work.
2018-03-23 17:45:50 +08:00
Tan Jiang
7238efd9ae Integrate new jobservice into docker-compose template
This commit doesn't integrate redis.  No change to makefile b/c it
should work once the temporary jobservice_v2 folder is renamed to jobservice.
2018-03-22 19:48:22 +08:00
Daniel Jiang
f885de0913
Merge pull request #4449 from ywk253100/180314_filter_by_label
Add unique constraint to columns label_id, resource_id, resource_name and resource_type in table harbor_resource_label
2018-03-21 18:37:04 +08:00
Wenkai Yin
83cfe9814d Add unique constraint to columns label_id, resource_id, resource_name and resource_type in table harbor_resource_label 2018-03-21 16:24:21 +08:00
Yan
5edbd00318
Revise harbor build version (#4445) 2018-03-21 13:03:49 +08:00
Wenkai Yin
838b439560 Implement filter repository and tags by label API 2018-03-21 10:51:06 +08:00
Daniel Jiang
62f25be709
Merge pull request #4423 from reasonerjt/config-migration
Provide migration scripts for harbor.cfg
2018-03-18 19:56:30 +08:00
Tan Jiang
c8265a8d53 Provide migration scripts for harbor.cfg
Default target version is 1.5.0
This is mainly for VIC-appliance upgrade, and should be considered
experimental for oss due to limited test.
Tested with 1.2 and 1.3 harbor.cfg from VIC appliance.
2018-03-16 14:38:50 +08:00
stonezdj
44fc373c6d Add LDAP Group Search Configure Param 2018-03-15 06:16:47 +08:00
Wenkai Yin
bcf81224ad Update according to the comments 2018-03-14 13:42:19 +08:00
Wenkai Yin
36b9c4e458 Implement adding/removing labels to/from repositories and images API 2018-03-12 19:30:05 +08:00
Wenkai Yin
379f113452 Implement label management API 2018-03-09 12:17:27 +08:00
Tan Jiang
f83c65bcc5 Reduce the output of build.
The following are done to avoid travis-ci failing due to too much log
size.
1) Update Makefile and scripts to make go build less verbose.
2) Make tdnf less verbose
2018-02-27 20:54:52 +08:00
stonezdj
f138067242 Refactor project member 2018-02-09 10:38:51 +08:00
Tan Jiang
07251181b9 Remove extra-hosts from docker-compose template 2018-02-05 00:02:37 +08:00
yixingj
6c8bb9c73f Refactor DB configuration in harbor.cfg
Put harbor_db, clair_db configuration together
2018-02-02 17:14:52 +08:00
Wenkai Yin
2221e114fa Add SELinux label for all volumes 2018-01-30 14:25:43 +08:00
yixingj
9c8706b0ce Fix images tag issue.
move the image tag out
only remove the first : in the keypair.
2018-01-25 17:34:47 +08:00
Tan Jiang
5975e6b964 Add place holder for injecting UAA host
As this is for tile deployment only, so add a shortcut for tile/bosh
script to add entry in /etc/hosts inside the container.
Due to effort consideration I don't think we want to render
docker-compose in `prepare` script.
2018-01-25 13:22:43 +08:00
yixingjia
f676a71422
Merge pull request #4110 from yixingjia/updatekeepalived
Update keepalived config file
2018-01-24 12:24:44 +08:00
yixingj
44df82a82d Update keepalived config file
Update the keepalived config file
2018-01-24 10:57:39 +08:00
Jesse Hu
8fe8c2b4ac Add a space after ':' when generating registry storage yaml config
yaml requires 1 or more spaces between the key and value. This patch
is useful in case the user inputs 'key:value' instead of 'key: value'.
2018-01-22 20:14:33 +08:00
yixingjia
208cb02d5c
Merge pull request #4054 from yixingjia/HA_Monitor
Keepalived real server check script
2018-01-17 19:03:14 +08:00
yixingj
ffa69bb256 Enhance monitor script for HA
Add keepalived real server check script
2018-01-17 14:23:26 +08:00
Wenkai Yin
8cda2d8d65
Merge pull request #4036 from ywk253100/180116_s3
Propagate registry storage driver name to adminserver and return it in /api/systeminfo
2018-01-16 18:41:08 +08:00
Daniel Jiang
1595200d05
Merge pull request #4028 from jessehu/reload_config
[Issue 4015] Read reload_config option from harbor.cfg
2018-01-16 17:35:04 +08:00
Wenkai Yin
53d5a2256a Propagate registry storage driver name to adminserver and return it in /api/systeminfo 2018-01-16 16:57:28 +08:00
Deng, Qian
b3e65ed71e Update migration tool for v1.4
1. Update database meta file
2. Add migration file for 1.4
2018-01-16 15:38:51 +08:00
Jesse Hu
b995881cd2 [Issue 4015] Read reload_config option from harbor.cfg
If reload_config is true, Harbor will reload all configuration
in harbor.cfg when restarting. It's false by default.
2018-01-16 12:42:30 +08:00
stonezdj
aa4e36c61a Change ldap scope after refactor 2018-01-08 16:59:15 +08:00
Daniel Jiang
64cc71ea12
Merge pull request #3941 from vmware/replication_enhancement
Replication enhancement
2018-01-08 10:56:39 +08:00
Jesse Hu
b1b316a97b
Add registry storage config in harbor.cfg (#3918)
Refer to https://docs.docker.com/registry/configuration/#storage
for all available configuration.
2018-01-07 17:23:18 +08:00
pfh
13308ce9d8 Merge remote-tracking branch 'upstream/master' into repEnhance 2018-01-05 14:09:03 +08:00
Jesse Hu
4fb947a155 Fix indentation in registry yaml files 2018-01-03 18:28:22 +08:00
Tan Jiang
e02de2068a Enable configuring the CA Certificate for UAA
Enable configuring the path of root cert of UAA in harbor.cfg.  It only
takes effects if the verify_cert is set to "true" If the file does not
exist, the configuration is skipped.
The intention for this commit is to support integration with nested UAA
in PAS or PKS, we don't expect user to manually configure this value,
though he can do it if he wants.
2018-01-03 16:21:29 +08:00
yixingjia
5340fed110
Merge pull request #3848 from yixingjia/ClairHA
Enable Clair in HA
2018-01-03 11:45:44 +08:00
Wenkai Yin
96a63c56b1 Merge remote-tracking branch 'upstream/master' into 180103_merge 2018-01-03 10:32:03 +08:00
stonezdj
b065f19f51 Add configure parameter ldap_verify_cert to harbor.cfg 2017-12-27 10:21:58 +08:00
Daniel Jiang
1c2d5e8036
Merge pull request #3638 from CMUH/k8s/bump-to-newer-resource
Bump many resources type to newer ones of Kubernetes
2017-12-26 11:44:29 +08:00
Deng, Qian
3187dcb5ae Upgrade Clarity to 0.10.x and Angular To 4.3.0 2017-12-25 16:10:51 +08:00
yixingj
ecd0bbf2dd Enable Clair in HA
Run clair with Core Harbor services
Add check logic for Clair DB
2017-12-21 23:23:35 +08:00
yixingj
f63588855f Make Clair DB configurable
Make the HOST,PORT,USERNAME,DB configurable for
Clair
2017-12-20 18:29:50 +08:00
Daniel Jiang
068d6a35df
Merge pull request #3832 from wy65701436/master
Remove the workaroud for avoiding photon distro-sync error
2017-12-20 15:06:43 +08:00
wangyan
b560f6c061 Remove the workaroud for avoiding photon distro-sync error 2017-12-19 21:10:32 -08:00
Daniel Jiang
052521b92c
Merge pull request #3821 from reasonerjt/uaa-restriction
Refactor the configuraiton of UAA
2017-12-19 19:36:09 +08:00
Tan Jiang
2ffc58a5d4 Refactor the configuraiton of UAA
Remove the attribute "uaa_ca_root" from harbor.cfg and introduce
"uaa_verify_cert".  Similar to LDAP settings, this allow user to
explicitly turn of the cert verification against UAA server, such that
the code will work with self-signed certificate.
2017-12-19 14:42:07 +08:00
wangyan
6b7df3636c Temporary workaround for photon distro-sync error 2017-12-18 22:18:21 -08:00
wangyan
1e750a1ed4 Unify images tags and build process 2017-12-14 23:52:18 -08:00
Wenkai Yin
a54b7dd4c0 Merge remote-tracking branch 'upstream/master' into 171219_merge 2017-12-15 08:48:57 +08:00
Wenkai Yin
665a54edc3 Merge remote-tracking branch 'upstream/master' into 171213_merge 2017-12-13 13:40:24 +08:00
yixingjia
ec269047c7
Merge pull request #3736 from yixingjia/HA
HA installation script
2017-12-12 16:27:28 +08:00
yixingjia
f4d0fd4d23
Merge pull request #3640 from yixingjia/moveconftoDB
Add database driver for Harbor configurations
2017-12-11 10:42:05 +08:00
yixingj
d328e2586e HA installation script
Add --ha options when install Harbor.

Currently it does nothing.
2017-12-07 22:56:57 +08:00
wangyan
8cd5ac5171 Update prepare chmod to support python3 2017-12-07 00:31:02 -08:00
yixingj
9b03c93afd Add database driver for Harbor configurations
1>Add a new database driver for configurations
2> change the current default driver from json
to database
2017-12-06 13:06:54 +08:00
stone
30e536b18b
Merge pull request #3683 from stonezdj/local_ldap_enhance
Ldap enhancement
2017-11-27 14:36:20 +08:00
Wenkai Yin
6b0ee138e5 Implement immediate trigger and the methods of WatchList 2017-11-27 14:23:21 +08:00
A31882(Wu Yi Chung)
0ae6eccde4 Replace Nginx-Proxy with Ingress 2017-11-27 09:25:24 +08:00
A31882(Wu Yi Chung)
ae2b702ea5 Replace ReplicationController with Deployment 2017-11-24 16:50:09 +08:00
A31882(Wu Yi Chung)
ee8144b98a change config path to fit with what defined in Dockerfile 2017-11-24 16:28:17 +08:00
stonezdj
16243cfbbc Add LDAP remote certifcate validation
push test

Add unit test for ldap verify cert

remove common.VerifyRemoteCert

Update code with PR review comments

Add change ldaps config and add UT testcase for TLS feature

add ldap verfiy cert checkbox about #3513

Draft harbor ova install guide

Search and import ldap user when add project members

Add unit test case for SearchAndImportUser

ova guide

Add ova install guide

Add ova install guide 2

Add ova install guide 3

Call ValidateLdapConf before search ldap

trim space in username

Remove leading space in openLdap username

Remove doc change in this branch

Update unit test for ldap search and import user

Add test case about ldap verify cert checkbox

Modify ldap testcase
2017-11-24 12:41:51 +08:00
yixingj
0af4e3a41d Fix clair permission issue
Clair will call bzr, without -H in sudo it will usr root user's
Home envrionment.
2017-11-23 20:03:57 +08:00
Daniel Jiang
a409cf8088
Merge pull request #3648 from reasonerjt/rebuild-images
[Upload Build]Bump up mariaDB's version
2017-11-21 23:47:45 +08:00