Commit Graph

131 Commits

Author SHA1 Message Date
Daniel Jiang
c8cb2f8481 Create shcema migration table in DB container
The migrate tool will try to create table schema_migration upon opening
the connection to DB.  This will cause error when there are multiple
instance of adminserver trying to access the migrator upon start.
This commit move the creation of the table during the initialization of
the DB container.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-10-12 17:17:31 +08:00
Daniel Jiang
31096a35af Run chown to job log directory
This commit revoke part of the change introduced in commit #1fc4142, by
calling chown to job log directory within the container when the job
service bootstraps.  The reason is we are seeing permission issue in
helm-chart deployment, and we want to reduce effort to handle the
permission on different deployment approaches.

There are some code in `prepare` script to change the ownership of the
JOB_LOG directory, it will be left for now to avoid regression in VIC
integration.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-09-22 12:05:37 +08:00
Qian Deng
7873a0312a Rename harbor-ui to harbor-core
1. Update the nginx.conf
2. Update Makefile
3. Update docker-compose
4. Update image name
5. Rename folder ui to core
6. Change the harbor-ui's package name to core
7. Remove unused static file on harbor-core
8. Remove unused code for harbor-portal

Signed-off-by: Qian Deng <dengq@vmware.com>
2018-09-19 16:35:13 +08:00
Qian Deng
097da4bb8d Fix typo
adminiserver to adminserver

Signed-off-by: Qian Deng <dengq@vmware.com>
2018-09-07 16:20:03 +08:00
Qian Deng
ba4762095f Update dockerfile of portal based on angular6 upgrade
1. Update entrypoint to reflect angular 6 upgrade

Signed-off-by: Qian Deng <dengq@vmware.com>
2018-09-07 15:06:15 +08:00
Qian Deng
870653a5fb Update nginx config to redirect traffic to specific backend
1. Update nginx.conf file
2. Update photon makefile
3. Update global makefile

Signed-off-by: Qian Deng <dengq@vmware.com>
2018-09-07 13:21:27 +08:00
Qian Deng
dc21f3f5e2 Add container for harbor-portal
1. Add dockerfile for building harbor-portal
2. change the name from ui_ng to harbor-portal

Signed-off-by: Qian Deng <dengq@vmware.com>
2018-09-07 13:20:08 +08:00
Yan
4eba01fc31 Clean make file unused code and unify docker build method (#59)
Signed-off-by: Yan <wangyan@vmware.com>
2018-09-04 17:18:15 +08:00
Daniel Jiang
823a9d11e9 Bump Clair to v2.0.5
The PR to fix the Alpine issue has been merged to Clair's release-2.0
branch, and released v2.0.5.
This commit updates Harbor to include that change and re-enable
Clair's updaters by default.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-09-02 12:27:04 +08:00
Steven Zou
3e241be34f
Merge pull request #5739 from steven-zou/fix_s3_storage_issue
Fix issues related with chart storage
2018-08-28 15:16:34 +08:00
Steven Zou
43ecf62c25 Fix issues related with chart storage
- inject custom CA bundle into chart repo
- update prepare script to inject credentials

Signed-off-by: Steven Zou <szou@vmware.com>
2018-08-28 14:10:50 +08:00
wang yan
854f606f63 upgrade notary to latest release v0.6.1
Move the notary-server and notary signer into ./notary/release-${notaryversion} as this will not impact the
release branches, the binaries in ./notary are v0.5.1.

Signed-off-by: wang yan <wangyan@vmware.com>
2018-08-28 13:51:55 +08:00
Wenkai Yin
0673e7c0a9 Add VOLUME definition in Dockerfile of chart museum
The VOLUME definition in Dockerfile of chart museum will mount a volume automatically by docker if no specific volume is provided.

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2018-08-20 17:00:40 +08:00
Daniel Jiang
bda3878ab8 Update registry image to enable inject root cert
In some user's environment, there's local object storage hosted with
self-signed certificate.
Because registry process runs in a photon container, it has to trust
the certificate in the photon level such that the registry can access
the storage service.

This commit updates the registry image to append custom cert to the root
bundle when the container is started.  And make the customer cert
configurable in `harbor.cfg`

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-08-16 18:36:21 +08:00
wangyan
e2ff77c4cd Fix namespace when to build harbor images 2018-08-09 23:41:30 -07:00
wangyan
063e44c486 Remove the tdnf error eater in docker files 2018-08-07 22:58:01 -07:00
Steven Zou
bb380e6dbc
Merge pull request #5314 from steven-zou/chart_repo_supporting
Refactor chart API endpoints
2018-07-20 20:43:55 +08:00
Steven Zou
0227a1315a Keep the chart server related configurations in adminserver
append chart server related config options to the supporting list of adminserver
provide chart server related config access method in the API layer
update prepare script and ui env template file to enable cache driver config for chart server API
append flag info in the systeminfo API to indicate if chart server is deployed with Harbor
refactor the response rewriting logic to return structual error object
add api init method to initilizing objects required in API handlers
chage owner of the storage folder
update offline/online package scripts in Harbor-Util.robot
2018-07-20 19:40:33 +08:00
Daniel Jiang
6c664ee993 Update photon base images (#5346)
This commit update the base photon image from vmware/photon:1.0 to
photon:1.0, per suggestion by photon team.
2018-07-19 20:45:20 +08:00
Steven Zou
726d81803b Fix conflicts in Makefiles and prepare script files with upstream 2018-07-19 03:17:05 -07:00
Steven Zou
22ea7dd91f Update the related build scripts to package the chart repo server
add env file template for chart repo server in make/common/config/chartserver
update the Makefiles to support build chart repo server
add docker file and related build scripts for upstream chart server - chartmuseum
update prepare to support generating chart server related configs
add docker compose file for the chart server
add build/install command options to install with/without chart repo server
update install.sh to support chart repo server installation
2018-07-19 16:47:05 +08:00
Yan
d5b85a6748
Add the registry controller httpserver, it's responsible for controlling (#5265)
docker regsitry. This version has the API to call regsitry GC with jobservice
secret. Seprates it into a standalone container as do not want to invoke two
processes in one container.

It needs to mount the registry storage into this container in order to do GC,
and needs to copy the registry binary into it.
2018-07-16 16:50:28 +08:00
Wenkai Yin
8a92019e8e
Merge pull request #5310 from reasonerjt/adminserver-update-schema
Let adminserver initialise the DB schema.
2018-07-16 12:52:11 +08:00
Daniel Jiang
0d6ea995e1 Let adminserver initialise the DB schema.
This commit make update to remove the code from ui container to init the
DB schema.  As UI has dependency on admin server, so it's safe to assume
adminserver has to be ready first.  Regardless the setting of the config
store of admin server, it will try to access and intialize the schema of
database.
2018-07-13 17:32:17 +08:00
Wenkai Yin
e0f2a3d5ce Fix permission deny issue when Redis starting
This commit changes the ownership of directory that Redis uses to user redis when starting up
2018-07-13 14:20:20 +08:00
Daniel Jiang
bc1969156e Bump up clair to v2.0.4
This commit bump up clair to v2.0.4.  The current build process is
download the binary from google storage, the update of the binary in
google storage is not reflected in this commit.
2018-07-12 13:59:51 +08:00
Yan
d366134fe8
Fix bug of packaging offline installer (#5245)
The init sql script name nad path was changed by PR #5197, this
commit is to update these and log the package command to console,
make it more easy to debug in future. Also remove the action to
pull migrator as it will built each time locally.
2018-07-04 20:03:44 +08:00
Daniel Jiang
c04d99b1ab Workaround the Clair issue in ubuntu updater
This commit is a temp fix to workaround coreos/clair#562
Recompiled the code at the tip of release-2.0 branch of clair and
updated Makefile.
Once clair provides a new release, we'll need to make update in
Makefiles and Dockerfiles again to consume it.
2018-07-04 17:28:47 +08:00
Daniel Jiang
c9b1962b1e Initialise Harbor DB schema in Harbor UI/adminserver container
This commit fixes #5040, the harbor-db image will only contain empty
databases, and harbor ui container will use migrate tool to run initial
SQL scripts to do initialization.  This is helpful for the case to
configure Harbor against external DB or DBaaS like RDS for HA deployment
However, this change will results some confusion as there are two tables
to track schema versions have been using alembic for migration, for this
release we'll try to use alembic to mock a `migration` table during
upgrade so the migrator will be bypassed, in future we'll consider to
consolidate to the golang based migrator.
Another issue is that the UI and adminserver containers will access DB
after start up in different congurations, can't ensure the sequence, so
both of them will try to update the schema when started up.
2018-06-28 16:22:53 +08:00
Wenkai Yin 79628
0c56493fb6 Soft delete label
Modify the deletion of label to soft deletion, in this way the names of deleted labels referenced by replication rules can be shown to users
2018-06-07 17:14:12 +08:00
Yan
6d800cabbd
enable migrator to support 1.5.0 migration from mysql to pgsql (#5029)
This commit is to enable data migrator to support migrates data
from mysql to pgsql, this is a specific step for user to upgrade
harbor across v1.5.0, as we have move harbor DB to pgsql from
1.5.0. It supports both harbor and notary db data migration,
and be split into two steps with dependency.

It also fix issue #4847, add build DB migrator in make process.
2018-06-01 14:58:43 +08:00
Daniel Jiang
9f13453d5f
chown only to the folder to store the config.json (#4978)
Narrow down the scope of `chown` in adminserver because the
/etc/adminserver/config/ is the location to store the config.json file.
And /etc/adminserver/key should be readonly.
2018-05-16 15:36:22 +08:00
Tan Jiang
21ec4808ec Collect log of redis
Previously the log file was set to a hard coded file, but given this
redis should run in container, the update is made to have the process
output log messages to standard output, and redirect it to syslog in
docker-compose template.
2018-04-30 18:16:11 +08:00
Wenkai Yin
fa8bbe821a Modify unique constraint of table harbor_label in pgsql (#4811)
Add unique constraint to column name, scope and project_id of table harbor_label to make creating same name labels under different projects valid
2018-04-27 08:01:20 -07:00
Yan
ae257433cc
Fully migrate harbor db to postgresql (#4689)
* Merge harbor db to postgres
2018-04-27 02:27:12 -07:00
Wenkai Yin
73babbf1ab Modify unique constraint of table harbor_label
Add unique constraint to column name, scope and project_id  of table harbor_label to make creating same name labels under different projects valid
2017-12-19 22:15:56 +08:00
Tan Jiang
1fc4142e1a Do not call chown to config files
This commit fixes a recently discovered issue on Kubernetes #4496
It make necessary to avoid calling `chown` to config files during the
bootstrap of the containers.
2018-04-20 13:44:21 +08:00
Daniel Jiang
ba4c7f5731
Merge pull request #4514 from yixingjia/usemonitorapi
Use new health check api for docker files
2018-04-08 10:03:49 +08:00
Steven Zou
44808650be Merge branch 'master' into job_service 2018-04-03 16:28:26 +08:00
Daniel Jiang
53cea404fd
Merge pull request #4523 from ninjadq/migrator_1_5
Upgrade migrator to 1.5.0
2018-04-02 09:25:11 +08:00
Deng, Qian
93c96da18b Upgrade migrator to 1.5.0 2018-03-30 22:45:32 +08:00
Steven Zou
250360307b Modify docker compose file template and make file to enable new job service
Fix typo in Makefile under photon

Fix version tag issue of redis container

Assign container name for redis container

Update docker compose template to enable network for redis

Remove exposed ports of redis from compose yaml tpl
2018-03-30 16:52:55 +08:00
Steven Zou
d5a696d821 Merge branch 'master' into job_service 2018-03-30 11:23:20 +08:00
Steven Zou
afd3ffd63c
Merge pull request #4472 from yixingjia/redisserver
Add Redis docker file
2018-03-30 11:21:23 +08:00
Steven Zou
d1899c840d Merge branch 'master' into job_service 2018-03-29 23:25:20 +08:00
yixingj
22f682c020 Use new health check api for docker files
ui
nginx
adminserver
postgresql
2018-03-29 13:00:28 +08:00
Tan Jiang
b6df6cf169 Add indexes to job tables and bump up schema version. 2018-03-28 16:15:54 +08:00
Tan Jiang
745b21abbc Merge remote-tracking branch 'upstream/master' into scan-job-migrate 2018-03-26 15:39:42 +08:00
stone
203b1b52bb
Merge pull request #4415 from stonezdj/user_group_and_project_member
Add DAO for user group and project member
2018-03-26 15:21:20 +08:00
Daniel Jiang
93568854d9
Merge pull request #4474 from reasonerjt/job_uuid
Update SQL script to add uuid to job tables.
2018-03-26 15:17:53 +08:00