Commit Graph

429 Commits

Author SHA1 Message Date
Yan
d5b85a6748
Add the registry controller httpserver, it's responsible for controlling (#5265)
docker regsitry. This version has the API to call regsitry GC with jobservice
secret. Seprates it into a standalone container as do not want to invoke two
processes in one container.

It needs to mount the registry storage into this container in order to do GC,
and needs to copy the registry binary into it.
2018-07-16 16:50:28 +08:00
Wenkai Yin
8a92019e8e
Merge pull request #5310 from reasonerjt/adminserver-update-schema
Let adminserver initialise the DB schema.
2018-07-16 12:52:11 +08:00
Daniel Jiang
0d6ea995e1 Let adminserver initialise the DB schema.
This commit make update to remove the code from ui container to init the
DB schema.  As UI has dependency on admin server, so it's safe to assume
adminserver has to be ready first.  Regardless the setting of the config
store of admin server, it will try to access and intialize the schema of
database.
2018-07-13 17:32:17 +08:00
Daniel Jiang
bd92b165c8
Merge pull request #5309 from ywk253100/180713_redis
Fix permission deny issue when Redis starting
2018-07-13 15:33:51 +08:00
Wenkai Yin
e0f2a3d5ce Fix permission deny issue when Redis starting
This commit changes the ownership of directory that Redis uses to user redis when starting up
2018-07-13 14:20:20 +08:00
timchenxiaoyu
a912a55ac2 add sync registry env (#5294)
By default Harbor will call catalog API of registry and sync the result to DB, this becomes problematic when registry is configured to custom storage service, there maybe inconsistent result and the whole process may be very time consuming.
So in this commit a env var SYNC_REGISTRY is introduced if user want Harbor to sync the repo when it starts up, by default it's false.
2018-07-13 11:15:41 +08:00
Daniel Jiang
f7a29363ed
Merge pull request #5296 from reasonerjt/clair-bump-up-v2.0.4
Bump up clair to v2.0.4
2018-07-12 15:09:17 +08:00
Daniel Jiang
bc1969156e Bump up clair to v2.0.4
This commit bump up clair to v2.0.4.  The current build process is
download the binary from google storage, the update of the binary in
google storage is not reflected in this commit.
2018-07-12 13:59:51 +08:00
Steven Zou
0dfc273ee8
Merge pull request #5246 from kulong0105/master
make/docker-compose.tpl: fix wrong mount configuration(#5208)
2018-07-12 13:54:35 +08:00
wangyan
bba96b3669 Update docker registry cache from inmemory to redis.
It gives Harbor the capability to controll the cache of docker
registry, and the workaround for cache invalidation bug caused
by garbage collection, that is clean cache in GC job.

For more details, see Harbor issue #5078.
2018-07-09 02:32:07 -07:00
Daniel Jiang
3bb4e2c921
Merge pull request #5260 from halfa/master
Change empty_subj to fix #2920 openssl issue
2018-07-06 15:08:10 +08:00
stonezdj
62acdb14f3 Add settings to define admin with LDAP group DN 2018-07-05 14:46:44 +08:00
Yan
d366134fe8
Fix bug of packaging offline installer (#5245)
The init sql script name nad path was changed by PR #5197, this
commit is to update these and log the package command to console,
make it more easy to debug in future. Also remove the action to
pull migrator as it will built each time locally.
2018-07-04 20:03:44 +08:00
Daniel Jiang
c04d99b1ab Workaround the Clair issue in ubuntu updater
This commit is a temp fix to workaround coreos/clair#562
Recompiled the code at the tip of release-2.0 branch of clair and
updated Makefile.
Once clair provides a new release, we'll need to make update in
Makefiles and Dockerfiles again to consume it.
2018-07-04 17:28:47 +08:00
Yilong Ren
15d6145f5c make/docker-compose.tpl: fix wrong mount configuration(#5208) 2018-07-04 14:12:10 +08:00
Daniel Jiang
a161f2c95b
Merge pull request #4965 from jouve/reg_upstream
remove unused upstream
2018-07-03 16:29:08 +08:00
Steven Zou
6dfccc7dea
Merge pull request #5074 from ninjadq/ldap_search_ui
Add LDAP search UI
2018-07-03 15:30:18 +08:00
Deng, Qian
72dfdd552f Add ldap serach
1. Add group management
2. Add rewrite import user to member ui
3. Add import group to member
4. Add new items in configuration page
2018-07-03 14:00:59 +08:00
Daniel Jiang
cb0acbace4 Restrict the CPU usage of Clair (#5217)
This commit fixes #5072
Due to an issue in bzr, Clair container may consume a lot of CPU
resource while updating the vuln data.  This commit mitigates the impact
by setting the cpu_quota of clair container. (default value of
cpu_period is 100000 in v2 docker-compose template)
2018-07-03 11:23:56 +08:00
Deng, Qian
edbe2fe620 Update migrator to 1 6 0
1. Add new alembic_pg folder for postgres
2. Add migration file for 1.6.0
3. Update version to 1.6.0
4. update migrator dockerfile
2018-07-02 21:23:47 +08:00
Daniel Jiang
aef3213dfa
Merge pull request #5190 from stonezdj/reload_config
Fix issue that harbor tile can not save customized settings
2018-06-29 13:04:36 +08:00
Daniel Jiang
c9b1962b1e Initialise Harbor DB schema in Harbor UI/adminserver container
This commit fixes #5040, the harbor-db image will only contain empty
databases, and harbor ui container will use migrate tool to run initial
SQL scripts to do initialization.  This is helpful for the case to
configure Harbor against external DB or DBaaS like RDS for HA deployment
However, this change will results some confusion as there are two tables
to track schema versions have been using alembic for migration, for this
release we'll try to use alembic to mock a `migration` table during
upgrade so the migrator will be bypassed, in future we'll consider to
consolidate to the golang based migrator.
Another issue is that the UI and adminserver containers will access DB
after start up in different congurations, can't ensure the sequence, so
both of them will try to update the schema when started up.
2018-06-28 16:22:53 +08:00
stonezdj
72e9b22e10 Fix issue that harbor tile can not save customized settings 2018-06-28 16:20:10 +08:00
mricher
ee60eaec16
Change empty_subj to fix openssl issue 2018-06-27 16:50:26 +02:00
Daniel Jiang
cfc95c69e6 Fix failure of running prepare with python3
This commit fixes #5053.
It removes the usage of `string.strip` which will fail in python3.
2018-06-13 18:17:00 +08:00
Daniel Jiang
ccbd23d14e Change owner of the secret file in prepare script
The secretkey file will be loaded by adminserver which is run by non-root
user (uid:10000) previously the entrypoint script will run `chown` to a
lot files, and there's a breakage in upgrade when we skip running
`chown` inside container.
This commit will fix the issue during upgrade by changing the owner of
the secretkey file.
2018-06-08 16:43:16 +08:00
Wenkai Yin 79628
0c56493fb6 Soft delete label
Modify the deletion of label to soft deletion, in this way the names of deleted labels referenced by replication rules can be shown to users
2018-06-07 17:14:12 +08:00
Yan
6d800cabbd
enable migrator to support 1.5.0 migration from mysql to pgsql (#5029)
This commit is to enable data migrator to support migrates data
from mysql to pgsql, this is a specific step for user to upgrade
harbor across v1.5.0, as we have move harbor DB to pgsql from
1.5.0. It supports both harbor and notary db data migration,
and be split into two steps with dependency.

It also fix issue #4847, add build DB migrator in make process.
2018-06-01 14:58:43 +08:00
Daniel Jiang
9f13453d5f
chown only to the folder to store the config.json (#4978)
Narrow down the scope of `chown` in adminserver because the
/etc/adminserver/config/ is the location to store the config.json file.
And /etc/adminserver/key should be readonly.
2018-05-16 15:36:22 +08:00
Cyril Jouve
086ebbfe84 remove unused upstream 2018-05-15 14:11:28 +02:00
Deng, Qian
282a63f57f Fix legacy issues that html and css file are written on ts file.
Currently, our html and css files are written as string on .ts file. This pr is to solve the legacy issue.
2018-05-10 18:39:48 +08:00
Tan Jiang
5ff07cf619 Fix permission issue on VIC appliance
The job logs directory's permission is not changed by prepare script
because the everything is moved from /data to /storage/data on VIC
appliance.  This commit will make sure both cases the directory is
readable by user 10000:10000.
This PR also makes sure the config json of notary signer has 0644
permission.
2018-05-06 22:02:19 +08:00
Tan Jiang
21ec4808ec Collect log of redis
Previously the log file was set to a hard coded file, but given this
redis should run in container, the update is made to have the process
output log messages to standard output, and redirect it to syslog in
docker-compose template.
2018-04-30 18:16:11 +08:00
Wenkai Yin
fa8bbe821a Modify unique constraint of table harbor_label in pgsql (#4811)
Add unique constraint to column name, scope and project_id of table harbor_label to make creating same name labels under different projects valid
2018-04-27 08:01:20 -07:00
Yan
ae257433cc
Fully migrate harbor db to postgresql (#4689)
* Merge harbor db to postgres
2018-04-27 02:27:12 -07:00
Wenkai Yin
73babbf1ab Modify unique constraint of table harbor_label
Add unique constraint to column name, scope and project_id  of table harbor_label to make creating same name labels under different projects valid
2017-12-19 22:15:56 +08:00
Tan Jiang
1fc4142e1a Do not call chown to config files
This commit fixes a recently discovered issue on Kubernetes #4496
It make necessary to avoid calling `chown` to config files during the
bootstrap of the containers.
2018-04-20 13:44:21 +08:00
Steven Zou
9d13842a29 Fix the issue of missing copying the setting.json to container image (#4714)
add `cp ./src/setting.json ../ui/static/`
2018-04-18 17:43:24 +08:00
Steven Zou
43018dc755 Update the entrypoint script of clarity builder image to fix customized image copying issue 2018-04-17 19:10:43 +08:00
Steven Zou
ca8d3bdcc9
Merge pull request #4638 from vmware/use_redis_url_addr
Use redis URL address to replace host:port when connecting to redis server
2018-04-17 10:12:51 +08:00
Steven Zou
adc2f8f124 Use redis URL address to replace host:port when connecting to redis server
replace tcp host:port with
'redis://arbitrary_usrname:password@ipaddress:port/database_index'

update prepare to generate config yaml file of job service based on harbor.cfg

update harbor.cfg default values
2018-04-13 19:19:56 +08:00
yixingj
221a8b0892 Update HA tempalte
Update the template.
Update the installation guide.
2018-04-13 11:12:48 +08:00
Yan
946b4b4ad8
Update harbor default log level to info (#4639) 2018-04-12 19:04:44 +08:00
Daniel Jiang
ebc9d04479
Merge pull request #4624 from yixingjia/clairdbconfigurable
Add dbname paramter in dburl
2018-04-11 18:43:52 +08:00
yixingj
5b7f176c1d Add dbname paramter in dburl
add dbname to dburl
2018-04-11 17:38:45 +08:00
Steven Zou
a7c2e049cc Update the worker concurrency of job service from 10 to 50 to improve performance 2018-04-11 17:12:09 +08:00
Daniel Jiang
ba4c7f5731
Merge pull request #4514 from yixingjia/usemonitorapi
Use new health check api for docker files
2018-04-08 10:03:49 +08:00
Steven Zou
44808650be Merge branch 'master' into job_service 2018-04-03 16:28:26 +08:00
Daniel Jiang
53cea404fd
Merge pull request #4523 from ninjadq/migrator_1_5
Upgrade migrator to 1.5.0
2018-04-02 09:25:11 +08:00
Deng, Qian
93c96da18b Upgrade migrator to 1.5.0 2018-03-30 22:45:32 +08:00