The migrate tool will try to create table schema_migration upon opening
the connection to DB. This will cause error when there are multiple
instance of adminserver trying to access the migrator upon start.
This commit move the creation of the table during the initialization of
the DB container.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
This commit revoke part of the change introduced in commit #1fc4142, by
calling chown to job log directory within the container when the job
service bootstraps. The reason is we are seeing permission issue in
helm-chart deployment, and we want to reduce effort to handle the
permission on different deployment approaches.
There are some code in `prepare` script to change the ownership of the
JOB_LOG directory, it will be left for now to avoid regression in VIC
integration.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
1. Update the nginx.conf
2. Update Makefile
3. Update docker-compose
4. Update image name
5. Rename folder ui to core
6. Change the harbor-ui's package name to core
7. Remove unused static file on harbor-core
8. Remove unused code for harbor-portal
Signed-off-by: Qian Deng <dengq@vmware.com>
The sslmode of the connection with postgresql is hardcoded as "disable" currently, this commit expose it as an environment variable so that users can configure it
Signed-off-by: Wenkai Yin <yinw@vmware.com>
The PR to fix the Alpine issue has been merged to Clair's release-2.0
branch, and released v2.0.5.
This commit updates Harbor to include that change and re-enable
Clair's updaters by default.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
Move the notary-server and notary signer into ./notary/release-${notaryversion} as this will not impact the
release branches, the binaries in ./notary are v0.5.1.
Signed-off-by: wang yan <wangyan@vmware.com>
Set the updater interval to "0" to mitigate the impact of Apline URL
change that cause clair keep polling vuln data.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
The VOLUME definition in Dockerfile of chart museum will mount a volume automatically by docker if no specific volume is provided.
Signed-off-by: Wenkai Yin <yinw@vmware.com>
In some user's environment, there's local object storage hosted with
self-signed certificate.
Because registry process runs in a photon container, it has to trust
the certificate in the photon level such that the registry can access
the storage service.
This commit updates the registry image to append custom cert to the root
bundle when the container is started. And make the customer cert
configurable in `harbor.cfg`
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
Add clean registry cache to gc job
To workaround the issue: https://github.com/docker/distribution/issues/2094
GC needs to clean cache before to call the docker reigstry api to delete blobs.
Otherwise, the following docker push will not be performed as docker registry
does not clean cache in GC, it thinks the image is still there, and the new
blobs will be uploaded.
After UI compilling should unlink harbor-ui.
Because it create a link with root user in container that will cause permission issues next time you want to aceess this file.
Root cause: Use default 'now'::timestamp will not generate timestamp for each transaction,
PG will convert now to a timestamp as soon as the constant is parsed. To fix it, update it
to defult CURRENT_TIMESTAMP, thie setting is the same as default now(), which returns the
start time of current transaction because ther are fuction calls, hey will give the desired
behavior of defaulting to the time of row insertion.
Reference: https://www.postgresql.org/docs/9.6/static/functions-datetime.html#FUNCTIONS-DATETIME-CURRENT
PG version: 9.6.9
In 1.6, there will be only one DB process in the default deployment.
The migrator will try to handle the setting by "guessing" whether Harbor
was pointed to external DB.
Verified 1.5->1.6 and 1.4->1.6 migration.
When proxy is set for Clair, there may be issue when Clair pulls image
from the registryif the `no_proxy` attribute is not updated. This
commit adds `registry` to the default setting.
append chart server related config options to the supporting list of adminserver
provide chart server related config access method in the API layer
update prepare script and ui env template file to enable cache driver config for chart server API
append flag info in the systeminfo API to indicate if chart server is deployed with Harbor
refactor the response rewriting logic to return structual error object
add api init method to initilizing objects required in API handlers
chage owner of the storage folder
update offline/online package scripts in Harbor-Util.robot
It supports Harbor admin to trigger job either manual or
schedule. The job will be populated to job service to execute.
The api includes:
1. POST /api/system/gc
2, GET /api/system/gc/:id
3, GET /api/system/gc/:id/log
4, PUT/GET/POST /api/system/gc/schedule
add env file template for chart repo server in make/common/config/chartserver
update the Makefiles to support build chart repo server
add docker file and related build scripts for upstream chart server - chartmuseum
update prepare to support generating chart server related configs
add docker compose file for the chart server
add build/install command options to install with/without chart repo server
update install.sh to support chart repo server installation
docker regsitry. This version has the API to call regsitry GC with jobservice
secret. Seprates it into a standalone container as do not want to invoke two
processes in one container.
It needs to mount the registry storage into this container in order to do GC,
and needs to copy the registry binary into it.
This commit make update to remove the code from ui container to init the
DB schema. As UI has dependency on admin server, so it's safe to assume
adminserver has to be ready first. Regardless the setting of the config
store of admin server, it will try to access and intialize the schema of
database.
By default Harbor will call catalog API of registry and sync the result to DB, this becomes problematic when registry is configured to custom storage service, there maybe inconsistent result and the whole process may be very time consuming.
So in this commit a env var SYNC_REGISTRY is introduced if user want Harbor to sync the repo when it starts up, by default it's false.
This commit bump up clair to v2.0.4. The current build process is
download the binary from google storage, the update of the binary in
google storage is not reflected in this commit.
It gives Harbor the capability to controll the cache of docker
registry, and the workaround for cache invalidation bug caused
by garbage collection, that is clean cache in GC job.
For more details, see Harbor issue #5078.
The init sql script name nad path was changed by PR #5197, this
commit is to update these and log the package command to console,
make it more easy to debug in future. Also remove the action to
pull migrator as it will built each time locally.
This commit is a temp fix to workaround coreos/clair#562
Recompiled the code at the tip of release-2.0 branch of clair and
updated Makefile.
Once clair provides a new release, we'll need to make update in
Makefiles and Dockerfiles again to consume it.
This commit fixes#5072
Due to an issue in bzr, Clair container may consume a lot of CPU
resource while updating the vuln data. This commit mitigates the impact
by setting the cpu_quota of clair container. (default value of
cpu_period is 100000 in v2 docker-compose template)
This commit fixes#5040, the harbor-db image will only contain empty
databases, and harbor ui container will use migrate tool to run initial
SQL scripts to do initialization. This is helpful for the case to
configure Harbor against external DB or DBaaS like RDS for HA deployment
However, this change will results some confusion as there are two tables
to track schema versions have been using alembic for migration, for this
release we'll try to use alembic to mock a `migration` table during
upgrade so the migrator will be bypassed, in future we'll consider to
consolidate to the golang based migrator.
Another issue is that the UI and adminserver containers will access DB
after start up in different congurations, can't ensure the sequence, so
both of them will try to update the schema when started up.
The secretkey file will be loaded by adminserver which is run by non-root
user (uid:10000) previously the entrypoint script will run `chown` to a
lot files, and there's a breakage in upgrade when we skip running
`chown` inside container.
This commit will fix the issue during upgrade by changing the owner of
the secretkey file.
This commit is to enable data migrator to support migrates data
from mysql to pgsql, this is a specific step for user to upgrade
harbor across v1.5.0, as we have move harbor DB to pgsql from
1.5.0. It supports both harbor and notary db data migration,
and be split into two steps with dependency.
It also fix issue #4847, add build DB migrator in make process.
Narrow down the scope of `chown` in adminserver because the
/etc/adminserver/config/ is the location to store the config.json file.
And /etc/adminserver/key should be readonly.
The job logs directory's permission is not changed by prepare script
because the everything is moved from /data to /storage/data on VIC
appliance. This commit will make sure both cases the directory is
readable by user 10000:10000.
This PR also makes sure the config json of notary signer has 0644
permission.
Previously the log file was set to a hard coded file, but given this
redis should run in container, the update is made to have the process
output log messages to standard output, and redirect it to syslog in
docker-compose template.
This commit fixes a recently discovered issue on Kubernetes #4496
It make necessary to avoid calling `chown` to config files during the
bootstrap of the containers.
replace tcp host:port with
'redis://arbitrary_usrname:password@ipaddress:port/database_index'
update prepare to generate config yaml file of job service based on harbor.cfg
update harbor.cfg default values
