Commit Graph

215 Commits

Author SHA1 Message Date
wang yan
506376dd1a let compiler can specify any parameters supported by go build
Signed-off-by: wang yan <wangyan@vmware.com>
2019-10-31 15:41:05 +08:00
Steven Zou
7b6e83090e create API folder to keep API swagger files
- create API folder
- move harbor API swagger file to API/harbor
- add scanner adapter open API swagger file to API/scanner
- update protal build Dockerfile
- update swagger explorer build command in Makefile

Signed-off-by: Steven Zou <szou@vmware.com>
2019-10-30 21:41:03 +08:00
wang yan
253e87d186 inject ldflags for harbor compiler and linker
1, replace the UIVERSION file with ldflags, which is generarted by make to inject into the UI core.
2, inject additional ldflags for harbor compiler

Signed-off-by: wang yan <wangyan@vmware.com>
2019-10-30 18:31:42 +08:00
He Weiwei
28e0c0693b Upgrade clair adapter to v1.0.0
1. Upgrade clair adapter to v1.0.0.
2. Make the clair adapter which installed by harbor immutable and using internal registry address.
3. Add support to build clair adapter image from binary.
4. Switch to ScannerPull action when make authorization for the scan request.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-10-26 17:25:36 +00:00
Daniel Jiang
dbe6ebceec Bump up golang to 1.12.12
Bump up the golang for compiling the binaries to 1.12.12
This commit also includes some minor changes to Makefile to fix issue in
building the binary files.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-10-21 15:55:58 +08:00
He Weiwei
8964a8697a build(clair): internal clair adapter when install with clair
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-10-17 12:00:51 +08:00
Daniel Jiang
49f12d0b16
Merge pull request #8786 from reasonerjt/fix-8622
Extract shared func for checkenv and install scripts
2019-10-10 16:53:51 +08:00
He Weiwei
6fbb77d65a
build(portal): npm registry configurable and build cache support (#9356)
1. Introduce NPM_REGISTRY in Makefile to support npm registry
configuration when build portal image.
2. Install npm pkgs before copy portal src so that build cache works for
npm install in portal image.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-10-10 15:29:50 +08:00
Daniel Jiang
b9154a858b Extract shared func for checkenv and install scripts
This commit fixes #8622 by extract shared func into common.sh to avoid
inconsistency in future.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-10-10 15:07:09 +08:00
wang yan
3cf7e702be patch regsitry fix of issue 2553
This commit is target to fix harbor issue #9186, which root cause is mentioned by
https://github.com/docker/distribution/issues/2553, and fixed by https://github.com/docker/distribution/pull/2879.

As the latest distribution release(v2.7.1) does not contain this fix, but it will break the quota migraion process on S3 storage, we have to path this fix into Harbor regsitry binary.

[Tag Version]
It uses the issue number(2553) as the tag naming convention, like v2.7.1-patch-2553, means that we patch the fix of issue 2553 into v2.7.1.

[Note]
So far, this fix is only targets on docker regsitry v2.7.1. If the registry has this fix in new release, we'll move on.

Signed-off-by: wang yan <wangyan@vmware.com>
2019-09-26 18:27:53 +08:00
Daniel Jiang
b34fda173c Bump up Clair to v2.0.9
Fixes #8584

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-08-19 16:19:29 +08:00
Qian Deng
dacb1fc79e Add healthcheck in Dockerfile* redis* jobservice
Signed-off-by: Qian Deng <dengq@vmware.com>
2019-08-06 13:16:12 +00:00
Qian Deng
5cd3594f20 Upgrade chartmuseum from v0.8.1 to v0.9.0
Signed-off-by: Qian Deng <dengqian0826@gmail.com>
2019-07-17 06:45:23 +00:00
Mark Peek
e26ac54bba Fix a couple of error message typos in the Makefile
Signed-off-by: Mark Peek <markpeek@vmware.com>
2019-06-26 14:33:32 -07:00
Daniel Jiang
737eaa396c Bump up to go 1.12.5 enable go.mod
This commit bumps up the version of Go to compile the code to v1.12.5,
and shifts to go.mod for managing depedency.
Some code from "harbor/tests" to "harbor/src/testing" to avoid depedency
loop of modules.

Note that in short term we will still vendor the dependency.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-06-11 00:42:20 +08:00
Daniel Jiang
ca3561c8b8 Bump up Clair to v2.0.8 (#7246)
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-04-22 10:43:12 +08:00
wang yan
469473b31c Patch regsitry v2.7.1 with fix on distribution issue 2819
This commit is to build a regsitry bases on v2.7.1 code and introduces
an fix on issue #2819, this is a P0 bug on v2.7.1 which causes GCS doesn't
work well on v2.7.1

For more details, refer to https://github.com/docker/distribution/pull/2821

Signed-off-by: wang yan <wangyan@vmware.com>
2019-04-09 18:42:29 +08:00
Qian Deng
deba378842 Enhance: Refacotr Registry config file
1. Refactor registry configs
2. cp gcs keyfile is exist

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-04-09 14:40:41 +08:00
Qian Deng
fcdab4d4af Fix: packaging offline in new prepare
This new prepare script now support offline packaging

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-03-22 13:56:15 +08:00
Qian Deng
93af296eeb Enhance: refactor the mount dirs and workflow of generate cert
mount a temp dir input for all input files and configs
generated secrets file stored in data volumns keys dir
certs file stored in data volumns nginx dir

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-03-15 15:51:11 +08:00
Qian Deng
ab7c81dac6 Fix: the adminserver caused regression
Remove some code related to adminserver
Fix some issues by adminserver removeing

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-03-08 16:46:13 +08:00
Qian Deng
5f80fe7b8a Refacotr the prepare script base on the proposal https://github.com/goharbor/community/pull/22
Signed-off-by: Qian Deng <dengq@vmware.com>
2019-03-08 16:46:13 +08:00
Daniel Jiang
894f3774b1
Merge pull request #6933 from michmike/master
Delete open_source_license because it interferes with the CNCF recognized license for Harbor
2019-02-25 13:55:02 +08:00
stonezdj
0cba36d79f Remove everything of adminserver
Signed-off-by: stonezdj <stonezdj@gmail.com>
2019-02-22 16:34:39 +08:00
Daniel Jiang
1832699e93 Bump up the migrate tool of notary
fixes #5863
The migrate binary that we include in notary is quite out dated.
Additionally it introduced a breaking change, more details see #5863

In this commit a go program was added to workaround this issue to ensure the
migration process works, and refined bootstrap scripts and make process accordingly.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-02-21 00:36:24 -08:00
Michael Michael
cc814c69de Update Makefile
Signed-off-by: Michael Michael <michmike@users.noreply.github.com>
2019-02-18 17:04:00 -06:00
Qian Deng
4a4ebc2fba Enhhance: bump chartmuseum version to 0.8.1
bump the version of chartmuseum to 0.8.1

Signed-off-by: Qian Deng <dengq@vmware.com>
2019-02-15 15:19:53 +08:00
wang yan
c77b387c53 Upgrade registry binary to v2.7.0
Signed-off-by: wang yan <wangyan@vmware.com>
2019-02-13 10:24:08 +08:00
Daniel Jiang
5d59d6fab8 Bump up golang to 1.11.2
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-01-11 14:44:32 +08:00
wang yan
21ece6e413 Bump up clair version to v2.0.7
This commit is to bump up clair version to v2.0.7, this release
includes a fix for mapping Ubuntu Cosmic to the proper namespace.

Signed-off-by: wang yan <wangyan@vmware.com>
2018-11-26 14:46:16 +08:00
De Chen
e3b061c415 Fix the Makefile for MacOS to run harbor locally for dev (#5502)
* fix sed -i in MacOS, BSD sed and Linux sed are different, use sed -i -e instead
* change target name from version to ui_version
* fix cleandockercomposefile target to clean all generated docker-compose files

Signed-off-by: 陈德 <chende@caicloud.io>
2018-09-25 15:12:43 +08:00
Daniel Jiang
595304151a Bump up Clair to v2.0.6 (#5940)
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-09-22 13:08:17 +08:00
wang yan
18411349b5 Add installer pipeline in travis
Signed-off-by: wang yan <wangyan@vmware.com>
2018-09-21 17:20:54 +08:00
De Chen
970d84f1b9 Add comment format check to makefile and travis (#5832)
* Add comment format check to makefile and travis

Signed-off-by: 陈德 <chende@caicloud.io>
2018-09-20 11:48:38 +08:00
Qian Deng
7873a0312a Rename harbor-ui to harbor-core
1. Update the nginx.conf
2. Update Makefile
3. Update docker-compose
4. Update image name
5. Rename folder ui to core
6. Change the harbor-ui's package name to core
7. Remove unused static file on harbor-core
8. Remove unused code for harbor-portal

Signed-off-by: Qian Deng <dengq@vmware.com>
2018-09-19 16:35:13 +08:00
Qian Deng
ac6c9d79ba Remove nodeclarity
Remove the nodeclarity container related code and ui_builder parameter no longer needed when build clarity

Signed-off-by: Qian Deng <dengq@vmware.com>
2018-09-08 20:33:21 +08:00
Qian Deng
d797c50438 Fix trivial issues about rename ui_ng to portal
Update ui_ng to portal

Signed-off-by: Qian Deng <dengq@vmware.com>
2018-09-07 13:25:41 +08:00
Qian Deng
870653a5fb Update nginx config to redirect traffic to specific backend
1. Update nginx.conf file
2. Update photon makefile
3. Update global makefile

Signed-off-by: Qian Deng <dengq@vmware.com>
2018-09-07 13:21:27 +08:00
Yan
4eba01fc31 Clean make file unused code and unify docker build method (#59)
Signed-off-by: Yan <wangyan@vmware.com>
2018-09-04 17:18:15 +08:00
Daniel Jiang
179d7be60a Remove 'ha' directory reference in Makefile (#5783)
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-09-03 14:00:38 +08:00
Daniel Jiang
823a9d11e9 Bump Clair to v2.0.5
The PR to fix the Alpine issue has been merged to Clair's release-2.0
branch, and released v2.0.5.
This commit updates Harbor to include that change and re-enable
Clair's updaters by default.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-09-02 12:27:04 +08:00
wang yan
f8b964d8cf Extend configuration length to 1024 in DB
Signed-off-by: wang yan <wangyan@vmware.com>
2018-08-31 13:42:24 +08:00
wang yan
fed46b7a0b Revise and cleanup travis script, and misspell/gofmt to makefile
Signed-off-by: wang yan <wangyan@vmware.com>
2018-08-29 16:09:46 +08:00
Daniel Jiang
c4eaf25ed1
Merge pull request #5742 from reasonerjt/remove-compose-ha
Remove reference of docker-compsoe based HA
2018-08-28 15:09:22 +08:00
Daniel Jiang
e1153eec0a Remove reference of docker-compsoe based HA
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-08-28 14:04:44 +08:00
wang yan
854f606f63 upgrade notary to latest release v0.6.1
Move the notary-server and notary signer into ./notary/release-${notaryversion} as this will not impact the
release branches, the binaries in ./notary are v0.5.1.

Signed-off-by: wang yan <wangyan@vmware.com>
2018-08-28 13:51:55 +08:00
Daniel Jiang
dcf4e2ee78 Update import path in go code
vmware -> goharbor

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-08-23 17:50:53 +08:00
Wenkai Yin
95c0c259ec Add E2E API test cases for replication based on labels (#5639)
This commit creates a new E2E API test case to verify the replication based on label

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2018-08-16 18:36:27 +08:00
wangyan
29d0d51403 Signed-off-by: wangyan <wangyan@vmware.com>
Add clean registry cache to gc job

To workaround the issue: https://github.com/docker/distribution/issues/2094
GC needs to clean cache before to call the docker reigstry api to delete blobs.
Otherwise, the following docker push will not be performed as docker registry
does not clean cache in GC, it thinks the image is still there, and the new
blobs will be uploaded.
2018-08-13 02:58:27 -07:00
wangyan
7713764aec Batch update docker image namespace to goharbor
This commit is to move all the images of harbor from vmware to goharbor
2018-08-09 23:24:21 -07:00
wangyan
1c8d77a446 Add secure go result into CI pipeline 2018-08-06 23:53:31 -07:00
stonezdj
edcb2fbf05 Add Harbor API Test based on python API
Correct swagger.yaml format
Add make task swagger_client to gen and install python client.
Add LDAP related Test
2018-08-01 10:43:23 +08:00
Yan
9524c70f5e
Fix error in package offline installe (#5395)
It has a redundant slash in the end of docker save command,
that causes the command with merge the following lines into
save cmd, and then to fail package offline installer.
2018-07-26 10:09:22 +08:00
Daniel Jiang
c6c5745183 Remove vmware/photon image from offline package (#5347)
We used to package vmware/photon:1.0 in the offline package for product
integration.  Currently this is no longer needed by downstream product.
This commit removes it.
2018-07-25 11:03:00 +08:00
Steven Zou
0227a1315a Keep the chart server related configurations in adminserver
append chart server related config options to the supporting list of adminserver
provide chart server related config access method in the API layer
update prepare script and ui env template file to enable cache driver config for chart server API
append flag info in the systeminfo API to indicate if chart server is deployed with Harbor
refactor the response rewriting logic to return structual error object
add api init method to initilizing objects required in API handlers
chage owner of the storage folder
update offline/online package scripts in Harbor-Util.robot
2018-07-20 19:40:33 +08:00
Steven Zou
726d81803b Fix conflicts in Makefiles and prepare script files with upstream 2018-07-19 03:17:05 -07:00
Steven Zou
22ea7dd91f Update the related build scripts to package the chart repo server
add env file template for chart repo server in make/common/config/chartserver
update the Makefiles to support build chart repo server
add docker file and related build scripts for upstream chart server - chartmuseum
update prepare to support generating chart server related configs
add docker compose file for the chart server
add build/install command options to install with/without chart repo server
update install.sh to support chart repo server installation
2018-07-19 16:47:05 +08:00
Daniel Jiang
6b92a538b4 Update Makefile and OSL (#5337)
This commit adds the OSL to repo, and updates Makefile to package it.
2018-07-19 15:47:37 +08:00
Yan
d5b85a6748
Add the registry controller httpserver, it's responsible for controlling (#5265)
docker regsitry. This version has the API to call regsitry GC with jobservice
secret. Seprates it into a standalone container as do not want to invoke two
processes in one container.

It needs to mount the registry storage into this container in order to do GC,
and needs to copy the registry binary into it.
2018-07-16 16:50:28 +08:00
Daniel Jiang
bc1969156e Bump up clair to v2.0.4
This commit bump up clair to v2.0.4.  The current build process is
download the binary from google storage, the update of the binary in
google storage is not reflected in this commit.
2018-07-12 13:59:51 +08:00
Yan
d366134fe8
Fix bug of packaging offline installer (#5245)
The init sql script name nad path was changed by PR #5197, this
commit is to update these and log the package command to console,
make it more easy to debug in future. Also remove the action to
pull migrator as it will built each time locally.
2018-07-04 20:03:44 +08:00
Deng, Qian
edbe2fe620 Update migrator to 1 6 0
1. Add new alembic_pg folder for postgres
2. Add migration file for 1.6.0
3. Update version to 1.6.0
4. update migrator dockerfile
2018-07-02 21:23:47 +08:00
Yan
6d800cabbd
enable migrator to support 1.5.0 migration from mysql to pgsql (#5029)
This commit is to enable data migrator to support migrates data
from mysql to pgsql, this is a specific step for user to upgrade
harbor across v1.5.0, as we have move harbor DB to pgsql from
1.5.0. It supports both harbor and notary db data migration,
and be split into two steps with dependency.

It also fix issue #4847, add build DB migrator in make process.
2018-06-01 14:58:43 +08:00
Deng, Qian
2ef1a92a93 Add ng-packagr to packaging harbor-ui npm repo
Using ng-packagr to build and publish npm repo
2018-05-09 10:59:22 +08:00
Daniel Jiang
a8737116a6
Merge pull request #4836 from wy65701436/fix-offline-installer
Remove useless db images from the offline installer
2018-05-02 14:06:36 +08:00
wangyan
d6da200ccb Remove not used db images from the offline installer 2018-05-01 19:57:41 -07:00
Yan
ae257433cc
Fully migrate harbor db to postgresql (#4689)
* Merge harbor db to postgres
2018-04-27 02:27:12 -07:00
Yan
11ea4c5f00
Update migrator image tag to v1.5.0 (#4679) 2018-04-16 19:45:34 +08:00
Yan
cd584822c1 Add harbor migrator into offline installer (#4596) 2018-04-09 10:30:50 +08:00
Yan
ba231aa070
Add radis image into offline installer (#4582) 2018-04-04 01:20:24 -07:00
Steven Zou
250360307b Modify docker compose file template and make file to enable new job service
Fix typo in Makefile under photon

Fix version tag issue of redis container

Assign container name for redis container

Update docker compose template to enable network for redis

Remove exposed ports of redis from compose yaml tpl
2018-03-30 16:52:55 +08:00
Yan
2095fb0c0a
Update harbor build naming convention (#4461) 2018-03-23 02:56:29 -07:00
Yan
5edbd00318
Revise harbor build version (#4445) 2018-03-21 13:03:49 +08:00
Tan Jiang
f83c65bcc5 Reduce the output of build.
The following are done to avoid travis-ci failing due to too much log
size.
1) Update Makefile and scripts to make go build less verbose.
2) Make tdnf less verbose
2018-02-27 20:54:52 +08:00
yixingj
9c8706b0ce Fix images tag issue.
move the image tag out
only remove the first : in the keypair.
2018-01-25 17:34:47 +08:00
wangyan
bb10e018a5 Change harbor build number to drone number plus git commit 2018-01-22 02:13:42 -08:00
Deng, Qian
b3e65ed71e Update migration tool for v1.4
1. Update database meta file
2. Add migration file for 1.4
2018-01-16 15:38:51 +08:00
yixingj
ecd0bbf2dd Enable Clair in HA
Run clair with Core Harbor services
Add check logic for Clair DB
2017-12-21 23:23:35 +08:00
wangyan
cfce2ca4e9 Refine image tag name and script path when to build offline bundle 2017-12-18 23:50:48 -08:00
wangyan
1e750a1ed4 Unify images tags and build process 2017-12-14 23:52:18 -08:00
yixingj
d328e2586e HA installation script
Add --ha options when install Harbor.

Currently it does nothing.
2017-12-07 22:56:57 +08:00
reasonerjt
074aa352ba Bump up mariaDB's version 2017-11-21 06:42:41 -08:00
yixingj
cb47b44ee8 Fix clair images version issue
Update clair image version in Makefile
2017-11-10 12:10:08 +08:00
Daniel Jiang
7a4a66cbd9
Merge pull request #3521 from wy65701436/fix-offline-installer
Return error on packaging offline installer failure
2017-11-07 10:53:27 +08:00
Yan
3d1713ac70
Merge pull request #3543 from wy65701436/update-migrator-13
Update migrator version to 1.3
2017-11-03 15:29:47 +08:00
root
6f335bdb1a Deprivilege harobr-log, harbor-db, registry image.
This change involves using non-root user to run the process of the
docker images.  Also made update in Dockerfile to make the containers
support "read-only" and introduce "HEALTHCHECK". Note the "read-only"
options are not enabled in docker-compose, to cover the very corner
case when user wants to update the container filesystem manually.

Remove read only option from docker-compose template by default
2017-11-02 23:35:06 -07:00
wangyan
ec8216a7db Update migrator version to 1.3 2017-11-02 23:23:19 -07:00
Daniel Jiang
f7967e22ab
Merge pull request #3482 from yixingjia/ossclairupdate
Update Clair base images
2017-11-02 14:23:33 +08:00
wangyan
6c092e2172 Return error on packaging offline installer failure 2017-11-01 03:09:45 -07:00
wangyan
9e1049e3c9 Update package offline installer with vmware/photon 2017-10-31 22:17:46 -07:00
yixingj
651cb81389 Update Clair base images
1>update clair base iamges to vmware/photon:1.0
2>update oss packages to latest
2017-10-30 17:04:14 +08:00
yixingj
2953ca9967 Update Notary base images and oss package
1> Update Notary server image to vmware/photon
2> Update NOtary signer image to vmware/photon
3> update oss package to latest
2017-10-30 14:46:33 +08:00
Tan Jiang
1871011a5d Provide Dockerfile and artifacts for building mariadb on photon OS.
Also update the docker-compose template such that the notary db instance
will be provisioned via the mariadb-photon image.
2017-10-20 14:41:36 +08:00
yixingj
3dc0f65fb3 Update OSS in postgresql image
1> update OSS in postgresql image
2> update postgresql to 9.6.5
2017-10-12 17:08:47 +08:00
Masataka Mizukoshi
96bc432b4f fix typo 2017-08-31 04:57:02 +00:00
yixingjia
549ad02ac7 Update postgresql to 9.6.4 (#3093) 2017-08-21 18:16:57 +08:00
wangyan
14d23b8a63 add migrator into offline installer 2017-07-31 18:06:30 +08:00
Yan
686b477775 update registry to 2.6.2 (#2851)
rm dockerfile

update

add comments
2017-07-24 02:19:32 -07:00
Yan
7040e5ad5a Clair photon (#2721)
* update

* update

* modify docker compose

* update tag

* update
2017-07-10 21:00:44 -07:00
yixingj
b2ac6c37d2 Also update clair in offline package 2017-07-05 18:28:41 +08:00