Commit Graph

55 Commits

Author SHA1 Message Date
Mitsuru Kariya
02d042a6b9
Reduce image size
Do following actions to reduce the size of the generated images.

- Change `COPY` command + `chown`/`chmod` command to `COPY` command +
  `--chown`/`--chmod` option.
  To prevent both files before/after `chown`/`chmod` commands from being
  recorded on different layers.
- Put all `tdnf` commands in a single `RUN` command and move `tdnf clean all`
  command to the end.
  To prevent the `tdnf` cache from being recorded on a layer, and the `tdnf`
  database from being recorded on multiple layers.
- Add `--link` option to `COPY` command.
  This does not contribute to image size reduction, but makes image building
  more efficient.
- Move `chown /etc/pki/tls/certs` to the `Dockerfile.base`.
  This does not contribute to image size reduction, but is used in
  conjunction with `COPY --link` to make image building more efficient.

The target images and their sizes are as follows. (The sizes are the value
when built locally)

- harbor-core : 185MB -> 118MB
- harbor-db : 285MB -> 263MB
- harbor-exporter : 108MB -> 79.1MB
- harbor-jobservice : 159MB -> 105MB
- harbor-registryctl : 160MB -> 104MB
- redis-photon : 179MB -> 170 MB
- standalone-db-migrator : 328MB -> 284MB

Note that harbor-log, harbor-portal, and nginx-photon have almost no effect,
and prepare is not directly executed by the user, so they are not included.

Also, registry-photon and trivy-adapter-photon are not included, since
PR#20622 and PR#20623 include equivalent action for these two, respectively.

Signed-off-by: Mitsuru Kariya <mitsuru.kariya@nttdata.com>
2024-06-21 02:08:34 +09:00
Yang Jiao
aefe1bce9d
Bump up PostgreSQL from 14 to 15 (#19789)
Fix #19512

Signed-off-by: Yang Jiao <yang.jiao@broadcom.com>
Co-authored-by: Yang Jiao <yang.jiao@broadcom.com>
2024-01-04 14:01:00 +08:00
Yang Jiao
5e130bd06c
Bump up photon version from 4.0 to 5.0 (#19006)
Signed-off-by: Yang Jiao <jiaoya@vmware.com>
2023-08-01 11:52:33 +08:00
Yang Jiao
70554fe4f1
Fix build harbor-db-base error (#19003)
Fix #19002

Signed-off-by: Yang Jiao <jiaoya@vmware.com>
2023-07-25 16:45:23 +08:00
Wang Yan
bf7c82b9a8
remove the notary from backend (#18668)
Since harbor deprecates notary since v2.9.0, this pull request targets to remove the code related with notary.

Signed-off-by: Wang Yan <wangyan@vmware.com>
2023-05-18 18:47:42 +08:00
Yang Jiao
cb8a3d0c74
Fix build db base image symlink error (#18673)
Fix #18672

Signed-off-by: Yang Jiao <jiaoya@vmware.com>
2023-05-12 15:45:27 +08:00
Yang Jiao
aaea068cce
Upgrade the internal PostgreSQL to 14 in 2.9.0 (#18612)
Fix #18338

Signed-off-by: Yang Jiao <jiaoya@vmware.com>
Co-authored-by: Wang Yan <wangyan@vmware.com>
2023-05-05 10:32:39 +08:00
Yang Jiao
f4f80ca8fb
Update DB Dockerfile.base (#18249)
1. Update the path to postgresql.conf.sample as it has changed in postgresql 13.10
2. Remove creating symbolic link as it already exists in postgresql 13.10

Signed-off-by: Yang Jiao <jiaoya@vmware.com>
2023-02-20 17:20:53 +08:00
Yang Jiao
b44c9d01dd
Drop build postgresql 9.6 (#17458)
Drop pg9.6, will not support upgrading from pg9.6 to pg13, if you use pg9.6 you can upgrade to 2.5 and upgrade to 2.7

Signed-off-by: Yang Jiao <jiaoya@vmware.com>
2022-09-21 17:59:25 +08:00
Yang Jiao
709a154724 Specify postgresql version to 13
Signed-off-by: Yang Jiao <jiaoya@vmware.com>
2022-01-27 06:46:15 +00:00
Yang Jiao
b4de95941b Specify postgresql version to 13
Signed-off-by: Yang Jiao <jiaoya@vmware.com>
2022-01-12 08:55:42 +00:00
Yurii Paneiko
be1e762b70
For some reason this script hangs on bzip2 presence checking (#15647)
Steps to reproduce:

Clone repo from master branch
Run: make install COMPILETAG=compile_golangimage
Wait until script will check that bzip2 is installed

Signed-off-by: YuriiPaneiko <yurapaneyko@gmail.com>
2021-10-14 20:11:14 +08:00
He Weiwei
28921e2997 fix: listen to 5443 when clean unexpected status of db
Closes #15081

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-06-08 06:41:23 +00:00
Wang Yan
ae06ac2fae
fix db issue on helm upgrade (#15028)
fixes #15000

It needs to double confirm that old pg is stopped before migration

Signed-off-by: wang yan <wangyan@vmware.com>
2021-06-01 15:59:41 +08:00
Wang Yan
66b8a8f8dd
add build arch parameter in Makefile (#14995)
* add build arch parameter in Makefile

Add parameter BUILDARCH for make file. DB base builds pg96 for x86_64 only

Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-06-01 10:38:05 +08:00
Wang Yan
39bdd7b506
pg upgrade failure handling (#14934)
To ensure the upgrade execution idempotence, it needs to clean the $PGDATANEW on pg_upgrade failure.
Otherwise, the upgrade will skip the upgrade process from the second time launch as the exist of $PGDATANEW.

Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-05-21 13:53:39 +08:00
Wang Yan
86185989cf
support pg upgrade (#14846)
1, use the pg source and photon spec to build postgres 9.6
2, install 9.6 on the photon 4.0
3, then leverage pg_upgrade to handle the pg major version migration

Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-05-20 16:25:50 +08:00
DQ
ffed6459c7 Fix: Use local host on db's healthcheck
hostname -i will malfunction in some cases like the `nsswitch.conf` file does'nt exist

Signed-off-by: DQ <dengq@vmware.com>
2021-04-16 18:37:24 +08:00
Wang Yan
d03a29e531 bump up photon to 4.0
Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-03-31 13:25:29 +08:00
Stefan Nica
1c768d0bf1 Use exec in harbor database entrypoint
The harbor-db pod takes a long time to terminate. Using an `exec`
command in the entrypoint ensures that Unix signals reach the
postgres process [1].

[1] https://docs.docker.com/engine/reference/builder/#exec-form-entrypoint-example

Signed-off-by: Stefan Nica <snica@suse.com>
2020-08-25 20:24:52 +02:00
DQ
b015440074 Remove expose port in dockerfiles
The export is dynamical now because of introduce of internal TLS

Signed-off-by: DQ <dengq@vmware.com>
2020-08-05 10:42:46 +08:00
He Weiwei
2a6fe801bc chore(db): change max_connections of postgres to 1024
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-07-14 07:34:37 +00:00
Daniel Jiang
6271da471b
Update health check script for harbor-db (#12103)
This patch remove the trailing space of the hostname introduced by
`hostname -i`.

The trailing space will cause resolution error after this patch is
applied to glibc in photon:
https://github.com/vmware/photon/blob/2.0/SPECS/glibc/glibc-fix-CVE-2019-10739.patch

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-05-30 14:05:39 +08:00
Ziming Zhang
572ebef685 feat(cicd) parameterize docker base image and external url
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2020-04-08 00:21:47 +08:00
wang yan
47793e77e3 update base file name ane pass base version to build file
Signed-off-by: wang yan <wangyan@vmware.com>
2019-11-12 19:12:49 +08:00
Wang Yan
544cc98971 add base images when to build harbor assets
* add base images when to build harbor assets

Signed-off-by: wang yan <wangyan@vmware.com>
2019-11-12 15:38:51 +08:00
Qian Deng
dacb1fc79e Add healthcheck in Dockerfile* redis* jobservice
Signed-off-by: Qian Deng <dengq@vmware.com>
2019-08-06 13:16:12 +00:00
Qian Deng
303471563f DB container run as non-root
Signed-off-by: Qian Deng <dengq@vmware.com>
2019-08-06 05:21:47 +00:00
Che-Wei Lin
7aa00aee87 fix hostname command not found (#7045)
Signed-off-by: mycroftlin <mycroftlin@tencent.com>
2019-03-15 10:52:47 +08:00
stonezdj
8c37b0877a Fix issue when query psql cli failed on more command
Signed-off-by: stonezdj <stonezdj@gmail.com>
2018-11-06 13:09:20 +08:00
Daniel Jiang
3d09089a9c Rebuild Harbor images based on photon:2.0 (#6054)
Make necessary change to make things work with photon 2.0 docker image.
Remove distro-sync to mitigate the build issue and add `--pull` to docker build
command to make sure the latest photon:2.0 will be pulled during build process.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-10-18 11:03:28 +08:00
Daniel Jiang
c8cb2f8481 Create shcema migration table in DB container
The migrate tool will try to create table schema_migration upon opening
the connection to DB.  This will cause error when there are multiple
instance of adminserver trying to access the migrator upon start.
This commit move the creation of the table during the initialization of
the DB container.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-10-12 17:17:31 +08:00
Yan
4eba01fc31 Clean make file unused code and unify docker build method (#59)
Signed-off-by: Yan <wangyan@vmware.com>
2018-09-04 17:18:15 +08:00
Daniel Jiang
6c664ee993 Update photon base images (#5346)
This commit update the base photon image from vmware/photon:1.0 to
photon:1.0, per suggestion by photon team.
2018-07-19 20:45:20 +08:00
Daniel Jiang
c9b1962b1e Initialise Harbor DB schema in Harbor UI/adminserver container
This commit fixes #5040, the harbor-db image will only contain empty
databases, and harbor ui container will use migrate tool to run initial
SQL scripts to do initialization.  This is helpful for the case to
configure Harbor against external DB or DBaaS like RDS for HA deployment
However, this change will results some confusion as there are two tables
to track schema versions have been using alembic for migration, for this
release we'll try to use alembic to mock a `migration` table during
upgrade so the migrator will be bypassed, in future we'll consider to
consolidate to the golang based migrator.
Another issue is that the UI and adminserver containers will access DB
after start up in different congurations, can't ensure the sequence, so
both of them will try to update the schema when started up.
2018-06-28 16:22:53 +08:00
Wenkai Yin 79628
0c56493fb6 Soft delete label
Modify the deletion of label to soft deletion, in this way the names of deleted labels referenced by replication rules can be shown to users
2018-06-07 17:14:12 +08:00
Yan
6d800cabbd
enable migrator to support 1.5.0 migration from mysql to pgsql (#5029)
This commit is to enable data migrator to support migrates data
from mysql to pgsql, this is a specific step for user to upgrade
harbor across v1.5.0, as we have move harbor DB to pgsql from
1.5.0. It supports both harbor and notary db data migration,
and be split into two steps with dependency.

It also fix issue #4847, add build DB migrator in make process.
2018-06-01 14:58:43 +08:00
Wenkai Yin
fa8bbe821a Modify unique constraint of table harbor_label in pgsql (#4811)
Add unique constraint to column name, scope and project_id of table harbor_label to make creating same name labels under different projects valid
2018-04-27 08:01:20 -07:00
Yan
ae257433cc
Fully migrate harbor db to postgresql (#4689)
* Merge harbor db to postgres
2018-04-27 02:27:12 -07:00
Wenkai Yin
73babbf1ab Modify unique constraint of table harbor_label
Add unique constraint to column name, scope and project_id  of table harbor_label to make creating same name labels under different projects valid
2017-12-19 22:15:56 +08:00
Deng, Qian
93c96da18b Upgrade migrator to 1.5.0 2018-03-30 22:45:32 +08:00
Tan Jiang
b6df6cf169 Add indexes to job tables and bump up schema version. 2018-03-28 16:15:54 +08:00
stone
203b1b52bb
Merge pull request #4415 from stonezdj/user_group_and_project_member
Add DAO for user group and project member
2018-03-26 15:21:20 +08:00
stonezdj
49d960b060 Add DAO for project member and user group 2018-03-26 14:38:32 +08:00
Tan Jiang
df69f7e410 Fix error in registry_sqlite.sql 2018-03-25 13:47:03 +08:00
Tan Jiang
6f1c46624f Update SQL script to add uuid to job tables.
We have to add the uuid/id mapping as new job service will only store uuid.

Further work is in feature branch for now, commit this change to
accelerate migration work.
2018-03-23 17:45:50 +08:00
Wenkai Yin
83cfe9814d Add unique constraint to columns label_id, resource_id, resource_name and resource_type in table harbor_resource_label 2018-03-21 16:24:21 +08:00
Wenkai Yin
838b439560 Implement filter repository and tags by label API 2018-03-21 10:51:06 +08:00
Wenkai Yin
bcf81224ad Update according to the comments 2018-03-14 13:42:19 +08:00
Wenkai Yin
36b9c4e458 Implement adding/removing labels to/from repositories and images API 2018-03-12 19:30:05 +08:00