harbor/docs/administration/vulnerability-scanning/import-vulnerability-data.md

66 lines
4.0 KiB
Markdown
Raw Normal View History

---
title: Import Vulnerability Data to an Offline Harbor instance
2020-02-11 16:24:43 +01:00
weight: 45
---
2019-10-17 15:47:25 +02:00
2020-01-13 14:48:49 +01:00
If Harbor is installed in an environment without an internet connection, Clair cannot fetch data from the public vulnerability database. In this case, the Harbor administrator must update the Clair database manually.
2019-10-17 15:47:25 +02:00
2020-01-13 14:48:49 +01:00
## Preparation
2019-10-17 15:47:25 +02:00
2020-01-13 14:48:49 +01:00
- You have an instance of Clair that has an internet connection. If you have another instance of Harbor that has internet access, this also works.
- Check whether your Clair instance has already updated its vulnerability database to the latest version.
2019-10-17 15:47:25 +02:00
2020-01-13 14:48:49 +01:00
1. Use `docker ps` to find out the container ID of the Clair service.
1. Run `docker logs <container_id>` to check the log of the Clair container. If you are using Harbor you can find the latest Clair logs under `/var/log/harbor/2017-xx-xx/clair.log`.
1. Look for logs that look like the following:
2019-10-17 15:47:25 +02:00
```
Jul 3 20:40:45 172.18.0.1 clair[3516]: {"Event":"finished fetching","Level":"info","Location":"updater.go:227","Time":"2017-07-04 03:40:45.890364","updater name":"rhel"}
Jul 3 20:40:46 172.18.0.1 clair[3516]: {"Event":"finished fetching","Level":"info","Location":"updater.go:227","Time":"2017-07-04 03:40:46.768924","updater name":"alpine"}
Jul 3 20:40:47 172.18.0.1 clair[3516]: {"Event":"finished fetching","Level":"info","Location":"updater.go:227","Time":"2017-07-04 03:40:47.190982","updater name":"oracle"}
Jul 3 20:41:07 172.18.0.1 clair[3516]: {"Event":"Debian buster is not mapped to any version number (eg. Jessie-\u003e8). Please update me.","Level":"warning","Location":"debian.go:128","Time":"2017-07-04 03:41:07.833720"}
Jul 3 20:41:07 172.18.0.1 clair[3516]: {"Event":"finished fetching","Level":"info","Location":"updater.go:227","Time":"2017-07-04 03:41:07.833975","updater name":"debian"}
Jul 4 00:26:17 172.18.0.1 clair[3516]: {"Event":"finished fetching","Level":"info","Location":"updater.go:227","Time":"2017-07-04 07:26:17.596986","updater name":"ubuntu"}
Jul 4 00:26:18 172.18.0.1 clair[3516]: {"Event":"adding metadata to vulnerabilities","Level":"info","Location":"updater.go:253","Time":"2017-07-04 07:26:18.060810"}
Jul 4 00:38:05 172.18.0.1 clair[3516]: {"Event":"update finished","Level":"info","Location":"updater.go:198","Time":"2017-07-04 07:38:05.251580"}
```
2020-01-13 14:48:49 +01:00
The phrase `finished fetching` indicates that Clair has finished a round of vulnerability updates from an endpoint. Make sure all of the `rhel`, `alpine`, `oracle`, `debian`, and `ubuntu` endpoints are updated correctly. If they have not, wait for Clair to get the data.
2019-10-17 15:47:25 +02:00
2020-01-13 14:48:49 +01:00
## Dump Vulnerability Data
2019-10-17 15:47:25 +02:00
2020-01-13 14:48:49 +01:00
1. Log in to the host, that is connected to Internet, on which the Postgres Clair database is running.
1. Dump Clair's vulnerability database by running the following commands.
2019-10-17 15:47:25 +02:00
2020-02-20 13:02:41 +01:00
{{< note >}}
The container name `clair-db` is a placeholder for the database container used by the internet-connected instance of Clair.
{{< /note >}}
2019-10-17 15:47:25 +02:00
```shell
$ docker exec clair-db /bin/sh -c "pg_dump -U postgres -a -t feature -t keyvalue -t namespace -t schema_migrations -t vulnerability -t vulnerability_fixedin_feature" > vulnerability.sql
$ docker exec clair-db /bin/sh -c "pg_dump -U postgres -c -s" > clear.sql
```
2019-10-17 15:47:25 +02:00
2020-01-13 14:48:49 +01:00
The files `vulnerability.sql` and `clear.sql` are generated.
2019-10-17 15:47:25 +02:00
2020-01-13 14:48:49 +01:00
## Back Up the Harbor Clair Database
2019-10-17 15:47:25 +02:00
Before importing the data, it is strongly recommended to back up the Clair database in Harbor.
2020-01-13 14:48:49 +01:00
```shell
docker exec harbor-db /bin/sh -c "pg_dump -U postgres -c" > all.sql
2019-10-17 15:47:25 +02:00
```
2020-01-13 14:48:49 +01:00
## Update the Harbor Clair Database
2019-10-17 15:47:25 +02:00
2020-01-13 14:48:49 +01:00
1. Copy the `vulnerability.sql` and `clear.sql` files to the host on which Harbor is running.
1. Run the following commands to import the data to the Harbor Clair database:
```shell
docker exec -i harbor-db psql -U postgres < clear.sql
docker exec -i harbor-db psql -U postgres < vulnerability.sql
```
2020-01-13 14:48:49 +01:00
## Rescan the Images
2019-10-17 15:47:25 +02:00
2020-02-20 13:02:41 +01:00
After importing the data, trigger the scanning process in the Harbor interface. For information about running a scan, see [Scan All Images](scan-all-images.md).