Commit Graph

993 Commits

Author SHA1 Message Date
DQ
1eeea6b888 Fix: fix logrotate is dir issue
Change it to bind command

Signed-off-by: DQ <dengq@vmware.com>
2020-03-13 14:58:45 +08:00
Wenkai Yin
a4a1913598 Repair the count usage during the upgrading
As the count quota is against artifact rather than tag in 2.0, the count usage should be recalculated

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-03-13 13:59:48 +08:00
Ted Guan
4ac31c6d46
Add API for query supported event types and notify types; Return policy name in last trigger info; Remove project_id unique constraint in table notification_policy (#11029)
Signed-off-by: guanxiatao <guanxiatao@corp.netease.com>
2020-03-11 18:06:58 +08:00
Wenkai Yin(尹文开)
8452100148
Merge pull request #10942 from ywk253100/200305_reference
Persistent the URLs and annotations of artifact references in database
2020-03-11 16:20:18 +08:00
Wang Yan
bd7940217a
upgrade golang version to v1.13.8 (#11006)
The vesrion contains two security bug fix - CVE-2020-0601, CVE-2020-7919

More details, see the golang milestone:

https://github.com/golang/go/issues?q=milestone%3AGo1.13.8+label%3ACherryPickApproved

Signed-off-by: wang yan <wangyan@vmware.com>
2020-03-11 12:20:06 +08:00
Ziming Zhang
695a2559be feat(cicd) use unified version as tag name, clean more
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2020-03-09 17:13:28 +08:00
Ziming Zhang
200c352c35 feat(cicd) use unified version as tag name
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2020-03-09 15:30:03 +08:00
Daniel Jiang
6d89553c4d
Merge pull request #10937 from reasonerjt/csrf-2.0
Update CSRF mechanism
2020-03-09 12:31:08 +08:00
Wenkai Yin(尹文开)
75eb7a8c5a
Merge pull request #10955 from wy65701436/migrate-access
add sql for migrating access log
2020-03-09 10:00:08 +08:00
Daniel Jiang
ae5ffce83a Update CSRF mechanism
This commit replaces beego's CSRF mechanism with gorilla's csrf library.
The criteria for requests to skip the csrf check remain the same.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-03-09 01:15:54 +08:00
wang yan
288c7790d0 add sql for migrating access log
1, loop each access log, change to resource/resource_type, and insert into audit log
2, loop each first push operation, change it to create repository and insert into audit log.

Signed-off-by: wang yan <wangyan@vmware.com>
2020-03-06 12:06:12 +08:00
wang yan
2b0b7576b2 Fix gc issue on clean the artifact trash
1, enable dao test for artifact trash
2, set default flush trash table to false
3, hanlder empty parameter in API call
4, add registry auth info into jobservice container

Signed-off-by: wang yan <wangyan@vmware.com>
2020-03-06 03:11:31 +08:00
stonezdj(Daojun Zhang)
49619e1907
Merge pull request #10939 from wy65701436/access-log-mgr
add audit logs API
2020-03-05 16:24:21 +08:00
wang yan
df237a5b17 add audit logs API
1, add API entry for get audit logs
2. add audit log manager to hanlder CRUD

Use the new format of audit log to cover differernt resource, artifact/tag/repostory/project

Signed-off-by: wang yan <wangyan@vmware.com>
2020-03-05 11:40:51 +08:00
Wenkai Yin
76c04b0219 Persistent the URLs and annotations of artifact references in database
Persistent the URLs and annotations of artifact references in database

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-03-05 10:54:45 +08:00
Will Sun
a5d9a3b65d
Merge pull request #10863 from AllForNothing/api-center
Fix Api cennter
2020-03-05 10:00:15 +08:00
Wenkai Yin(尹文开)
4fa4c4e74c
Merge pull request #10815 from cd1989/redis-idle-timeout
Set redis idle timeout for core
2020-03-03 14:10:22 +08:00
Wenkai Yin
4c9b59c904 Migrate artifact data in 2.0
Abstract extra attributes and annotations for artifacts stored in database

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-28 18:09:02 +08:00
jwangyangls
572510c82d
Merge pull request #10755 from Snaacker/master
Fix typo
2020-02-28 11:59:51 +08:00
Daniel Jiang
1823c984f7
Merge branch 'master' into redis-idle-timeout 2020-02-27 22:01:22 +08:00
AllForNothing
d41c5496a2 Fix Api cennter
Signed-off-by: AllForNothing <sshijun@vmware.com>
2020-02-27 15:55:20 +08:00
Wenkai Yin
02c2647e1e Use the repository name of artifact model
As we store the repository name in the artifact table, we can use it direclty in the code to reduce the database query

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-26 13:37:09 +08:00
stonezdj(Daojun Zhang)
a7e5873f46
Merge pull request #10821 from stonezdj/20200224_remove_notification
Remove registry notification and change core health check url
2020-02-25 13:34:37 +08:00
Ziming Zhang
94230b5e19 feat(cicd) fix some build problem
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2020-02-25 12:05:39 +08:00
stonezdj
6005101c95 Remove registry notification and change /api/ping
Update config.yaml.jinja to remove notification
Change api/ping in core/Dockerfile

Signed-off-by: stonezdj <stonezdj@gmail.com>
2020-02-25 11:24:21 +08:00
Wang Yan
948d45604c Revise the GC job flow,
1, set harbor to readonly
2, select the candidate artifacts from Harbor DB.
3, call registry API(--delete-untagged=false) to delete manifest bases on the results of #2
4, clean keys of redis DB of registry, clean artifact trash and untagged from DB.
5, roll back readonly.

Signed-off-by: wang yan <wangyan@vmware.com>
2020-02-24 18:29:55 +08:00
Wenkai Yin
bd204464f3 Remove dead code
Remove dead code

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-23 17:11:46 +08:00
dechen
e642a73280 Set redis idle timeout for core
Signed-off-by: dechen <xxyydream@gmail.com>
2020-02-23 12:31:56 +08:00
Wenkai Yin
9312b788dc Upgrade the artifact table
Split the table artifact into artifact and tags, and populate related data

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-21 20:37:31 +08:00
Wang Yan
22021a988e
Merge pull request #10753 from wy65701436/artifact-trash
add artifact trash manager
2020-02-19 19:14:48 +08:00
Steven Zou
f1374737f6
Merge pull request #10694 from danielpacak/feature/install_with_trivy
chore(install): Add --with-trivy arg to the installation script
2020-02-19 16:27:57 +08:00
Wang Yan
f160505686 add artifact trash manager
1, move the deleted artifact into trash
2, disable GC to delete the untagged manifest

Signed-off-by: wang yan <wangyan@vmware.com>
2020-02-19 14:52:58 +08:00
Tran Huy
2aed96851e Fix typo
Signed-off-by: Tran Huy <Tran.Huy@ericsson.com>
2020-02-18 13:26:59 +01:00
Wenkai Yin(尹文开)
341cb88cba
Merge pull request #10727 from ywk253100/200214_bump_up_legacy_api_version
Bump up legacy api version to v2.0
2020-02-18 10:37:25 +08:00
Wenkai Yin(尹文开)
d7903fcf1b
Merge pull request #10682 from ywk253100/200126_label
Support add/remove label to/from artifact
2020-02-17 11:33:45 +08:00
Daniel Pacak
1b60bb255c refactor(Makefile): Add variables for download URLs
Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
2020-02-14 11:58:59 +01:00
Wenkai Yin
94787ea60d Bump up the version of legacy APIs to v2.0
Bump up the version of legacy APIs to v2.0

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-14 13:16:30 +08:00
Daniel Pacak
70dda1387a chore: Configure Redis URL for Trivy adapter
Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
2020-02-13 17:57:02 +01:00
Daniel Pacak
4755439b75 chore: Build Trivy adapter from sources
Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
2020-02-13 15:32:57 +01:00
Wenkai Yin
93731eeb2e Support add/remove label to/from artifact
This commit add supporting for adding/removing label to/from artifacts and populates labels when listing artifacts

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-13 10:46:23 +08:00
Daniel Pacak
a642667ffc chore(install): Add --with-trivy arg to the installation script
Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
2020-02-12 23:47:56 +01:00
Ziming Zhang
6047a8102a fix(feat) fix prepare version in installer
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2020-02-06 10:58:29 +08:00
Will Sun
acfcd2d175
Merge pull request #10489 from AllForNothing/postinstall
Fix postinstall script in Docker
2020-02-03 14:13:12 +08:00
Daniel Jiang
2064a1cd6d Switch to basic authentication for registry
1. Add basic authorizer for registry which modify the request
to add basic authorization header to request based on configuration.
2. Set basic auth header for proxy when accessing registry
3. Switche the registry to use basic auth by default and use the basic
authorizer to access Harbor.
4. Make necessary change to test cases, particularly
"test_robot_account.py" and "docker_api.py", because the error is
changed after siwtched to basic auth from token auth.  #10604 is opened
to track the follow up work.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-01-31 21:46:47 +09:00
Wenkai Yin
7dc28bcff9 Add foreign key to avoid the concurrent issue
Add foreign key to avoid the concurrent issue when operating the artifacts, tags and references

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-01-21 16:03:51 +08:00
sshijun
3175d5f646 Fix postinstall script in Docker
Signed-off-by: sshijun <sshijun@vmware.com>
2020-01-15 16:28:57 +08:00
Daniel Jiang
a087ba02e3 Populate basic auth information for registry
This commit updates `prepare` and templates to populate the credential
for registry for basic authentication.

A temporary flag `registry_use_basic_auth` was added to avoid breakage.
It MUST be removed before the release.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-12-31 14:50:46 +08:00
Wenkai Yin(尹文开)
5f11ca4df6
Merge pull request #10248 from ywk253100/191213_controller
Define the interface for artifact manager
2019-12-24 17:44:18 +08:00
Wenkai Yin
ac605db5da Define the controller/manager interface for artifact and tag
1. Define the controller/manager interface for artifact and tag
    2. Provide a null implementation for artifact manager

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-12-24 17:12:55 +08:00
Ziming
e32649adb4 enhance[cicd] introduce github action for CICD
In order to replace travis.
Implement 5 CI jobs
- UTTEST
- APITEST_DB
- APITEST_LDAP
- OFFLINE
- UI_UT

Signed-off-by: Ziming Zhang <zziming@vmware.com>
2019-12-17 18:36:33 +08:00
Wenkai Yin
7b41c900a7 Create the models for OCI supporting
This commits does some basic init work for supporting OCI:
1. Create the artifact and tag model
2. Create database tables

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-12-12 16:05:38 +08:00
Steven Zou
7bf9372f32 chore[api]:rename API folder to api
- update swagger yaml file reference in `Makefile`
- update swagger yaml file reference in `README`
- update swagger yaml file reference in `docs/configure_swagger.md`
- update swagger yaml file reference in `make/photon/portal/Dockerfile`
- update swagger yaml file reference in `tests/swaggerchecker.sh`

Signed-off-by: Steven Zou <szou@vmware.com>
2019-12-11 17:41:27 +08:00
Wang Yan
550d690997
Merge pull request #10135 from bitsf/upgrade_clair
upgrade clair to v2.1.1
2019-12-06 11:52:10 +08:00
Ziming
9cad403762 fix(build): npm install with special endpoint (#10168)
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: Iaaf33310a2621d58cdc3b9d3359607a961fef05e
2019-12-06 11:45:48 +08:00
Wang Yan
2a63382236
Merge pull request #10047 from bitsf/makefile_clean
optimize the makefile process
2019-12-05 19:03:19 +08:00
He Weiwei
4ea5c41553
chore(scanner): upgrade clair scanner to 1.0.1 (#10147)
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-12-05 17:52:37 +08:00
Ziming Zhang
332f88ec8c add make clean
Change-Id: Ibe806972a19cd69bfd90be051cdc340c4d7c6afb
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2019-12-05 14:44:07 +08:00
Ziming Zhang
744ae62831 upgrade clair to v2.1.1
Change-Id: Idb2ad0470a51666d75895d8c5e68d80a67e05276
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2019-12-05 14:27:26 +08:00
Wenkai Yin(尹文开)
d145f4baf4
Merge pull request #10034 from ywk253100/191128_clean
Clean up admiral-related code
2019-12-04 17:33:31 +08:00
stonezdj(Daojun Zhang)
339c1d4cab
Merge pull request #10088 from reasonerjt/authproxy-cert-setting
Support pinning to authproxy server's cert
2019-12-04 14:03:27 +08:00
Daniel Jiang
7bb71db478
Merge pull request #10003 from ninjadq/migrator_miss_component_no_proxy
Add default domainname for no_proxy
2019-12-03 10:50:32 +08:00
Daniel Jiang
902598fabd Support pinning to authproxy server's cert
This commit add an attribute to configurations, whose value is the
certificate of authproxy server.  When this attribute is set Harbor will
pin to this cert when connecting authproxy.
This value will also be part of the response of systemInfo API.

This commit will be cherrypicked to 1.10 and 1.9 branch.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-12-03 07:31:26 +08:00
Qian Deng
e5f8c2d779
Merge pull request #10022 from ninjadq/fix_ca_bundle_path_join
Fix ca bundle path join issue
2019-12-02 11:31:23 +08:00
Wenkai Yin
dd2bc0ecef Clean up admiral-related code
Clean up admiral-related code as it's useless

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-11-28 17:28:54 +08:00
DQ
79344887b9 Fix ca bundle path join issue
CA bundle name start with '/' will break the os path join

Signed-off-by: DQ <dengq@vmware.com>
2019-11-27 18:37:45 +08:00
Will Sun
a52b99e180
Merge branch 'master' into remove-lib 2019-11-27 17:44:30 +08:00
DQ
ed6438cf69 Add default domainname for no_proxy
All internal service and known internal hostname shuold add to no_proxy by default

Signed-off-by: DQ <dengq@vmware.com>
2019-11-27 15:10:42 +08:00
sshijun
c692f5c67e Move lib into src for better UI building
Signed-off-by: sshijun <sshijun@vmware.com>
2019-11-27 09:59:06 +08:00
wang yan
7b664f64f1 Bump up golang version to v1.13.4
Signed-off-by: wang yan <wangyan@vmware.com>
2019-11-26 19:18:45 +08:00
Wang Yan
60101c4ea1
Merge pull request #9964 from heww/fix-clair-updaters-disable
fix(prepaire,clair): disable clair updaters when its interval is 0
2019-11-22 13:38:34 +08:00
He Weiwei
b8308f41a0 fix(prepaire,clair): disable clair updaters when its interval is 0
Closes #9961

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-11-22 03:31:20 +00:00
stonezdj
938168b8ad Failed to start harbor when proxy is set
Fix #9614, all communication between internal components should bypass the proxy
Add chartmuseum, notary-server,clair-adapter to the no_proxy list in harbor.yml

Signed-off-by: stonezdj <stonezdj@gmail.com>
2019-11-22 11:22:51 +08:00
stonezdj(Daojun Zhang)
2b0ede5341
Merge pull request #9829 from reasonerjt/rm-k8s-install
Remove scripts to deploy Harbor on k8s
2019-11-19 15:53:49 +08:00
Wang Yan
eab974419c
Merge pull request #9825 from stonezdj/bug_9681
Avoid to create duplicated immutable tag rules in the same project
2019-11-18 17:26:22 +08:00
stonezdj
15898f2069 Avoid to create duplicated immutable tag rules in the same project
Fix #9681, add constraint on immutable_tag_rule and catch the error

Signed-off-by: stonezdj <stonezdj@gmail.com>
2019-11-15 14:46:23 +08:00
Daniel Jiang
2fcd174e4b
Merge pull request #9828 from wy65701436/cii-docker-base
add base images when to build harbor assets
2019-11-15 14:24:11 +08:00
He Weiwei
fe69a5df99 build(scanner-adapter): bump up clair adapter to v1.0.1-rc2
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-11-13 02:35:21 +00:00
wang yan
47793e77e3 update base file name ane pass base version to build file
Signed-off-by: wang yan <wangyan@vmware.com>
2019-11-12 19:12:49 +08:00
Wang Yan
544cc98971 add base images when to build harbor assets
* add base images when to build harbor assets

Signed-off-by: wang yan <wangyan@vmware.com>
2019-11-12 15:38:51 +08:00
Daniel Jiang
39a22d4470 Remove scripts to deploy Harbor on k8s
This commit removes scripts have been deprecated for a while and users have to use
helm chart to deploy Harbor on top of k8s cluster.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-11-11 19:23:22 +08:00
Yogi_Wang
cddc1149f1 Modify the memory of nodejs used from 8192MB to 2048MB
Signed-off-by: Yogi_Wang <yawang@vmware.com>
2019-11-11 17:05:42 +08:00
Wang Yan
6da183d576
Merge pull request #9800 from ninjadq/failure_earlier_of_ca_bundle_permission_check
Failure earlier of ca bundle permission check
2019-11-11 14:09:21 +08:00
Wang Yan
0b09bd14b1
Merge pull request #9756 from ninjadq/add_ignore_media_type
Add ignore mediatypes for registry
2019-11-08 18:34:13 +08:00
DQ
80c3e76b5a check the permission of ca bundle file
CA bundle need check before use

Signed-off-by: DQ <dengq@vmware.com>
2019-11-08 15:34:17 +08:00
DQ
7237067d63 Bump config version
Bump version to 1.10

Signed-off-by: DQ <dengq@vmware.com>
2019-11-07 17:06:20 +08:00
Daniel Jiang
06e4e124d8
Refine request handle process (#9760)
* Refine request handle process

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-11-07 13:02:17 +08:00
DQ
45868107aa Add ignore mediatypes for registry
Add these mediatypes to reduce the amount of registry event

Signed-off-by: DQ <dengq@vmware.com>
2019-11-06 21:39:08 +08:00
Wang Yan
a9e8c6a430
Merge pull request #9738 from ninjadq/fix_install_script
Fix install script
2019-11-06 14:24:07 +08:00
Steven Zou
240f771006
Merge pull request #9733 from steven-zou/fix/scan_all_enhancements
do improvements to the scan all job
2019-11-05 16:22:50 +08:00
Steven Zou
ebc5d2482b do improvements to the scan all job
- update scan all job to avoid sending too many HTTP requets
- update scan controller to support scan options
- update the db schema of the scan report to introduce requester
- introduce scan all metrics to report the overall progress of scan all job
- fix the status updating bug in scan report
- enhance the admin job status updats
- add duplicate checking before triggering generic admin job
- update the db scheme of admin job

fix #9705
fix #9722
fix #9670

Signed-off-by: Steven Zou <szou@vmware.com>
2019-11-05 15:12:07 +08:00
Wang Yan
27cb25cc04
Merge pull request #9400 from ninjadq/inject_certs_to_non_root
Inject certs to non root
2019-11-05 14:49:08 +08:00
DQ
75c91273bc Fix install script
Move load images to above

Signed-off-by: DQ <dengq@vmware.com>
2019-11-05 11:22:30 +08:00
DQ
ece321a53a Change certs's owner to 10000
Signed-off-by: DQ <dengq@vmware.com>
2019-11-04 17:38:41 +08:00
Wang Yan
3f39b0ba4f
Merge pull request #9550 from ninjadq/enable_https_by_default
Enable https by default
2019-11-04 16:51:33 +08:00
DQ
a0462f0baa Change the clair container to non root user
Signed-off-by: DQ <dengq@vmware.com>
2019-11-04 11:36:39 +08:00
DQ
d0ed075b91 Change chartmuseum container to non-root
Signed-off-by: DQ <dengq@vmware.com>
2019-11-04 11:36:39 +08:00
DQ
1c76d52152 Add registryctl to non-root
And the install_cert.sh will changed for non-root too

Signed-off-by: DQ <dengq@vmware.com>
2019-11-04 11:36:39 +08:00
Qian Deng
336dbfd3e1
Merge pull request #9452 from ninjadq/add_certs_in_specific_dir
All certs in /harbor_cust_certs will appended to ca_bundle
2019-11-01 13:13:18 +08:00
Daniel Jiang
02dab35a43
Merge pull request #9683 from ninjadq/upgrade_python_rand_gen
Replance python ran lib to secrets
2019-10-31 21:51:38 +08:00
DQ
873d9f5b82 Enable https by default
1. Umcomment https related configs
2. Remove the https prepare related thing in ci

Signed-off-by: DQ <dengq@vmware.com>
2019-10-31 20:58:09 +08:00
DQ
2529f69fba All certs in /harbor_cust_certs will appended to ca_bundle
Signed-off-by: DQ <dengq@vmware.com>
2019-10-31 20:51:08 +08:00
Daniel Jiang
bc65609a10
Merge pull request #9657 from wy65701436/quota-sync-switcher
add a switcher for quota sync on core launch
2019-10-31 19:22:23 +08:00
Wang Yan
fa784d7514
Merge pull request #9649 from wy65701436/fix-9081
add ldflags for harbor compiler and linker
2019-10-31 19:14:16 +08:00
DQ
6c01049d94 Replance python ran lib to secrets
Secrets is included in python 3.6, so just import and use it

Signed-off-by: DQ <dengq@vmware.com>
2019-10-31 17:23:19 +08:00
wang yan
c46d7e856a add a switcher for quota sync on core launch
As the quota sync is default called by harbor-core on every launch, and it will break the launch process if any failure throwed.

1, The commit is to provide an switcher for the system admin to bypass the quota sync.
2, In case Harbor goes into the restarting cycle.

Harbor already provides an internal API to sync quota data, in the failure case,
system admin can launch harbor and call the /api/internal/syncquota to sync quota.

Signed-off-by: wang yan <wangyan@vmware.com>
2019-10-31 16:17:27 +08:00
Steven Zou
7b6e83090e create API folder to keep API swagger files
- create API folder
- move harbor API swagger file to API/harbor
- add scanner adapter open API swagger file to API/scanner
- update protal build Dockerfile
- update swagger explorer build command in Makefile

Signed-off-by: Steven Zou <szou@vmware.com>
2019-10-30 21:41:03 +08:00
wang yan
253e87d186 inject ldflags for harbor compiler and linker
1, replace the UIVERSION file with ldflags, which is generarted by make to inject into the UI core.
2, inject additional ldflags for harbor compiler

Signed-off-by: wang yan <wangyan@vmware.com>
2019-10-30 18:31:42 +08:00
He Weiwei
b0f7404231
chore(log): log level support for clair adapter (#9640)
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-10-29 16:50:26 +08:00
He Weiwei
28e0c0693b Upgrade clair adapter to v1.0.0
1. Upgrade clair adapter to v1.0.0.
2. Make the clair adapter which installed by harbor immutable and using internal registry address.
3. Add support to build clair adapter image from binary.
4. Switch to ScannerPull action when make authorization for the scan request.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-10-26 17:25:36 +00:00
wang yan
f9996663d8 update immutable rule API
1, unify disable and enable
2, fix update rule error

Signed-off-by: wang yan <wangyan@vmware.com>
2019-10-25 14:11:07 +08:00
Steven Zou
cb59ba3bbc support using internal registry addr to perform scan
- do changes to the sql schema
- add `UseInternalAddr` and `Immutable` properties to scanner registration
- support multiple authentication type
  - basic
  - bearer token

Signed-off-by: Steven Zou <szou@vmware.com>
2019-10-24 18:28:35 +08:00
Wang Yan
d503f2a245
Merge pull request #9489 from reasonerjt/bump-up-golang
Bump up golang to 1.12.12
2019-10-22 10:54:35 +08:00
Daniel Jiang
6e131d511c Hide DB URL from notary migrator script
This commit modify the log message from upstream notary DB migrator, to
make sure the DB URL is not displayed.
Fixes #7510

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-10-21 23:10:27 +08:00
Daniel Jiang
dbe6ebceec Bump up golang to 1.12.12
Bump up the golang for compiling the binaries to 1.12.12
This commit also includes some minor changes to Makefile to fix issue in
building the binary files.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-10-21 15:55:58 +08:00
He Weiwei
bf6a14c9ad
feat(role): introduce a limited guest role (#9403)
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-10-20 14:21:28 +08:00
wang yan
8c4cf17129 disable noglob after import common
Signed-off-by: wang yan <wangyan@vmware.com>
2019-10-18 17:55:33 +08:00
Steven Zou
0f16913635 rebase: resolve the code confilcts with master
Signed-off-by: Steven Zou <szou@vmware.com>
2019-10-17 17:42:41 +08:00
He Weiwei
8964a8697a build(clair): internal clair adapter when install with clair
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-10-17 12:00:51 +08:00
Steven Zou
f18afc0a3f do changes to let the vul policy check compatiable with new framework
- update the scan/scanner controller
- enhance the report summary generation
- do changes to the vulnerable handler
- remove the unused clair related code
- add more UT cases
- update the scan web hook event
- drop the unsed tables/index/triggers in sql schema

Signed-off-by: Steven Zou <szou@vmware.com>
2019-10-16 23:15:26 +08:00
stonezdj(Daojun Zhang)
0fa4934679
Merge pull request #8596 from JakubOnderka/patch-4
nginx: Remove TLSv1.1 support
2019-10-16 11:39:55 +08:00
wang yan
25f638a989 Merge branch 'master' of https://github.com/goharbor/harbor into robot-invisiable 2019-10-14 14:35:45 +08:00
wang yan
3e81bd7f1d add visible attribute to robot account
The commit is to make robot controller could create invisible robot account for internal use

Signed-off-by: wang yan <wangyan@vmware.com>
2019-10-12 00:51:48 +08:00
Steven Zou
9fd8b6306c refactor code to reflect code review comments
- refactor the db schema \
- refactor  permission checking in API handlers \

to follow the latest code/interface changes

Signed-off-by: Steven Zou <szou@vmware.com>
2019-10-11 18:07:47 +08:00
Steven Zou
58afd8e14b [stage3] support pluggable scanner
- implement scan controller
- add scan resource and update role bindings
- update registration model and related interfaces

Signed-off-by: Steven Zou <szou@vmware.com>

- implement scan API to do scan/get report/get log
- update repository rest API to produce scan report summary
- update scan job hook handler
- update some UT cases

- update robot account making content
- hidden credential in the job log

Commnet scan related API test cases which will be re-activate later
fix #8985

fix the issues found by codacy

Signed-off-by: Steven Zou <szou@vmware.com>
2019-10-11 12:53:02 +08:00
Daniel Jiang
49f12d0b16
Merge pull request #8786 from reasonerjt/fix-8622
Extract shared func for checkenv and install scripts
2019-10-10 16:53:51 +08:00
He Weiwei
6fbb77d65a
build(portal): npm registry configurable and build cache support (#9356)
1. Introduce NPM_REGISTRY in Makefile to support npm registry
configuration when build portal image.
2. Install npm pkgs before copy portal src so that build cache works for
npm install in portal image.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-10-10 15:29:50 +08:00
Daniel Jiang
b9154a858b Extract shared func for checkenv and install scripts
This commit fixes #8622 by extract shared func into common.sh to avoid
inconsistency in future.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-10-10 15:07:09 +08:00
Wang Yan
7e73dfb754
Merge pull request #9221 from wy65701436/fix-9186
patch registry fix of issue 2553
2019-09-26 19:34:18 +08:00
wang yan
3cf7e702be patch regsitry fix of issue 2553
This commit is target to fix harbor issue #9186, which root cause is mentioned by
https://github.com/docker/distribution/issues/2553, and fixed by https://github.com/docker/distribution/pull/2879.

As the latest distribution release(v2.7.1) does not contain this fix, but it will break the quota migraion process on S3 storage, we have to path this fix into Harbor regsitry binary.

[Tag Version]
It uses the issue number(2553) as the tag naming convention, like v2.7.1-patch-2553, means that we patch the fix of issue 2553 into v2.7.1.

[Note]
So far, this fix is only targets on docker regsitry v2.7.1. If the registry has this fix in new release, we'll move on.

Signed-off-by: wang yan <wangyan@vmware.com>
2019-09-26 18:27:53 +08:00
Qian Deng
578adaa064
Merge pull request #9240 from ninjadq/add_extra_headers_in_nginx
Add headers in nginx config file
2019-09-26 10:27:08 +08:00
DQ
2bc11200d6 Move db change to new migration file
DB should move to new migration file cause 1.9 is already released

Signed-off-by: DQ <dengq@vmware.com>
2019-09-25 15:57:03 +08:00
DQ
e7394041ab Add headers in nginx config file
extra headered added in https and http config

Signed-off-by: DQ <dengq@vmware.com>
2019-09-24 17:50:40 +08:00
stonezdj(Daojun Zhang)
ec559b0585
Merge pull request #9123 from stonezdj/immutable_tags
Add DAO for immutable tags
2019-09-23 21:46:07 +08:00
Steven Zou
a73f896f23
Merge pull request #9154 from steven-zou/feature/pluggable_scanner_s2
[stage2]support pluggable scanner
2019-09-23 21:12:27 +08:00
stonezdj
29d2bcce99 Add DAO for immutable tags
Signed-off-by: stonezdj <stonezdj@gmail.com>
2019-09-23 16:45:07 +08:00
Steven Zou
d616bc3509 add scan report CRUD supporting and
- change error collection in scan job
- add dead client checking in client pool
- change key word type to interface{} for q.Query
- update bearer authorizer
- add required UT cases

Signed-off-by: Steven Zou <szou@vmware.com>
2019-09-23 16:21:39 +08:00
Steven Zou
0c19eba8c2 [stage2]support pluggable scanner
- add scanner rest API v1 spec
- implement v1 client which is used to talk to scanner adapter
- adjust data/orm models
- adjust code package structure

Signed-off-by: Steven Zou <szou@vmware.com>

- implement scan client which is used to talk to scanner adapter
- implement scan job which take the work of communicating with scanner
- update scanner mgmt API routes
- add corresponding UT cases
2019-09-23 09:37:54 +08:00
Daniel Jiang
3e5973fc6e Add Secure flag to cookie
This commit modifies nginx configuration file to make sure the secure
flag is added to "Set-Cookie" header when Harbor is serving https

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-09-19 21:04:37 +08:00
Steven Zou
4c4897aef1
Merge pull request #9134 from steven-zou/feature/pluggable_scanners
support pluggable scanner
2019-09-19 16:08:24 +08:00
Steven Zou
e324a4d623 support pluggable scanner
- add DAO layer for scanner registration
- add CURD manager for scanner registration
- add API controller for plug scanner
- add REST APIs for CURD of plug scanner
- add migration sql:0011_1.10.0
- add scan interface definition (no implementations)
- add related UT cases with testify

fix #8979 #8990

Signed-off-by: Steven Zou <szou@vmware.com>
2019-09-18 21:56:45 +08:00
jwangyangls
6dd2ae90a0
Merge pull request #9011 from jwangyangls/upgrade_clarity-2.1
Upgrade angular from 7.1.3 to 8.2.0 and clarity from 1.0 to 2.2
2019-09-18 10:45:40 +08:00
Yogi_Wang
a7c7a8e675 Upgrade angualr from 7.1.3 to 8.2.0 and clarity from 1.0 to 2.2
Signed-off-by: Yogi_Wang <yawang@vmware.com>

Signed-off-by: Meina Zhou <meinaz@vmware.com>
Signed-off-by: sshijun <sshijun@vmware.com>
2019-09-18 10:12:20 +08:00
Daniel Jiang
753219834e
Merge pull request #8960 from ninjadq/upgrade_hash_alg_for_pswd
Upgrade hash alg for pswd
2019-09-12 11:22:39 +08:00
DQ
ea5c27fcd5 Enhance: Upgrade encrypt alg to sha256
previous sha1 will still used for old password

Signed-off-by: DQ <dengq@vmware.com>
2019-09-09 21:48:21 +08:00
stonezdj(Daojun Zhang)
ca97c85279
Merge pull request #8927 from ninjadq/fix_config_with_components
Add logic to read clair and notary config
2019-09-09 15:50:09 +08:00
DQ
495a257ab5 Add logic to read clair and notary config
Signed-off-by: DQ <dengq@vmware.com>
2019-09-05 12:49:32 +08:00
Daniel Jiang
b75cbe1a7e
Merge pull request #8912 from ninjadq/no_cache_index_html
Add no-cache to index.html
2019-09-03 13:01:55 +08:00
Qian Deng
97c40df40f
Merge pull request #8593 from ninjadq/fix_wording_in_doc
Update config file names
2019-09-03 10:53:23 +08:00
DQ
d50df0f0db Add no-cache to index.html
shouldn't cache index.html for access fresh page after upgrade.

Signed-off-by: DQ <dengq@vmware.com>
2019-09-02 18:48:02 +08:00
DQ
aef93af21f Fix docker-compose version to 1.18.0
Signed-off-by: DQ <dengq@vmware.com>
2019-09-02 18:37:42 +08:00