Commit Graph

1632 Commits

Author SHA1 Message Date
Steven Zou
9f0f959749 Fix the multipart form data content missing issue by refill the content again before passing to the backend server
refill the multipart form data at harbor API layer
export write error functions from the chartserver package
refactor the error format to be compatiable with push plugin
2018-07-24 10:32:20 +08:00
Steven Zou
405e89158a
Merge pull request #5364 from steven-zou/fix_file_missing_issue
Fix the issue of returning '* file not found' error
2018-07-23 15:32:02 +08:00
Steven Zou
2f0b706f84 Fix the issue of returning '* file not found' error
return default empty when file is not existing
2018-07-23 14:53:40 +08:00
Daniel Jiang
b9762afbdd Clean up vendor folder
Add `prune` section to Gopkg.toml to remove the unused packages from
vendor folder.
Update the reference of `semver` and `distribution` repo.
2018-07-23 14:40:25 +08:00
Steven Zou
bb380e6dbc
Merge pull request #5314 from steven-zou/chart_repo_supporting
Refactor chart API endpoints
2018-07-20 20:43:55 +08:00
Steven Zou
0227a1315a Keep the chart server related configurations in adminserver
append chart server related config options to the supporting list of adminserver
provide chart server related config access method in the API layer
update prepare script and ui env template file to enable cache driver config for chart server API
append flag info in the systeminfo API to indicate if chart server is deployed with Harbor
refactor the response rewriting logic to return structual error object
add api init method to initilizing objects required in API handlers
chage owner of the storage folder
update offline/online package scripts in Harbor-Util.robot
2018-07-20 19:40:33 +08:00
Yan
efdb57548f
add admin job api (#5344)
It supports Harbor admin to trigger job either manual or
schedule. The job will be populated to job service to execute. 
The api includes:
1. POST /api/system/gc
2, GET /api/system/gc/:id 
3, GET /api/system/gc/:id/log
4, PUT/GET/POST /api/system/gc/schedule
2018-07-20 19:22:37 +08:00
Steven Zou
726d81803b Fix conflicts in Makefiles and prepare script files with upstream 2018-07-19 03:17:05 -07:00
stonezdj
f5e82f75a7 Add SafeCast function and set default value for some sytem configure
Add SafeCastString, SafeCastInt, SafeCastFloat64, SafeCastBool function to check
the type matched and avoid panic in runtime

Add default value to configure settings to avoid cannot save configure
issue
2018-07-17 17:00:06 +08:00
Yan
9e65499c10
Add garbage collection job implemention, this job could (#5268)
be triggered by manual and schedule. It calls registrtctl
to do the GC job, and log the output.
2018-07-16 18:08:40 +08:00
Yan
d5b85a6748
Add the registry controller httpserver, it's responsible for controlling (#5265)
docker regsitry. This version has the API to call regsitry GC with jobservice
secret. Seprates it into a standalone container as do not want to invoke two
processes in one container.

It needs to mount the registry storage into this container in order to do GC,
and needs to copy the registry binary into it.
2018-07-16 16:50:28 +08:00
Steven Zou
b572e64a68 Refactor chart API endpoints
add /chartrepo after /api
add /chartrepo before /index.yaml,/:repo/index.yaml and /:repo/charts/:filename
add debug messages to the potential debug points
add more supporting mime types "multipart/form-data", "application/octet-stream" to the API endpoints

update travis file to inject ENV for UT cases of chart repo
2018-07-16 14:33:39 +08:00
Wenkai Yin
8a92019e8e
Merge pull request #5310 from reasonerjt/adminserver-update-schema
Let adminserver initialise the DB schema.
2018-07-16 12:52:11 +08:00
Daniel Jiang
0d6ea995e1 Let adminserver initialise the DB schema.
This commit make update to remove the code from ui container to init the
DB schema.  As UI has dependency on admin server, so it's safe to assume
adminserver has to be ready first.  Regardless the setting of the config
store of admin server, it will try to access and intialize the schema of
database.
2018-07-13 17:32:17 +08:00
Daniel Jiang
8f8a6a1fc9
Merge pull request #5304 from steven-zou/chart_repo_supporting
Add API routes of chart repo services in harbor API router
2018-07-13 16:55:49 +08:00
Steven Zou
3a44f76b94 Add API routes of chart repo services in harbor API router
add chart repo API controller
update GetChartVersion to extract repo name from the request
update the corresponding UI cases

update the chart controller initialization process
2018-07-13 16:10:46 +08:00
timchenxiaoyu
a912a55ac2 add sync registry env (#5294)
By default Harbor will call catalog API of registry and sync the result to DB, this becomes problematic when registry is configured to custom storage service, there maybe inconsistent result and the whole process may be very time consuming.
So in this commit a env var SYNC_REGISTRY is introduced if user want Harbor to sync the repo when it starts up, by default it's false.
2018-07-13 11:15:41 +08:00
Daniel Jiang
e415d56099
Merge pull request #5258 from steven-zou/chart_repo_supporting
Implement the API services to support chart repo functions
2018-07-11 18:46:36 +08:00
Steven Zou
6ea9291c09 Refactor code after PR reviewing
pass cache config to the constructor of chart cache
use break loop to instead of return in GetIndexYaml
fix typo reids to redis
pass credential to the reverse proxy and chart client via constructors
2018-07-11 17:31:34 +08:00
Steven Zou
da4359f56b Add UT cases for the code of supporting chart repo server
add UT cases for cache
add UT cases for ChartOperator
add UT cases for the main API service of chart server
export some consts in the security.go of filter package to let them visible in chartserver package
2018-07-11 13:55:35 +08:00
Steven Zou
44cbb29664 Add cache support for the querying of chart details
support in memory cache
support redis cache
2018-07-08 11:17:55 +08:00
Steven Zou
7d782c41fc Implement the functions of getting chart version details and chart lists
getting chart list returns chart list instead of map and provides total versions info
getting chart details will return dependencies and values besides the metatdata of the chart version
2018-07-06 20:53:13 +08:00
Daniel Jiang
ad7a85da51
Merge pull request #5200 from stonezdj/add_group_search_api
Add REST API to search LDAP by Group DN
2018-07-06 15:10:31 +08:00
Steven Zou
b65cc34c70 Implement a unified index.yaml entrypoint service to support namespace isolations
replace the helm package from github.com to k8s.io
implement /index.yaml service API
2018-07-05 18:07:00 +08:00
stonezdj
62acdb14f3 Add settings to define admin with LDAP group DN 2018-07-05 14:46:44 +08:00
stonezdj
2494a7aaab Add REST API to search LDAP by Group DN
Used to verify and search DN when adding group member to project
2018-07-05 12:55:44 +08:00
Daniel Jiang
e1474ac50b
Merge pull request #5242 from steven-zou/chart_repo_supporting
Implement the related handler methods of the transparent operations
2018-07-04 21:16:07 +08:00
Steven Zou
ce4e5ac01d Implement the related handler methods of the transparent operations based on proxy
support index file
support object downloading
support health checking
support related manipulating operations except getting chart version
2018-07-03 21:36:20 +08:00
Steven Zou
6dfccc7dea
Merge pull request #5074 from ninjadq/ldap_search_ui
Add LDAP search UI
2018-07-03 15:30:18 +08:00
Deng, Qian
72dfdd552f Add ldap serach
1. Add group management
2. Add rewrite import user to member ui
3. Add import group to member
4. Add new items in configuration page
2018-07-03 14:00:59 +08:00
Daniel Jiang
b02d100fb5
Merge pull request #5234 from steven-zou/chart_repo_supporting
Define the related handlers to handler the requests of chart server/repo
2018-07-03 11:05:11 +08:00
Deng, Qian
edbe2fe620 Update migrator to 1 6 0
1. Add new alembic_pg folder for postgres
2. Add migration file for 1.6.0
3. Update version to 1.6.0
4. update migrator dockerfile
2018-07-02 21:23:47 +08:00
Steven Zou
54a0d10994 Define the related handlers to handler the requests of chart server/repo
add chartserver directory to put chart server code
add controller to coordinate the flow
add base/repo/manipulation handlers to handle requests
add operator to parse more details of chart version
call dep ensure
2018-07-02 19:34:04 +08:00
Daniel Jiang
1afb09b6cf
Merge pull request #5194 from ywk253100/180626_search
Fix bug in search API
2018-06-29 21:13:09 +08:00
Qian Deng
3ba302622c
Merge pull request #5213 from vmware/replication_by_label_ui
Label filter UI for replication
2018-06-29 07:44:26 -04:00
Daniel Jiang
aef3213dfa
Merge pull request #5190 from stonezdj/reload_config
Fix issue that harbor tile can not save customized settings
2018-06-29 13:04:36 +08:00
Daniel Jiang
c9b1962b1e Initialise Harbor DB schema in Harbor UI/adminserver container
This commit fixes #5040, the harbor-db image will only contain empty
databases, and harbor ui container will use migrate tool to run initial
SQL scripts to do initialization.  This is helpful for the case to
configure Harbor against external DB or DBaaS like RDS for HA deployment
However, this change will results some confusion as there are two tables
to track schema versions have been using alembic for migration, for this
release we'll try to use alembic to mock a `migration` table during
upgrade so the migrator will be bypassed, in future we'll consider to
consolidate to the golang based migrator.
Another issue is that the UI and adminserver containers will access DB
after start up in different congurations, can't ensure the sequence, so
both of them will try to update the schema when started up.
2018-06-28 16:22:53 +08:00
stonezdj
72e9b22e10 Fix issue that harbor tile can not save customized settings 2018-06-28 16:20:10 +08:00
Wenkai Yin
982760a132 Fix bug in search API
Refactor the logic of search API to fix bug mentioned in issue #5156 and #3838
2018-06-27 12:51:08 +08:00
Wenkai Yin
66811b3289
Merge pull request #5160 from ywk253100/180619_response_code
Parse the error got from jobservice, fix #5047
2018-06-22 17:17:26 +08:00
stonezdj
d6a4d79a03 Handle Invalid syntax and not found error 2018-06-20 14:27:29 +08:00
Wenkai Yin 79628
078360a021 Parse the error got from jobservice, fix #5047
Parse the error got from jobservice, and return the corresponding HTTP code if it's a HTTP error to caller
2018-06-19 13:30:06 +08:00
Daniel Jiang
1fae22add1
Merge pull request #5157 from pengpengshui/repLabelNew
Add replication rule label filter function #4861
2018-06-16 04:40:56 +08:00
pfh
ffbcdf62a0 Modify small bugs about code 2018-06-15 17:46:00 +08:00
pfh
963830b9d1 Add replication rule label filter 2018-06-15 11:03:47 +08:00
clouderati
60e88351b6 Small change to clarify wording deployment security toggle setting in project(s). 2018-06-14 17:37:05 -04:00
Wenkai Yin
952bba6d6c
Merge pull request #5141 from ywk253100/180613_label_resource
Provide an API to return the resources that a label is referenced by
2018-06-14 12:56:17 +08:00
Wenkai Yin 79628
3298c05157 Provide an API to return the resources that a label is referenced by
The API returns the resources that the label is referenced by. The resources contain only replication policies for now.
2018-06-14 11:52:19 +08:00
stonezdj
f025f28d12 If the LDAP user group already exist return 409 when onboarding an LDAP group
Do not adding the LDAP user group again.
2018-06-12 19:17:02 +08:00
Daniel Jiang
255a6d6f95
Merge pull request #5070 from ywk253100/180601_label_fuzzy_match
Fix #4742: fuzzy match label name
2018-06-12 14:39:35 +08:00
pfh
6f8ad44d5c set default border when add label without color 2018-06-11 16:58:32 +08:00
Qian Deng
9b5cd7d046
Merge pull request #5115 from pengpengshui/replicaton
Add name validation when creat new replication rule #5050
2018-06-11 04:03:44 -04:00
Qian Deng
7cbd3a4a96
Merge pull request #5110 from pengpengshui/leftNav
Hidden pwd button if auth_mode is not db_auth
2018-06-11 03:09:42 -04:00
pfh
2366f6a39f Add name validate when creat new replication rule 2018-06-11 11:34:17 +08:00
Daniel Jiang
7254b438ca
Merge pull request #5109 from ywk253100/180607_label
Soft delete label
2018-06-07 22:56:43 -07:00
Qian Deng
19e9fbf446
Merge pull request #5026 from pengpengshui/spelling
Modify spelling about 'serverity' to severity #4983
2018-06-08 01:47:30 -04:00
Wenkai Yin 79628
0c56493fb6 Soft delete label
Modify the deletion of label to soft deletion, in this way the names of deleted labels referenced by replication rules can be shown to users
2018-06-07 17:14:12 +08:00
pfh
2c2211dcfb Hidden pwd button if auth_mode is not db_auth 2018-06-07 14:20:12 +08:00
Deng, Qian
e1814597d8 Change the admin rename UX
Change the rename operation from double click icon to click button
2018-06-05 22:52:45 +08:00
pfh
81271c8e31 Mofify admin can't change existing pwd 2018-06-04 16:08:46 +08:00
pfh
4cca83ee9a Merge upstream master 2018-06-01 16:57:41 +08:00
Yan
6d800cabbd
enable migrator to support 1.5.0 migration from mysql to pgsql (#5029)
This commit is to enable data migrator to support migrates data
from mysql to pgsql, this is a specific step for user to upgrade
harbor across v1.5.0, as we have move harbor DB to pgsql from
1.5.0. It supports both harbor and notary db data migration,
and be split into two steps with dependency.

It also fix issue #4847, add build DB migrator in make process.
2018-06-01 14:58:43 +08:00
Wenkai Yin 79628
07c092c9be Fix #4742
This commit provides the support of fuzzy matching for label name when listing labels
2018-06-01 07:38:21 +08:00
pfh
8b2b3b1594 Add func about after create label close create panel 2018-05-31 15:07:48 +08:00
pfh
76cbe3b5d2 Modify tag detail alert when without clarity 2018-05-31 14:30:57 +08:00
Qian Deng
82b57971c0
Merge pull request #5057 from pengpengshui/projectcheck
Fix create project name doesn't have tooltip #5000
2018-05-30 23:25:27 -04:00
Qian Deng
aec9fabefb
Merge pull request #5051 from pengpengshui/colorpanel
Add dropdown for label color panel and modify color disappear on Mac
2018-05-30 23:23:14 -04:00
pfh
56e438b9c3 Fix create project name doesn't tooltip 2018-05-30 18:32:02 +08:00
pfh
1ec0ea01dc Add func about admin reset user`s pwd 2018-05-30 16:29:43 +08:00
pfh
73c23dbe45 add dropdown for color panel and delete type=button 2018-05-30 11:05:01 +08:00
Daniel Jiang
2f4950b80c
Merge pull request #5034 from ywk253100/180524_relativeurl
Fix replication issue when the remote registry enables relativeurls
2018-05-25 19:33:29 +08:00
Daniel Jiang
4eec586425
Merge pull request #5030 from ywk253100/180522_change_password
Update change password API
2018-05-25 11:05:12 +08:00
Wenkai Yin 79628
29a4a3335e Fix replication issue when the remote registry enables relativeurls
The location header returned by the remote registry contains no scheme and host parts if "relativeurls" is enabled,
this commit fix it by adding them at the beginning of location.
2018-05-24 15:34:31 +08:00
Steven Zou
9943c0cd07
Merge pull request #5021 from pengpengshui/asynctask
Add async task progress and delete dialog task progress #4371
2018-05-23 15:20:13 +08:00
Wenkai Yin 79628
76274dbf84 Update change password API
Modify the changing password API to support that admin user can change the password of normal users without old password
2018-05-22 19:02:20 +08:00
pengpengshui
9989a67d09
Merge pull request #5022 from pengpengshui/hightlights
Modify leftnav and tab have no hightLight about #4996
2018-05-22 18:12:00 +08:00
pfh
1121c8a76b Add async task progress and delete dialog task progress #4371
norm code
2018-05-22 14:42:06 +08:00
pfh
9992241163 Modify spelling about serverity 2018-05-22 14:21:45 +08:00
Wenkai Yin 79628
ab41977dcf Log error message in server side
Record the errors which are only returned to frontend in the server side for easy debugging.
2018-05-22 11:36:08 +08:00
stonezdj(Daojun Zhang)
2844d137d3
Merge pull request #5020 from stonezdj/fix_login_500
Fix error 500 with error username
2018-05-21 21:14:30 +08:00
pfh
a51af4a1c5 Modify leftnav and tab have no hightLight 2018-05-21 18:57:36 +08:00
Steven Zou
fb33f83b9a
Merge pull request #5003 from ywk253100/180517_label_filter_api
Add label filter in replication policy API
2018-05-21 18:55:02 +08:00
stonezdj
a1b668921b Fix error 500 with error username 2018-05-21 18:23:10 +08:00
pengpengshui
d16cddda04
Merge pull request #4986 from pengpengshui/leftNav
Filter out existimg members when add a new member #4364
2018-05-21 18:19:39 +08:00
pfh
14614b3572 Filter out existimg members when add a new member #4364 2018-05-21 17:27:20 +08:00
Wenkai Yin 79628
ad62fffb0e Add label filter in replication policy API
The label filter can be used to filter images when do the replication, this commit provides the filter in replication policy
2018-05-21 15:01:17 +08:00
Daniel Jiang
c2c667c6d6
Merge pull request #4976 from ywk253100/180514_parallel_pull_tag
Request repository and tag list in parallel
2018-05-21 15:00:44 +08:00
Deng, Qian
b3749bd4a4 Deploy button send event based on latest tag
If latest tag exist sent provision event. Otherwise send Add additional info event.
2018-05-17 17:07:27 +08:00
stone
d3930ae17c Put user info into session (#4885)
Fix the following issues.
1) GroupList is not found in SecurityContext user info
2) Retrieve multiple memberof information from LDAP.
3) If user is in two groups with guest, administrator role separately, display the max privilege role.
2018-05-17 16:23:51 +08:00
Steven Zou
6a7ba4609f
Merge pull request #4977 from ninjadq/fix_observable_issues
Fix 'xxx' is not a function err
2018-05-17 14:07:19 +08:00
Deng, Qian
a8025d2b32 Fix 'xxx' is not a function err
Because new version of rxjs you need import function manually, update code to avoid related errors.
Issues #4923 #4904 are caused by this problem.
2018-05-16 14:00:19 +08:00
Steven Zou
3917512d3e
Merge pull request #4962 from ywk253100/180509_label_filter
Add label filter to filter chain
2018-05-15 20:12:34 +08:00
Wenkai Yin 79628
c1ebf0ba1e Add label filter to filter chain
Label filters are used to filter the resources according to the labels, adding the filters to the chain to enable the function.
2018-05-15 19:19:01 +08:00
Deng, Qian
43c8e9f589 Fix all tslint warnings
1. Fix tslint warnings
2. Add tslint to travis
2018-05-15 16:28:23 +08:00
stonezdj
44006247a6 Fix issue of import LDAP user 404 message
Some user requires return json format message in 404
2018-05-15 16:05:11 +08:00
Wenkai Yin 79628
6243fed8f5 Request repository and tag list in parallel
When using S3 or others as the registry storage, the request for repository and tag list may be very slow. This commit fixes this by parallel requesting the resources.
2018-05-15 15:47:32 +08:00
Daniel Jiang
b537ea96f3
Merge pull request #4925 from stonezdj/fix_ldap_import_404
Fix issue of import LDAP user 404 message
2018-05-14 16:15:49 +08:00
stonezdj
931e17330c Fix issue of import LDAP user 404 message
Some user requires return json format message in 404
2018-05-11 15:33:35 +08:00
Steven Zou
9b01d3a2f6
Merge pull request #4899 from ninjadq/mv_string_to_html_n_css
Fix legacy issues that html and css file are written on ts file.
2018-05-11 15:22:37 +08:00
Deng, Qian
282a63f57f Fix legacy issues that html and css file are written on ts file.
Currently, our html and css files are written as string on .ts file. This pr is to solve the legacy issue.
2018-05-10 18:39:48 +08:00