Commit Graph

467 Commits

Author SHA1 Message Date
Colstuwjx
e49a9de2f4 Fix TestTCPConn break issue.
Signed-off-by: Colstuwjx <Colstuwjx@gmail.com>
2018-09-06 14:58:04 +08:00
stonezdj
9dca49ba6e LDAP group DN should be case insensitive
Fix issue #5776, LDAP servers are case insensitive. because only LDAP
group DN is used to compare/equal operation, lowercase all LDAP group DN
when retrieves it from LDAP server, and lowercase them before save in DB

Signed-off-by: stonezdj <stonezdj@gmail.com>
2018-09-06 11:33:05 +08:00
Wenkai Yin
5427c0064c
Merge pull request #5731 from Colstuwjx/fix-log-test
Fix logger test case, add SetSkipLine func.
2018-09-06 08:17:17 +08:00
Colstuwjx
bab203c0f4 Fix logger test case.
Signed-off-by: Colstuwjx <Colstuwjx@gmail.com>
2018-09-05 19:25:17 +08:00
陈德
0582db9a82 Apply consistent format for comments
Signed-off-by: 陈德 <chende@caicloud.io>
2018-09-05 16:16:31 +08:00
Wenkai Yin
49bb5cfafb Test TCP connection before upgrading database schema
This commit moves the database schema upgrading after database initialization. The init will test TCP connection.

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2018-09-05 02:07:47 +08:00
陈德
6eb972c383 Add pull scope to post/put/patch method
Signed-off-by: 陈德 <chende@caicloud.io>
2018-09-03 11:12:11 +08:00
陈德
a59af8ce82 fix gofmt
Signed-off-by: 陈德 <chende@caicloud.io>
2018-08-30 14:11:18 +08:00
陈德
666bd692fe Support repo list sorting
Signed-off-by: 陈德 <chende@caicloud.io>
2018-08-30 10:56:50 +08:00
wang yan
aab761ac8a Fix gofmt check results
Signed-off-by: wang yan <wangyan@vmware.com>
2018-08-29 11:50:00 +08:00
Yan
fca2bb3a6b
Fix misspell checking results (#5749)
Signed-off-by: wang yan <wangyan@vmware.com>
2018-08-29 10:25:42 +08:00
Daniel Jiang
dcf4e2ee78 Update import path in go code
vmware -> goharbor

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-08-23 17:50:53 +08:00
wangyan
9a95f14918 Cherry-pick -- Fix security issue found by gas 2018-08-03 01:16:53 -07:00
Daniel Jiang
6062bf279b Set default creation_time and update_time at model
This commit set the default value of creation_time and update_time to
data objects by adding `orm:add_now` annotations.
2018-07-31 12:56:14 +08:00
stonezdj
9b209858f4 Ldap_group_admin_dn can not updated via rest 2018-07-24 17:47:57 +08:00
silenceshell
7745b79b2e var name should not be error (#5332)
Rename the variable names from "error" to "err"
2018-07-24 11:33:21 +08:00
Steven Zou
bb380e6dbc
Merge pull request #5314 from steven-zou/chart_repo_supporting
Refactor chart API endpoints
2018-07-20 20:43:55 +08:00
Steven Zou
0227a1315a Keep the chart server related configurations in adminserver
append chart server related config options to the supporting list of adminserver
provide chart server related config access method in the API layer
update prepare script and ui env template file to enable cache driver config for chart server API
append flag info in the systeminfo API to indicate if chart server is deployed with Harbor
refactor the response rewriting logic to return structual error object
add api init method to initilizing objects required in API handlers
chage owner of the storage folder
update offline/online package scripts in Harbor-Util.robot
2018-07-20 19:40:33 +08:00
Yan
efdb57548f
add admin job api (#5344)
It supports Harbor admin to trigger job either manual or
schedule. The job will be populated to job service to execute. 
The api includes:
1. POST /api/system/gc
2, GET /api/system/gc/:id 
3, GET /api/system/gc/:id/log
4, PUT/GET/POST /api/system/gc/schedule
2018-07-20 19:22:37 +08:00
stonezdj
f5e82f75a7 Add SafeCast function and set default value for some sytem configure
Add SafeCastString, SafeCastInt, SafeCastFloat64, SafeCastBool function to check
the type matched and avoid panic in runtime

Add default value to configure settings to avoid cannot save configure
issue
2018-07-17 17:00:06 +08:00
Yan
9e65499c10
Add garbage collection job implemention, this job could (#5268)
be triggered by manual and schedule. It calls registrtctl
to do the GC job, and log the output.
2018-07-16 18:08:40 +08:00
Yan
d5b85a6748
Add the registry controller httpserver, it's responsible for controlling (#5265)
docker regsitry. This version has the API to call regsitry GC with jobservice
secret. Seprates it into a standalone container as do not want to invoke two
processes in one container.

It needs to mount the registry storage into this container in order to do GC,
and needs to copy the registry binary into it.
2018-07-16 16:50:28 +08:00
Daniel Jiang
0d6ea995e1 Let adminserver initialise the DB schema.
This commit make update to remove the code from ui container to init the
DB schema.  As UI has dependency on admin server, so it's safe to assume
adminserver has to be ready first.  Regardless the setting of the config
store of admin server, it will try to access and intialize the schema of
database.
2018-07-13 17:32:17 +08:00
stonezdj
62acdb14f3 Add settings to define admin with LDAP group DN 2018-07-05 14:46:44 +08:00
Deng, Qian
edbe2fe620 Update migrator to 1 6 0
1. Add new alembic_pg folder for postgres
2. Add migration file for 1.6.0
3. Update version to 1.6.0
4. update migrator dockerfile
2018-07-02 21:23:47 +08:00
Daniel Jiang
1afb09b6cf
Merge pull request #5194 from ywk253100/180626_search
Fix bug in search API
2018-06-29 21:13:09 +08:00
Daniel Jiang
aef3213dfa
Merge pull request #5190 from stonezdj/reload_config
Fix issue that harbor tile can not save customized settings
2018-06-29 13:04:36 +08:00
Daniel Jiang
c9b1962b1e Initialise Harbor DB schema in Harbor UI/adminserver container
This commit fixes #5040, the harbor-db image will only contain empty
databases, and harbor ui container will use migrate tool to run initial
SQL scripts to do initialization.  This is helpful for the case to
configure Harbor against external DB or DBaaS like RDS for HA deployment
However, this change will results some confusion as there are two tables
to track schema versions have been using alembic for migration, for this
release we'll try to use alembic to mock a `migration` table during
upgrade so the migrator will be bypassed, in future we'll consider to
consolidate to the golang based migrator.
Another issue is that the UI and adminserver containers will access DB
after start up in different congurations, can't ensure the sequence, so
both of them will try to update the schema when started up.
2018-06-28 16:22:53 +08:00
stonezdj
72e9b22e10 Fix issue that harbor tile can not save customized settings 2018-06-28 16:20:10 +08:00
Wenkai Yin
982760a132 Fix bug in search API
Refactor the logic of search API to fix bug mentioned in issue #5156 and #3838
2018-06-27 12:51:08 +08:00
stonezdj
d6a4d79a03 Handle Invalid syntax and not found error 2018-06-20 14:27:29 +08:00
Daniel Jiang
255a6d6f95
Merge pull request #5070 from ywk253100/180601_label_fuzzy_match
Fix #4742: fuzzy match label name
2018-06-12 14:39:35 +08:00
Wenkai Yin 79628
0c56493fb6 Soft delete label
Modify the deletion of label to soft deletion, in this way the names of deleted labels referenced by replication rules can be shown to users
2018-06-07 17:14:12 +08:00
Wenkai Yin 79628
07c092c9be Fix #4742
This commit provides the support of fuzzy matching for label name when listing labels
2018-06-01 07:38:21 +08:00
Daniel Jiang
2f4950b80c
Merge pull request #5034 from ywk253100/180524_relativeurl
Fix replication issue when the remote registry enables relativeurls
2018-05-25 19:33:29 +08:00
Wenkai Yin 79628
29a4a3335e Fix replication issue when the remote registry enables relativeurls
The location header returned by the remote registry contains no scheme and host parts if "relativeurls" is enabled,
this commit fix it by adding them at the beginning of location.
2018-05-24 15:34:31 +08:00
Wenkai Yin 79628
76274dbf84 Update change password API
Modify the changing password API to support that admin user can change the password of normal users without old password
2018-05-22 19:02:20 +08:00
stone
d3930ae17c Put user info into session (#4885)
Fix the following issues.
1) GroupList is not found in SecurityContext user info
2) Retrieve multiple memberof information from LDAP.
3) If user is in two groups with guest, administrator role separately, display the max privilege role.
2018-05-17 16:23:51 +08:00
stonezdj
b8a48d0326 Update security context for assign role to project group member
The project list will contain all public projects, user is a member of this project, or user is in the group which is a member of this projects.
Change the behaviour of user roles, if the user is not a member of this project, then return the user's groups role of current project
2018-05-03 16:49:16 +08:00
Yan
ae257433cc
Fully migrate harbor db to postgresql (#4689)
* Merge harbor db to postgres
2018-04-27 02:27:12 -07:00
Wenkai Yin
f77e4167ac
Merge pull request #4802 from ywk253100/180427_label_db
Modify unique constraint of table harbor_label
2018-04-27 17:10:05 +08:00
Wenkai Yin
73babbf1ab Modify unique constraint of table harbor_label
Add unique constraint to column name, scope and project_id  of table harbor_label to make creating same name labels under different projects valid
2017-12-19 22:15:56 +08:00
Tan Jiang
93c448d91b Fix mis-interpretation of severity in Clair
Currently "Critical" vulnerablity is treated as "Unknown" in Harbor.
This commit provides a quickfix that it will be interpret as "High".  In
future, we should consider introduce "Critical" and enable UI to handle
it to be more consistent with CVSS spec.
2018-04-25 10:31:12 +08:00
Tan Jiang
1fc4142e1a Do not call chown to config files
This commit fixes a recently discovered issue on Kubernetes #4496
It make necessary to avoid calling `chown` to config files during the
bootstrap of the containers.
2018-04-20 13:44:21 +08:00
Daniel Jiang
f8d577994f
Merge pull request #4719 from reasonerjt/fix-db-broken-pipe
Fix intermittent `broken pipe` issue in log
2018-04-19 10:23:59 +08:00
Tan Jiang
7fa8261661 Fix intermittent broken pipe issue in log
This commit fixes #4713, by adopting the suggested fix in:
https://github.com/go-sql-driver/mysql/issues/529
When creating the DB instance in orm, call `SetConnMaxLifetime()`
2018-04-18 17:39:13 +08:00
Wenkai Yin 79628
f6bd2f245d Fix bug #4688
Fix bug: the user can push images although he have no permisson by checking empty value before assign permissions.
2018-04-18 12:03:06 +08:00
stonezdj
7e57c685ac Add project member search by name
Previous implementation contains the search user by name feature. This implementation can search user and user group by name.
2018-04-16 18:38:10 +08:00
stonezdj
de49165427 Refactor project member API
1) Remove the previous /api/projects/?:project_id/members/:userid
    2) Move the /api/projects/:project_id/projectmembers/?:pmid to
        /api/projects/:project_id/members/?:pmid
    3) Change the project member maintain ui to call new REST API
2018-04-11 17:49:33 +08:00
Tan Jiang
ff06ec05c3 Store secret in header instead of cookie 2018-04-07 22:02:06 +08:00
Daniel Jiang
b5f52bc3fb
Merge pull request #4586 from reasonerjt/job-context-props
Read the system properties from scan job context
2018-04-07 10:23:30 +08:00
Tan Jiang
15580a5e8c Read the system properties from scan job context 2018-04-04 19:58:54 +08:00
stone
df63a73fd4
Merge pull request #4483 from stonezdj/api4assign_role_to_group4
Add REST API for assign role to group
2018-04-04 16:19:37 +08:00
stonezdj
9bcfaedc0e Add REST API for assign role to group 2018-04-04 13:39:42 +08:00
Wenkai Yin
79f0ca96bc
Merge pull request #4572 from ywk253100/180402_job
Hide schedule job when listing replication jobs
2018-04-03 16:12:10 +08:00
Wenkai Yin
500651a5a1 Hide schedule job when listing replication jobs 2018-04-03 01:11:55 +08:00
Tan Jiang
7aec4d9f21 Add UT and remove useless code. 2018-04-02 21:28:46 +08:00
Wenkai Yin
3436729d52
Merge pull request #4547 from ywk253100/180328_schedule_replication_job
Move schedule replication job to new jobservice
2018-03-30 21:03:45 +08:00
Steven Zou
ba91fc2861 Merge master into job_service and fix conflicts 2018-03-30 19:26:04 +08:00
Wenkai Yin
dd40f187ec Move schedule replication job to new jobservice 2018-03-30 17:44:05 +08:00
Steven Zou
d1899c840d Merge branch 'master' into job_service 2018-03-29 23:25:20 +08:00
Tan Jiang
5dd75bb0b0 Trust Root CA of VIC appliance when accessing Admiral 2018-03-29 19:53:41 +08:00
Daniel Jiang
2c2cbd9c52
Merge pull request #4510 from reasonerjt/master
Add indexes to job tables and bump up schema version.
2018-03-28 16:58:16 +08:00
Tan Jiang
b6df6cf169 Add indexes to job tables and bump up schema version. 2018-03-28 16:15:54 +08:00
stone
203b1b52bb
Merge pull request #4415 from stonezdj/user_group_and_project_member
Add DAO for user group and project member
2018-03-26 15:21:20 +08:00
stonezdj
49d960b060 Add DAO for project member and user group 2018-03-26 14:38:32 +08:00
yixingj
cb64ad96ff Make endpoint configurable
Move all the endpoint to harbor.cfg
2018-03-26 10:50:18 +08:00
Wenkai Yin
dd156ca243 Handle replication job status hook 2018-03-24 21:18:58 +08:00
Tan Jiang
be97a91650 Integrate with jobservice webhook 2018-03-27 21:27:52 +08:00
Tan Jiang
c859616e25 fix golint and go vet issue 2018-03-26 22:10:01 +08:00
Tan Jiang
582deea9e7 resolve conflict 2018-03-26 18:11:39 +08:00
Tan Jiang
41ce0891ab Trigger scan job from UI. 2018-03-26 18:07:21 +08:00
Wenkai Yin
38568a1d2c
Merge pull request #4485 from ywk253100/180326_period_job
Create a job to call UI's replication API to do the period replication job
2018-03-26 17:31:10 +08:00
Tan Jiang
745b21abbc Merge remote-tracking branch 'upstream/master' into scan-job-migrate 2018-03-26 15:39:42 +08:00
Tan Jiang
381ecc3521 Merge with master 2018-03-26 10:37:17 +08:00
Wenkai Yin
85f357ec6b Delete the mapping relationship between resources and labels when the label is deleted 2018-03-24 02:22:51 +08:00
Wenkai Yin
e63d5a1c06 Create a job to call UI's replication API to do the period replication job 2018-03-23 23:53:15 +08:00
Wenkai Yin
ed08a42e4b Migrate replication job to the new jobservice 2018-03-23 18:36:37 +08:00
Yan
cbcca015b0
add read only mode to stop docker push (#4433) 2018-03-23 03:16:08 -07:00
Wenkai Yin
c6e65d2ded Fuzzy matching repository name in GET repositories API 2018-03-22 13:38:26 +08:00
Tan Jiang
b8ec419c8e Add missed package to fix compilation issue 2018-03-21 16:59:32 +08:00
Tan Jiang
613464bc16 Migrate scan job to job service V1 phase1 2018-03-21 16:25:32 +08:00
Wenkai Yin
838b439560 Implement filter repository and tags by label API 2018-03-21 10:51:06 +08:00
stonezdj
44fc373c6d Add LDAP Group Search Configure Param 2018-03-15 06:16:47 +08:00
Wenkai Yin
36b9c4e458 Implement adding/removing labels to/from repositories and images API 2018-03-12 19:30:05 +08:00
Wenkai Yin
379f113452 Implement label management API 2018-03-09 12:17:27 +08:00
Wenkai Yin
0a8929b85e Do the authentication with CRAM-MD5 when the connection is insecure 2018-03-08 14:21:44 +08:00
stonezdj
4c6d1488bd Add UT 2018-02-09 15:29:08 +08:00
stonezdj
f138067242 Refactor project member 2018-02-09 10:38:51 +08:00
Wenkai Yin
10f56d26fe Change codes to make everything OK after upgrading to beego 1.9.0 2018-02-05 13:07:52 +08:00
Wenkai Yin
9022abfc13 Fix code issues found by Gas 2018-01-29 15:17:03 +08:00
Wenkai Yin
515cac010a
Merge pull request #4071 from ywk253100/180117_policy_pagination
Add pagination support in listing replication policy API
2018-01-19 15:19:45 +08:00
Wenkai Yin
611709a7be Add pagination support in listing replication policy API 2018-01-18 15:54:12 +08:00
stone
c815dc01dd
Merge pull request #4043 from reasonerjt/uaa-bugfix
Read Email from UAA while onboarding user.
2018-01-18 14:04:35 +08:00
Tan Jiang
d5d913f51d Read Email from UAA while onboarding user.
Will call the userinfo API of UAA to get user info and generage user
model based on the response.  Also this commit include a change that
whenever the UAA Client is to be used it will update the configuraiton,
this is needed as we enable user to update the configuration of UAA via
UI.
2018-01-17 10:28:49 +08:00
Wenkai Yin
8cda2d8d65
Merge pull request #4036 from ywk253100/180116_s3
Propagate registry storage driver name to adminserver and return it in /api/systeminfo
2018-01-16 18:41:08 +08:00
Qian Deng
5017670d00
Merge pull request #4005 from ninjadq/db_migrate_from_1_3_to_1_4
Update migration tool for v1.4
2018-01-16 17:04:54 +08:00
Wenkai Yin
53d5a2256a Propagate registry storage driver name to adminserver and return it in /api/systeminfo 2018-01-16 16:57:28 +08:00
Deng, Qian
b3e65ed71e Update migration tool for v1.4
1. Update database meta file
2. Add migration file for 1.4
2018-01-16 15:38:51 +08:00
Wenkai Yin
a1dd8c3bff
Merge pull request #4004 from ywk253100/180111_jobservice
Provide a mechanism to stop pending and retrying jobs
2018-01-15 12:55:44 +08:00