Commit Graph

3713 Commits

Author SHA1 Message Date
He Weiwei
75bbf9d0bf
fix(middleware): escape and clean path for the skipper (#10674)
1. Escape and clean request path for `legacyAPISkipper`.
2. Escape and clean request path for `MethodAndPathSkipper`.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-02-11 22:29:55 +08:00
Wenkai Yin(尹文开)
d66c1a4a21
Merge pull request #10612 from ywk253100/200202_replication_basic_auth
Do enhancement for the registry authorizer
2020-02-11 22:09:40 +08:00
Wenkai Yin
a4ebbc6ecf Do enhancement for the registry authorizer
This commit introduces a new wrapper authorizer which can authorize the request according to the auth scheme automatically

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-11 21:21:49 +08:00
Will Sun
647f11f26b
Merge pull request #10688 from AllForNothing/relative
UI should use relative path for back-end APIs
2020-02-11 18:06:42 +08:00
Daniel Jiang
9fb676c219 Fix nilpointer issue in v2 auth middleware
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-02-11 15:49:39 +08:00
Wang Yan
582fac5dae
Merge pull request #10687 from wy65701436/fix-catalog
fix catalog pagination issue
2020-02-11 10:40:35 +08:00
He Weiwei
c4f77069c8
Hide projects in global logs where user has limited guest role (#10639)
Signed-off-by: Mark Huang <mhuang@pivotal.io>

Co-authored-by: Mark Huang <mhuang@pivotal.io>
2020-02-11 10:14:30 +08:00
Wang Yan
e68b474dbc Merge pull request #10581 from ywk253100/100123_list_artifact
Add query string to listing artifact API to support specify only show the tagged artifacts
2020-02-10 18:12:06 +08:00
wang yan
beb7664b33 fix catalog pagenation issue
Signed-off-by: wang yan <wangyan@vmware.com>
2020-02-10 18:03:33 +08:00
Wang Yan
65dc54b059 Merge pull request #10626 from ywk253100/200125_handle_error
Unify the method/style to handle error in handler/middleware
2020-02-10 17:47:12 +08:00
AllForNothing
c932ca9d2b UI should use relative path for back-end APIs
Signed-off-by: AllForNothing <sshijun@vmware.com>
2020-02-10 15:37:09 +08:00
Wenkai Yin
0373e08e58 Add query string to listing artifact API to support specify only show the tagged artifacts
Specify the query string "tags=NOT_NULL" to the listing artifact API to only return the tagged artifacts

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-10 14:53:51 +08:00
Wang Yan
71414a9bc3 Merge pull request #10670 from wy65701436/upgrade-beego-12.1
update beego version to v1.12.1
2020-02-10 13:57:07 +08:00
wy65701436
b2e3761f62 update beego version to v1.12.1
Signed-off-by: wang yan <wangyan@vmware.com>
2020-02-09 16:39:48 +08:00
Wenkai Yin
af4dd142bc Unify the method/style to handle error in handler/middleware
This commit provides a "SendError" method to unify the way to handle error in handlers/middlewares

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-08 07:37:00 +08:00
Wenkai Yin
0f6057a22c Implement get addition API for image
This commit implements the API to get build history of image with manifest version 2 and populates the addition links when listing/getting the artifact

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-07 20:21:38 +08:00
Wenkai Yin(尹文开)
8a74fcb074
Merge pull request #10652 from wy65701436/fix-10552
Use controller rather than manager in the API handler and middleware
2020-02-07 19:30:51 +08:00
Wang Yan
6bad9f62ba
Merge pull request #10662 from ywk253100/200206_delete_repo
Implement repository deletion API
2020-02-07 17:51:03 +08:00
Wang Yan
5679c174c6
Merge pull request #10643 from ywk253100/200204_auth
Add permission check for artifact related APIs
2020-02-07 11:25:37 +08:00
wang yan
0fbbd674c2 Use controller rather than manager in the API handler and middleware
Signed-off-by: wang yan <wangyan@vmware.com>
2020-02-07 11:23:17 +08:00
Wenkai Yin
c267aaa474 Implement repository deletion API
Implement repository deletion API based on the new design

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-07 09:36:35 +08:00
Wang Yan
72f9e02dd9
Merge pull request #10628 from wy65701436/af-immu
set the immutable status on getting/listting tag
2020-02-05 21:03:29 +08:00
wang yan
596a6261ca set the immutable status on getting/listting tag
Signed-off-by: wang yan <wangyan@vmware.com>
2020-02-05 16:24:13 +08:00
Wenkai Yin
6087647895 Add permission check for artifact related APIs
Add permission check for artifact related APIs

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-05 15:58:39 +08:00
Wenkai Yin(尹文开)
260d43db4f
Merge pull request #10627 from ywk253100/200202_reference
Add child artifact digest in reference model
2020-02-05 13:31:12 +08:00
wang yan
a53df4863d fix issue on listing robot accounts
Update the query FuzzyMatch when to list robot accounts per project

Signed-off-by: wang yan <wangyan@vmware.com>
2020-02-05 11:36:18 +08:00
Wenkai Yin
7930e4f38a Add child artifact digest in reference model
As we only provide the API to get artifact information via project name, repository name and digest, the digest of child artifact must be returned

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-04 15:59:06 +08:00
Wenkai Yin
c1746cc675 Update the method called by registry handler
Use ListTags instead in the registry tag listing handler

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-04 15:16:28 +08:00
Wang Yan
fa41168e8e
Merge pull request #10587 from ywk253100/200122_route
Register registry handler with the new methods of Route
2020-02-04 14:28:15 +08:00
Wenkai Yin(尹文开)
26ade207d3
Merge pull request #10588 from ywk253100/200126_delete_get_artifact
Implement artifact/tag related API
2020-02-04 14:14:28 +08:00
Wenkai Yin
793b23a444 Implement artifact/tag related API
Implement APIs:
1. Get artifact
2. Delete artifact
3. Create tag
4. Delete tag

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-04 13:46:55 +08:00
He Weiwei
791439086d
feat(api,permission): add HasPermission, HasProjectPermission in BaseAPI (#10618)
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-02-04 08:42:00 +08:00
He Weiwei
b1437c1341
refactor(security): add NewContext and FromContext to security pkg (#10617)
1. Add `NewContext` and `FromContext` funcs in security pkg.
2. Add `Name` func in `security.Context` interface to make the checking
for the `/api/internal/configurations` API clear.
3. Get the security from the context to prepare change the security
filter to middleware.
4. Remove `GetSecurityContext` in filter pkg.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-02-03 17:43:36 +08:00
Wenkai Yin
ef3af85a5b Register registry handler with the new methods of Route
Register registry handler with the new methods of Route

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-02-03 16:55:08 +08:00
Wang Yan
9c0d400817
Merge pull request #10549 from wy65701436/middleware-pc
add Middleware policycheck into v2 handler
2020-02-03 15:58:45 +08:00
Will Sun
acfcd2d175
Merge pull request #10489 from AllForNothing/postinstall
Fix postinstall script in Docker
2020-02-03 14:13:12 +08:00
wang yan
661f7a7902 resolve conflict with latest code
Signed-off-by: wang yan <wangyan@vmware.com>
2020-02-03 10:34:09 +08:00
Daniel Jiang
2064a1cd6d Switch to basic authentication for registry
1. Add basic authorizer for registry which modify the request
to add basic authorization header to request based on configuration.
2. Set basic auth header for proxy when accessing registry
3. Switche the registry to use basic auth by default and use the basic
authorizer to access Harbor.
4. Make necessary change to test cases, particularly
"test_robot_account.py" and "docker_api.py", because the error is
changed after siwtched to basic auth from token auth.  #10604 is opened
to track the follow up work.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-01-31 21:46:47 +09:00
Wenkai Yin(尹文开)
a1b25e1fec
Switch to new registry API handlers (#10596)
Switch to new registry API handlers

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-01-30 21:21:07 +08:00
Wenkai Yin
d0ac70d26c Implement the get/delete handler for registry API
Implement the get/delete handler for registry API

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-01-28 11:01:17 +08:00
Wang Yan
9db84f2880 add Middleware policycheck into v2 handler
1, add content trust middleware in new v2 handler
2, add vulnable middlware in new v2 hanlder

Signed-off-by: wang yan <wangyan@vmware.com>
2020-01-28 00:24:58 +09:00
Daniel Jiang
5f8acc3896 Add middlewares for permission checking for v2 API
When the registry shifts from token auth to basic auth, we'll use the middleware to check permission.
This commit add middlewares for populate the artifact info and check
permission based on request to /v2/* api via security context

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-01-27 12:53:15 +08:00
He Weiwei
b594861658 feat(middleware): add transaction middleware for v2 and v2.0 APIs
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-01-22 05:00:39 +00:00
Wenkai Yin(尹文开)
f9d8951aed
Merge pull request #10519 from ywk253100/200116_router
Wrap the beego router and provide a unified view for users to register routes
2020-01-22 10:20:36 +08:00
Wenkai Yin(尹文开)
a774a19823
Merge pull request #10537 from ywk253100/200115_artifact_api
Implement the listing artifact API
2020-01-22 08:05:12 +08:00
Wenkai Yin
19f4bad042 Implement the listing artifact API
Implement the listing artifact API

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-01-22 07:35:29 +08:00
Wenkai Yin(尹文开)
17318aedc3
Merge pull request #10547 from ywk253100/200120_artifact
Add foreign key to avoid the concurrent issue
2020-01-21 17:44:27 +08:00
Wenkai Yin
8aeabc7717 Wrap the beego router and provide a unified view for users to register routes
Wrap the beego router and provide a unified view for users to register routes

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-01-21 17:26:20 +08:00
Wenkai Yin
7dc28bcff9 Add foreign key to avoid the concurrent issue
Add foreign key to avoid the concurrent issue when operating the artifacts, tags and references

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-01-21 16:03:51 +08:00
Wenkai Yin(尹文开)
8b3313a1ce
Merge pull request #10525 from ywk253100/200117_chart_resolver
Implement the resolver for helm chart
2020-01-21 15:12:57 +08:00
Wenkai Yin(尹文开)
4dc59c5e39
Merge pull request #10541 from ywk253100/200120_default_resolver
Try to parse the type of the artifact based on the media type when no resolver found
2020-01-21 14:41:35 +08:00
Wenkai Yin
f759c8fd64 Implement the resolver for helm chart
Implement the resolver for helm chart

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-01-21 14:38:53 +08:00
Wenkai Yin
1f3fcbde36 Try to parse the type of the artifact based on the media type when no resolver found
Try to parse the type of the artifact based on the media type when no resolver found, if parse failed, set it's type to unknown. This won't block the pushing for artifacts that has no resolver registered in Harbor

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-01-21 14:15:33 +08:00
wang yan
8fae15b803 add immutable tag middleware into new v2 handler
Signed-off-by: wang yan <wangyan@vmware.com>
2020-01-20 22:41:08 +08:00
Steven Zou
e8a617e0bd
Merge pull request #10429 from steven-zou/fix/job_hung_issue
fix[jobservice]:job status is hung after restart
2020-01-20 16:46:22 +08:00
He Weiwei
33dfa1ea11
feat(beego): upgrade beego to v1.12 which support middleware (#10524)
1. Upgrade beego to v1.12.0
2. Add RequestID middleware to all HTTP requests.
3. Add Orm middleware to v2 and v2.0 APIs.
4. Remove OrmFilter from all HTTP requests.
5. Fix some test cases which cause panic in API controllers.
6. Enable XSRF for test cases of CommonController.
7. Imporve ReadOnly middleware.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-01-20 16:41:49 +08:00
Wenkai Yin(尹文开)
603cc0f5f3
Merge pull request #10526 from heww/send-error
feat(swagger): implement SendError for BaseAPI handler
2020-01-20 14:51:01 +08:00
wang yan
7ebb337e3f add regtoken middleware in new v2 handler
To move the regtoken middlware to new v2 handler framework, it parses the docker pull bearer token and check whether it's a scanner pull.

Signed-off-by: wang yan <wangyan@vmware.com>
2020-01-20 11:02:10 +08:00
Wang Yan
63ef743ba7
Merge pull request #10482 from wy65701436/api-list-repo
add code for catalog and list tag API
2020-01-19 15:26:13 +08:00
wang yan
986989af3c add code for catalog and list tag API
Signed-off-by: wang yan <wangyan@vmware.com>
2020-01-19 13:00:02 +08:00
wang yan
bffbc4009b add get manifest information middleware in new v2 hanlder
To add a new middleware to resolve the manifest information from request path, then pass
it into the request context for later use.

Signed-off-by: wang yan <wangyan@vmware.com>
2020-01-19 12:31:38 +08:00
He Weiwei
5daa3f4fb3 feat(swagger): implement SendError for BaseAPI handler
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-01-19 00:46:18 +00:00
Wenkai Yin(尹文开)
8809b7aa5d
Merge pull request #9887 from chlins/fix/replication-huawei-adapter-authorization
fix(replication): huawei adapter multi authorization
2020-01-17 19:04:52 +08:00
Wang Yan
2ea41ca78a
Merge pull request #10481 from ywk253100/200114_orm
Get the ormer from context instead of creating a new one
2020-01-17 17:54:00 +08:00
Wang Yan
a27a5986ab
Merge pull request #10448 from ywk253100/200111_index
Implement the index/manifest list image resolver
2020-01-17 14:39:01 +08:00
Wenkai Yin(尹文开)
6e3733aa7f
Merge pull request #10490 from wy65701436/middleware-readonly
move readonly middleware to new v2 handler
2020-01-17 14:09:20 +08:00
Daniel Jiang
a200947666
Merge pull request #10449 from ywk253100/200111_manifest_v1
Implement the manifest v1 image resolver
2020-01-17 11:13:50 +08:00
wang yan
935303a476 move readonly middleware to new v2 handler
Signed-off-by: wang yan <wangyan@vmware.com>
2020-01-17 11:12:10 +08:00
sshijun
3175d5f646 Fix postinstall script in Docker
Signed-off-by: sshijun <sshijun@vmware.com>
2020-01-15 16:28:57 +08:00
Will Sun
a49c0640fd
Merge pull request #10434 from AllForNothing/swagger
Add ng-swagger-gen to automatically generate models and services with swagger.yaml
2020-01-14 17:58:32 +08:00
Wenkai Yin(尹文开)
4f3024191b
Merge pull request #10458 from ywk253100/20013_middleware
Provide a util function to apply middlewares to handler
2020-01-14 17:11:59 +08:00
Wenkai Yin
ff1a03cccc Get the ormer from context instead of creating a new one
This commit updates the dao methods with the ormer got from context

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-01-14 16:35:36 +08:00
Will Sun
88e6b0a743
Merge pull request #10443 from AllForNothing/express
Add express module
2020-01-14 15:22:23 +08:00
sshijun
679a6e0b05 Add ng-swagger-gen to automatically generate models and services with
swagger.yaml

Signed-off-by: sshijun <sshijun@vmware.com>
2020-01-14 15:18:32 +08:00
Will Sun
a3e84380fa
Merge pull request #10436 from AllForNothing/nightly
Improve tag-retention and project search function
2020-01-14 13:54:40 +08:00
Wenkai Yin(尹文开)
7e8bc0173d
Merge pull request #10390 from heww/gen-apis
feat(swagger): generate apis v2.0 from swagger.yaml
2020-01-14 13:39:59 +08:00
sshijun
95287db4b5 Improve tag-retention and project search function
Signed-off-by: sshijun <sshijun@vmware.com>

Improve tag-retention and project search function

Signed-off-by: sshijun <sshijun@vmware.com>
2020-01-14 13:26:30 +08:00
He Weiwei
c729e3b9e0 feat(swagger): generate apis v2.0 from swagger.yaml
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-01-14 02:20:18 +00:00
Wenkai Yin
c5cc7cf49e Provide a util function to apply middlewares to handler
Provide a util function to apply middlewares to handler

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-01-13 19:27:00 +08:00
Ziming Zhang
ed3bf041e2 fix(replication): aws ecr delete image
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I5e38b813c2840e0270973c38680cb8f815e5ece9
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2020-01-13 10:58:13 +08:00
Wenkai Yin
e91dbea76d Implement the manifest v1 image resolver
The manifest v1 image resolver populates the architecture info into the artifact

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-01-11 16:33:02 +08:00
Wenkai Yin
40d5043d5c Implement the index/manifest list image resolver
The index/manifest list image resolver populates the references of the artifact

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-01-11 16:02:16 +08:00
Wenkai Yin
df551e1310 Implement the artifact abstractor and resolver
1. Define the interface for artifact abstractor and resolver
2. Implement the artifact abstractor
3. Implement the resolver for image with manifest v2

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-01-11 15:09:08 +08:00
sshijun
8afc524adb Add express module
Signed-off-by: sshijun <sshijun@vmware.com>
2020-01-10 16:21:16 +08:00
Steven Zou
e899d659f3 fix[jobservice]:job status is hung after restart
- improve the status hook sending/resending approach
- improve the status compare and set approach
- simplify the relevant flow
- add reaper to fix the out of sync jobs
- fix #10244 , fix #9963

Signed-off-by: Steven Zou <szou@vmware.com>
2020-01-09 17:10:35 +08:00
jwangyangls
fb82cd6bd1
Merge pull request #10412 from jwangyangls/refactor-styles.css
Refactor styles.css
2020-01-09 17:06:48 +08:00
Qian Deng
ceb4bbf849
Merge pull request #10394 from heww/cherry-pick-pr-10393
[Cherry pick]fix(scanner): use new created ormer for transaction
2020-01-09 14:29:47 +08:00
Wenkai Yin(尹文开)
1ea9e68178
Merge pull request #10366 from ywk253100/191219_manager
Implement tag/artifact manager and artifact controller
2020-01-08 21:24:52 +08:00
Wenkai Yin
400a47a5c5 Implement tag/artifact manager and artifact controller
1. Implement tag/artifact manager
2. Implement artifact controller
3. Onboard the artifact when pushing artifacts

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-01-08 20:19:48 +08:00
Will Sun
593b99d1c1
Merge pull request #10408 from AllForNothing/modify-project-tab1
Add more UT
2020-01-07 18:05:54 +08:00
Yogi_Wang
2308519596 Refactor styles.css
1.refactor styles.css
2.modify theme mode
3.fix bug #9939
Signed-off-by: Yogi_Wang <yawang@vmware.com>
2020-01-07 14:37:26 +08:00
sshijun
fc78aeacc9 Add more UT
Signed-off-by: sshijun <sshijun@vmware.com>
2020-01-07 09:23:33 +08:00
wang yan
a0f3709b3c add expiration data time when to create a robot account
Update API of creating robot accout, user can specify expiration time per account.

Signed-off-by: wang yan <wangyan@vmware.com>
2020-01-03 13:47:06 +08:00
He Weiwei
b90ac468c5 fix(scanner): use new created ormer for transaction
Use new created ormer for transaction in SetDefaultRegistration

Closes #10284

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-01-03 05:25:32 +00:00
Will Sun
76dcedb4f3
Merge pull request #10375 from AllForNothing/more-ut-v2
Add more UT for project-quota-component
2020-01-02 13:43:56 +08:00
Will Sun
061b37764a
Merge pull request #10384 from AllForNothing/modify-project-tab1
Improve tabs UI for project detail page
2020-01-02 13:40:25 +08:00
Yogi_Wang
ad67e45932 Refine the tabs in ellipsis code
Signed-off-by: AllForNothing <sshijun@vmware.com>
Signed-off-by: Yogi_Wang <yawang@vmware.com>
2020-01-02 10:43:09 +08:00
He Weiwei
7a4cb17450
feat(orm): add orm support with context (#10337)
1. Get and set orm from context.
2. Add WithTransaction decorator make func run in transaction.
3. Support nested transaction by Savepoint.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-12-31 18:30:52 +08:00
Wenkai Yin(尹文开)
803e676ee7
Merge pull request #10385 from ywk253100/191231_replication
Remove the workaround for blocking manifest list in replication
2019-12-31 18:01:30 +08:00
Daniel Jiang
99e052aeb0
Merge pull request #10369 from reasonerjt/registry-basic-auth
Populate basic auth information for registry
2019-12-31 17:56:57 +08:00