The way Harbor handles notification is problematic.
It currently triggers rescan, which will cause problem when there are
lot of images in the registry.
Such as #7316
This commit removes the notifier and we need to revisit the notification
to figure out how to map the notification to a particular image if need
the notification mechanism in future.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
this is for internal registry api call, the request should be intercpeted by quota middlerwares, like retag and delete.
Note: The api developer has to know that if the internal registry call in your api, please consider to use
NewRepositoryClientForLocal() to init the repository client, which can handle quota change.
Signed-off-by: wang yan <wangyan@vmware.com>
This commit fixes#8432
When querying the role of group ID, all matched roles should be returned
instead of the minimal role ID.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
[Add]:
1, size middleware for quota size
2, count middleware for quota artifact count
[Support]:
1, put, patch, mount blob
2, put manifest
[Refactor]:
1, Add handle response for middlerware
2, Remove the modifyResponse for registry proxy
3, Use the custom response writer to recored status
Signed-off-by: wang yan <wangyan@vmware.com>
If harbor is built (or `go test`'d) in a different folder than the repo
root, the call to common/utils/log/line(...) will panic with an index
out of range runtime error because the separator can't find `harbor/src`
in the path.
Signed-off-by: Nathan Lowe <public@nlowe.me>
Signed-off-by: wang yan <wangyan@vmware.com>
Add dao for quota
Signed-off-by: He Weiwei <hweiwei@vmware.com>
fix govet
Signed-off-by: wang yan <wangyan@vmware.com>
This commit update the project API to support "reuse_sys_cve_whitelist"
setting in project metadata and "cve_whitelist" in project request.
Also modify the interceptor to support project level CVE whitelist if
the reuse flag is false.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
Interceptor will filter the vulnerability in whitelist while calculating
the serverity of an image and determine whether or not to block client
form pulling it.
It will use the system level whitelist in this commit, another commit
will switch to project level whitelist based on setting in a project.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
When the auth mode is OIDC, when a user login via Harbor's login form.
If the user does not exist or the user is onboarded via OIDC, he will be
redirected to the OIDC login page.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
* add scan all and gc schedule migration
Signed-off-by: wang yan <wangyan@vmware.com>
* Fix gofmt errors
Signed-off-by: wang yan <wangyan@vmware.com>
* Update code according to review comments
Signed-off-by: wang yan <wangyan@vmware.com>
* remove convertschedule return name just return value
Signed-off-by: wang yan <wangyan@vmware.com>
* add periodic job UUID to upstream job id and use execution log as the periodic log
Signed-off-by: wang yan <wangyan@vmware.com>
* add comments to fix codacy
Signed-off-by: wang yan <wangyan@vmware.com>
* Update code per comments
Signed-off-by: wang yan <wangyan@vmware.com>
This commit update the response off OIDC callback when there's error in exchange token.
Additionally add comments to clarify that by default 500 error will not
contain any details.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
* Return 404 when the log of task doesn't exist
Return 404 when the log of task doesn't exist
Signed-off-by: Wenkai Yin <yinw@vmware.com>
* Upgrade the distribution and notary library
Upgrade the distribution library to 2.7.1, the notary library to 0.6.1
Signed-off-by: Wenkai Yin <yinw@vmware.com>
This commit make sure the token is persist to DB after every time after
a user logs in via OIDC provider, to make sure the secret is usable for
the OIDC providers that don't provide refresh token.
It also updates the authorize URL for google to make sure the refresh
token will be returned.
Also some misc refinement included, including add comment to the
OIDC onboarded user, preset the username in onboard dialog.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
Ths commmit ensures that when user's token is invalidated OIDC provider, he
cannot access protected resource in Harbor with the user info in his session.
We share the code path with secret verification b/c the refresh token
can be used only once, so it has to be stored in one place.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
This commit tweaks the attribute for auth proxy mode and OIDC auth mode.
To change it from "Skip verify cert" to "verify cert" so they are more
consistent with other modes.
Additionally it removes a workaround in `SearchUser` in auth proxy
authenticator.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
As CLI does not support oauth flow, we'll use secret for help OIDC user
to authenticate via CLI.
Add column to store secret and token, and add code to support
verify/refresh token associates with secret. Such that when the user is
removed from OIDC provider the secret will no longer work.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
This commit adjust the code and fix some bugs to make onboard process
work.
Only thing missed is that the UI will need to initiate the redirection,
because the request of onboarding a user was sent via ajax call and didn't
handle the 302.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
Only add the authentication info when the username is provided to support pulling public images from docker hub without login
Signed-off-by: Wenkai Yin <yinw@vmware.com>
This commit add callback controller to handle the redirection from
successful OIDC authentication.
For E2E case this requires callback controller to kick off onboard
process, which will be covered in subsequent commits.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
The controller will redirect user to the OIDC login page based on
configuration.
Additionally this commit add some basic code to wrap `oauth2` package
and `provider` in `go-oidc`, and fixed an issue in UT to make
InMemoryDriver for config management thread-safe.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
* add authn proxy docker login support
User could use the web hook token issued by k8s api server to login to harbor.
The username should add a specific prefix.
Signed-off-by: wang yan <wangyan@vmware.com>
* update code per review comments
Signed-off-by: wang yan <wangyan@vmware.com>
* Add UT for auth proxy modifier
Signed-off-by: wang yan <wangyan@vmware.com>
* Refactor scan all api
This commit is to let scan all api using admin job to handle schedule
management. After the PR, GC and scan all share unified code path.
Signed-off-by: wang yan <wangyan@vmware.com>
* update admin job api code according to review comments
Signed-off-by: wang yan <wangyan@vmware.com>
* Update test code and comments per review
Signed-off-by: wang yan <wangyan@vmware.com>
2. Add ng replication policy CURD methods.
3. Implement ng policy manger.
Signed-off-by: Frank Kung <kfanjian@gmail.com>
Signed-off-by: 慕薇疯魔 <kfanjian@gmail.com>
Previously the settings of HTTP authproxy were set in environment
variable.
This commit move them to the configuration API
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
This commit is to make the expiration of robot account configurable
1, The expiration could be set by system admin in the configuation page or
by /api/config with robot_token_expiration=60, the default value is 30 days.
2, The expiration could be shown in the robot account infor both on UI and API.
Signed-off-by: wang yan <wangyan@vmware.com>
This commit is to do:
1, Add post response on creating robot account
2, Lower-case the attribute of response
Signed-off-by: wang yan <wangyan@vmware.com>
An HTTP authenticator verifies the credentials by sending a POST request
to an HTTP endpoint. After successful authentication he will be
onboarded to Harbor's local DB and assigned a role in a project.
This commit provides the initial implementation.
Currently one limitation is that we don't have clear definition about
how we would "search" a user via this HTTP authenticator, a flag for
"alway onboard" is provided to skip the search, otherwise, a user has
to login first before he can be assigned a role in Harbor.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
* Support master role for project member create and update apis
Signed-off-by: He Weiwei <hweiwei@vmware.com>
* Fix description for role_id in swagger.yaml
Signed-off-by: He Weiwei <hweiwei@vmware.com>
* Add Can method to securty.Context interface
Signed-off-by: He Weiwei <hweiwei@vmware.com>
* Improve mockSecurityContext Can method
Signed-off-by: He Weiwei <hweiwei@vmware.com>
Add API implementation of robot account
1. POST /api/project/pid/robots
2, GET /api/project/pid/robots/id?
3, PUT /api/project/pid/robots/id
4, DELETE /api/project/pid/robots/id
Signed-off-by: wang yan <wangyan@vmware.com>
There are two different types to represent http error in the current code. This commit updates the codes to keep only one.
Signed-off-by: Wenkai Yin <yinw@vmware.com>
Previously there was a in-memory marker to prevent user from frequently
calling the "scan all" API. This has become problematic in HA
deployment, and is no longer needed after enhancement in jobservice.
This commit removes the marker for "scan all" api, however, we need to
review the mechanism and rework to make it stateless.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
1. Fix#5102 by checking the existence of name when creating/editing replication rule
2. Add unique constraint to the name of replication policy and target
3. Fix bugs of testing library
Signed-off-by: Wenkai Yin <yinw@vmware.com>
Fixes#6115
As for the change in migration sql file, in 1.7 we'll switch to
jobservice for scheduling "scan all" job. To avoid inconsistency,
this item will be reset and user will need to configure the policy again.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
This commit mitigates the situation when more then one adminserver is
deployed and there may be duplication error when they try to initialize
the configuration to DB.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
In various parts of the code, we used insecure transport in http Client
when we assume the endpoint is http. This causes complaints form
security scanner. We should use secure transport in such cases.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
This commit leverage the jobservice to trigger "scan all" and
gets rid of the local scheduler to make the harbor-core container
stateless.
It keeps using the notifer mechanism to handle the configuration change.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
- add new interface method to get total count of charts under namespaces by calling get index
- add new field 'chart_count' in project model
- append chart count to the project model in project API
Signed-off-by: Steven Zou <szou@vmware.com>
This commit adds the job to scan all images on registry.
It also makes necessary change to Secret based security context, to
job service has higher permission to call the API of core service.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
1. Update the nginx.conf
2. Update Makefile
3. Update docker-compose
4. Update image name
5. Rename folder ui to core
6. Change the harbor-ui's package name to core
7. Remove unused static file on harbor-core
8. Remove unused code for harbor-portal
Signed-off-by: Qian Deng <dengq@vmware.com>
This commit is to update gc api to fix issues found by UI implemention:
1, Return json format of gc schedule
2, Unify capital and small letter
3，Return gc records by desc
Signed-off-by: wang yan <wangyan@vmware.com>
- add related chart label API entries
- extract label related functionalities to a separate manager interface
- add a base controller for label related actions
- add related UT cases
Signed-off-by: Steven Zou <szou@vmware.com>
The sslmode of the connection with postgresql is hardcoded as "disable" currently, this commit expose it as an environment variable so that users can configure it
Signed-off-by: Wenkai Yin <yinw@vmware.com>
Fix issue #5776, LDAP servers are case insensitive. because only LDAP
group DN is used to compare/equal operation, lowercase all LDAP group DN
when retrieves it from LDAP server, and lowercase them before save in DB
Signed-off-by: stonezdj <stonezdj@gmail.com>
