Upstream/harbor
1
0
Fork 0
mirror of https://github.com/goharbor/harbor.git synced 2024-12-22 16:48:30 +01:00
Code Issues Projects Releases Wiki Activity

Merge remote-tracking branch 'upstream/dev' into 170224_merge_config

Browse Source 
Conflicts:
	make/docker-compose.tpl
	src/ui/service/token/authutils.go
This commit is contained in:
Wenkai Yin 2017-02-24 13:52:19 +08:00
commit 414e8a8bcf
128 changed files with 13520 additions and 366 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
.github/ISSUE_TEMPLATE vendored
View File

@ -1,5 +1,5 @@
If you are reporting a problem, please make sure the following information are provided:
1)Version of docker engine and docker-compose
2)Config files of harbor, you can get them by packaging "Deploy/config" directory
3)Log files, you can get them by package the /var/log/harbor/
1)Version of docker engine and docker-compose.
2)Config files of harbor, you can get them by packaging "harbor.cfg" and files in the same directory, including subdirectory.
3)Log files, you can get them by package the /var/log/harbor/ .

2
Makefile
View File

@ -219,7 +219,7 @@ compile:check_environment $(COMPILETAG)
prepare:
@echo "preparing..."
@$(MAKEPATH)/$(PREPARECMD) -conf $(CONFIGPATH)/$(CONFIGFILE)
@$(MAKEPATH)/$(PREPARECMD) --conf $(CONFIGPATH)/$(CONFIGFILE)
build_common: version
@echo "buildging db container for photon..."

13
README.md
View File

@ -32,7 +32,8 @@ Refer to **[User Guide](docs/user_guide.md)** for more details on how to use Har
### Community
**Slack:** Join Harbor's community here: [VMware {code}](https://code.vmware.com/join/), Channel: #harbor.
**Email:** harbor@ vmware.com .
**WeChat Group:** Add WeChat id *connect1688* to join WeChat discussion group.
**WeChat Group:** Add WeChat id *connect1688* to join WeChat discussion group.
More info on [partners and users](partners.md).
### Contribution
We welcome contributions from the community. If you wish to contribute code and you have not signed our contributor license agreement (CLA), our bot will update the issue when you open a pull request. For any questions about the CLA process, please refer to our [FAQ](https://cla.vmware.com/faq). Contact us for any questions: harbor @vmware.com .
@ -46,13 +47,3 @@ This project uses open source components which have additional licensing terms.
* Docker Registry 2.5: [docker image](https://hub.docker.com/_/registry/), [license](https://github.com/docker/distribution/blob/master/LICENSE)
* MySQL 5.6: [docker image](https://hub.docker.com/_/mysql/), [license](https://github.com/docker-library/mysql/blob/master/LICENSE)
* NGINX 1.11.5: [docker image](https://hub.docker.com/_/nginx/), [license](https://github.com/nginxinc/docker-nginx/blob/master/LICENSE)
### Partners
<a href="https://www.shurenyun.com/" border="0" target="_blank"><img alt="DataMan" src="docs/img/dataman.png"></a> &nbsp; &nbsp; <a href="http://www.slamtec.com" target="_blank" border="0"><img alt="SlamTec" src="docs/img/slamteclogo.png"></a>
&nbsp; &nbsp; <a href="https://www.caicloud.io" border="0"><img alt="CaiCloud" src="docs/img/caicloudLogoWeb.png"></a>
### Users
<a href="https://www.madailicai.com/" border="0" target="_blank"><img alt="MaDaiLiCai" src="docs/img/UserMaDai.jpg"></a> <a href="https://www.dianrong.com/" border="0" target="_blank"><img alt="Dianrong" src="docs/img/dianrong.png"></a>
### Supporting Technologies
<img alt="beego" src="docs/img/beegoLogo.png"> Harbor is powered by <a href="http://beego.me/">Beego</a>.

29
contrib/registryapi/README.md Normal file
View File

@ -0,0 +1,29 @@
# registryapi
api for docker registry by token authorization
+ a simple api class which lies in registryapi.py, which simulates the interactions
between docker registry and the vendor authorization platform like harbor.
```
usage:
from registryapi import RegistryApi
api = RegistryApi('username', 'password', 'http://www.your_registry_url.com/')
repos = api.getRepositoryList()
tags = api.getTagList('public/ubuntu')
manifest = api.getManifest('public/ubuntu', 'latest')
res = api.deleteManifest('public/ubuntu', '23424545**4343')
```
+ a simple client tool based on api class, which contains basic read and delete
operations for repo, tag, manifest
```
usage:
./cli.py --username username --password passwrod --registry_endpoint http://www.your_registry_url.com/ target action params
target can be: repo, tag, manifest
action can be: list, get, delete
params can be: --repo --ref --tag
more see: ./cli.py -h
```

135
contrib/registryapi/cli.py Executable file
View File

@ -0,0 +1,135 @@
#!/usr/bin/env python
# -*- coding:utf-8 -*-
# bug-report: feilengcui008@gmail.com
""" cli tool """
import argparse
import sys
import json
from registry import RegistryApi
class ApiProxy(object):
""" user RegistryApi """
def __init__(self, registry, args):
self.registry = registry
self.args = args
self.callbacks = dict()
self.register_callback("repo", "list", self.list_repo)
self.register_callback("tag", "list", self.list_tag)
self.register_callback("tag", "delete", self.delete_tag)
self.register_callback("manifest", "list", self.list_manifest)
self.register_callback("manifest", "delete", self.delete_manifest)
self.register_callback("manifest", "get", self.get_manifest)
def register_callback(self, target, action, func):
""" register real actions """
if not target in self.callbacks.keys():
self.callbacks[target] = {action: func}
return
self.callbacks[target][action] = func
def execute(self, target, action):
""" execute """
print json.dumps(self.callbacks[target][action](), indent=4, sort_keys=True)
def list_repo(self):
""" list repo """
return self.registry.getRepositoryList(self.args.num)
def list_tag(self):
""" list tag """
return self.registry.getTagList(self.args.repo)
def delete_tag(self):
""" delete tag """
(_, ref) = self.registry.existManifest(self.args.repo, self.args.tag)
if ref is not None:
return self.registry.deleteManifest(self.args.repo, ref)
return False
def list_manifest(self):
""" list manifest """
tags = self.registry.getTagList(self.args.repo)["tags"]
manifests = list()
if tags is None:
return None
for i in tags:
content = self.registry.getManifestWithConf(self.args.repo, i)
manifests.append({i: content})
return manifests
def delete_manifest(self):
""" delete manifest """
return self.registry.deleteManifest(self.args.repo, self.args.ref)
def get_manifest(self):
""" get manifest """
return self.registry.getManifestWithConf(self.args.repo, self.args.tag)
# since just a script tool, we do not construct whole target->action->args
# structure with oo abstractions which has more flexibility, just register
# parser directly
def get_parser():
""" return a parser """
parser = argparse.ArgumentParser("cli")
parser.add_argument('--username', action='store', required=True, help='username')
parser.add_argument('--password', action='store', required=True, help='password')
parser.add_argument('--registry_endpoint', action='store', required=True,
help='registry endpoint')
subparsers = parser.add_subparsers(dest='target', help='target to operate on')
# repo target
repo_target_parser = subparsers.add_parser('repo', help='target repository')
repo_target_subparsers = repo_target_parser.add_subparsers(dest='action',
help='repository subcommand')
repo_cmd_parser = repo_target_subparsers.add_parser('list', help='list repositories')
repo_cmd_parser.add_argument('--num', action='store', required=False, default=None,
help='the number of data to return')
# tag target
tag_target_parser = subparsers.add_parser('tag', help='target tag')
tag_target_subparsers = tag_target_parser.add_subparsers(dest='action',
help='tag subcommand')
tag_list_parser = tag_target_subparsers.add_parser('list', help='list tags')
tag_list_parser.add_argument('--repo', action='store', required=True, help='list tags')
tag_delete_parser = tag_target_subparsers.add_parser('delete', help='delete tag')
tag_delete_parser.add_argument('--repo', action='store', required=True, help='delete tags')
tag_delete_parser.add_argument('--tag', action='store', required=True,
help='tag reference')
# manifest target
manifest_target_parser = subparsers.add_parser('manifest', help='target manifest')
manifest_target_subparsers = manifest_target_parser.add_subparsers(dest='action',
help='manifest subcommand')
manifest_list_parser = manifest_target_subparsers.add_parser('list', help='list manifests')
manifest_list_parser.add_argument('--repo', action='store', required=True,
help='list manifests')
manifest_delete_parser = manifest_target_subparsers.add_parser('delete', help='delete manifest')
manifest_delete_parser.add_argument('--repo', action='store', required=True,
help='delete manifest')
manifest_delete_parser.add_argument('--ref', action='store', required=True,
help='manifest reference')
manifest_get_parser = manifest_target_subparsers.add_parser('get', help='get manifest content')
manifest_get_parser.add_argument('--repo', action='store', required=True, help='delete tags')
manifest_get_parser.add_argument('--tag', action='store', required=True,
help='manifest reference')
return parser
def main():
""" main entrance """
parser = get_parser()
options = parser.parse_args(sys.argv[1:])
registry = RegistryApi(options.username, options.password, options.registry_endpoint)
proxy = ApiProxy(registry, options)
proxy.execute(options.target, options.action)
if __name__ == '__main__':
main()

165
contrib/registryapi/registry.py Normal file
View File

@ -0,0 +1,165 @@
#!/usr/bin/env python
# -*- coding:utf-8 -*-
# bug-report: feilengcui008@gmail.com
""" api for docker registry """
import urllib2
import urllib
import json
import base64
class RegistryException(Exception):
""" registry api related exception """
pass
class RegistryApi(object):
""" interact with docker registry and harbor """
def __init__(self, username, password, registry_endpoint):
self.username = username
self.password = password
self.basic_token = base64.encodestring("%s:%s" % (str(username), str(password)))[0:-1]
self.registry_endpoint = registry_endpoint.rstrip('/')
auth = self.pingRegistry("%s/v2/_catalog" % (self.registry_endpoint,))
if auth is None:
raise RegistryException("get token realm and service failed")
self.token_endpoint = auth[0]
self.service = auth[1]
def pingRegistry(self, registry_endpoint):
""" ping v2 registry and get realm and service """
headers = dict()
try:
res = urllib2.urlopen(registry_endpoint)
except urllib2.HTTPError as e:
headers = e.hdrs.dict
try:
(realm, service, _) = headers['www-authenticate'].split(',')
return (realm[14:-1:], service[9:-1])
except Exception as e:
return None
def getBearerTokenForScope(self, scope):
""" get bearer token from harbor """
payload = urllib.urlencode({'service': self.service, 'scope': scope})
url = "%s?%s" % (self.token_endpoint, payload)
req = urllib2.Request(url)
req.add_header('Authorization', 'Basic %s' % (self.basic_token,))
try:
response = urllib2.urlopen(req)
return json.loads(response.read())["token"]
except Exception as e:
return None
def getRepositoryList(self, n=None):
""" get repository list """
scope = "registry:catalog:*"
bear_token = self.getBearerTokenForScope(scope)
if bear_token is None:
return None
url = "%s/v2/_catalog" % (self.registry_endpoint,)
if n is not None:
url = "%s?n=%s" % (url, str(n))
req = urllib2.Request(url)
req.add_header('Authorization', r'Bearer %s' % (bear_token,))
try:
response = urllib2.urlopen(req)
return json.loads(response.read())
except Exception as e:
return None
def getTagList(self, repository):
""" get tag list for repository """
scope = "repository:%s:pull" % (repository,)
bear_token = self.getBearerTokenForScope(scope)
if bear_token is None:
return None
url = "%s/v2/%s/tags/list" % (self.registry_endpoint, repository)
req = urllib2.Request(url)
req.add_header('Authorization', r'Bearer %s' % (bear_token,))
try:
response = urllib2.urlopen(req)
return json.loads(response.read())
except Exception as e:
return None
def getManifest(self, repository, reference="latest", v1=False):
""" get manifest for tag or digest """
scope = "repository:%s:pull" % (repository,)
bear_token = self.getBearerTokenForScope(scope)
if bear_token is None:
return None
url = "%s/v2/%s/manifests/%s" % (self.registry_endpoint, repository, reference)
req = urllib2.Request(url)
req.get_method = lambda: 'GET'
req.add_header('Authorization', r'Bearer %s' % (bear_token,))
req.add_header('Accept', 'application/vnd.docker.distribution.manifest.v2+json')
if v1:
req.add_header('Accept', 'application/vnd.docker.distribution.manifest.v1+json')
try:
response = urllib2.urlopen(req)
return json.loads(response.read())
except Exception as e:
return None
def existManifest(self, repository, reference, v1=False):
""" check to see it manifest exist """
scope = "repository:%s:pull" % (repository,)
bear_token = self.getBearerTokenForScope(scope)
if bear_token is None:
raise RegistryException("manifestExist failed due to token error")
url = "%s/v2/%s/manifests/%s" % (self.registry_endpoint, repository, reference)
req = urllib2.Request(url)
req.get_method = lambda: 'HEAD'
req.add_header('Authorization', r'Bearer %s' % (bear_token,))
req.add_header('Accept', 'application/vnd.docker.distribution.manifest.v2+json')
if v1:
req.add_header('Accept', 'application/vnd.docker.distribution.manifest.v1+json')
try:
response = urllib2.urlopen(req)
return (True, response.headers.dict["docker-content-digest"])
except Exception as e:
return (False, None)
def deleteManifest(self, repository, reference):
""" delete manifest by tag """
(is_exist, digest) = self.existManifest(repository, reference)
if not is_exist:
raise RegistryException("manifest not exist")
scope = "repository:%s:pull,push" % (repository,)
bear_token = self.getBearerTokenForScope(scope)
if bear_token is None:
raise RegistryException("delete manifest failed due to token error")
url = "%s/v2/%s/manifests/%s" % (self.registry_endpoint, repository, digest)
req = urllib2.Request(url)
req.get_method = lambda: 'DELETE'
req.add_header('Authorization', r'Bearer %s' % (bear_token,))
try:
urllib2.urlopen(req)
except Exception as e:
return False
return True
def getManifestWithConf(self, repository, reference="latest"):
""" get manifest for tag or digest """
manifest = self.getManifest(repository, reference)
if manifest is None:
raise RegistryException("manifest for %s %s not exist" % (repository, reference))
config_digest = manifest["config"]["digest"]
scope = "repository:%s:pull" % (repository,)
bear_token = self.getBearerTokenForScope(scope)
if bear_token is None:
return None
url = "%s/v2/%s/blobs/%s" % (self.registry_endpoint, repository, config_digest)
req = urllib2.Request(url)
req.get_method = lambda: 'GET'
req.add_header('Authorization', r'Bearer %s' % (bear_token,))
req.add_header('Accept', 'application/vnd.docker.distribution.manifest.v2+json')
try:
response = urllib2.urlopen(req)
manifest["configContent"] = json.loads(response.read())
return manifest
except Exception as e:
return None

3
docs/README.md
View File

@ -26,6 +26,9 @@ Guide to deploy Harbor on Kubenetes. (maintained by community)
[Architecture Overview of Harbor](https://github.com/vmware/harbor/wiki/Architecture-Overview-of-Harbor)
Developers read this first.
[Build Harbor from Source](compile_guide.md)
How to build Harbor from source code.
[Harbor API Specs by Swagger](configure_swagger.md)
Use Swagger to find out the specs of Harbor API.

193
docs/kubernetes_deployment.md
View File

@ -1,65 +1,142 @@
## makeing Harbor on Kubernetes
To deploy Harbor on Kubernetes, it requires some additional steps because
1. When Harbor registry uses https, so we need cert or workaround to avoid errors like this:
```
Error response from daemon: invalid registry endpoint https://{HOST}/v0/: unable to ping registry endpoint https://{HOST}/v0/
v2 ping attempt failed with error: Get https://{HOST}/v2/: EOF
v1 ping attempt failed with error: Get https://{HOST}/v1/_ping: EOF. If this private registry supports only HTTP or HTTPS with an unknown CA certificate, please add `--insecure-registry {HOST}` to the daemon's arguments. In the case of HTTPS, if you have access to the registry's CA certificate, no need for the flag; simply place the CA certificate at /etc/docker/certs.d/{HOST}/ca.crt
```
There is a workaround if you don't have a cert. The workaround is to add the host into the list of insecure registry by editting the ```/etc/default/docker``` file:
```
sudo vi /etc/default/docker
```
add the line at the end of file:
```
DOCKER_OPTS="$DOCKER_OPTS --insecure-registry={HOST}"
```
restart docker service
```
sudo service docker restart
```
2. The registry config file needs to have the IP (or DNS name) of the registry, but on Kubernetes, you don't know the IP before the service is created. There are several workarounds to solve this problem for now:
- Use DNS name and link the DNS name with the IP after the service is created.
- Rebuild the registry image with the service IP after the service is created and use ```kubectl rolling-update``` to update to the new image.
To start Harbor on Kubernetes, you first need to build the docker images. The docker images for deploying Harbor on Kubernetes depends on the docker images to deploy Harbor with docker-compose. So the first step is to build docker images with docker-compose. Before actually building the images, you need to first adjust the [configuration](https://github.com/vmware/harbor/blob/master/make/harbor.cfg):
- Change the [hostname](https://github.com/vmware/harbor/blob/master/make/harbor.cfg#L5) to ```localhost```
- Adjust the [email settings](https://github.com/vmware/harbor/blob/master/make/harbor.cfg#L11) according to your needs.
## Integration with Kubernetes
This Document decribes how to deploy Harbor on Kubernetes.
### Prerequisite
* You need to download docker images of Harbor.
* Download the offline installer of Harbor from the [release](https://github.com/vmware/harbor/releases) page.
* Uncompress the offline installer and get the images tgz file harbor.*.tgz.
* Load the images into docker:
```
docker load -i harbor.*.tgz
```
* You should have domain knowledge about Kubernetes (Replication Controller, Service, Persistent Volume, Persistent Volume Claim, Config Map).
### Configuration
We provide a python script `make/kubernetes/prepare` to generate Kubernetes ConfigMap files.
The script is written in python, so you need a version of python in your deployment environment.
Also the script need `openssl` to generate private key and certification, make sure you have a workable `openssl`.
There are some args of the python script:
- -f: Default Value is `../harbor.cfg`. You can specify other config file of Harbor.
- -k: Path to https private key. This arg can overwrite the value of `ssl_cert_key` in `harbor.cfg`.
- -c: Path to https certification. This arg can overwrite the value of `ssl_cert` in `harbor.cfg`.
- -s: Path to secret key. Must be 16 characters. If you don't set it, the script will generate it automatically.
#### Basic Configuration
These Basic Configuration must be set. Otherwise you can't deploy Harbor on Kubernetes.
- `make/harbor.cfg`: Basic config of Harbor. Please refer to `harbor.cfg`.
- `make/kubernetes/**/*.rc.yaml`: Specify configs of containers.
You need to specify the path to your images in all `*.rc.yaml`. example:
```
containers:
- name: nginx-app
# it's very importent that you need modify the path of image.
image: harbor/nginx
```
- `make/kubernetes/pv/*.pvc.yaml`: Persistent Volume Claim.
You can set capacity of storage in these files. example:
```
resources:
requests:
# you can set another value to adapt to your needs
storage: 100Gi
```
- `make/kubernetes/pv/*.pv.yaml`: Persistent Volume. Be bound with `*.pvc.yaml`.
PVs and PVCs are one to one correspondence. If you changed capacity of PVC, you need to set capacity of PV together.
example:
```
capacity:
# same value with PVC
storage: 100Gi
```
In PV, you should set another way to store data rather than `hostPath`:
```
# it's default value, you should use others like nfs.
hostPath:
path: /data/registry
```
For more infomation about store ways, Please check [Kubernetes Document](http://kubernetes.io/docs/user-guide/persistent-volumes/)
Then you can generate ConfigMap files by :
Then you can run the following commends to build docker images:
```
cd make
./prepare
docker-compose build
docker build -f kubernetes/dockerfiles/proxy-dockerfile -t {your_account}/proxy .
docker build -f kubernetes/dockerfiles/registry-dockerfile -t {your_account}/registry .
docker build -f kubernetes/dockerfiles/ui-dockerfile -t {your_account}/deploy_ui .
docker tag deploy_mysql {your_account}/deploy_mysql
docker push {your_account}/proxy
docker push {your_account}/registry
docker push {your_account}/deploy_ui
docker push {your_account}/deploy_mysql
```
where "your_account" is your own registry. Then you need to update the "image" field in the ```*-rc.yaml``` files at:
```
make/kubernetes/mysql-rc.yaml
make/kubernetes/proxy-rc.yaml
make/kubernetes/registry-rc.yaml
make/kubernetes/ui-rc.yaml
python make/kubernetes/prepare
```
Further more, the following configuration could be changed according to your need:
- **harbor_admin_password**: The password for the administrator of Harbor, by default the password is Harbor12345. You can changed it [here](https://github.com/vmware/harbor/blob/master/make/kubernetes/ui-rc.yaml#L36).
- **auth_mode**: The authentication mode of Harbor. By default it is *db_auth*, i.e. the credentials are stored in a database. Please set it to *ldap_auth* if you want to verify user's credentials against an LDAP server. You can change the configuration [here](https://github.com/vmware/harbor/blob/master/make/kubernetes/ui-rc.yaml#L40).
- **ldap_url**: The URL for LDAP endpoint, for example ldaps://ldap.mydomain.com. It is only used when **auth_mode** is set to *ldap_auth*. It could be changed [here](https://github.com/vmware/harbor/blob/master/make/kubernetes/ui-rc.yaml#L42).
- **ldap_basedn**: The basedn template for verifying the user's credentials against LDAP, for example uid=%s,ou=people,dc=mydomain,dc=com. It is only used when **auth_mode** is set to *ldap_auth*. It could be changed [here](https://github.com/vmware/harbor/blob/master/make/kubernetes/ui-rc.yaml#L44).
- **db_password**: The password of root user of mySQL database. Change this password for any production use. You need to change both [here](https://github.com/vmware/harbor/blob/master/make/kubernetes/ui-rc.yaml#L28) and [here](https://github.com/vmware/harbor/blob/master/make/harbor.cfg#L32) to make the change. Please note, you need to change the ```harbor.cfg``` before building the docker images.
These files will be generated:
- make/kubernetes/jobservice/jobservice.cm.yaml
- make/kubernetes/mysql/mysql.cm.yaml
- make/kubernetes/nginx/nginx.cm.yaml
- make/kubernetes/registry/registry.cm.yaml
- make/kubernetes/ui/ui.cm.yaml
Finally you can start the jobs by running:
```
kubectl create -f make/kubernetes
```
#### Advanced Configuration
If Basic Configuration was not covering your requirements, you can read this section for more details.
`./prepare` has a specify format of placeholder:
- `{{key}}`: It means we should replace the placeholder with the value in `config.cfg` which name is `key`.
- `{{num key}}`: It's used for multiple lines text. It will add `num` spaces to the leading of every line in text.
You can find all configs of Harbor in `make/kubernetes/templates/`. There are specifications of these files:
- `jobservice.cm.yaml`: ENV and web config of jobservice
- `mysql.cm.yaml`: Root passowrd of MySQL
- `nginx.cm.yaml`: Https certification and nginx config. If you are fimiliar with nginx, you can modify it.
- `registry.cm.yaml`: Token service certification and registry config
Registry use filesystem to store data of images. You can find it like:
```
storage:
filesystem:
rootdirectory: /storage
```
If you want use another storage backend, please see [Docker Doc](https://docs.docker.com/datacenter/dtr/2.1/guides/configure/configure-storage/)
- `ui.cm.yaml`: Token service private key, ENV and web config of ui
`ui` and `jobservice` are powered by beego. If you are fimiliar with beego, you can modify configs in `jobservice.cm.yaml` and `ui.cm.yaml`.
### Running
When you finished your configuring and generated ConfigMap files, you can run Harbor on kubernetes with these commands:
```
# create pv & pvc
kubectl apply -f make/kubernetes/pv/log.pv.yaml
kubectl apply -f make/kubernetes/pv/registry.pv.yaml
kubectl apply -f make/kubernetes/pv/storage.pv.yaml
kubectl apply -f make/kubernetes/pv/log.pvc.yaml
kubectl apply -f make/kubernetes/pv/registry.pvc.yaml
kubectl apply -f make/kubernetes/pv/storage.pvc.yaml
# create config map
kubectl apply -f make/kubernetes/jobservice/jobservice.cm.yaml
kubectl apply -f make/kubernetes/mysql/mysql.cm.yaml
kubectl apply -f make/kubernetes/nginx/nginx.cm.yaml
kubectl apply -f make/kubernetes/registry/registry.cm.yaml
kubectl apply -f make/kubernetes/ui/ui.cm.yaml
# create service
kubectl apply -f make/kubernetes/jobservice/jobservice.svc.yaml
kubectl apply -f make/kubernetes/mysql/mysql.svc.yaml
kubectl apply -f make/kubernetes/nginx/nginx.svc.yaml
kubectl apply -f make/kubernetes/registry/registry.svc.yaml
kubectl apply -f make/kubernetes/ui/ui.svc.yaml
# create k8s rc
kubectl apply -f make/kubernetes/registry/registry.rc.yaml
kubectl apply -f make/kubernetes/mysql/mysql.rc.yaml
kubectl apply -f make/kubernetes/jobservice/jobservice.rc.yaml
kubectl apply -f make/kubernetes/ui/ui.rc.yaml
kubectl apply -f make/kubernetes/nginx/nginx.rc.yaml
```

44
docs/migration_guide.md
View File

@ -9,23 +9,21 @@ When upgrading your existing Habor instance to a newer version, you may need to
### Upgrading Harbor and migrating data
1. Log in to the machine that Harbor runs on, stop and remove existing Harbor service if it is still running:
1. Log in to the host that Harbor runs on, stop and remove existing Harbor instance if it is still running:
```
cd make/
cd harbor
docker-compose down
```
2. Back up Harbor's current source code so that you can roll back to the current version when it is necessary.
2. Back up Harbor's current files so that you can roll back to the current version when it is necessary.
```sh
cd ../..
cd ..
mv harbor /tmp/harbor
```
3. Get the lastest source code from Github:
```sh
git clone https://github.com/vmware/harbor
```
3. Get the lastest Harbor release package from Github:
https://github.com/vmware/harbor/releases
4. Before upgrading Harbor, perform database migration first. The migration tool is delivered as a docker image, so you should pull the image from docker hub:
@ -45,24 +43,21 @@ When upgrading your existing Habor instance to a newer version, you may need to
docker run -ti --rm -e DB_USR=root -e DB_PWD=xxxx -v /data/database:/var/lib/mysql vmware/harbor-db-migrator up head
```
7. Change to `make/` directory, configure Harbor by modifying the file `harbor.cfg`, you may need to refer to the configuration files you've backed up during step 2. Refer to [Installation & Configuration Guide ](../docs/installation_guide.md) for more info.
7. Unzip the new Harbor package and change to `./harbor` as the working directory. Configure Harbor by modifying the file `harbor.cfg`,
you may need to refer to the configuration files you've backed up during step 2.
Refer to [Installation & Configuration Guide ](../docs/installation_guide.md) for more information.
Since the content and format of `harbor.cfg` may have been changed in the new release, **DO NOT directly copy `harbor.cfg` from previous version of Harbor.**
8. Under the directory `make/`, run the `./prepare` script to generate necessary config files.
8. Under the directory `./harbor`, run the `./install.sh` script to install the new Harbor instance.
9. Rebuild Harbor and restart the registry service
```
docker-compose up --build -d
```
### Roll back from an upgrade
For any reason, if you want to roll back to the previous version of Harbor, follow the below steps:
1. Stop and remove the current Harbor service if it is still running.
```
cd make/
cd harbor
docker-compose down
```
2. Restore database from backup file in `/path/to/backup` .
@ -71,19 +66,26 @@ For any reason, if you want to roll back to the previous version of Harbor, foll
docker run -ti --rm -e DB_USR=root -e DB_PWD=xxxx -v /data/database:/var/lib/mysql -v /path/to/backup:/harbor-migration/backup vmware/harbor-db-migrator restore
```
3. Remove current source code of Harbor.
3. Remove current Harbor instance.
```
rm -rf harbor
```
4. Restore the source code of an older version of Harbor.
4. Restore the older version package of Harbor.
```sh
mv /tmp/harbor harbor
```
5. Restart Harbor service using the previous configuration.
5. Restart Harbor service using the previous configuration.
If previous version of Harbor was installed by a release build:
```sh
cd make/
cd harbor
./install.sh
```
If your previous version of Harbor was installed from source code:
```sh
cd harbor
docker-compose up --build -d
```

14
make/common/templates/nginx/nginx.https.conf
View File

@ -8,11 +8,11 @@ events {
http {
tcp_nodelay on;
include /etc/nginx/conf.d/*.conf;
# this is necessary for us to be able to disable request buffering in all cases
proxy_http_version 1.1;
upstream registry {
server registry:5000;
}
@ -62,6 +62,18 @@ http {
return 404;
}
location ~ ^/v2/(.*)/_trust/(.*) {
proxy_pass http://notary-server/v2/$$1/_trust/$$2;
proxy_set_header Host $$http_host;
proxy_set_header X-Real-IP $$remote_addr;
proxy_set_header X-Forwarded-For $$proxy_add_x_forwarded_for;
# When setting up Harbor behind other proxy, such as an Nginx instance, remove the below line if the proxy already has similar settings.
proxy_set_header X-Forwarded-Proto $$scheme;
proxy_buffering off;
proxy_request_buffering off;
}
location /v2/ {
proxy_pass http://registry/v2/;
proxy_set_header Host $$http_host;

4
make/common/templates/nginx/nginx.notary.conf Normal file
View File

@ -0,0 +1,4 @@
upstream notary-server {
server notary-server:4443;
}

7
make/common/templates/notary/mysql-initdb.d/initial-notaryserver.sql Normal file
View File

@ -0,0 +1,7 @@
CREATE DATABASE IF NOT EXISTS `notaryserver`;
CREATE USER "server"@"%" IDENTIFIED BY "";
GRANT
ALL PRIVILEGES ON `notaryserver`.*
TO "server"@"%";

7
make/common/templates/notary/mysql-initdb.d/initial-notarysigner.sql Normal file
View File

@ -0,0 +1,7 @@
CREATE DATABASE IF NOT EXISTS `notarysigner`;
CREATE USER "signer"@"%" IDENTIFIED BY "";
GRANT
ALL PRIVILEGES ON `notarysigner`.*
TO "signer"@"%";

210
make/common/templates/notary/mysql-initdb.d/notary.sql Normal file
View File

@ -0,0 +1,210 @@
-- MySQL dump 10.16 Distrib 10.1.10-MariaDB, for debian-linux-gnu (x86_64)
--
-- Host: localhost Database:
-- ------------------------------------------------------
-- Server version 10.1.10-MariaDB-1~jessie
/*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT */;
/*!40101 SET @OLD_CHARACTER_SET_RESULTS=@@CHARACTER_SET_RESULTS */;
/*!40101 SET @OLD_COLLATION_CONNECTION=@@COLLATION_CONNECTION */;
/*!40101 SET NAMES utf8 */;
/*!40103 SET @OLD_TIME_ZONE=@@TIME_ZONE */;
/*!40103 SET TIME_ZONE='+00:00' */;
/*!40014 SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0 */;
/*!40014 SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0 */;
/*!40101 SET @OLD_SQL_MODE=@@SQL_MODE, SQL_MODE='NO_AUTO_VALUE_ON_ZERO' */;
/*!40111 SET @OLD_SQL_NOTES=@@SQL_NOTES, SQL_NOTES=0 */;
--
-- Current Database: `notaryserver`
--
CREATE DATABASE /*!32312 IF NOT EXISTS*/ `notaryserver` /*!40100 DEFAULT CHARACTER SET latin1 */;
USE `notaryserver`;
--
-- Table structure for table `change_category`
--
DROP TABLE IF EXISTS `change_category`;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE `change_category` (
`category` varchar(20) NOT NULL,
PRIMARY KEY (`category`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
/*!40101 SET character_set_client = @saved_cs_client */;
--
-- Dumping data for table `change_category`
--
LOCK TABLES `change_category` WRITE;
/*!40000 ALTER TABLE `change_category` DISABLE KEYS */;
INSERT INTO `change_category` VALUES ('deletion'),('update');
/*!40000 ALTER TABLE `change_category` ENABLE KEYS */;
UNLOCK TABLES;
--
-- Table structure for table `changefeed`
--
DROP TABLE IF EXISTS `changefeed`;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE `changefeed` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`gun` varchar(255) NOT NULL,
`version` int(11) NOT NULL,
`sha256` char(64) DEFAULT NULL,
`category` varchar(20) NOT NULL DEFAULT 'update',
PRIMARY KEY (`id`),
KEY `category` (`category`),
KEY `idx_changefeed_gun` (`gun`),
CONSTRAINT `changefeed_ibfk_1` FOREIGN KEY (`category`) REFERENCES `change_category` (`category`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
/*!40101 SET character_set_client = @saved_cs_client */;
--
-- Dumping data for table `changefeed`
--
LOCK TABLES `changefeed` WRITE;
/*!40000 ALTER TABLE `changefeed` DISABLE KEYS */;
/*!40000 ALTER TABLE `changefeed` ENABLE KEYS */;
UNLOCK TABLES;
--
-- Table structure for table `schema_migrations`
--
DROP TABLE IF EXISTS `schema_migrations`;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE `schema_migrations` (
`version` int(11) NOT NULL,
PRIMARY KEY (`version`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
/*!40101 SET character_set_client = @saved_cs_client */;
--
-- Dumping data for table `schema_migrations`
--
LOCK TABLES `schema_migrations` WRITE;
/*!40000 ALTER TABLE `schema_migrations` DISABLE KEYS */;
INSERT INTO `schema_migrations` VALUES (1),(2),(3),(4),(5);
/*!40000 ALTER TABLE `schema_migrations` ENABLE KEYS */;
UNLOCK TABLES;
--
-- Table structure for table `tuf_files`
--
DROP TABLE IF EXISTS `tuf_files`;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE `tuf_files` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`created_at` timestamp NULL DEFAULT NULL,
`updated_at` timestamp NULL DEFAULT NULL,
`deleted_at` timestamp NULL DEFAULT NULL,
`gun` varchar(255) NOT NULL,
`role` varchar(255) NOT NULL,
`version` int(11) NOT NULL,
`data` longblob NOT NULL,
`sha256` char(64) DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `gun` (`gun`,`role`,`version`),
KEY `sha256` (`sha256`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
/*!40101 SET character_set_client = @saved_cs_client */;
--
-- Dumping data for table `tuf_files`
--
LOCK TABLES `tuf_files` WRITE;
/*!40000 ALTER TABLE `tuf_files` DISABLE KEYS */;
/*!40000 ALTER TABLE `tuf_files` ENABLE KEYS */;
UNLOCK TABLES;
--
-- Current Database: `notarysigner`
--
CREATE DATABASE /*!32312 IF NOT EXISTS*/ `notarysigner` /*!40100 DEFAULT CHARACTER SET latin1 */;
USE `notarysigner`;
--
-- Table structure for table `private_keys`
--
DROP TABLE IF EXISTS `private_keys`;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE `private_keys` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`created_at` timestamp NULL DEFAULT NULL,
`updated_at` timestamp NULL DEFAULT NULL,
`deleted_at` timestamp NULL DEFAULT NULL,
`key_id` varchar(255) NOT NULL,
`encryption_alg` varchar(255) NOT NULL,
`keywrap_alg` varchar(255) NOT NULL,
`algorithm` varchar(50) NOT NULL,
`passphrase_alias` varchar(50) NOT NULL,
`public` blob NOT NULL,
`private` blob NOT NULL,
`gun` varchar(255) NOT NULL,
`role` varchar(255) NOT NULL,
`last_used` datetime DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `key_id` (`key_id`),
UNIQUE KEY `key_id_2` (`key_id`,`algorithm`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
/*!40101 SET character_set_client = @saved_cs_client */;
--
-- Dumping data for table `private_keys`
--
LOCK TABLES `private_keys` WRITE;
/*!40000 ALTER TABLE `private_keys` DISABLE KEYS */;
/*!40000 ALTER TABLE `private_keys` ENABLE KEYS */;
UNLOCK TABLES;
--
-- Table structure for table `schema_migrations`
--
DROP TABLE IF EXISTS `schema_migrations`;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE `schema_migrations` (
`version` int(11) NOT NULL,
PRIMARY KEY (`version`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
/*!40101 SET character_set_client = @saved_cs_client */;
--
-- Dumping data for table `schema_migrations`
--
LOCK TABLES `schema_migrations` WRITE;
/*!40000 ALTER TABLE `schema_migrations` DISABLE KEYS */;
INSERT INTO `schema_migrations` VALUES (1),(2);
/*!40000 ALTER TABLE `schema_migrations` ENABLE KEYS */;
UNLOCK TABLES;
/*!40103 SET TIME_ZONE=@OLD_TIME_ZONE */;
/*!40101 SET SQL_MODE=@OLD_SQL_MODE */;
/*!40014 SET FOREIGN_KEY_CHECKS=@OLD_FOREIGN_KEY_CHECKS */;
/*!40014 SET UNIQUE_CHECKS=@OLD_UNIQUE_CHECKS */;
/*!40101 SET CHARACTER_SET_CLIENT=@OLD_CHARACTER_SET_CLIENT */;
/*!40101 SET CHARACTER_SET_RESULTS=@OLD_CHARACTER_SET_RESULTS */;
/*!40101 SET COLLATION_CONNECTION=@OLD_COLLATION_CONNECTION */;
/*!40111 SET SQL_NOTES=@OLD_SQL_NOTES */;
-- Dump completed on 2017-02-14 6:32:48

63
make/common/templates/notary/notary-signer.crt Normal file
View File

@ -0,0 +1,63 @@
-----BEGIN CERTIFICATE-----
MIIFBDCCAuygAwIBAgIJAMbWdVJcKhXYMA0GCSqGSIb3DQEBCwUAMGwxCzAJBgNV
BAYTAlVTMQswCQYDVQQIDAJDQTEWMBQGA1UEBwwNU2FuIEZyYW5jaXNjbzEPMA0G
A1UECgwGRG9ja2VyMScwJQYDVQQDDB5Ob3RhcnkgSW50ZXJtZWRpYXRlIFRlc3Rp
bmcgQ0EwHhcNMTcwMTIzMDYwMzM3WhcNMTkwMjEyMDYwMzM3WjBbMQswCQYDVQQG
EwJVUzELMAkGA1UECAwCQ0ExFjAUBgNVBAcMDVNhbiBGcmFuY2lzY28xDzANBgNV
BAoMBkRvY2tlcjEWMBQGA1UEAwwNbm90YXJ5LXNpZ25lcjCCASIwDQYJKoZIhvcN
AQEBBQADggEPADCCAQoCggEBANhO8+K9xT6M9dQC90Hxs6bmTXWQzE5oV2kLeVKq
OjwAvGt6wBE2XJCAbTS3FORIOyoOVQDVCv2Pk2lZXGWqSrH8SY2umjRJIhPDiqN9
V5M/gcmMm2EUgwmp2l4bsDk1MQ6GSbud5kjYGZcp9uXxAVO8tfLVLQF7ohJYqiex
JN+fZkQyxTgSqrI7MKK1pUvGX/fa6EXzpKwxTQPJXiG/ZQW0Pn+gdrz+/Cf0PcVy
V/Ghc2RR+WjKzqqAiDUJoEtKm/xQVRcSPbagVLCe0KZr7VmtDWnHsUv9ZB9BRNlI
lRVDOhVDCCcMu/zEtcxuH8ja7fafi5xNt6vCBmHuCXQtTUsCAwEAAaOBuTCBtjAf
BgNVHSMEGDAWgBQjgpNYJjU9Ei7nadpOhHm59FPiKTAMBgNVHRMBAf8EAjAAMB0G
A1UdJQQWMBQGCCsGAQUFBwMBBggrBgEFBQcDAjAOBgNVHQ8BAf8EBAMCBaAwNwYD
VR0RBDAwLoINbm90YXJ5LXNpZ25lcoIMbm90YXJ5c2lnbmVygglsb2NhbGhvc3SH
BAp1BI4wHQYDVR0OBBYEFLv4/22eN7pe8IzCbL+gKr2i/o6VMA0GCSqGSIb3DQEB
CwUAA4ICAQBzBcFgcwtr7oNP7WPyG64mRXHFs1qGCoDZO3D2dZPF/vUKnyPWI6+i
Ozu1Lmvd6QUQ5C0m91D6RidKKy3ENz2MgUo8NNj3QY3XzassiLnNOtpo1ed6U3BG
2w05gaLTTFywnpOgPy180U6f5uNSHGxY/fq9dN+8YR/MqGOht74q36x0swkPegG/
+0SLloKOJw1wBzZ4nCLmED08DWNnuNTAj5IIVjApzqZbTh4+z6H1lmN3b7XwmiWw
+y7Jx8k74h5JmqKQnV+3lN0DlCc1BCbtH2fbKOmAKeu4gMniw5FBo75wYrPIet+Z
E3G2Zg+T6fjTXAnLGT3S0RVn/CW1lLR6RgkoFgURRZoJyTWrg+1yu4ZOgEz+bot2
/hMAr/fjo+Dd6ReFrgGkpTyWYtPhYusori1W8KW138CVrJmSs6p2ss1Ixh8uIOaQ
iFmlX/ZXXbvkz3FGQS9LfBdESO3MGjiJTcnXE0DTnXf6RmdlUfNwxsZbIliFa0TQ
E/JjIJYQzWmtkJbUdC02GUMjUJAM7SxmP7tU9CmMmjUI28Nno0XtPN2WsAszaiLh
JYLJCi7rqaLo0oZuaXVIrgBpQ0qEC1XXS5sCQL+xvMSYvke/rhwIPItWt7Ww/9yj
QDIi1nzzX86lbKd095pNX4sUfFx6j4caR8iENgJDfWnqynAzj1Y21A==
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIF1TCCA72gAwIBAgIJAMk2DFRLRSRRMA0GCSqGSIb3DQEBCwUAMF8xCzAJBgNV
BAYTAlVTMQswCQYDVQQIDAJDQTEWMBQGA1UEBwwNU2FuIEZyYW5jaXNjbzEPMA0G
A1UECgwGRG9ja2VyMRowGAYDVQQDDBFOb3RhcnkgVGVzdGluZyBDQTAeFw0xNzAx
MjMwNjAzMzdaFw0yNzAxMjEwNjAzMzdaMGwxCzAJBgNVBAYTAlVTMQswCQYDVQQI
DAJDQTEWMBQGA1UEBwwNU2FuIEZyYW5jaXNjbzEPMA0GA1UECgwGRG9ja2VyMScw
JQYDVQQDDB5Ob3RhcnkgSW50ZXJtZWRpYXRlIFRlc3RpbmcgQ0EwggIiMA0GCSqG
SIb3DQEBAQUAA4ICDwAwggIKAoICAQCu+ldASegXuhXrA7mnk4nybTEomHnV8zJ/
uU6+8bWIo+htD8zgiONuk1uEww0p/nWtIZqm7xpLsklMp0CWRA8EAeUnxfNJ37ks
7nZuJ+YDtw77fC0IUJSWqFbro75nPMyegMqajT7IDWfLeTrIlgUmDu/45AWdbE2w
BrRgejqkL1yeQPaldgr97g00swbTd7wzWn1o6025Frm0kDEIqMJlkB61cHiVGZNu
oeDBZcFiwa/Ek/keDG3Y2R6cDQzZa8aEZG9i3Cmo0nGviojr+06JxQ8IkVc5P72e
Fb/jgX/NvRaqeBnJrZoiPnuMoMag/ynGC9fuIAGz25fKOuGOf52x+swzQB2ZVtxA
BIgIZIbMTURKknqbl6LAh46onQUVF+3h9E9Te3a4Oh7SvSGLYfEbWprPKo1J3lI9
ApU19TBhKUrj7dsJT3gri7f71NC2RLraZbpK3d8PWKMc/q4ffoRCeW+TPjYreC/d
7LdykAwYB2AGyHCLHkkkJC86n6wAsk/TaoTgjflyyQ35FNikUYqNF/rVuc+0Oj5R
odPk8y2vB7VvPvWWlttcr7OMqVVAymQvDOTb+5T6EI/LdHejjDMMI5lt6rVUU+uq
kGMYGiHtWG5JqQdhUBpISYuF74cS5aVRmnhK6O2ylMpmlWYq4128SRv8EEAPNcN9
V/RrOF9RsQIDAQABo4GGMIGDMB8GA1UdIwQYMBaAFJZZtwJ5t4SBmVaTb+T5puH5
sQWkMBIGA1UdEwEB/wQIMAYBAf8CAQAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsG
AQUFBwMCMA4GA1UdDwEB/wQEAwIBRjAdBgNVHQ4EFgQUI4KTWCY1PRIu52naToR5
ufRT4ikwDQYJKoZIhvcNAQELBQADggIBAI64zW1o24R8K7qsE8FO3UHJQdizR1RC
FvMDgXGDSYMUg4QkEvHYYOoFH1zMd1HNUuLDO231dtw23kshNY/kdKfdFJktT3Dz
50r/hl2090uZIOk9aLv7swG0voA6A8CI2qyXEXW9Le8xrnrJUU5T+3YDxseHokTT
XT9hLd1iSNH5gi3tOaJ4KNbHc2zhKtQSUZbxguapUIUXStiQLz06itQu3i1fLdMd
L3yRJID4aWU+Dmm5AQ6F3ticIpzFmJyAsTM2BMiTnlSJPK3LA2WYMBOVD6r9yo08
cEpi6Vo8pZdsnHWaIaIkO4UR7iBwmkT0h8HfNZ4uEoViiMsxqNVsQBfJR/9DzAXz
ctO6JtNJdNwn2zlv4NCIcV0AdncVf049uOtTBWIqRn1IHQ8d119lQAMXZZMSNKBI
lIYFCKMh95XI6mK6VFsFKs2wSDiSH4ZOqIwr4urmr1opLNJ5T5Ck18YwJafgCH4p
1BcgR06wuw5ckIuUyUwiakiGINZcrzUnAoRtEKsVi/PQAC+45veo8Lcvwnj5X0vg
PKudwiJivo7Umvj1xEVyVIy+22cyDk/yLTVI0sS2Kpuwd+PLE16C5+nPr8wKEWqL
ccotlod4ZDVb6vNU5VRUSu4bSYBry/FbftPNgAwfH8ufSddeJMjTQ+V69XrQZ5Ex
XJCKYD/1jYIB
-----END CERTIFICATE-----

28
make/common/templates/notary/notary-signer.key Normal file
View File

@ -0,0 +1,28 @@
-----BEGIN RSA PRIVATE KEY-----
MIIEowIBAAKCAQEA2E7z4r3FPoz11AL3QfGzpuZNdZDMTmhXaQt5Uqo6PAC8a3rA
ETZckIBtNLcU5Eg7Kg5VANUK/Y+TaVlcZapKsfxJja6aNEkiE8OKo31Xkz+ByYyb
YRSDCanaXhuwOTUxDoZJu53mSNgZlyn25fEBU7y18tUtAXuiEliqJ7Ek359mRDLF
OBKqsjsworWlS8Zf99roRfOkrDFNA8leIb9lBbQ+f6B2vP78J/Q9xXJX8aFzZFH5
aMrOqoCINQmgS0qb/FBVFxI9tqBUsJ7QpmvtWa0NacexS/1kH0FE2UiVFUM6FUMI
Jwy7/MS1zG4fyNrt9p+LnE23q8IGYe4JdC1NSwIDAQABAoIBAHykYhyRxYrZpv3Y
B6pUIHVX1+Ka4V98+IFrPynHNW9F7UzxmqNQc95AYq0xojQ4+v6s64ZjPMYHaaYW
/AsJKamN+sRNjEX8rko9LzIuE7yhp6QABbjXHPsAiPgZdF5CrFX2Q558yinHfFeC
sualDWK3JxEajaiBGU8BEGt2xAymuWACGblrM1aAEZa8B84TW3CzzcdyzAkn8P3e
piJCe+DWMc33441r0KlV5GruwF9ewXiWzZtXAOiP/0xEDICFdlFWbO39myMpxDdU
Y0uZ+zmn2G3gz2tz25thH0Wl7mDQ3AA0VlHurgPBBEekeZPQmjiKW+F4slCzXvuy
kW/urIECgYEA/LhY+OWlZVXzIEly7z1/cU9/WImqTs2uRKDeQHMwZrd7D9BXkJuQ
jPN+jZlMYBBrxoaCywbMrgB80Z3MgGHaSx9OIDEZmaxyuQv0zQJCMogysYkbCcaD
mHYnyAf7OXa708Z168WAisEhrwa/DXBn3/hPoBkrbMsuPF/J+tEP7lsCgYEA2x2g
86SitgPVeNV3iuZ6D/SV0QIbDWOYoST2GQn2LnfALIOrzpXRClOSQZ2pGtg9gYo1
owUyyOSv2Fke93p3ufHv3Gqvjl55lzBVV0siHkEXwHcol36DDGQcskVnXJqaL3IF
tiOisuJS9A7PW7gEi0miyGzzB/kh/IEWHKqLL9ECgYEAoBOFB+MuqMmQftsHWlLx
7qwUVdidb90IjZ/4J4rPFcESyimFzas8HIv/lWGM5yx/l/iL0F42N+FHLt9tMcTJ
qNvjeLChLp307RGNtm2/0JJEyf+2iLKdmGz/Nc0YbIWw46vJ9dXcXgeHdn4ndjPF
GDEI/rfysa7hUoy6O41BMhECgYBPJsLPgHdufLAOeD44pM0PGnFMERCoo4OtImbr
4JdXbdazvdTASYo7yriYj1VY5yhAtSZu/x+7RjDnXDo9d7XsK6NT4g4Mxb/yh3ks
kW1/tE/aLLEzGHZKcZeUJlISN57e6Ld7dh/9spf4pajuHuk1T6JH+GNKTAqk5hSQ
wmKJIQKBgCGBWGvJrCeT5X9oHdrlHj2YoKvIIG1eibagcjcKemD7sWzi7Q4P7JIo
xeX8K1WVxdBpo4/RiQcGFmwSmSUKwwr1dO00xtjxIl7ip4DU+WAM7CdmcOIOMbr4
rP9T/wy1ZBkERCIw2ElybTzB8yuOlNLuOMhUeU55xUMFNYYrWEp2
-----END RSA PRIVATE KEY-----

32
make/common/templates/notary/root-ca.crt Normal file
View File

@ -0,0 +1,32 @@
-----BEGIN CERTIFICATE-----
MIIFhjCCA26gAwIBAgIJALJdsE+BUxypMA0GCSqGSIb3DQEBCwUAMF8xCzAJBgNV
BAYTAlVTMQswCQYDVQQIDAJDQTEWMBQGA1UEBwwNU2FuIEZyYW5jaXNjbzEPMA0G
A1UECgwGRG9ja2VyMRowGAYDVQQDDBFOb3RhcnkgVGVzdGluZyBDQTAeFw0xNzAx
MjMwNjAzMzZaFw0yNzAxMjEwNjAzMzZaMF8xCzAJBgNVBAYTAlVTMQswCQYDVQQI
DAJDQTEWMBQGA1UEBwwNU2FuIEZyYW5jaXNjbzEPMA0GA1UECgwGRG9ja2VyMRow
GAYDVQQDDBFOb3RhcnkgVGVzdGluZyBDQTCCAiIwDQYJKoZIhvcNAQEBBQADggIP
ADCCAgoCggIBALIZNBcIoQDJql5w+XULXq9W3tmD47xnf+IG4u7hkDVPCT4xRG74
LBoSuFyPUrfT+tsibMlNG6XRtSfLQdNNeQuyIuiilNXV0kXB0RR3TrhxCaKdhRU5
oQGfpYMvbPNFB7WU/5aAiQutHH85hEMPECf1qPjq8YlUaXJLGFY3WRkW+OOBZ78U
00PqKlvC1kR/NbsV3IkMrO+vWWJQrPFusyYjQ511eQXnRtt8P0Qic0azPffQDVxC
WUe47hmdQ1AULbxQ9AZcPlMI7UFqo+/w/4hPEGJMeOWirLvHLXg4nsOwy7DfWl/n
MqLdJOC/KNfQVAQtkteeZZkkIIV1gxTPYsJqPNwkP9GdJK1A8NW1ef75v7xbQCPY
03QQonBEK7ny7b1xXGGgJzXvK9RP0UUwjt/815c4d0cgUHsy4yuvl2F44EObRshk
fjJVsN/0wrtq4QLE5ZvbeO+7to8dLcRxkmB8axhxahega7akUyY0WxZ+iSn6fzft
/xeCcs/L10V5z0kK4PbiNnooDzV4B6Dy/5oyNExw0jgpD0mzOK5aLb0tXGqFT/ZJ
9vydelBq5q4jLV7SHhHM1dBJSv1fl7vOpDlEr7LBd4YAO2BowoyGLHtLhgYybXF+
CZ9ywPb1dIIcdK5IVeZECNHMSBuhCRZUu+aun8tRcdSgLEX7mQ/GKWELAgMBAAGj
RTBDMBIGA1UdEwEB/wQIMAYBAf8CAQEwDgYDVR0PAQH/BAQDAgFGMB0GA1UdDgQW
BBSWWbcCebeEgZlWk2/k+abh+bEFpDANBgkqhkiG9w0BAQsFAAOCAgEAQ9gA3Q4b
r2+ZJdIDoDzCNdtHQbb/d1NiUP/Na1MFo7omR3MnKGXy3dIp9IrQq6ROhlqUhDvl
pZegYhTbunTVv1KKJ+5n1hY6pG/Jr8oLY3b9i4qwDLKfQGm5PmrfwAtqbLSfY2M0
2AZyAhCdGbqB7WpTdG1J7DzGbVVWAtS05e24Mu0qZJvpHdtl4+t89vXgJ/bPrPxF
cpAlT9DOtobTEqrXZeS937F1qNyIgyBki+7mtxkwng5cf3zQM2BJ9lSFQJOBSRDr
haMcnaPI4pknO7OfYf5W9LaS1Dx/U/NeMBfnVBd9NjUw+TMjy2MdMLUaLa9EF7Jo
Gjk+fKaTaUgO8I487wHPMeoEA4A4dEePzGrybRLfl1ZYGQ0xcgunz64n2xfQIy2y
swiyaofYlLxzHzOL0N+Y76P0ic37t9R2F5ggNhfbXhClK2h4HmdjRRRt3VkxR4AD
7OM09bEhlZby34HOlCaC0PHKwYBMjneAG3ycPN88YTMYR2/KizExe71ayNwX2KHL
ib1nOZgZT6s+YvgsZ7lRmMD4iqjuAEh5SRAcWlolVif8bAy09BkY1vwrtgV73q88
heEbsCE1fsfk1OfH5W4yjjiSDZFRt5oTCPQWJp+2P0RJ9LCxcbf0RrCg3hg5rD9N
lVTA0dsixv5zF3wTuad9inhk9Rmlq1KoaqA=
-----END CERTIFICATE-----

28
make/common/templates/notary/server-config.json Normal file
View File

@ -0,0 +1,28 @@
{
"server": {
"http_addr": ":4443"
},
"trust_service": {
"type": "remote",
"hostname": "notarysigner",
"port": "7899",
"tls_ca_file": "./root-ca.crt",
"key_algorithm": "ecdsa"
},
"logging": {
"level": "debug"
},
"storage": {
"backend": "mysql",
"db_url": "server@tcp(mysql:3306)/notaryserver?parseTime=True"
},
"auth": {
"type": "token",
"options": {
"realm": "$token_endpoint/service/token",
"service": "token-service",
"issuer": "registry-token-issuer",
"rootcertbundle": "/config/root.crt"
}
}
}

15
make/common/templates/notary/signer-config.json Normal file
View File

@ -0,0 +1,15 @@
{
"server": {
"grpc_addr": ":7899",
"tls_cert_file": "./notary-signer.crt",
"tls_key_file": "./notary-signer.key"
},
"logging": {
"level": "debug"
},
"storage": {
"backend": "mysql",
"db_url": "signer@tcp(mysql:3306)/notarysigner?parseTime=True",
"default_alias":"defaultalias"
}
}

2
make/common/templates/notary/signer_env Normal file
View File

@ -0,0 +1,2 @@
NOTARY_SIGNER_DEFAULTALIAS=$alias

61
make/docker-compose.notary.yml Normal file
View File

@ -0,0 +1,61 @@
version: '2'
services:
ui:
networks:
- harbor-notary
proxy:
networks:
- harbor-notary
notary-server:
image: jiangd/notary:server-0.5.0-fix
container_name: notary-server
networks:
- notary-mdb
- notary-sig
- harbor-notary
volumes:
- ./common/config/notary:/config
entrypoint: /usr/bin/env sh
command: -c "notary-server -config=/config/server-config.json -logf=logfmt"
depends_on:
- notary-db
- notary-signer
notary-signer:
image: notary:signer-0.5.0
container_name: notary-signer
networks:
notary-mdb:
notary-sig:
aliases:
- notarysigner
volumes:
- ./common/config/notary:/config
env_file:
- ./common/config/notary/signer_env
entrypoint: /usr/bin/env sh
command: -c "notary-signer -config=/config/signer-config.json -logf=logfmt"
depends_on:
- notary-db
notary-db:
image: mariadb:10.1.10
container_name: notary-db
networks:
notary-mdb:
aliases:
- mysql
volumes:
- ./common/config/notary/mysql-initdb.d:/docker-entrypoint-initdb.d
- /data/notary-db:/var/lib/mysql
ports:
- "3306:3306"
environment:
- TERM=dumb
- MYSQL_ALLOW_EMPTY_PASSWORD="true"
command: mysqld --innodb_file_per_table
networks:
harbor-notary:
external: false
notary-mdb:
external: false
notary-sig:
external: false

16
make/docker-compose.tpl
View File

@ -8,6 +8,8 @@ services:
- /var/log/harbor/:/var/log/docker/
ports:
- 1514:514
networks:
- harbor
registry:
image: library/registry:2.5.1
container_name: registry
@ -15,6 +17,8 @@ services:
volumes:
- /data/registry:/storage
- ./common/config/registry/:/etc/registry/
networks:
- harbor
environment:
- GODEBUG=netdns=cgo
command:
@ -32,6 +36,8 @@ services:
restart: always
volumes:
- /data/database:/var/lib/mysql
networks:
- harbor
env_file:
- ./common/config/db/env
depends_on:
@ -68,6 +74,8 @@ services:
- ./common/config/ui/private_key.pem:/etc/ui/private_key.pem
- /data:/harbor_storage
- /data/secretkey:/etc/ui/key
networks:
- harbor
depends_on:
- log
- adminserver
@ -87,6 +95,8 @@ services:
- /data/job_logs:/var/log/jobs
- ./common/config/jobservice/app.conf:/etc/jobservice/app.conf
- /data/secretkey:/etc/jobservice/key
networks:
- harbor
depends_on:
- ui
- adminserver
@ -101,6 +111,8 @@ services:
restart: always
volumes:
- ./common/config/nginx:/etc/nginx
networks:
- harbor
ports:
- 80:80
- 443:443
@ -114,3 +126,7 @@ services:
options:
syslog-address: "tcp://127.0.0.1:1514"
tag: "proxy"
networks:
harbor:
external: false

3
make/kubernetes/dockerfiles/proxy-dockerfile
View File

@ -1,3 +0,0 @@
FROM library/nginx:1.11.5
ADD ./config/nginx /etc/nginx

33
make/kubernetes/dockerfiles/registry-config.yml
View File

@ -1,33 +0,0 @@
version: 0.1
log:
level: debug
fields:
service: registry
storage:
cache:
layerinfo: inmemory
filesystem:
rootdirectory: /storage
maintenance:
uploadpurging:
enabled: false
http:
addr: :5000
secret: placeholder
debug:
addr: localhost:5001
auth:
token:
issuer: registry-token-issuer
realm: http://harbor.caicloud.io/service/token
rootcertbundle: /etc/registry/root.crt
service: token-service
notifications:
endpoints:
- name: harbor
disabled: false
url: http://harbor.caicloud.io/service/notifications
timeout: 500
threshold: 5
backoff: 1000

6
make/kubernetes/dockerfiles/registry-dockerfile
View File

@ -1,6 +0,0 @@
FROM library/registry:2.3.0
ADD ./config/registry/ /etc/registry/
ADD ./kubernetes/dockerfiles/registry-config.yml /etc/registry/config.yml
CMD ["/etc/registry/config.yml"]

4
make/kubernetes/dockerfiles/ui-dockerfile
View File

@ -1,4 +0,0 @@
FROM deploy_ui
ADD ./config/ui/app.conf /etc/ui/app.conf
ADD ./config/ui/private_key.pem /etc/ui/private_key.pem

112
make/kubernetes/jobservice/jobservice.rc.yaml Normal file
View File

@ -0,0 +1,112 @@
apiVersion: v1
kind: ReplicationController
metadata:
name: jobservice-rc
labels:
name: jobservice-rc
spec:
replicas: 1
selector:
name: jobservice-apps
template:
metadata:
labels:
name: jobservice-apps
spec:
containers:
- name: jobservice-app
image: harbor/jobservice
imagePullPolicy: IfNotPresent
env:
- name: MYSQL_HOST
valueFrom:
configMapKeyRef:
name: harbor-jobservice-config
key: MYSQL_HOST
- name: MYSQL_PORT
valueFrom:
configMapKeyRef:
name: harbor-jobservice-config
key: MYSQL_PORT
- name: MYSQL_USR
valueFrom:
configMapKeyRef:
name: harbor-jobservice-config
key: MYSQL_USR
- name: MYSQL_PWD
valueFrom:
configMapKeyRef:
name: harbor-jobservice-config
key: MYSQL_PWD
- name: UI_SECRET
valueFrom:
configMapKeyRef:
name: harbor-jobservice-config
key: UI_SECRET
- name: SECRET_KEY
valueFrom:
configMapKeyRef:
name: harbor-jobservice-config
key: SECRET_KEY
- name: CONFIG_PATH
valueFrom:
configMapKeyRef:
name: harbor-jobservice-config
key: CONFIG_PATH
- name: REGISTRY_URL
valueFrom:
configMapKeyRef:
name: harbor-jobservice-config
key: REGISTRY_URL
- name: VERIFY_REMOTE_CERT
valueFrom:
configMapKeyRef:
name: harbor-jobservice-config
key: VERIFY_REMOTE_CERT
- name: MAX_JOB_WORKERS
valueFrom:
configMapKeyRef:
name: harbor-jobservice-config
key: MAX_JOB_WORKERS
- name: LOG_LEVEL
valueFrom:
configMapKeyRef:
name: harbor-jobservice-config
key: LOG_LEVEL
- name: LOG_DIR
valueFrom:
configMapKeyRef:
name: harbor-jobservice-config
key: LOG_DIR
- name: GODEBUG
valueFrom:
configMapKeyRef:
name: harbor-jobservice-config
key: GODEBUG
- name: EXT_ENDPOINT
valueFrom:
configMapKeyRef:
name: harbor-jobservice-config
key: EXT_ENDPOINT
- name: TOKEN_URL
valueFrom:
configMapKeyRef:
name: harbor-jobservice-config
key: TOKEN_URL
ports:
- containerPort: 80
volumeMounts:
- name: config
mountPath: /etc/jobservice
- name: logs
mountPath: /var/log/jobs
volumes:
- name: config
configMap:
name: harbor-jobservice-config
items:
- key: config
path: app.conf
- name: logs
persistentVolumeClaim:
claimName: log-pvc

9
make/kubernetes/jobservice/jobservice.svc.yaml Normal file
View File

@ -0,0 +1,9 @@
apiVersion: v1
kind: Service
metadata:
name: jobservice
spec:
ports:
- port: 80
selector:
name: jobservice-apps

22
make/kubernetes/mysql-rc.yaml → make/kubernetes/mysql/mysql.rc.yaml
View File

@ -1,30 +1,34 @@
apiVersion: v1
kind: ReplicationController
metadata:
name: mysql
name: mysql-rc
labels:
name: mysql
name: mysql-rc
spec:
replicas: 1
selector:
name: mysql
name: mysql-apps
template:
metadata:
labels:
name: mysql
name: mysql-apps
spec:
containers:
- name: mysql
image: caicloud/harbor_deploy_mysql:latest
imagePullPolicy: Always
- name: mysql-app
image: harbor/mysql
imagePullPolicy: IfNotPresent
ports:
- containerPort: 3306
env:
- name: MYSQL_ROOT_PASSWORD
value: root123
valueFrom:
configMapKeyRef:
name: harbor-mysql-config
key: MYSQL_ROOT_PASSWORD
volumeMounts:
- name: mysql-storage
mountPath: /var/lib/mysql
volumes:
- name: mysql-storage
emptyDir: {}
persistentVolumeClaim:
claimName: storage-pvc

4
make/kubernetes/mysql-svc.yaml → make/kubernetes/mysql/mysql.svc.yaml
View File

@ -2,10 +2,8 @@ apiVersion: v1
kind: Service
metadata:
name: mysql
labels:
name: mysql
spec:
ports:
- port: 3306
selector:
name: mysql
name: mysql-apps

36
make/kubernetes/nginx/nginx.rc.yaml Normal file
View File

@ -0,0 +1,36 @@
apiVersion: v1
kind: ReplicationController
metadata:
name: nginx-rc
labels:
name: nginx-rc
spec:
replicas: 1
selector:
name: nginx-apps
template:
metadata:
labels:
name: nginx-apps
spec:
containers:
- name: nginx-app
image: harbor/nginx
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
- containerPort: 443
volumeMounts:
- name: config
mountPath: /etc/nginx
volumes:
- name: config
configMap:
name: harbor-nginx-config
items:
- key: config
path: nginx.conf
- key: pkey
path: https.key
- key: cert
path: https.crt

7
make/kubernetes/proxy-svc.yaml → make/kubernetes/nginx/nginx.svc.yaml
View File

@ -1,15 +1,12 @@
apiVersion: v1
kind: Service
metadata:
name: proxy
labels:
name: proxy
name: nginx
spec:
type: LoadBalancer
ports:
- name: http
port: 80
- name: https
port: 443
selector:
name: proxy
name: nginx-apps

201
make/kubernetes/prepare Normal file
View File

@ -0,0 +1,201 @@
#!/usr/bin/env python
from __future__ import print_function, unicode_literals # We require Python 2.6 or later
import sys
import argparse
import io
import os
import random
import re
import string
import subprocess
if sys.version_info[:3][0] == 2:
import ConfigParser as configparser
import StringIO as io
if sys.version_info[:3][0] == 3:
import configparser as configparser
import io as io
# prepare base dir
base_dir = os.path.dirname(os.path.abspath(__file__))
parser = argparse.ArgumentParser(description='Generate *.cm.yaml')
parser.add_argument('-f', default=os.path.join(base_dir, '../harbor.cfg'),
dest='config_file', help='[Optional] path of harbor config file')
parser.add_argument('-k', default='',
dest='private_key', help='[Optional] path of harbor https private key(pem)')
parser.add_argument('-c', default='',
dest='cert', help='[Optional] harbor path of https cert(pem)')
parser.add_argument('-s', default='',
dest='secret_key', help="[Optional] path of harbor secret key(16 characters)")
args = parser.parse_args()
# read config file
config_str = ''
if os.path.isfile(args.config_file):
with open(args.config_file) as conf:
config_str = conf.read()
else:
raise Exception('Error: No such file(' + args.config_file + ')')
config_str = '[harbor]\n' + config_str
fp = io.StringIO()
fp.write(config_str)
fp.seek(0, os.SEEK_SET)
config = configparser.RawConfigParser()
config.readfp(fp)
def get_config(key):
"""get value by key
"""
if config.has_option('harbor', key):
return config.get('harbor', key)
print('Warning: Key(' + key + ') is not existing. Use empty string as default')
return ''
def set_config(key, value):
"""set key & value
"""
config.set('harbor', key, value)
# relative path with config file
def rel_path(p):
if p[0] == '/':
return p
config_path = args.config_file
if config_path[0] != '/':
config_path = os.path.join(os.getcwd(), config_path)
return os.path.join(os.path.dirname(config_path), p)
# path of private key
pk_path = args.private_key
if pk_path == '':
pk_path = get_config('ssl_cert_key')
if pk_path != '':
pk_path = rel_path(pk_path)
# path of cert
cert_path = args.cert
if cert_path == '':
cert_path = get_config('ssl_cert')
if cert_path != '':
cert_path = rel_path(cert_path)
# validate
if get_config('ui_url_protocol') == 'https':
if pk_path == '':
raise Exception("Error: The protocol is https but attribute ssl_cert_key is not set")
if cert_path == '':
raise Exception("Error: The protocol is https but attribute ssl_cert is not set")
else:
pk_path = ''
cert_path = ''
# read secret key
if args.secret_key != '':
if os.path.isfile(args.secret_key):
key = ''
with open(args.secret_key, 'r') as skey:
key = skey.read()
if len(key) != 16:
raise Exception('Error: The length of secret key has to be 16 characters!')
set_config('secret_key', key)
else:
set_config('secret_key', ''.join(random.choice(
string.ascii_letters + string.digits) for i in range(16)))
# read https pkey & cert
if pk_path != '':
if os.path.isfile(pk_path):
with open(pk_path, 'r') as pkey:
set_config('https_pkey', pkey.read())
else:
raise Exception('Error: https private key is not existing')
else:
set_config('https_pkey', 'USE_HTTP')
if cert_path != '':
if os.path.isfile(cert_path):
with open(cert_path, 'r') as cert:
set_config('https_cert', cert.read())
else:
raise Exception('Error: https cert is not existing')
else:
set_config('https_cert', 'USE_HTTP')
# add configs
set_config('ui_url', get_config('ui_url_protocol') +
'://' + get_config('hostname'))
set_config('ui_secret', ''.join(random.choice(
string.ascii_letters + string.digits) for i in range(16)))
# generate auth pkey & cert
with open(os.devnull, 'w') as devnull:
openssl = subprocess.call(['which','openssl'], stdout=devnull, stderr=devnull)
if openssl == 0:
pkey = subprocess.check_output(['openssl','genrsa','4096'], stderr=devnull)
subj = '/C={0}/ST={1}/L={2}/O={3}/OU={4}/CN={5}/emailAddress={6}'.format(get_config('crt_country'),
get_config('crt_state'), get_config('crt_location'), get_config('crt_organization'),
get_config('crt_organizationalunit'), get_config('crt_commonname'), get_config('crt_email'))
openssl = subprocess.Popen(['openssl', 'req', '-new', '-x509', '-key', '/dev/stdin', '-days', '3650', '-subj', subj],
stdout=subprocess.PIPE, stdin=subprocess.PIPE, stderr=devnull)
cert = openssl.communicate(input=pkey)[0]
set_config('auth_pkey', pkey.decode())
set_config('auth_cert', cert.decode())
else:
set_config('auth_pkey', 'NEED_SET')
set_config('auth_cert', 'NEED_SET')
print('Warning: auth_pkey and auth_cert cannot be generated automatically without openssl. Please set it manually')
variable = re.compile(r'{{.+?}}')
detail = re.compile(r'((\d+) )?([a-zA-Z_0-9-]+)')
def render_template(tmpl):
"""render template
replace {{(number of leading spaces)name}} with config
examples:
config:
hostname='test\ntest'
{{hostname}} -> 'test\ntest'
{{4 hostname}} -> 'test\n test'
"""
matches = variable.findall(tmpl)
for match in matches:
segs = detail.search(match)
if segs.group() == '':
raise Exception('Error: Invalid template item(' + match + ')')
value = get_config(segs.group(3))
spaces = segs.group(2)
if spaces != '' and spaces != None:
leading = ''.join(' ' for i in range(int(spaces)))
value = str(value).replace('\n', '\n' + leading)
tmpl = tmpl.replace(match, value)
return tmpl
def generate_template(tmpl, dest):
"""generate file
"""
with open(tmpl) as tmpl:
with open(dest, 'w') as dest:
dest.write(render_template(tmpl.read()))
template_dir = os.path.join(base_dir, 'templates')
output_dir = base_dir
generate_template(os.path.join(template_dir, 'ui.cm.yaml'), os.path.join(output_dir, 'ui/ui.cm.yaml'))
generate_template(os.path.join(template_dir, 'jobservice.cm.yaml'), os.path.join(output_dir, 'jobservice/jobservice.cm.yaml'))
generate_template(os.path.join(template_dir, 'mysql.cm.yaml'), os.path.join(output_dir, 'mysql/mysql.cm.yaml'))
generate_template(os.path.join(template_dir, 'nginx.cm.yaml'), os.path.join(output_dir, 'nginx/nginx.cm.yaml'))
generate_template(os.path.join(template_dir, 'registry.cm.yaml'), os.path.join(output_dir, 'registry/registry.cm.yaml'))

22
make/kubernetes/proxy-rc.yaml
View File

@ -1,22 +0,0 @@
apiVersion: v1
kind: ReplicationController
metadata:
name: proxy
labels:
name: proxy
spec:
replicas: 1
selector:
name: proxy
template:
metadata:
labels:
name: proxy
spec:
containers:
- name: proxy
image: caicloud/harbor_proxy:latest
imagePullPolicy: Always
ports:
- containerPort: 80
- containerPort: 443

14
make/kubernetes/pv/log.pv.yaml Normal file
View File

@ -0,0 +1,14 @@
apiVersion: v1
kind: PersistentVolume
metadata:
name: log-pv
labels:
type: log
spec:
capacity:
storage: 5Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
hostPath:
path: /data/logs

13
make/kubernetes/pv/log.pvc.yaml Normal file
View File

@ -0,0 +1,13 @@
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: log-pvc
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
selector:
matchLabels:
type: log

14
make/kubernetes/pv/registry.pv.yaml Normal file
View File

@ -0,0 +1,14 @@
apiVersion: v1
kind: PersistentVolume
metadata:
name: registry-pv
labels:
type: registry
spec:
capacity:
storage: 100Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
hostPath:
path: /data/registry

13
make/kubernetes/pv/registry.pvc.yaml Normal file
View File

@ -0,0 +1,13 @@
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: registry-pvc
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 100Gi
selector:
matchLabels:
type: registry

14
make/kubernetes/pv/storage.pv.yaml Normal file
View File

@ -0,0 +1,14 @@
apiVersion: v1
kind: PersistentVolume
metadata:
name: storage-pv
labels:
type: storage
spec:
capacity:
storage: 5Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
hostPath:
path: /data/storage

13
make/kubernetes/pv/storage.pvc.yaml Normal file
View File

@ -0,0 +1,13 @@
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: storage-pvc
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
selector:
matchLabels:
type: storage

28
make/kubernetes/registry-rc.yaml
View File

@ -1,28 +0,0 @@
apiVersion: v1
kind: ReplicationController
metadata:
name: registry
labels:
name: registry
spec:
replicas: 1
selector:
name: registry
template:
metadata:
labels:
name: registry
spec:
containers:
- name: registry
image: caicloud/harbor_registry:2.3.0
imagePullPolicy: Always
ports:
- containerPort: 5000
- containerPort: 5001
volumeMounts:
- name: storage
mountPath: /storage
volumes:
- name: storage
emptyDir: {}

39
make/kubernetes/registry/registry.rc.yaml Normal file
View File

@ -0,0 +1,39 @@
apiVersion: v1
kind: ReplicationController
metadata:
name: registry-rc
labels:
name: registry-rc
spec:
replicas: 1
selector:
name: registry-apps
template:
metadata:
labels:
name: registry-apps
spec:
containers:
- name: registry-app
image: harbor/registry
imagePullPolicy: IfNotPresent
ports:
- containerPort: 5000
- containerPort: 5001
volumeMounts:
- name: config
mountPath: /etc/docker/registry
- name: storage
mountPath: /storage
volumes:
- name: config
configMap:
name: harbor-registry-config
items:
- key: config
path: config.yml
- key: cert
path: root.crt
- name: storage
persistentVolumeClaim:
claimName: registry-pvc

8
make/kubernetes/registry-svc.yaml → make/kubernetes/registry/registry.svc.yaml
View File

@ -2,13 +2,11 @@ apiVersion: v1
kind: Service
metadata:
name: registry
labels:
name: registry
spec:
ports:
- name: internal
- name: repo
port: 5000
- name: external
- name: debug
port: 5001
selector:
name: registry
name: registry-apps

25
make/kubernetes/templates/jobservice.cm.yaml Normal file
View File

@ -0,0 +1,25 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: harbor-jobservice-config
data:
MYSQL_HOST: mysql
MYSQL_PORT: "3306"
MYSQL_USR: root
MYSQL_PWD: "{{db_password}}"
UI_SECRET: "{{ui_secret}}"
SECRET_KEY: "{{secret_key}}"
CONFIG_PATH: /etc/jobservice/app.conf
REGISTRY_URL: http://registry:5000
VERIFY_REMOTE_CERT: "{{verify_remote_cert}}"
MAX_JOB_WORKERS: "{{max_job_workers}}"
LOG_LEVEL: debug
LOG_DIR: /var/log/jobs
GODEBUG: netdns=cgo
EXT_ENDPOINT: "{{ui_url}}"
TOKEN_URL: http://ui
config: |
appname = jobservice
runmode = dev
[dev]
httpport = 80

6
make/kubernetes/templates/mysql.cm.yaml Normal file
View File

@ -0,0 +1,6 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: harbor-mysql-config
data:
MYSQL_ROOT_PASSWORD: "{{db_password}}"

155
make/kubernetes/templates/nginx.cm.yaml Normal file
View File

@ -0,0 +1,155 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: harbor-nginx-config
data:
config: |
worker_processes auto;
events {
worker_connections 1024;
use epoll;
multi_accept on;
}
http {
tcp_nodelay on;
# this is necessary for us to be able to disable request buffering in all cases
proxy_http_version 1.1;
upstream registry {
server registry:5000;
}
upstream ui {
server ui:80;
}
server {
listen 443 ssl;
server_name {{hostname}};
# SSL
ssl_certificate /etc/nginx/https.crt;
ssl_certificate_key /etc/nginx/https.key;
# Recommendations from https://raymii.org/s/tutorials/Strong_SSL_Security_On_nginx.html
ssl_protocols TLSv1.1 TLSv1.2;
ssl_ciphers 'EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH';
ssl_prefer_server_ciphers on;
ssl_session_cache shared:SSL:10m;
# disable any limits to avoid HTTP 413 for large image uploads
client_max_body_size 0;
# required to avoid HTTP 411: see Issue #1486 (https://github.com/docker/docker/issues/1486)
chunked_transfer_encoding on;
location / {
proxy_pass http://ui/;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# When setting up Harbor behind other proxy, such as an Nginx instance, remove the below line if the proxy already has similar settings.
proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering off;
proxy_request_buffering off;
}
location /v1/ {
return 404;
}
location /v2/ {
proxy_pass http://registry/v2/;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# When setting up Harbor behind other proxy, such as an Nginx instance, remove the below line if the proxy already has similar settings.
proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering off;
proxy_request_buffering off;
}
location /service/ {
proxy_pass http://ui/service/;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# When setting up Harbor behind other proxy, such as an Nginx instance, remove the below line if the proxy already has similar settings.
proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering off;
proxy_request_buffering off;
}
}
server {
listen 80;
server_name {{hostname}};
# disable any limits to avoid HTTP 413 for large image uploads
client_max_body_size 0;
# required to avoid HTTP 411: see Issue #1486 (https://github.com/docker/docker/issues/1486)
chunked_transfer_encoding on;
# rewrite ^/(.*) https://$server_name:443/$1 permanent;
location / {
proxy_pass http://ui/;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# When setting up Harbor behind other proxy, such as an Nginx instance, remove the below line if the proxy already has similar settings.
proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering off;
proxy_request_buffering off;
}
location /v1/ {
return 404;
}
location /v2/ {
proxy_pass http://registry/v2/;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# When setting up Harbor behind other proxy, such as an Nginx instance, remove the below line if the proxy already has similar settings.
proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering off;
proxy_request_buffering off;
}
location /service/ {
proxy_pass http://ui/service/;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# When setting up Harbor behind other proxy, such as an Nginx instance, remove the below line if the proxy already has similar settings.
proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering off;
proxy_request_buffering off;
}
}
}
pkey: |
{{4 https_pkey}}
cert: |
{{4 https_cert}}

44
make/kubernetes/templates/registry.cm.yaml Normal file
View File

@ -0,0 +1,44 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: harbor-registry-config
data:
config: |
version: 0.1
log:
level: debug
fields:
service: registry
storage:
filesystem:
rootdirectory: /storage
cache:
layerinfo: inmemory
maintenance:
uploadpurging:
enabled: false
delete:
enabled: true
http:
addr: :5000
secret: placeholder
debug:
addr: localhost:5001
auth:
token:
issuer: registry-token-issuer
realm: {{ui_url}}/service/token
rootcertbundle: /etc/docker/registry/root.crt
service: token-service
notifications:
endpoints:
- name: harbor
disabled: false
url: http://ui/service/notifications
timeout: 3000ms
threshold: 5
backoff: 1s
cert: |
{{4 auth_cert}}

52
make/kubernetes/templates/ui.cm.yaml Normal file
View File

@ -0,0 +1,52 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: harbor-ui-config
data:
MYSQL_HOST: mysql
MYSQL_PORT: "3306"
MYSQL_USR: root
MYSQL_PWD: "{{db_password}}"
REGISTRY_URL: http://registry:5000
CONFIG_PATH: /etc/ui/app.conf
HARBOR_REG_URL: "{{hostname}}"
HARBOR_ADMIN_PASSWORD: "{{harbor_admin_password}}"
HARBOR_URL: http://ui
AUTH_MODE: "{{auth_mode}}"
LDAP_URL: "{{ldap_url}}"
LDAP_SEARCH_DN: "{{ldap_searchdn}}"
LDAP_SEARCH_PWD: "{{ldap_search_pwd}}"
LDAP_BASE_DN: "{{ldap_basedn}}"
LDAP_FILTER: "{{ldap_filter}}"
LDAP_UID: "{{ldap_uid}}"
LDAP_SCOPE: "{{ldap_scope}}"
LOG_LEVEL: debug
UI_SECRET: "{{ui_secret}}"
SECRET_KEY: "{{secret_key}}"
GODEBUG: netdns=cgo
EXT_ENDPOINT: "{{ui_url}}"
TOKEN_URL: http://ui
SELF_REGISTRATION: "{{self_registration}}"
USE_COMPRESSED_JS: "{{use_compressed_js}}"
VERIFY_REMOTE_CERT: "{{verify_remote_cert}}"
TOKEN_EXPIRATION: "{{token_expiration}}"
EXT_REG_URL: "{{hostname}}"
config: |
appname = registry
runmode = dev
[lang]
types = en-US|zh-CN
names = en-US|zh-CN
[dev]
httpport = 80
[mail]
host = {{email_server}}
port = {{email_server_port}}
username = {{email_username}}
password = {{email_password}}
from = {{email_from}}
ssl = {{email_ssl}}
pkey: |
{{4 auth_pkey}}

49
make/kubernetes/ui-rc.yaml
View File

@ -1,49 +0,0 @@
apiVersion: v1
kind: ReplicationController
metadata:
name: ui
labels:
name: ui
spec:
replicas: 1
selector:
name: ui
template:
metadata:
labels:
name: ui
spec:
containers:
- name: ui
image: caicloud/harbor_deploy_ui:latest
imagePullPolicy: Always
env:
- name: MYSQL_HOST
value: mysql
- name: MYSQL_PORT
value: "3306"
- name: MYSQL_USR
value: root
- name: MYSQL_PWD
value: root123
- name: REGISTRY_URL
value: http://registry:5000
- name: CONFIG_PATH
value: /etc/ui/app.conf
- name: HARBOR_REG_URL
value: localhost
- name: HARBOR_ADMIN_PASSWORD
value: Harbor12345
- name: HARBOR_URL
value: http://localhost
- name: AUTH_MODE
value: db_auth
- name: LDAP_URL
value: ldaps://ldap.mydomain.com
- name: LDAP_BASE_DN
value: uid=%s,ou=people,dc=mydomain,dc=com
- name: LOG_LEVEL
value: debug
ports:
- containerPort: 80

174
make/kubernetes/ui/ui.rc.yaml Normal file
View File

@ -0,0 +1,174 @@
apiVersion: v1
kind: ReplicationController
metadata:
name: ui-rc
labels:
name: ui-rc
spec:
replicas: 1
selector:
name: ui-apps
template:
metadata:
labels:
name: ui-apps
spec:
containers:
- name: ui-app
image: harbor/ui
imagePullPolicy: IfNotPresent
env:
- name: MYSQL_HOST
valueFrom:
configMapKeyRef:
name: harbor-ui-config
key: MYSQL_HOST
- name: MYSQL_PORT
valueFrom:
configMapKeyRef:
name: harbor-ui-config
key: MYSQL_PORT
- name: MYSQL_USR
valueFrom:
configMapKeyRef:
name: harbor-ui-config
key: MYSQL_USR
- name: MYSQL_PWD
valueFrom:
configMapKeyRef:
name: harbor-ui-config
key: MYSQL_PWD
- name: REGISTRY_URL
valueFrom:
configMapKeyRef:
name: harbor-ui-config
key: REGISTRY_URL
- name: CONFIG_PATH
valueFrom:
configMapKeyRef:
name: harbor-ui-config
key: CONFIG_PATH
- name: HARBOR_REG_URL
valueFrom:
configMapKeyRef:
name: harbor-ui-config
key: HARBOR_REG_URL
- name: HARBOR_ADMIN_PASSWORD
valueFrom:
configMapKeyRef:
name: harbor-ui-config
key: HARBOR_ADMIN_PASSWORD
- name: HARBOR_URL
valueFrom:
configMapKeyRef:
name: harbor-ui-config
key: HARBOR_URL
- name: AUTH_MODE
valueFrom:
configMapKeyRef:
name: harbor-ui-config
key: AUTH_MODE
- name: LDAP_URL
valueFrom:
configMapKeyRef:
name: harbor-ui-config
key: LDAP_URL
- name: LDAP_SEARCH_DN
valueFrom:
configMapKeyRef:
name: harbor-ui-config
key: LDAP_SEARCH_DN
- name: LDAP_SEARCH_PWD
valueFrom:
configMapKeyRef:
name: harbor-ui-config
key: LDAP_SEARCH_PWD
- name: LDAP_BASE_DN
valueFrom:
configMapKeyRef:
name: harbor-ui-config
key: LDAP_BASE_DN
- name: LDAP_FILTER
valueFrom:
configMapKeyRef:
name: harbor-ui-config
key: LDAP_FILTER
- name: LDAP_UID
valueFrom:
configMapKeyRef:
name: harbor-ui-config
key: LDAP_UID
- name: LDAP_SCOPE
valueFrom:
configMapKeyRef:
name: harbor-ui-config
key: LDAP_SCOPE
- name: LOG_LEVEL
valueFrom:
configMapKeyRef:
name: harbor-ui-config
key: LOG_LEVEL
- name: UI_SECRET
valueFrom:
configMapKeyRef:
name: harbor-ui-config
key: UI_SECRET
- name: SECRET_KEY
valueFrom:
configMapKeyRef:
name: harbor-ui-config
key: SECRET_KEY
- name: GODEBUG
valueFrom:
configMapKeyRef:
name: harbor-ui-config
key: GODEBUG
- name: EXT_ENDPOINT
valueFrom:
configMapKeyRef:
name: harbor-ui-config
key: EXT_ENDPOINT
- name: TOKEN_URL
valueFrom:
configMapKeyRef:
name: harbor-ui-config
key: TOKEN_URL
- name: SELF_REGISTRATION
valueFrom:
configMapKeyRef:
name: harbor-ui-config
key: SELF_REGISTRATION
- name: USE_COMPRESSED_JS
valueFrom:
configMapKeyRef:
name: harbor-ui-config
key: USE_COMPRESSED_JS
- name: VERIFY_REMOTE_CERT
valueFrom:
configMapKeyRef:
name: harbor-ui-config
key: VERIFY_REMOTE_CERT
- name: TOKEN_EXPIRATION
valueFrom:
configMapKeyRef:
name: harbor-ui-config
key: TOKEN_EXPIRATION
- name: EXT_REG_URL
valueFrom:
configMapKeyRef:
name: harbor-ui-config
key: EXT_REG_URL
ports:
- containerPort: 80
volumeMounts:
- name: config
mountPath: /etc/ui
volumes:
- name: config
configMap:
name: harbor-ui-config
items:
- key: config
path: app.conf
- key: pkey
path: private_key.pem

4
make/kubernetes/ui-svc.yaml → make/kubernetes/ui/ui.svc.yaml
View File

@ -2,10 +2,8 @@ apiVersion: v1
kind: Service
metadata:
name: ui
labels:
name: ui
spec:
ports:
- port: 80
selector:
name: ui
name: ui-apps

106
make/prepare
View File

@ -19,8 +19,10 @@ if sys.version_info[:3][0] == 3:
import configparser as ConfigParser
import io as StringIO
def validate(conf):
def validate(conf, args):
protocol = rcp.get("configuration", "ui_url_protocol")
if protocol != "https" and args.notary_mode:
raise Exception("Error: the protocol must be https when Harbor is deployed with Notary")
if protocol == "https":
if not rcp.has_option("configuration", "ssl_cert"):
raise Exception("Error: The protocol is https but attribute ssl_cert is not set")
@ -54,15 +56,39 @@ def get_secret_key(path):
print("generated and saved secret key")
return key
def prep_conf_dir(root, name):
absolute_path = os.path.join(root, name)
if not os.path.exists(absolute_path):
os.makedirs(absolute_path)
return absolute_path
def render(src, dest, **kw):
t = Template(open(src, 'r').read())
with open(dest, 'w') as f:
f.write(t.substitute(**kw))
print("Generated configuration file: %s" % dest)
base_dir = os.path.dirname(__file__)
config_dir = os.path.join(base_dir, "common/config")
templates_dir = os.path.join(base_dir, "common/templates")
parser = argparse.ArgumentParser()
parser.add_argument('-conf', dest='cfgfile', default=base_dir+'/harbor.cfg',type=str,help="the path of Harbor configuration file")
def delfile(src):
if os.path.isfile(src):
try:
os.remove(src)
print("Clearing the configuration file: %s" % src)
except:
pass
elif os.path.isdir(src):
for item in os.listdir(src):
itemsrc=os.path.join(src,item)
delfile(itemsrc)
parser = argparse.ArgumentParser()
parser.add_argument('--conf', dest='cfgfile', default=base_dir+'/harbor.cfg',type=str,help="the path of Harbor configuration file")
parser.add_argument('--with-notary', dest='notary_mode', default=False, action='store_true', help="the Harbor instance is to be deployed with notary")
args = parser.parse_args()
delfile(config_dir)
#Read configurations
conf = StringIO.StringIO()
conf.write("[configuration]\n")
@ -71,7 +97,7 @@ conf.seek(0, os.SEEK_SET)
rcp = ConfigParser.RawConfigParser()
rcp.readfp(conf)
validate(rcp)
validate(rcp, args)
hostname = rcp.get("configuration", "hostname")
protocol = rcp.get("configuration", "ui_url_protocol")
@ -131,31 +157,12 @@ adminserver_config_dir = os.path.join(config_dir,"adminserver")
if not os.path.exists(adminserver_config_dir):
os.makedirs(os.path.join(config_dir, "adminserver"))
ui_config_dir = os.path.join(config_dir,"ui")
if not os.path.exists(ui_config_dir):
os.makedirs(os.path.join(config_dir, "ui"))
db_config_dir = os.path.join(config_dir, "db")
if not os.path.exists(db_config_dir):
os.makedirs(os.path.join(config_dir, "db"))
job_config_dir = os.path.join(config_dir, "jobservice")
if not os.path.exists(job_config_dir):
os.makedirs(job_config_dir)
registry_config_dir = os.path.join(config_dir, "registry")
if not os.path.exists(registry_config_dir):
os.makedirs(registry_config_dir)
nginx_config_dir = os.path.join(config_dir, "nginx")
if not os.path.exists(nginx_config_dir):
os.makedirs(nginx_config_dir)
def render(src, dest, **kw):
t = Template(open(src, 'r').read())
with open(dest, 'w') as f:
f.write(t.substitute(**kw))
print("Generated configuration file: %s" % dest)
ui_config_dir = prep_conf_dir(config_dir,"ui")
db_config_dir = prep_conf_dir(config_dir, "db")
job_config_dir = prep_conf_dir(config_dir, "jobservice")
registry_config_dir = prep_conf_dir(config_dir, "registry")
nginx_config_dir = prep_conf_dir (config_dir, "nginx")
nginx_conf_d = prep_conf_dir(nginx_config_dir, "conf.d")
adminserver_conf_env = os.path.join(config_dir, "adminserver", "env")
ui_conf_env = os.path.join(config_dir, "ui", "env")
@ -166,18 +173,6 @@ db_conf_env = os.path.join(config_dir, "db", "env")
job_conf_env = os.path.join(config_dir, "jobservice", "env")
nginx_conf = os.path.join(config_dir, "nginx", "nginx.conf")
cert_dir = os.path.join(config_dir, "nginx", "cert")
def delfile(src):
if os.path.isfile(src):
try:
os.remove(src)
print("Clearing the configuration file: %s" % src)
except:
pass
elif os.path.isdir(src):
for item in os.listdir(src):
itemsrc=os.path.join(src,item)
delfile(itemsrc)
delfile(config_dir)
if protocol == "https":
target_cert_path = os.path.join(cert_dir, os.path.basename(cert_path))
@ -300,7 +295,6 @@ if customize_crt == 'on':
check_private_key_stat(path=private_key_pem)
check_certificate_stat(path=root_crt)
else:
print("Generated configuration file: %s" % ui_config_dir + "private_key.pem")
shutil.copyfile(os.path.join(templates_dir, "ui", "private_key.pem"), os.path.join(ui_config_dir, "private_key.pem"))
@ -308,5 +302,31 @@ else:
shutil.copyfile(os.path.join(templates_dir, "registry", "root.crt"), os.path.join(registry_config_dir, "root.crt"))
FNULL.close()
if args.notary_mode:
notary_config_dir = prep_conf_dir(config_dir, "notary")
notary_temp_dir = os.path.join(templates_dir, "notary")
print("Copying sql file for notary DB")
if os.path.exists(os.path.join(notary_config_dir, "mysql-initdb.d")):
shutil.rmtree(os.path.join(notary_config_dir, "mysql-initdb.d"))
shutil.copytree(os.path.join(notary_temp_dir, "mysql-initdb.d"), os.path.join(notary_config_dir, "mysql-initdb.d"))
#TODO:generate certs?
print ("Copying certs for notary signer")
shutil.copy2(os.path.join(notary_temp_dir, "notary-signer.crt"), notary_config_dir)
shutil.copy2(os.path.join(notary_temp_dir, "notary-signer.key"), notary_config_dir)
shutil.copy2(os.path.join(notary_temp_dir, "root-ca.crt"), notary_config_dir)
shutil.copy2(os.path.join(registry_config_dir, "root.crt"), notary_config_dir)
print ("Copying notary signer configuration file")
shutil.copy2(os.path.join(notary_temp_dir, "signer-config.json"), notary_config_dir)
render(os.path.join(notary_temp_dir, "server-config.json"),
os.path.join(notary_config_dir, "server-config.json"),
token_endpoint=ui_url)
print ("Copying nginx configuration file for notary")
shutil.copy2(os.path.join(templates_dir, "nginx", "nginx.notary.conf"), nginx_conf_d)
default_alias = ''.join(random.choice(string.ascii_letters) for i in range(8))
render(os.path.join(notary_temp_dir, "signer_env"), os.path.join(notary_config_dir, "signer_env"), alias = default_alias)
print("The configuration files are ready, please use docker-compose to start the service.")

13
partners.md Normal file
View File

@ -0,0 +1,13 @@
# Community of Harbor
Below are some of the partners and users of Harbor projects ( update constantly ). If you wish to be listed on this page as reference partners or users, please contact us at harbor@ vmware.com .
## Partners
<a href="https://www.shurenyun.com/" border="0" target="_blank"><img alt="DataMan" src="docs/img/dataman.png"></a> &nbsp; &nbsp; <a href="http://www.slamtec.com" target="_blank" border="0"><img alt="SlamTec" src="docs/img/slamteclogo.png"></a>
&nbsp; &nbsp; <a href="https://www.caicloud.io" border="0"><img alt="CaiCloud" src="docs/img/caicloudLogoWeb.png"></a>
## Users
<a href="https://www.madailicai.com/" border="0" target="_blank"><img alt="MaDaiLiCai" src="docs/img/UserMaDai.jpg"></a> <a href="https://www.dianrong.com/" border="0" target="_blank"><img alt="Dianrong" src="docs/img/dianrong.png"></a>
## Supporting Technologies
<img alt="beego" src="docs/img/beegoLogo.png"> Harbor is powered by <a href="http://beego.me/">Beego</a>.

14
src/common/dao/repository.go
View File

@ -110,13 +110,17 @@ func GetTopRepos(userID int, count int) ([]models.TopRepo, error) {
where (
p.deleted = 0 and (
p.public = 1 or (
? <> ? and exists (
select 1 from project_member pm
where pm.project_id = p.project_id and pm.user_id = ?
))))
? <> ? and (
exists (
select 1 from user u
where u.user_id = ? and u.sysadmin_flag = 1
) or exists (
select 1 from project_member pm
where pm.project_id = p.project_id and pm.user_id = ?
)))))
order by r.pull_count desc, r.name limit ?`
repositories := []*models.RepoRecord{}
_, err := GetOrmer().Raw(sql, userID, NonExistUserID, userID, count).QueryRows(&repositories)
_, err := GetOrmer().Raw(sql, userID, NonExistUserID, userID, userID, count).QueryRows(&repositories)
if err != nil {
return topRepos, err
}

66
src/common/dao/repository_test.go
View File

@ -202,17 +202,15 @@ func TestGetTopRepos(t *testing.T) {
require.NoError(err)
project2 := models.Project{
OwnerID: admin.UserID,
OwnerID: user.UserID,
Name: "project2",
CreationTime: time.Now(),
OwnerName: admin.Username,
OwnerName: user.Username,
Public: 0,
}
project2.ProjectID, err = AddProject(project2)
require.NoError(err)
require.NoError(AddProjectMember(project2.ProjectID, user.UserID, models.PROJECTADMIN))
err = AddRepository(*repository)
require.NoError(err)
@ -268,12 +266,13 @@ func TestGetTopRepos(t *testing.T) {
}
err = AddRepository(*deletedPublicRepository1)
require.NoError(err)
DeleteProject(deletedPublicProject.ProjectID)
err = DeleteProject(deletedPublicProject.ProjectID)
require.NoError(err)
var topRepos []models.TopRepo
// anonymous should retrieve public non-deleted repositories
topRepos, err = GetTopRepos(NonExistUserID, 3)
topRepos, err = GetTopRepos(NonExistUserID, 100)
require.NoError(err)
require.Len(topRepos, 1)
require.Equal(topRepos, []models.TopRepo{
@ -283,7 +282,45 @@ func TestGetTopRepos(t *testing.T) {
},
})
// admin should retrieve all visible repositories limited by count
// admin should retrieve all repositories
topRepos, err = GetTopRepos(admin.UserID, 100)
require.NoError(err)
require.Len(topRepos, 4)
require.Equal(topRepos, []models.TopRepo{
models.TopRepo{
RepoName: repository3.Name,
AccessCount: repository3.PullCount,
},
models.TopRepo{
RepoName: repository2.Name,
AccessCount: repository2.PullCount,
},
models.TopRepo{
RepoName: repository1.Name,
AccessCount: repository1.PullCount,
},
models.TopRepo{
RepoName: repository.Name,
AccessCount: repository.PullCount,
},
})
// user should retrieve visible repositories
topRepos, err = GetTopRepos(user.UserID, 100)
require.NoError(err)
require.Len(topRepos, 2)
require.Equal(topRepos, []models.TopRepo{
models.TopRepo{
RepoName: repository3.Name,
AccessCount: repository3.PullCount,
},
models.TopRepo{
RepoName: repository.Name,
AccessCount: repository.PullCount,
},
})
// limit by count
topRepos, err = GetTopRepos(admin.UserID, 3)
require.NoError(err)
require.Len(topRepos, 3)
@ -301,21 +338,6 @@ func TestGetTopRepos(t *testing.T) {
AccessCount: repository1.PullCount,
},
})
// user should retrieve all visible repositories
topRepos, err = GetTopRepos(user.UserID, 3)
require.NoError(err)
require.Len(topRepos, 2)
require.Equal(topRepos, []models.TopRepo{
models.TopRepo{
RepoName: repository3.Name,
AccessCount: repository3.PullCount,
},
models.TopRepo{
RepoName: repository.Name,
AccessCount: repository.PullCount,
},
})
}
func addRepository(repository *models.RepoRecord) error {

2
src/common/utils/utils.go
View File

@ -57,7 +57,7 @@ func ParseRepository(repository string) (project, rest string) {
rest = repository
return
}
index := strings.LastIndex(repository, "/")
index := strings.Index(repository, "/")
project = repository[0:index]
rest = repository[index+1:]
return

4
src/common/utils/utils_test.go
View File

@ -66,10 +66,10 @@ func TestParseRepository(t *testing.T) {
repository = "library/test/ubuntu"
project, rest = ParseRepository(repository)
if project != "library/test" {
if project != "library" {
t.Errorf("unexpected project: %s != %s", project, "library/test")
}
if rest != "ubuntu" {
if rest != "test/ubuntu" {
t.Errorf("unexpected rest: %s != %s", rest, "ubuntu")
}

2
src/ui/api/utils.go
View File

@ -149,7 +149,7 @@ func TriggerReplication(policyID int64, repository string,
func GetPoliciesByRepository(repository string) ([]*models.RepPolicy, error) {
repository = strings.TrimSpace(repository)
repository = strings.TrimRight(repository, "/")
projectName := repository[:strings.LastIndex(repository, "/")]
projectName, _ := utils.ParseRepository(repository)
project, err := dao.GetProjectByName(projectName)
if err != nil {

19
src/ui/service/token/authutils.go
View File

@ -33,10 +33,25 @@ import (
)
const (
issuer = "registry-token-issuer"
privateKey = "/etc/ui/private_key.pem"
issuer = "registry-token-issuer"
defaultTokenExpiration = 30
)
var expiration int //minutes
var privateKey string
func init() {
var err error
expiration, err = config.TokenExpiration()
if err != nil {
log.Errorf("failed to get token expiration: %v, will use the default value %d",
err, defaultTokenExpiration)
expiration = defaultTokenExpiration
}
privateKey = "/etc/ui/private_key.pem"
log.Infof("token expiration: %d minutes", expiration)
}
// GetResourceActions ...
func GetResourceActions(scopes []string) []*token.ResourceActions {
log.Debugf("scopes: %+v", scopes)

51
src/ui/service/token/test/private_key.pem Normal file
View File

@ -0,0 +1,51 @@
-----BEGIN RSA PRIVATE KEY-----
MIIJKAIBAAKCAgEAtpMvyv153iSmwm6TrFpUOzsIGBEDbGtOOEZMEm08D8IC2n1G
d6/XOZ5FxPAD6gIpE0EAcMojY5O0Hl4CDoyV3e/iKcBqFOgYtpogNtan7yT5J8gw
KsPbU/8nBkK75GOq56nfvq4t9GVAclIDtHbuvmlh6O2n+fxtR0M9LbuotbSBdXYU
hzXqiSsMclBvLyIk/z327VP5l0nUNOzPuKIwQjuxYKDkvq1oGy98oVlE6wl0ldh2
ZYZLGAYbVhqBVUT1Un/PYqi9Nofa2RI5n1WOkUJQp87vb+PUPFhVOdvH/oAzV6/b
9dzyhA5paDM06lj2gsg9hQWxCgbFh1x39c6pSI8hmVe6x2d4tAtSyOm3Qwz+zO2l
bPDvkY8Svh5nxUYObrNreoO8wHr8MC6TGUQLnUt/RfdVKe5fYPFl6VYqJP/L3LDn
Xj771nFq6PKiYbhBwJw3TM49gpKNS/Of70TP2m7nVlyuyMdE5T1j3xyXNkixXqqn
JuSMqX/3Bmm0On9KEbemwn7KRYF/bqc50+RcGUdKNcOkN6vuMVZei4GbxALnVqac
s+/UQAiQP4212UO7iZFwMaCNJ3r/b4GOlyalI1yEA4odoZov7k5zVOzHu8O6QmCj
3R5TVOudpGiUh+lumRRpNqxDgjngLljvaWU6ttyIbjnAwCjnJoppZM2lkRkCAwEA
AQKCAgAvsvCPlf2a3fR7Y6xNISRUfS22K+u7DaXX6fXB8qv4afWY45Xfex89vG35
78L2Bi55C0h0LztjrpkmPeVHq88TtrJduhl88M5UFpxH93jUb9JwZErBQX4xyb2G
UzUHjEqAT89W3+a9rR5TP74cDd59/MZJtp1mIF7keVqochi3sDsKVxkx4hIuWALe
csk5hTApRyUWCBRzRCSe1yfF0wnMpA/JcP+SGXfTcmqbNNlelo/Q/kaga59+3UmT
C0Wy41s8fIvP+MnGT2QLxkkrqYyfwrWTweqoTtuKEIHjpdnwUcoYJKfQ6jKp8aH0
STyP5UIyFOKNuFjyh6ZfoPbuT1nGW+YKlUnK4hQ9N/GE0oMoecTaHTbqM+psQvbj
6+CG/1ukA5ZTQyogNyuOApArFBQ+RRmVudPKA3JYygIhwctuB2oItsVEOEZMELCn
g2aVFAVXGfGRDXvpa8oxs3Pc6RJEp/3tON6+w7cMCx0lwN/Jk2Ie6RgTzUycT3k6
MoTQJRoO6/ZHcx3hTut/CfnrWiltyAUZOsefLuLg+Pwf9GHhOycLRI6gHfgSwdIV
S77UbbELWdscVr1EoPIasUm1uYWBBcFRTturRW+GHJ8TZX+mcWSBcWwBhp15LjEl
tJf+9U6lWMOSB2LvT+vFmR0M9q56fo7UeKFIR7mo7/GpiVu5AQKCAQEA6Qs7G9mw
N/JZOSeQO6xIQakC+sKApPyXO58fa7WQzri+l2UrLNp0DEQfZCujqDgwys6OOzR/
xg8ZKQWVoad08Ind3ZwoJgnLn6QLENOcE6PpWxA/JjnVGP4JrXCYR98cP0sf9jEI
xkR1qT50GbeqU3RDFliI4kGRvbZ8cekzuWppfQcjstSBPdvuxqAcUVmTnTw83nvD
FmBbhlLiEgI3iKtJ97UB7480ivnWnOuusduk7FO4jF3hkrOa+YRidinTCi8JBo0Y
jx4Ci3Y5x6nvwkXhKzXapd7YmPNisUc5xA7/a+W71cyC0IKUwRc/8pYWLL3R3CpR
YiV8gf6gwzOckQKCAQEAyI9CSNoAQH4zpS8B9PF8zILqEEuun8m1f5JB3hQnfWzm
7uz/zg6I0TkcCE0AJVSKPHQm1V9+TRbF9+DiOWHEYYzPmK8h63SIufaWxZPqai4E
PUj6eQWykBUVJ96n6/AW0JHRZ+WrJ5RXBqCLuY7NP6wDhORrCJjBwaGMohNpbKPS
H3QewsoxCh+CEXKdKyy+/yU/f4E89PlHapkW1/bDJ5u7puSD+KvmiDDIXSBncdOO
uFT8n+XH5IwgjdXFSDim15rQ8jD2l2xLcwKboTpx5GeRl8oB1VGm0fUbBn1dvGPG
4WfHGyrp9VNZtP160WoHr+vRVPqvHNkoeAlCfEwQCQKCAQBN1dtzLN0HgqE8TrOE
ysEDdTCykj4nXNoiJr522hi4gsndhQPLolb6NdKKQW0S5Vmekyi8K4e1nhtYMS5N
5MFRCasZtmtOcR0af87WWucZRDjPmniNCunaxBZ1YFLsRl+H4E6Xir8UgY8O7PYY
FNkFsKIrl3x4nU/RHl8oKKyG9Dyxbq4Er6dPAuMYYiezIAkGjjUCVjHNindnQM2T
GDx2IEe/PSydV6ZD+LguhyU88FCAQmI0N7L8rZJIXmgIcWW0VAterceTHYHaFK2t
u1uB9pcDOKSDnA+Z3kiLT2/CxQOYhQ2clgbnH4YRi/Nm0awsW2X5dATklAKm5GXL
bLSRAoIBAQClaNnPQdTBXBR2IN3pSZ2XAkXPKMwdxvtk+phOc6raHA4eceLL7FrU
y9gd1HvRTfcwws8gXcDKDYU62gNaNhMELWEt2QsNqS/2x7Qzwbms1sTyUpUZaSSL
BohLOKyfv4ThgdIGcXoGi6Z2tcRnRqpq4BCK8uR/05TBgN5+8amaS0ZKYLfaCW4G
nlPk1fVgHWhtAChtnYZLuKg494fKmB7+NMfAbmmVlxjrq+gkPkxyqXvk9Vrg+V8y
VIuozu0Fkouv+GRpyw4ldtCHS1hV0eEK8ow2dwmqCMygDxm58X10mYn2b2PcOTl5
9sNerUw1GNC8O66K+rGgBk4FKgXmg8kZAoIBABBcuisK250fXAfjAWXGqIMs2+Di
vqAdT041SNZEOJSGNFsLJbhd/3TtCLf29PN/YXtnvBmC37rqryTsqjSbx/YT2Jbr
Bk3jOr9JVbmcoSubXl8d/uzf7IGs91qaCgBwPZHgeH+kK13FCLexz+U9zYMZ78fF
/yO82CpoekT+rcl1jzYn43b6gIklHABQU1uCD6MMyMhJ9Op2WmbDk3X+py359jMc
+Cr2zfzdHAIVff2dOV3OL+ZHEWbwtnn3htKUdOmjoTJrciFx0xNZJS5Q7QYHMONj
yPqbajyhopiN01aBQpCSGF1F1uRpWeIjTrAZPbrwLl9YSYXz0AT05QeFEFk=
-----END RSA PRIVATE KEY-----

35
src/ui/service/token/test/root.crt Normal file
View File

@ -0,0 +1,35 @@
-----BEGIN CERTIFICATE-----
MIIGBzCCA++gAwIBAgIJAKB8CNqCxhr7MA0GCSqGSIb3DQEBCwUAMIGZMQswCQYD
VQQGEwJDTjEOMAwGA1UECAwFU3RhdGUxCzAJBgNVBAcMAkNOMRUwEwYDVQQKDAxv
cmdhbml6YXRpb24xHDAaBgNVBAsME29yZ2FuaXphdGlvbmFsIHVuaXQxFDASBgNV
BAMMC2V4YW1wbGUuY29tMSIwIAYJKoZIhvcNAQkBFhNleGFtcGxlQGV4YW1wbGUu
Y29tMB4XDTE2MDUxNjAyNDY1NVoXDTI2MDUxNDAyNDY1NVowgZkxCzAJBgNVBAYT
AkNOMQ4wDAYDVQQIDAVTdGF0ZTELMAkGA1UEBwwCQ04xFTATBgNVBAoMDG9yZ2Fu
aXphdGlvbjEcMBoGA1UECwwTb3JnYW5pemF0aW9uYWwgdW5pdDEUMBIGA1UEAwwL
ZXhhbXBsZS5jb20xIjAgBgkqhkiG9w0BCQEWE2V4YW1wbGVAZXhhbXBsZS5jb20w
ggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQC2ky/K/XneJKbCbpOsWlQ7
OwgYEQNsa044RkwSbTwPwgLafUZ3r9c5nkXE8APqAikTQQBwyiNjk7QeXgIOjJXd
7+IpwGoU6Bi2miA21qfvJPknyDAqw9tT/ycGQrvkY6rnqd++ri30ZUByUgO0du6+
aWHo7af5/G1HQz0tu6i1tIF1dhSHNeqJKwxyUG8vIiT/PfbtU/mXSdQ07M+4ojBC
O7FgoOS+rWgbL3yhWUTrCXSV2HZlhksYBhtWGoFVRPVSf89iqL02h9rZEjmfVY6R
QlCnzu9v49Q8WFU528f+gDNXr9v13PKEDmloMzTqWPaCyD2FBbEKBsWHXHf1zqlI
jyGZV7rHZ3i0C1LI6bdDDP7M7aVs8O+RjxK+HmfFRg5us2t6g7zAevwwLpMZRAud
S39F91Up7l9g8WXpViok/8vcsOdePvvWcWro8qJhuEHAnDdMzj2Cko1L85/vRM/a
budWXK7Ix0TlPWPfHJc2SLFeqqcm5Iypf/cGabQ6f0oRt6bCfspFgX9upznT5FwZ
R0o1w6Q3q+4xVl6LgZvEAudWppyz79RACJA/jbXZQ7uJkXAxoI0nev9vgY6XJqUj
XIQDih2hmi/uTnNU7Me7w7pCYKPdHlNU652kaJSH6W6ZFGk2rEOCOeAuWO9pZTq2
3IhuOcDAKOcmimlkzaWRGQIDAQABo1AwTjAdBgNVHQ4EFgQUPJF++WMsv1OJvf7F
oCew37JTnfQwHwYDVR0jBBgwFoAUPJF++WMsv1OJvf7FoCew37JTnfQwDAYDVR0T
BAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAgEAb5LvqukMxWd5Zajbh3orfYsXmhWn
UWiwG176+bd3b5xMlG9iLd4vQ11lTZoIhFOfprRQzbizQ8BzR2JBQckpLcy+5hyA
D3M9vLL37OwA0wT6kxFnd6LtlFaH5gG++huw2ts2PDXFz0jqw+0YE/R8ov2+YdaZ
aPSEMunmAuEY1TbYWzz4u6PxycxhQzDQ34ZmJZ34Elvw1NYMfPMGTKp34PsxIcgT
ao5jqb9RMU6JAumfXrOvXRjjl573vX2hgMZzEU6OF2/+uyg95chn6nO1GUQrT2+F
/1xIqfHfFCm8+jujSDgqfBtGI+2C7No+Dq8LEyEINZe6wSQ81+ryt5jy5SZmAsnj
V4OsSIwlpR5fLUwrFStVoUWHEKl1DflkYki/cAC1TL0Om+ldJ219kcOnaXDNaq66
3I75BvRY7/88MYLl4Fgt7sn05Mn3uNPrCrci8d0R1tlXIcwMdCowIHeZdWHX43f7
NsVk/7VSOxJ343csgaQc+3WxEFK0tBxGO6GP+Xj0XmdVGLhalVBsEhPjnmx+Yyrn
oMsTA1Yrs88C8ItQn7zuO/30eKNGTnby0gptHiS6sa/c3O083Mpi8y33GPVZDvBl
l9PfSZT8LG7SvpjsdgdNZlyFvTY4vsB+Vd5Howh7gXYPVXdCs4k7HMyo7zvzliZS
ekCw9NGLoNqQqnA=
-----END CERTIFICATE-----

101
src/ui/service/token/token_test.go
View File

@ -1,9 +1,108 @@
package token
import (
jwt "github.com/dgrijalva/jwt-go"
"github.com/docker/distribution/registry/auth/token"
"github.com/stretchr/testify/assert"
"crypto/rsa"
"crypto/x509"
"encoding/pem"
"fmt"
"io/ioutil"
"os"
"path"
"runtime"
"testing"
)
func TestMain(t *testing.T) {
func TestMain(m *testing.M) {
result := m.Run()
if result != 0 {
os.Exit(result)
}
}
func TestGetResourceActions(t *testing.T) {
s := []string{"registry:catalog:*", "repository:10.117.4.142/notary-test/hello-world-2:pull,push"}
expectedRA := [2]token.ResourceActions{
token.ResourceActions{
Type: "registry",
Name: "catalog",
Actions: []string{"*"},
},
token.ResourceActions{
Type: "repository",
Name: "10.117.4.142/notary-test/hello-world-2",
Actions: []string{"pull", "push"},
},
}
ra := GetResourceActions(s)
assert.Equal(t, *ra[0], expectedRA[0], "The Resource Action mismatch")
assert.Equal(t, *ra[1], expectedRA[1], "The Resource Action mismatch")
}
func getKeyAndCertPath() (string, string) {
_, f, _, ok := runtime.Caller(0)
if !ok {
panic("Failed to get current directory")
}
return path.Join(path.Dir(f), "test/private_key.pem"), path.Join(path.Dir(f), "test/root.crt")
}
func getPublicKey(crtPath string) (*rsa.PublicKey, error) {
crt, err := ioutil.ReadFile(crtPath)
if err != nil {
return nil, err
}
block, _ := pem.Decode(crt)
cert, err := x509.ParseCertificate(block.Bytes)
if err != nil {
return nil, err
}
return cert.PublicKey.(*rsa.PublicKey), nil
}
type harborClaims struct {
jwt.StandardClaims
// Private claims
Access []*token.ResourceActions `json:"access"`
}
func TestMakeToken(t *testing.T) {
pk, crt := getKeyAndCertPath()
//overwrite the config values for testing.
privateKey = pk
expiration = 10
ra := []*token.ResourceActions{&token.ResourceActions{
Type: "repository",
Name: "10.117.4.142/notary-test/hello-world-2",
Actions: []string{"pull", "push"},
}}
svc := "harbor-registry"
u := "tester"
tokenString, _, _, err := MakeToken(u, svc, ra)
if err != nil {
t.Errorf("Error while making token: %v", err)
}
//t.Logf("privatekey: %s, crt: %s", tokenString, crt)
pubKey, err := getPublicKey(crt)
if err != nil {
t.Errorf("Error while getting public key from cert: %s", crt)
}
tok, err := jwt.ParseWithClaims(tokenString, &harborClaims{}, func(t *jwt.Token) (interface{}, error) {
if _, ok := t.Method.(*jwt.SigningMethodRSA); !ok {
return nil, fmt.Errorf("Unexpected signing method: %v", t.Header["alg"])
}
return pubKey, nil
})
t.Logf("validity: %v", tok.Valid)
if err != nil {
t.Errorf("Error while parsing the token: %v", err)
}
claims := tok.Claims.(*harborClaims)
t.Logf("claims: %+v", *claims)
assert.Equal(t, *(claims.Access[0]), *(ra[0]), "Access mismatch")
assert.Equal(t, claims.Audience, svc, "Audience mismatch")
}

22
src/ui/static/resources/js/components/repository/list-repository.directive.html
View File

@ -17,13 +17,21 @@
<div class="panel panel-default" ng-repeat="repo in vm.repositories">
<div class="panel-heading" role="tab" id="heading//$index + 1//">
<h4 class="panel-title">
<a ng-show="vm.tagCount[repo] > 0" role="button" data-toggle="collapse" data-parent="" href="?project_id=//vm.projectId//#collapse//$index + 1//" aria-expanded="true" aria-controls="collapse//$index+1//">
<span class="glyphicon glyphicon-book"></span> &nbsp;//repo// &nbsp;&nbsp;<span class="badge">//vm.tagCount[repo]//</span>
</a>
<a ng-if="vm.tagCount[repo] === 0" role="button" style="text-decoration: none;" data-toggle="collapse" data-parent="" aria-expanded="true" aria-controls="collapse//$index+1//">
<span class="glyphicon glyphicon-book"></span> &nbsp;//repo// &nbsp;&nbsp;<span class="badge">//vm.tagCount[repo]//</span>
</a>
<a ng-show="vm.tagCount[repo] > 0 && vm.roleId == 1" class="pull-right" style="margin-right: 75px;" href="javascript:void(0)" ng-click="vm.deleteByRepo(repo)" title="// 'delete_repo' | tr //" loading-progress hide-target="true" toggle-in-progress="vm.toggleInProgress[repo + '|']"><span class="glyphicon glyphicon-trash"></span></a>
<div class="row">
<div class="col-md-10" style="padding: 2px">
<a ng-show="vm.tagCount[repo] > 0" role="button" data-toggle="collapse" data-parent="" href="?project_id=//vm.projectId//#collapse//$index + 1//" aria-expanded="true" aria-controls="collapse//$index+1//">
<span class="glyphicon glyphicon-book"></span> &nbsp;//repo// &nbsp;&nbsp;<span class="badge">//vm.tagCount[repo]//</span>
</a>
<a ng-if="vm.tagCount[repo] === 0" role="button" style="text-decoration: none;" data-toggle="collapse" data-parent="" aria-expanded="true" aria-controls="collapse//$index+1//">
<span class="glyphicon glyphicon-book"></span> &nbsp;//repo// &nbsp;&nbsp;<span class="badge">//vm.tagCount[repo]//</span>
</a>
</div>
<div class="col-md-2" style="padding: 2px; padding-left: 6%;">
&nbsp;&nbsp;
<input ng-if="vm.tagCount[repo] > 0 && vm.roleId == 1" type="checkbox" ng-checked="vm.selectAll[repo]" ng-click="vm.selectAllTags(repo)">
<a ng-if="vm.tagCount[repo] > 0 && vm.roleId == 1 && vm.selectedTags[repo].length > 0" href="javascript:void(0)" ng-click="vm.deleteSelectedTagsByRepo(repo)" title="// 'delete_selected_tag' | tr //" loading-progress hide-target="true" toggle-in-progress="vm.toggleInProgress[repo + '|']"><span class="glyphicon glyphicon-trash"></span></a>
</div>
</div>
</h4>
</div>
<list-tag ng-show="vm.tagCount[repo] > 0" associate-id="$index + 1" repo-name="repo" tag-count="vm.tagCount" toggle-in-progress="vm.toggleInProgress" delete-by-tag="vm.deleteByTag()" role-id="//vm.roleId//"></list-tag>

75
src/ui/static/resources/js/components/repository/list-repository.directive.js
View File

@ -54,7 +54,7 @@
if(val) {
vm.projectId = getParameterByName('project_id', $location.absUrl());
vm.filterInput = '';
vm.retrieve();
vm.retrieve();
}
});
@ -72,6 +72,12 @@
}
});
$scope.$watch('vm.tagName', function(current) {
if(current) {
vm.selectedTags = [];
}
});
$scope.$on('repoName', function(e, val) {
vm.repoName = val;
});
@ -87,11 +93,20 @@
$scope.$on('tags', function(e, val) {
vm.tags = val;
});
vm.deleteByRepo = deleteByRepo;
vm.deleteByTag = deleteByTag;
vm.deleteSelectedTagsByRepo = deleteSelectedTagsByRepo;
vm.deleteImage = deleteImage;
vm.deleteSelectedTags = deleteSelectedTags;
vm.selectAll = [];
vm.selectAllTags = selectAllTags;
vm.selectedTags = [];
function retrieve(){
console.log('retrieve repositories, project_id:' + vm.projectId);
ListRepositoryService(vm.projectId, vm.filterInput, vm.page, vm.pageSize)
@ -101,6 +116,8 @@
function getRepositoryComplete(response) {
vm.repositories = response.data || [];
vm.totalCount = response.headers('X-Total-Count');
vm.selectAll[vm.repoName] = false;
vm.selectedTags = [];
}
function getRepositoryFailed(response) {
@ -143,7 +160,31 @@
$scope.$emit('raiseInfo', emitInfo);
}
function deleteSelectedTagsByRepo(repo) {
vm.repoName = repo;
$scope.$broadcast('gatherSelectedTags' + vm.repoName, true);
var emitInfo = {
'confirmOnly': false,
'contentType': 'text/html',
'action' : vm.deleteSelectedTags
};
$scope.$emit('modalTitle', $filter('tr')('alert_delete_tag_title'));
$scope.$emit('modalMessage', $filter('tr')('alert_delete_selected_tag'));
$scope.$emit('raiseInfo', emitInfo);
}
function selectAllTags(repo) {
vm.selectAll[repo] = !vm.selectAll[repo];
console.log('send to tags selectAll:' + vm.selectAll[repo]);
$scope.$broadcast('selectAll' + repo, {'status': vm.selectAll[repo], 'repoName': repo});
}
$scope.$on('selectedAll', function(e, val) {
console.log('received from tags selectedAll:' + angular.toJson(val));
vm.selectAll[val.repoName] = val.status;
});
function deleteByTag() {
$scope.$emit('modalTitle', $filter('tr')('alert_delete_tag_title', [vm.tag]));
var message;
@ -155,26 +196,46 @@
'contentType': 'text/html',
'action' : vm.deleteImage
};
$scope.$emit('raiseInfo', emitInfo);
}
}
function deleteImage() {
console.log('Delete image, repoName:' + vm.repoName + ', tag:' + vm.tag);
vm.toggleInProgress[vm.repoName + '|' + vm.tag] = true;
DeleteRepositoryService(vm.repoName, vm.tag)
.success(deleteRepositorySuccess)
.error(deleteRepositoryFailed);
}
$scope.$on('selectedTags', function(e, val) {
if(val) {
vm.selectedTags[val.repoName] = val.tags;
}
});
function deleteSelectedTags() {
console.log('Delete selected tags:' + angular.toJson(vm.selectedTags[vm.repoName]) + ' under repo:' + vm.repoName);
vm.toggleInProgress[vm.repoName + '|'] = true;
for(var i in vm.selectedTags[vm.repoName] || []) {
var tag = vm.selectedTags[vm.repoName][i];
if(tag !== '') {
vm.toggleInProgress[vm.repoName + '|' + tag] = true;
DeleteRepositoryService(vm.repoName, tag)
.success(deleteRepositorySuccess)
.error(deleteRepositoryFailed);
}
}
}
function deleteRepositorySuccess(data, status) {
vm.toggleInProgress[vm.repoName + '|'] = false;
vm.toggleInProgress[vm.repoName + '|' + vm.tag] = false;
vm.retrieve();
$scope.$broadcast('refreshTags', true);
}
function deleteRepositoryFailed(data, status) {
vm.toggleInProgress[vm.repoName + '|'] = false;
vm.toggleInProgress[vm.repoName + '|' + vm.tag] = false;
$scope.$emit('modalTitle', $filter('tr')('error'));

10
src/ui/static/resources/js/components/repository/list-tag.directive.html
View File

@ -1,6 +1,6 @@
<div id="collapse//vm.associateId//" class="panel-collapse collapse" role="tabpanel" aria-labelledby="heading//vm.associateId//">
<div class="panel-body">
<table class="repository-table">
<div class="panel-body table-responsive">
<table class="table" style="margin-bottom: 0;">
<thead>
<th width="20%"><span class="glyphicon glyphicon-tags"></span> // 'tag' | tr //</th>
<th width="15%" style="text-align: center;">// 'image_details' | tr //</th>
@ -14,7 +14,11 @@
<td>
<pull-command repo-name="//vm.repoName//" tag="//tag//"></pull-command>
</td>
<td style="text-align: center;"><a ng-if="vm.roleId == 1" href="javascript:void(0);" ng-click="vm.deleteTag({repoName: vm.repoName, tag: tag})" title="// 'delete_tag' | tr //" loading-progress hide-target="true" toggle-in-progress="vm.toggleInProgress[vm.repoName +'|'+ tag]"><span class="glyphicon glyphicon-trash"></span></a></td>
<td style="padding: 2px; padding-left: 6%;">
&nbsp;&nbsp;
<input ng-if="vm.roleId == 1" type="checkbox" ng-checked="vm.selected[vm.repoName][$index]" ng-click="vm.selectOne($index, tag)">
<a ng-show="vm.roleId == 1 && vm.selected[vm.repoName][$index]" href="javascript:void(0);" ng-click="vm.deleteTag({repoName: vm.repoName, tag: tag})" title="// 'delete_tag' | tr //" loading-progress hide-target="true" toggle-in-progress="vm.toggleInProgress[vm.repoName +'|'+ tag]"><span class="glyphicon glyphicon-trash"></span></a>
</td>
</tr>
</tbody>
</table>

74
src/ui/static/resources/js/components/repository/list-tag.directive.js
View File

@ -28,8 +28,14 @@
vm.tags = [];
vm.retrieve = retrieve;
vm.selected = [];
vm.selected[vm.repoName] = [];
vm.selectedTags = [];
$scope.$watch('vm.repoName', function(current, origin) {
if(current) {
console.log('vm.repoName triggered tags retrieval.');
vm.retrieve();
}
});
@ -37,10 +43,47 @@
$scope.$on('refreshTags', function(e, val) {
if(val) {
vm.retrieve();
vm.selectedCount[vm.repoName] = 0;
vm.selected[val.repoName] = [];
vm.selectedTags = [];
}
});
vm.deleteTag = deleteTag;
$scope.$watch('vm.selectedCount[vm.repoName]', function(current, previous) {
if(current !== previous) {
console.log('Watching vm.selectedCount:' + current);
$scope.$emit('selectedAll', {'status': (current === vm.tags.length), 'repoName': vm.repoName});
}
});
$scope.$on('gatherSelectedTags' + vm.repoName, function(e, val) {
if(val) {
console.log('RECEIVED gatherSelectedTags:' + val);
gatherSelectedTags();
}
});
$scope.$on('selectAll' + vm.repoName, function(e, val) {
(val.status) ? vm.selectedCount[val.repoName] = vm.tags.length : vm.selectedCount[val.repoName] = 0;
for(var i = 0; i < vm.tags.length; i++) {
vm.selected[val.repoName][i] = val.status;
}
gatherSelectedTags();
console.log('received selectAll:' + angular.toJson(val) + ', vm.selected:' + angular.toJson(vm.selected));
});
$scope.$watch('vm.tags', function(current) {
if(current) {
vm.tags = current;
}
});
vm.deleteTag = deleteTag;
vm.selectedCount = [];
vm.selectedCount[vm.repoName] = 0;
vm.selectOne = selectOne;
function retrieve() {
ListTagService(vm.repoName)
@ -49,7 +92,6 @@
}
function getTagSuccess(data) {
vm.tags = data || [];
vm.tagCount[vm.repoName] = vm.tags.length;
@ -59,6 +101,10 @@
angular.forEach(vm.tags, function(item) {
vm.toggleInProgress[vm.repoName + '|' + item] = false;
});
for(var i = 0; i < vm.tags.length; i++) {
vm.selected[vm.repoName][i] = false;
}
}
function getTagFailed(data) {
@ -72,6 +118,28 @@
$scope.$emit('repoName', e.repoName);
$scope.$emit('tag', e.tag);
vm.deleteByTag();
}
function selectOne(index, tagName) {
vm.selected[vm.repoName][index] = !vm.selected[vm.repoName][index];
(vm.selected[vm.repoName][index]) ? ++vm.selectedCount[vm.repoName] : --vm.selectedCount[vm.repoName];
console.log('selectOne, repoName:' + vm.repoName + ', vm.selected:' + vm.selected[vm.repoName][index] + ', index:' + index + ', length:' + vm.selectedCount[vm.repoName]);
gatherSelectedTags();
}
function gatherSelectedTags() {
vm.selectedTags[vm.repoName] = [];
for(var i = 0; i < vm.tags.length; i++) {
(vm.selected[vm.repoName][i]) ? vm.selectedTags[vm.repoName][i] = vm.tags[i] : vm.selectedTags[vm.repoName][i] = '';
}
var tagsToDelete = [];
for(var i in vm.selectedTags[vm.repoName]) {
var tag = vm.selectedTags[vm.repoName][i];
if(tag !== '') {
tagsToDelete.push(tag);
}
}
$scope.$emit('selectedTags', {'repoName': vm.repoName, 'tags': tagsToDelete});
}
}
@ -97,4 +165,4 @@
}
})();
})();

6
src/ui/static/resources/js/components/repository/pull-command.directive.html
View File

@ -1,7 +1,5 @@
<form class="form-inline">
<div class="form-group">
<div class="input-group">
<input type="text" class="form-control-static" style="padding-left: 10px;" value="docker pull //vm.harborRegUrl////vm.repoName//://vm.tag//" readonly="readonly" size="60">
</div>
<div class="form-group" style="width: 100%;">
<input type="text" class="form-control form-control-static" style="padding-left: 10px; width: 100%;" value="docker pull //vm.harborRegUrl////vm.repoName//://vm.tag//" readonly="readonly">
</div>
</form>

2
src/ui/static/resources/js/services/i18n/locale_messages_en-US.js
View File

@ -142,6 +142,7 @@ var locale_messages = {
'<br/>Delete repository "$0" now?',
'alert_delete_tag_title': 'Confirm Deletion',
'alert_delete_tag': 'Note: All tags under this repository will be deleted if they are pointing to this image.<br/><br/>Delete tag "$0" now?',
'alert_delete_selected_tag': 'Note: All selected tags under this repository will be deleted if they are pointing to this image.<br/><br/>Delete selected tags now?',
'close': 'Close',
'ok': 'OK',
'welcome': 'Welcome to Harbor!',
@ -217,6 +218,7 @@ var locale_messages = {
'select_all': 'Select All',
'delete_tag': 'Delete Tag',
'delete_repo': 'Delete Repo',
'delete_selected_tag': 'Delete Selected Tag(s)',
'download_log': 'View Logs',
'edit': 'Edit',
'delete': 'Delete',

2
src/ui/static/resources/js/services/i18n/locale_messages_zh-CN.js
View File

@ -142,6 +142,7 @@ var locale_messages = {
'<br/>是否删除镜像仓库 "$0" ?',
'alert_delete_tag_title': '确认删除',
'alert_delete_tag': '注意：此镜像仓库下所有指向该镜像的标签将会被删除。<br/><br/>删除镜像标签 "$0" ?',
'alert_delete_selected_tag': '注意：此镜像仓库下选中的指向该镜像的标签将会被删除。<br/><br/>删除选中的镜像标签?',
'close': '关闭',
'ok': '确认',
'welcome': '欢迎使用Harbor!',
@ -217,6 +218,7 @@ var locale_messages = {
'select_all': '全选',
'delete_tag': '删除镜像标签',
'delete_repo': '删除镜像仓库',
'delete_selected_tag': '删除选中镜像标签',
'download_log': '查看日志',
'edit': '修改',
'delete': '删除',

15
src/vendor/github.com/davecgh/go-spew/LICENSE generated vendored Normal file
View File

@ -0,0 +1,15 @@
ISC License
Copyright (c) 2012-2016 Dave Collins <dave@davec.name>
Permission to use, copy, modify, and distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

152
src/vendor/github.com/davecgh/go-spew/spew/bypass.go generated vendored Normal file
View File

@ -0,0 +1,152 @@
// Copyright (c) 2015-2016 Dave Collins <dave@davec.name>
//
// Permission to use, copy, modify, and distribute this software for any
// purpose with or without fee is hereby granted, provided that the above
// copyright notice and this permission notice appear in all copies.
//
// THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
// WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
// MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
// ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
// WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
// ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
// OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
// NOTE: Due to the following build constraints, this file will only be compiled
// when the code is not running on Google App Engine, compiled by GopherJS, and
// "-tags safe" is not added to the go build command line. The "disableunsafe"
// tag is deprecated and thus should not be used.
// +build !js,!appengine,!safe,!disableunsafe
package spew
import (
"reflect"
"unsafe"
)
const (
// UnsafeDisabled is a build-time constant which specifies whether or
// not access to the unsafe package is available.
UnsafeDisabled = false
// ptrSize is the size of a pointer on the current arch.
ptrSize = unsafe.Sizeof((*byte)(nil))
)
var (
// offsetPtr, offsetScalar, and offsetFlag are the offsets for the
// internal reflect.Value fields. These values are valid before golang
// commit ecccf07e7f9d which changed the format. The are also valid
// after commit 82f48826c6c7 which changed the format again to mirror
// the original format. Code in the init function updates these offsets
// as necessary.
offsetPtr = uintptr(ptrSize)
offsetScalar = uintptr(0)
offsetFlag = uintptr(ptrSize * 2)
// flagKindWidth and flagKindShift indicate various bits that the
// reflect package uses internally to track kind information.
//
// flagRO indicates whether or not the value field of a reflect.Value is
// read-only.
//
// flagIndir indicates whether the value field of a reflect.Value is
// the actual data or a pointer to the data.
//
// These values are valid before golang commit 90a7c3c86944 which
// changed their positions. Code in the init function updates these
// flags as necessary.
flagKindWidth = uintptr(5)
flagKindShift = uintptr(flagKindWidth - 1)
flagRO = uintptr(1 << 0)
flagIndir = uintptr(1 << 1)
)
func init() {
// Older versions of reflect.Value stored small integers directly in the
// ptr field (which is named val in the older versions). Versions
// between commits ecccf07e7f9d and 82f48826c6c7 added a new field named
// scalar for this purpose which unfortunately came before the flag
// field, so the offset of the flag field is different for those
// versions.
//
// This code constructs a new reflect.Value from a known small integer
// and checks if the size of the reflect.Value struct indicates it has
// the scalar field. When it does, the offsets are updated accordingly.
vv := reflect.ValueOf(0xf00)
if unsafe.Sizeof(vv) == (ptrSize * 4) {
offsetScalar = ptrSize * 2
offsetFlag = ptrSize * 3
}
// Commit 90a7c3c86944 changed the flag positions such that the low
// order bits are the kind. This code extracts the kind from the flags
// field and ensures it's the correct type. When it's not, the flag
// order has been changed to the newer format, so the flags are updated
// accordingly.
upf := unsafe.Pointer(uintptr(unsafe.Pointer(&vv)) + offsetFlag)
upfv := *(*uintptr)(upf)
flagKindMask := uintptr((1<<flagKindWidth - 1) << flagKindShift)
if (upfv&flagKindMask)>>flagKindShift != uintptr(reflect.Int) {
flagKindShift = 0
flagRO = 1 << 5
flagIndir = 1 << 6
// Commit adf9b30e5594 modified the flags to separate the
// flagRO flag into two bits which specifies whether or not the
// field is embedded. This causes flagIndir to move over a bit
// and means that flagRO is the combination of either of the
// original flagRO bit and the new bit.
//
// This code detects the change by extracting what used to be
// the indirect bit to ensure it's set. When it's not, the flag
// order has been changed to the newer format, so the flags are
// updated accordingly.
if upfv&flagIndir == 0 {
flagRO = 3 << 5
flagIndir = 1 << 7
}
}
}
// unsafeReflectValue converts the passed reflect.Value into a one that bypasses
// the typical safety restrictions preventing access to unaddressable and
// unexported data. It works by digging the raw pointer to the underlying
// value out of the protected value and generating a new unprotected (unsafe)
// reflect.Value to it.
//
// This allows us to check for implementations of the Stringer and error
// interfaces to be used for pretty printing ordinarily unaddressable and
// inaccessible values such as unexported struct fields.
func unsafeReflectValue(v reflect.Value) (rv reflect.Value) {
indirects := 1
vt := v.Type()
upv := unsafe.Pointer(uintptr(unsafe.Pointer(&v)) + offsetPtr)
rvf := *(*uintptr)(unsafe.Pointer(uintptr(unsafe.Pointer(&v)) + offsetFlag))
if rvf&flagIndir != 0 {
vt = reflect.PtrTo(v.Type())
indirects++
} else if offsetScalar != 0 {
// The value is in the scalar field when it's not one of the
// reference types.
switch vt.Kind() {
case reflect.Uintptr:
case reflect.Chan:
case reflect.Func:
case reflect.Map:
case reflect.Ptr:
case reflect.UnsafePointer:
default:
upv = unsafe.Pointer(uintptr(unsafe.Pointer(&v)) +
offsetScalar)
}
}
pv := reflect.NewAt(vt, upv)
rv = pv
for i := 0; i < indirects; i++ {
rv = rv.Elem()
}
return rv
}

38
src/vendor/github.com/davecgh/go-spew/spew/bypasssafe.go generated vendored Normal file
View File

@ -0,0 +1,38 @@
// Copyright (c) 2015-2016 Dave Collins <dave@davec.name>
//
// Permission to use, copy, modify, and distribute this software for any
// purpose with or without fee is hereby granted, provided that the above
// copyright notice and this permission notice appear in all copies.
//
// THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
// WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
// MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
// ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
// WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
// ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
// OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
// NOTE: Due to the following build constraints, this file will only be compiled
// when the code is running on Google App Engine, compiled by GopherJS, or
// "-tags safe" is added to the go build command line. The "disableunsafe"
// tag is deprecated and thus should not be used.
// +build js appengine safe disableunsafe
package spew
import "reflect"
const (
// UnsafeDisabled is a build-time constant which specifies whether or
// not access to the unsafe package is available.
UnsafeDisabled = true
)
// unsafeReflectValue typically converts the passed reflect.Value into a one
// that bypasses the typical safety restrictions preventing access to
// unaddressable and unexported data. However, doing this relies on access to
// the unsafe package. This is a stub version which simply returns the passed
// reflect.Value when the unsafe package is not available.
func unsafeReflectValue(v reflect.Value) reflect.Value {
return v
}

341
src/vendor/github.com/davecgh/go-spew/spew/common.go generated vendored Normal file
View File

@ -0,0 +1,341 @@
/*
* Copyright (c) 2013-2016 Dave Collins <dave@davec.name>
*
* Permission to use, copy, modify, and distribute this software for any
* purpose with or without fee is hereby granted, provided that the above
* copyright notice and this permission notice appear in all copies.
*
* THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
* WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
* MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
* ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
* WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
* ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
* OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
package spew
import (
"bytes"
"fmt"
"io"
"reflect"
"sort"
"strconv"
)
// Some constants in the form of bytes to avoid string overhead. This mirrors
// the technique used in the fmt package.
var (
panicBytes = []byte("(PANIC=")
plusBytes = []byte("+")
iBytes = []byte("i")
trueBytes = []byte("true")
falseBytes = []byte("false")
interfaceBytes = []byte("(interface {})")
commaNewlineBytes = []byte(",\n")
newlineBytes = []byte("\n")
openBraceBytes = []byte("{")
openBraceNewlineBytes = []byte("{\n")
closeBraceBytes = []byte("}")
asteriskBytes = []byte("*")
colonBytes = []byte(":")
colonSpaceBytes = []byte(": ")
openParenBytes = []byte("(")
closeParenBytes = []byte(")")
spaceBytes = []byte(" ")
pointerChainBytes = []byte("->")
nilAngleBytes = []byte("<nil>")
maxNewlineBytes = []byte("<max depth reached>\n")
maxShortBytes = []byte("<max>")
circularBytes = []byte("<already shown>")
circularShortBytes = []byte("<shown>")
invalidAngleBytes = []byte("<invalid>")
openBracketBytes = []byte("[")
closeBracketBytes = []byte("]")
percentBytes = []byte("%")
precisionBytes = []byte(".")
openAngleBytes = []byte("<")
closeAngleBytes = []byte(">")
openMapBytes = []byte("map[")
closeMapBytes = []byte("]")
lenEqualsBytes = []byte("len=")
capEqualsBytes = []byte("cap=")
)
// hexDigits is used to map a decimal value to a hex digit.
var hexDigits = "0123456789abcdef"
// catchPanic handles any panics that might occur during the handleMethods
// calls.
func catchPanic(w io.Writer, v reflect.Value) {
if err := recover(); err != nil {
w.Write(panicBytes)
fmt.Fprintf(w, "%v", err)
w.Write(closeParenBytes)
}
}
// handleMethods attempts to call the Error and String methods on the underlying
// type the passed reflect.Value represents and outputes the result to Writer w.
//
// It handles panics in any called methods by catching and displaying the error
// as the formatted value.
func handleMethods(cs *ConfigState, w io.Writer, v reflect.Value) (handled bool) {
// We need an interface to check if the type implements the error or
// Stringer interface. However, the reflect package won't give us an
// interface on certain things like unexported struct fields in order
// to enforce visibility rules. We use unsafe, when it's available,
// to bypass these restrictions since this package does not mutate the
// values.
if !v.CanInterface() {
if UnsafeDisabled {
return false
}
v = unsafeReflectValue(v)
}
// Choose whether or not to do error and Stringer interface lookups against
// the base type or a pointer to the base type depending on settings.
// Technically calling one of these methods with a pointer receiver can
// mutate the value, however, types which choose to satisify an error or
// Stringer interface with a pointer receiver should not be mutating their
// state inside these interface methods.
if !cs.DisablePointerMethods && !UnsafeDisabled && !v.CanAddr() {
v = unsafeReflectValue(v)
}
if v.CanAddr() {
v = v.Addr()
}
// Is it an error or Stringer?
switch iface := v.Interface().(type) {
case error:
defer catchPanic(w, v)
if cs.ContinueOnMethod {
w.Write(openParenBytes)
w.Write([]byte(iface.Error()))
w.Write(closeParenBytes)
w.Write(spaceBytes)
return false
}
w.Write([]byte(iface.Error()))
return true
case fmt.Stringer:
defer catchPanic(w, v)
if cs.ContinueOnMethod {
w.Write(openParenBytes)
w.Write([]byte(iface.String()))
w.Write(closeParenBytes)
w.Write(spaceBytes)
return false
}
w.Write([]byte(iface.String()))
return true
}
return false
}
// printBool outputs a boolean value as true or false to Writer w.
func printBool(w io.Writer, val bool) {
if val {
w.Write(trueBytes)
} else {
w.Write(falseBytes)
}
}
// printInt outputs a signed integer value to Writer w.
func printInt(w io.Writer, val int64, base int) {
w.Write([]byte(strconv.FormatInt(val, base)))
}
// printUint outputs an unsigned integer value to Writer w.
func printUint(w io.Writer, val uint64, base int) {
w.Write([]byte(strconv.FormatUint(val, base)))
}
// printFloat outputs a floating point value using the specified precision,
// which is expected to be 32 or 64bit, to Writer w.
func printFloat(w io.Writer, val float64, precision int) {
w.Write([]byte(strconv.FormatFloat(val, 'g', -1, precision)))
}
// printComplex outputs a complex value using the specified float precision
// for the real and imaginary parts to Writer w.
func printComplex(w io.Writer, c complex128, floatPrecision int) {
r := real(c)
w.Write(openParenBytes)
w.Write([]byte(strconv.FormatFloat(r, 'g', -1, floatPrecision)))
i := imag(c)
if i >= 0 {
w.Write(plusBytes)
}
w.Write([]byte(strconv.FormatFloat(i, 'g', -1, floatPrecision)))
w.Write(iBytes)
w.Write(closeParenBytes)
}
// printHexPtr outputs a uintptr formatted as hexidecimal with a leading '0x'
// prefix to Writer w.
func printHexPtr(w io.Writer, p uintptr) {
// Null pointer.
num := uint64(p)
if num == 0 {
w.Write(nilAngleBytes)
return
}
// Max uint64 is 16 bytes in hex + 2 bytes for '0x' prefix
buf := make([]byte, 18)
// It's simpler to construct the hex string right to left.
base := uint64(16)
i := len(buf) - 1
for num >= base {
buf[i] = hexDigits[num%base]
num /= base
i--
}
buf[i] = hexDigits[num]
// Add '0x' prefix.
i--
buf[i] = 'x'
i--
buf[i] = '0'
// Strip unused leading bytes.
buf = buf[i:]
w.Write(buf)
}
// valuesSorter implements sort.Interface to allow a slice of reflect.Value
// elements to be sorted.
type valuesSorter struct {
values []reflect.Value
strings []string // either nil or same len and values
cs *ConfigState
}
// newValuesSorter initializes a valuesSorter instance, which holds a set of
// surrogate keys on which the data should be sorted. It uses flags in
// ConfigState to decide if and how to populate those surrogate keys.
func newValuesSorter(values []reflect.Value, cs *ConfigState) sort.Interface {
vs := &valuesSorter{values: values, cs: cs}
if canSortSimply(vs.values[0].Kind()) {
return vs
}
if !cs.DisableMethods {
vs.strings = make([]string, len(values))
for i := range vs.values {
b := bytes.Buffer{}
if !handleMethods(cs, &b, vs.values[i]) {
vs.strings = nil
break
}
vs.strings[i] = b.String()
}
}
if vs.strings == nil && cs.SpewKeys {
vs.strings = make([]string, len(values))
for i := range vs.values {
vs.strings[i] = Sprintf("%#v", vs.values[i].Interface())
}
}
return vs
}
// canSortSimply tests whether a reflect.Kind is a primitive that can be sorted
// directly, or whether it should be considered for sorting by surrogate keys
// (if the ConfigState allows it).
func canSortSimply(kind reflect.Kind) bool {
// This switch parallels valueSortLess, except for the default case.
switch kind {
case reflect.Bool:
return true
case reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64, reflect.Int:
return true
case reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uint:
return true
case reflect.Float32, reflect.Float64:
return true
case reflect.String:
return true
case reflect.Uintptr:
return true
case reflect.Array:
return true
}
return false
}
// Len returns the number of values in the slice. It is part of the
// sort.Interface implementation.
func (s *valuesSorter) Len() int {
return len(s.values)
}
// Swap swaps the values at the passed indices. It is part of the
// sort.Interface implementation.
func (s *valuesSorter) Swap(i, j int) {
s.values[i], s.values[j] = s.values[j], s.values[i]
if s.strings != nil {
s.strings[i], s.strings[j] = s.strings[j], s.strings[i]
}
}
// valueSortLess returns whether the first value should sort before the second
// value. It is used by valueSorter.Less as part of the sort.Interface
// implementation.
func valueSortLess(a, b reflect.Value) bool {
switch a.Kind() {
case reflect.Bool:
return !a.Bool() && b.Bool()
case reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64, reflect.Int:
return a.Int() < b.Int()
case reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uint:
return a.Uint() < b.Uint()
case reflect.Float32, reflect.Float64:
return a.Float() < b.Float()
case reflect.String:
return a.String() < b.String()
case reflect.Uintptr:
return a.Uint() < b.Uint()
case reflect.Array:
// Compare the contents of both arrays.
l := a.Len()
for i := 0; i < l; i++ {
av := a.Index(i)
bv := b.Index(i)
if av.Interface() == bv.Interface() {
continue
}
return valueSortLess(av, bv)
}
}
return a.String() < b.String()
}
// Less returns whether the value at index i should sort before the
// value at index j. It is part of the sort.Interface implementation.
func (s *valuesSorter) Less(i, j int) bool {
if s.strings == nil {
return valueSortLess(s.values[i], s.values[j])
}
return s.strings[i] < s.strings[j]
}
// sortValues is a sort function that handles both native types and any type that
// can be converted to error or Stringer. Other inputs are sorted according to
// their Value.String() value to ensure display stability.
func sortValues(values []reflect.Value, cs *ConfigState) {
if len(values) == 0 {
return
}
sort.Sort(newValuesSorter(values, cs))
}

306
src/vendor/github.com/davecgh/go-spew/spew/config.go generated vendored Normal file
View File

@ -0,0 +1,306 @@
/*
* Copyright (c) 2013-2016 Dave Collins <dave@davec.name>
*
* Permission to use, copy, modify, and distribute this software for any
* purpose with or without fee is hereby granted, provided that the above
* copyright notice and this permission notice appear in all copies.
*
* THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
* WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
* MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
* ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
* WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
* ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
* OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
package spew
import (
"bytes"
"fmt"
"io"
"os"
)
// ConfigState houses the configuration options used by spew to format and
// display values. There is a global instance, Config, that is used to control
// all top-level Formatter and Dump functionality. Each ConfigState instance
// provides methods equivalent to the top-level functions.
//
// The zero value for ConfigState provides no indentation. You would typically
// want to set it to a space or a tab.
//
// Alternatively, you can use NewDefaultConfig to get a ConfigState instance
// with default settings. See the documentation of NewDefaultConfig for default
// values.
type ConfigState struct {
// Indent specifies the string to use for each indentation level. The
// global config instance that all top-level functions use set this to a
// single space by default. If you would like more indentation, you might
// set this to a tab with "\t" or perhaps two spaces with " ".
Indent string
// MaxDepth controls the maximum number of levels to descend into nested
// data structures. The default, 0, means there is no limit.
//
// NOTE: Circular data structures are properly detected, so it is not
// necessary to set this value unless you specifically want to limit deeply
// nested data structures.
MaxDepth int
// DisableMethods specifies whether or not error and Stringer interfaces are
// invoked for types that implement them.
DisableMethods bool
// DisablePointerMethods specifies whether or not to check for and invoke
// error and Stringer interfaces on types which only accept a pointer
// receiver when the current type is not a pointer.
//
// NOTE: This might be an unsafe action since calling one of these methods
// with a pointer receiver could technically mutate the value, however,
// in practice, types which choose to satisify an error or Stringer
// interface with a pointer receiver should not be mutating their state
// inside these interface methods. As a result, this option relies on
// access to the unsafe package, so it will not have any effect when
// running in environments without access to the unsafe package such as
// Google App Engine or with the "safe" build tag specified.
DisablePointerMethods bool
// DisablePointerAddresses specifies whether to disable the printing of
// pointer addresses. This is useful when diffing data structures in tests.
DisablePointerAddresses bool
// DisableCapacities specifies whether to disable the printing of capacities
// for arrays, slices, maps and channels. This is useful when diffing
// data structures in tests.
DisableCapacities bool
// ContinueOnMethod specifies whether or not recursion should continue once
// a custom error or Stringer interface is invoked. The default, false,
// means it will print the results of invoking the custom error or Stringer
// interface and return immediately instead of continuing to recurse into
// the internals of the data type.
//
// NOTE: This flag does not have any effect if method invocation is disabled
// via the DisableMethods or DisablePointerMethods options.
ContinueOnMethod bool
// SortKeys specifies map keys should be sorted before being printed. Use
// this to have a more deterministic, diffable output. Note that only
// native types (bool, int, uint, floats, uintptr and string) and types
// that support the error or Stringer interfaces (if methods are
// enabled) are supported, with other types sorted according to the
// reflect.Value.String() output which guarantees display stability.
SortKeys bool
// SpewKeys specifies that, as a last resort attempt, map keys should
// be spewed to strings and sorted by those strings. This is only
// considered if SortKeys is true.
SpewKeys bool
}
// Config is the active configuration of the top-level functions.
// The configuration can be changed by modifying the contents of spew.Config.
var Config = ConfigState{Indent: " "}
// Errorf is a wrapper for fmt.Errorf that treats each argument as if it were
// passed with a Formatter interface returned by c.NewFormatter. It returns
// the formatted string as a value that satisfies error. See NewFormatter
// for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Errorf(format, c.NewFormatter(a), c.NewFormatter(b))
func (c *ConfigState) Errorf(format string, a ...interface{}) (err error) {
return fmt.Errorf(format, c.convertArgs(a)...)
}
// Fprint is a wrapper for fmt.Fprint that treats each argument as if it were
// passed with a Formatter interface returned by c.NewFormatter. It returns
// the number of bytes written and any write error encountered. See
// NewFormatter for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Fprint(w, c.NewFormatter(a), c.NewFormatter(b))
func (c *ConfigState) Fprint(w io.Writer, a ...interface{}) (n int, err error) {
return fmt.Fprint(w, c.convertArgs(a)...)
}
// Fprintf is a wrapper for fmt.Fprintf that treats each argument as if it were
// passed with a Formatter interface returned by c.NewFormatter. It returns
// the number of bytes written and any write error encountered. See
// NewFormatter for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Fprintf(w, format, c.NewFormatter(a), c.NewFormatter(b))
func (c *ConfigState) Fprintf(w io.Writer, format string, a ...interface{}) (n int, err error) {
return fmt.Fprintf(w, format, c.convertArgs(a)...)
}
// Fprintln is a wrapper for fmt.Fprintln that treats each argument as if it
// passed with a Formatter interface returned by c.NewFormatter. See
// NewFormatter for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Fprintln(w, c.NewFormatter(a), c.NewFormatter(b))
func (c *ConfigState) Fprintln(w io.Writer, a ...interface{}) (n int, err error) {
return fmt.Fprintln(w, c.convertArgs(a)...)
}
// Print is a wrapper for fmt.Print that treats each argument as if it were
// passed with a Formatter interface returned by c.NewFormatter. It returns
// the number of bytes written and any write error encountered. See
// NewFormatter for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Print(c.NewFormatter(a), c.NewFormatter(b))
func (c *ConfigState) Print(a ...interface{}) (n int, err error) {
return fmt.Print(c.convertArgs(a)...)
}
// Printf is a wrapper for fmt.Printf that treats each argument as if it were
// passed with a Formatter interface returned by c.NewFormatter. It returns
// the number of bytes written and any write error encountered. See
// NewFormatter for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Printf(format, c.NewFormatter(a), c.NewFormatter(b))
func (c *ConfigState) Printf(format string, a ...interface{}) (n int, err error) {
return fmt.Printf(format, c.convertArgs(a)...)
}
// Println is a wrapper for fmt.Println that treats each argument as if it were
// passed with a Formatter interface returned by c.NewFormatter. It returns
// the number of bytes written and any write error encountered. See
// NewFormatter for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Println(c.NewFormatter(a), c.NewFormatter(b))
func (c *ConfigState) Println(a ...interface{}) (n int, err error) {
return fmt.Println(c.convertArgs(a)...)
}
// Sprint is a wrapper for fmt.Sprint that treats each argument as if it were
// passed with a Formatter interface returned by c.NewFormatter. It returns
// the resulting string. See NewFormatter for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Sprint(c.NewFormatter(a), c.NewFormatter(b))
func (c *ConfigState) Sprint(a ...interface{}) string {
return fmt.Sprint(c.convertArgs(a)...)
}
// Sprintf is a wrapper for fmt.Sprintf that treats each argument as if it were
// passed with a Formatter interface returned by c.NewFormatter. It returns
// the resulting string. See NewFormatter for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Sprintf(format, c.NewFormatter(a), c.NewFormatter(b))
func (c *ConfigState) Sprintf(format string, a ...interface{}) string {
return fmt.Sprintf(format, c.convertArgs(a)...)
}
// Sprintln is a wrapper for fmt.Sprintln that treats each argument as if it
// were passed with a Formatter interface returned by c.NewFormatter. It
// returns the resulting string. See NewFormatter for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Sprintln(c.NewFormatter(a), c.NewFormatter(b))
func (c *ConfigState) Sprintln(a ...interface{}) string {
return fmt.Sprintln(c.convertArgs(a)...)
}
/*
NewFormatter returns a custom formatter that satisfies the fmt.Formatter
interface. As a result, it integrates cleanly with standard fmt package
printing functions. The formatter is useful for inline printing of smaller data
types similar to the standard %v format specifier.
The custom formatter only responds to the %v (most compact), %+v (adds pointer
addresses), %#v (adds types), and %#+v (adds types and pointer addresses) verb
combinations. Any other verbs such as %x and %q will be sent to the the
standard fmt package for formatting. In addition, the custom formatter ignores
the width and precision arguments (however they will still work on the format
specifiers not handled by the custom formatter).
Typically this function shouldn't be called directly. It is much easier to make
use of the custom formatter by calling one of the convenience functions such as
c.Printf, c.Println, or c.Printf.
*/
func (c *ConfigState) NewFormatter(v interface{}) fmt.Formatter {
return newFormatter(c, v)
}
// Fdump formats and displays the passed arguments to io.Writer w. It formats
// exactly the same as Dump.
func (c *ConfigState) Fdump(w io.Writer, a ...interface{}) {
fdump(c, w, a...)
}
/*
Dump displays the passed parameters to standard out with newlines, customizable
indentation, and additional debug information such as complete types and all
pointer addresses used to indirect to the final value. It provides the
following features over the built-in printing facilities provided by the fmt
package:
* Pointers are dereferenced and followed
* Circular data structures are detected and handled properly
* Custom Stringer/error interfaces are optionally invoked, including
on unexported types
* Custom types which only implement the Stringer/error interfaces via
a pointer receiver are optionally invoked when passing non-pointer
variables
* Byte arrays and slices are dumped like the hexdump -C command which
includes offsets, byte values in hex, and ASCII output
The configuration options are controlled by modifying the public members
of c. See ConfigState for options documentation.
See Fdump if you would prefer dumping to an arbitrary io.Writer or Sdump to
get the formatted result as a string.
*/
func (c *ConfigState) Dump(a ...interface{}) {
fdump(c, os.Stdout, a...)
}
// Sdump returns a string with the passed arguments formatted exactly the same
// as Dump.
func (c *ConfigState) Sdump(a ...interface{}) string {
var buf bytes.Buffer
fdump(c, &buf, a...)
return buf.String()
}
// convertArgs accepts a slice of arguments and returns a slice of the same
// length with each argument converted to a spew Formatter interface using
// the ConfigState associated with s.
func (c *ConfigState) convertArgs(args []interface{}) (formatters []interface{}) {
formatters = make([]interface{}, len(args))
for index, arg := range args {
formatters[index] = newFormatter(c, arg)
}
return formatters
}
// NewDefaultConfig returns a ConfigState with the following default settings.
//
// Indent: " "
// MaxDepth: 0
// DisableMethods: false
// DisablePointerMethods: false
// ContinueOnMethod: false
// SortKeys: false
func NewDefaultConfig() *ConfigState {
return &ConfigState{Indent: " "}
}

211
src/vendor/github.com/davecgh/go-spew/spew/doc.go generated vendored Normal file
View File

@ -0,0 +1,211 @@
/*
* Copyright (c) 2013-2016 Dave Collins <dave@davec.name>
*
* Permission to use, copy, modify, and distribute this software for any
* purpose with or without fee is hereby granted, provided that the above
* copyright notice and this permission notice appear in all copies.
*
* THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
* WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
* MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
* ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
* WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
* ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
* OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
/*
Package spew implements a deep pretty printer for Go data structures to aid in
debugging.
A quick overview of the additional features spew provides over the built-in
printing facilities for Go data types are as follows:
* Pointers are dereferenced and followed
* Circular data structures are detected and handled properly
* Custom Stringer/error interfaces are optionally invoked, including
on unexported types
* Custom types which only implement the Stringer/error interfaces via
a pointer receiver are optionally invoked when passing non-pointer
variables
* Byte arrays and slices are dumped like the hexdump -C command which
includes offsets, byte values in hex, and ASCII output (only when using
Dump style)
There are two different approaches spew allows for dumping Go data structures:
* Dump style which prints with newlines, customizable indentation,
and additional debug information such as types and all pointer addresses
used to indirect to the final value
* A custom Formatter interface that integrates cleanly with the standard fmt
package and replaces %v, %+v, %#v, and %#+v to provide inline printing
similar to the default %v while providing the additional functionality
outlined above and passing unsupported format verbs such as %x and %q
along to fmt
Quick Start
This section demonstrates how to quickly get started with spew. See the
sections below for further details on formatting and configuration options.
To dump a variable with full newlines, indentation, type, and pointer
information use Dump, Fdump, or Sdump:
spew.Dump(myVar1, myVar2, ...)
spew.Fdump(someWriter, myVar1, myVar2, ...)
str := spew.Sdump(myVar1, myVar2, ...)
Alternatively, if you would prefer to use format strings with a compacted inline
printing style, use the convenience wrappers Printf, Fprintf, etc with
%v (most compact), %+v (adds pointer addresses), %#v (adds types), or
%#+v (adds types and pointer addresses):
spew.Printf("myVar1: %v -- myVar2: %+v", myVar1, myVar2)
spew.Printf("myVar3: %#v -- myVar4: %#+v", myVar3, myVar4)
spew.Fprintf(someWriter, "myVar1: %v -- myVar2: %+v", myVar1, myVar2)
spew.Fprintf(someWriter, "myVar3: %#v -- myVar4: %#+v", myVar3, myVar4)
Configuration Options
Configuration of spew is handled by fields in the ConfigState type. For
convenience, all of the top-level functions use a global state available
via the spew.Config global.
It is also possible to create a ConfigState instance that provides methods
equivalent to the top-level functions. This allows concurrent configuration
options. See the ConfigState documentation for more details.
The following configuration options are available:
* Indent
String to use for each indentation level for Dump functions.
It is a single space by default. A popular alternative is "\t".
* MaxDepth
Maximum number of levels to descend into nested data structures.
There is no limit by default.
* DisableMethods
Disables invocation of error and Stringer interface methods.
Method invocation is enabled by default.
* DisablePointerMethods
Disables invocation of error and Stringer interface methods on types
which only accept pointer receivers from non-pointer variables.
Pointer method invocation is enabled by default.
* DisablePointerAddresses
DisablePointerAddresses specifies whether to disable the printing of
pointer addresses. This is useful when diffing data structures in tests.
* DisableCapacities
DisableCapacities specifies whether to disable the printing of
capacities for arrays, slices, maps and channels. This is useful when
diffing data structures in tests.
* ContinueOnMethod
Enables recursion into types after invoking error and Stringer interface
methods. Recursion after method invocation is disabled by default.
* SortKeys
Specifies map keys should be sorted before being printed. Use
this to have a more deterministic, diffable output. Note that
only native types (bool, int, uint, floats, uintptr and string)
and types which implement error or Stringer interfaces are
supported with other types sorted according to the
reflect.Value.String() output which guarantees display
stability. Natural map order is used by default.
* SpewKeys
Specifies that, as a last resort attempt, map keys should be
spewed to strings and sorted by those strings. This is only
considered if SortKeys is true.
Dump Usage
Simply call spew.Dump with a list of variables you want to dump:
spew.Dump(myVar1, myVar2, ...)
You may also call spew.Fdump if you would prefer to output to an arbitrary
io.Writer. For example, to dump to standard error:
spew.Fdump(os.Stderr, myVar1, myVar2, ...)
A third option is to call spew.Sdump to get the formatted output as a string:
str := spew.Sdump(myVar1, myVar2, ...)
Sample Dump Output
See the Dump example for details on the setup of the types and variables being
shown here.
(main.Foo) {
unexportedField: (*main.Bar)(0xf84002e210)({
flag: (main.Flag) flagTwo,
data: (uintptr) <nil>
}),
ExportedField: (map[interface {}]interface {}) (len=1) {
(string) (len=3) "one": (bool) true
}
}
Byte (and uint8) arrays and slices are displayed uniquely like the hexdump -C
command as shown.
([]uint8) (len=32 cap=32) {
00000000 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 |............... |
00000010 21 22 23 24 25 26 27 28 29 2a 2b 2c 2d 2e 2f 30 |!"#$%&'()*+,-./0|
00000020 31 32 |12|
}
Custom Formatter
Spew provides a custom formatter that implements the fmt.Formatter interface
so that it integrates cleanly with standard fmt package printing functions. The
formatter is useful for inline printing of smaller data types similar to the
standard %v format specifier.
The custom formatter only responds to the %v (most compact), %+v (adds pointer
addresses), %#v (adds types), or %#+v (adds types and pointer addresses) verb
combinations. Any other verbs such as %x and %q will be sent to the the
standard fmt package for formatting. In addition, the custom formatter ignores
the width and precision arguments (however they will still work on the format
specifiers not handled by the custom formatter).
Custom Formatter Usage
The simplest way to make use of the spew custom formatter is to call one of the
convenience functions such as spew.Printf, spew.Println, or spew.Printf. The
functions have syntax you are most likely already familiar with:
spew.Printf("myVar1: %v -- myVar2: %+v", myVar1, myVar2)
spew.Printf("myVar3: %#v -- myVar4: %#+v", myVar3, myVar4)
spew.Println(myVar, myVar2)
spew.Fprintf(os.Stderr, "myVar1: %v -- myVar2: %+v", myVar1, myVar2)
spew.Fprintf(os.Stderr, "myVar3: %#v -- myVar4: %#+v", myVar3, myVar4)
See the Index for the full list convenience functions.
Sample Formatter Output
Double pointer to a uint8:
%v: <**>5
%+v: <**>(0xf8400420d0->0xf8400420c8)5
%#v: (**uint8)5
%#+v: (**uint8)(0xf8400420d0->0xf8400420c8)5
Pointer to circular struct with a uint8 field and a pointer to itself:
%v: <*>{1 <*><shown>}
%+v: <*>(0xf84003e260){ui8:1 c:<*>(0xf84003e260)<shown>}
%#v: (*main.circular){ui8:(uint8)1 c:(*main.circular)<shown>}
%#+v: (*main.circular)(0xf84003e260){ui8:(uint8)1 c:(*main.circular)(0xf84003e260)<shown>}
See the Printf example for details on the setup of variables being shown
here.
Errors
Since it is possible for custom Stringer/error interfaces to panic, spew
detects them and handles them internally by printing the panic information
inline with the output. Since spew is intended to provide deep pretty printing
capabilities on structures, it intentionally does not return any errors.
*/
package spew

509
src/vendor/github.com/davecgh/go-spew/spew/dump.go generated vendored Normal file
View File

@ -0,0 +1,509 @@
/*
* Copyright (c) 2013-2016 Dave Collins <dave@davec.name>
*
* Permission to use, copy, modify, and distribute this software for any
* purpose with or without fee is hereby granted, provided that the above
* copyright notice and this permission notice appear in all copies.
*
* THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
* WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
* MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
* ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
* WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
* ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
* OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
package spew
import (
"bytes"
"encoding/hex"
"fmt"
"io"
"os"
"reflect"
"regexp"
"strconv"
"strings"
)
var (
// uint8Type is a reflect.Type representing a uint8. It is used to
// convert cgo types to uint8 slices for hexdumping.
uint8Type = reflect.TypeOf(uint8(0))
// cCharRE is a regular expression that matches a cgo char.
// It is used to detect character arrays to hexdump them.
cCharRE = regexp.MustCompile("^.*\\._Ctype_char$")
// cUnsignedCharRE is a regular expression that matches a cgo unsigned
// char. It is used to detect unsigned character arrays to hexdump
// them.
cUnsignedCharRE = regexp.MustCompile("^.*\\._Ctype_unsignedchar$")
// cUint8tCharRE is a regular expression that matches a cgo uint8_t.
// It is used to detect uint8_t arrays to hexdump them.
cUint8tCharRE = regexp.MustCompile("^.*\\._Ctype_uint8_t$")
)
// dumpState contains information about the state of a dump operation.
type dumpState struct {
w io.Writer
depth int
pointers map[uintptr]int
ignoreNextType bool
ignoreNextIndent bool
cs *ConfigState
}
// indent performs indentation according to the depth level and cs.Indent
// option.
func (d *dumpState) indent() {
if d.ignoreNextIndent {
d.ignoreNextIndent = false
return
}
d.w.Write(bytes.Repeat([]byte(d.cs.Indent), d.depth))
}
// unpackValue returns values inside of non-nil interfaces when possible.
// This is useful for data types like structs, arrays, slices, and maps which
// can contain varying types packed inside an interface.
func (d *dumpState) unpackValue(v reflect.Value) reflect.Value {
if v.Kind() == reflect.Interface && !v.IsNil() {
v = v.Elem()
}
return v
}
// dumpPtr handles formatting of pointers by indirecting them as necessary.
func (d *dumpState) dumpPtr(v reflect.Value) {
// Remove pointers at or below the current depth from map used to detect
// circular refs.
for k, depth := range d.pointers {
if depth >= d.depth {
delete(d.pointers, k)
}
}
// Keep list of all dereferenced pointers to show later.
pointerChain := make([]uintptr, 0)
// Figure out how many levels of indirection there are by dereferencing
// pointers and unpacking interfaces down the chain while detecting circular
// references.
nilFound := false
cycleFound := false
indirects := 0
ve := v
for ve.Kind() == reflect.Ptr {
if ve.IsNil() {
nilFound = true
break
}
indirects++
addr := ve.Pointer()
pointerChain = append(pointerChain, addr)
if pd, ok := d.pointers[addr]; ok && pd < d.depth {
cycleFound = true
indirects--
break
}
d.pointers[addr] = d.depth
ve = ve.Elem()
if ve.Kind() == reflect.Interface {
if ve.IsNil() {
nilFound = true
break
}
ve = ve.Elem()
}
}
// Display type information.
d.w.Write(openParenBytes)
d.w.Write(bytes.Repeat(asteriskBytes, indirects))
d.w.Write([]byte(ve.Type().String()))
d.w.Write(closeParenBytes)
// Display pointer information.
if !d.cs.DisablePointerAddresses && len(pointerChain) > 0 {
d.w.Write(openParenBytes)
for i, addr := range pointerChain {
if i > 0 {
d.w.Write(pointerChainBytes)
}
printHexPtr(d.w, addr)
}
d.w.Write(closeParenBytes)
}
// Display dereferenced value.
d.w.Write(openParenBytes)
switch {
case nilFound == true:
d.w.Write(nilAngleBytes)
case cycleFound == true:
d.w.Write(circularBytes)
default:
d.ignoreNextType = true
d.dump(ve)
}
d.w.Write(closeParenBytes)
}
// dumpSlice handles formatting of arrays and slices. Byte (uint8 under
// reflection) arrays and slices are dumped in hexdump -C fashion.
func (d *dumpState) dumpSlice(v reflect.Value) {
// Determine whether this type should be hex dumped or not. Also,
// for types which should be hexdumped, try to use the underlying data
// first, then fall back to trying to convert them to a uint8 slice.
var buf []uint8
doConvert := false
doHexDump := false
numEntries := v.Len()
if numEntries > 0 {
vt := v.Index(0).Type()
vts := vt.String()
switch {
// C types that need to be converted.
case cCharRE.MatchString(vts):
fallthrough
case cUnsignedCharRE.MatchString(vts):
fallthrough
case cUint8tCharRE.MatchString(vts):
doConvert = true
// Try to use existing uint8 slices and fall back to converting
// and copying if that fails.
case vt.Kind() == reflect.Uint8:
// We need an addressable interface to convert the type
// to a byte slice. However, the reflect package won't
// give us an interface on certain things like
// unexported struct fields in order to enforce
// visibility rules. We use unsafe, when available, to
// bypass these restrictions since this package does not
// mutate the values.
vs := v
if !vs.CanInterface() || !vs.CanAddr() {
vs = unsafeReflectValue(vs)
}
if !UnsafeDisabled {
vs = vs.Slice(0, numEntries)
// Use the existing uint8 slice if it can be
// type asserted.
iface := vs.Interface()
if slice, ok := iface.([]uint8); ok {
buf = slice
doHexDump = true
break
}
}
// The underlying data needs to be converted if it can't
// be type asserted to a uint8 slice.
doConvert = true
}
// Copy and convert the underlying type if needed.
if doConvert && vt.ConvertibleTo(uint8Type) {
// Convert and copy each element into a uint8 byte
// slice.
buf = make([]uint8, numEntries)
for i := 0; i < numEntries; i++ {
vv := v.Index(i)
buf[i] = uint8(vv.Convert(uint8Type).Uint())
}
doHexDump = true
}
}
// Hexdump the entire slice as needed.
if doHexDump {
indent := strings.Repeat(d.cs.Indent, d.depth)
str := indent + hex.Dump(buf)
str = strings.Replace(str, "\n", "\n"+indent, -1)
str = strings.TrimRight(str, d.cs.Indent)
d.w.Write([]byte(str))
return
}
// Recursively call dump for each item.
for i := 0; i < numEntries; i++ {
d.dump(d.unpackValue(v.Index(i)))
if i < (numEntries - 1) {
d.w.Write(commaNewlineBytes)
} else {
d.w.Write(newlineBytes)
}
}
}
// dump is the main workhorse for dumping a value. It uses the passed reflect
// value to figure out what kind of object we are dealing with and formats it
// appropriately. It is a recursive function, however circular data structures
// are detected and handled properly.
func (d *dumpState) dump(v reflect.Value) {
// Handle invalid reflect values immediately.
kind := v.Kind()
if kind == reflect.Invalid {
d.w.Write(invalidAngleBytes)
return
}
// Handle pointers specially.
if kind == reflect.Ptr {
d.indent()
d.dumpPtr(v)
return
}
// Print type information unless already handled elsewhere.
if !d.ignoreNextType {
d.indent()
d.w.Write(openParenBytes)
d.w.Write([]byte(v.Type().String()))
d.w.Write(closeParenBytes)
d.w.Write(spaceBytes)
}
d.ignoreNextType = false
// Display length and capacity if the built-in len and cap functions
// work with the value's kind and the len/cap itself is non-zero.
valueLen, valueCap := 0, 0
switch v.Kind() {
case reflect.Array, reflect.Slice, reflect.Chan:
valueLen, valueCap = v.Len(), v.Cap()
case reflect.Map, reflect.String:
valueLen = v.Len()
}
if valueLen != 0 || !d.cs.DisableCapacities && valueCap != 0 {
d.w.Write(openParenBytes)
if valueLen != 0 {
d.w.Write(lenEqualsBytes)
printInt(d.w, int64(valueLen), 10)
}
if !d.cs.DisableCapacities && valueCap != 0 {
if valueLen != 0 {
d.w.Write(spaceBytes)
}
d.w.Write(capEqualsBytes)
printInt(d.w, int64(valueCap), 10)
}
d.w.Write(closeParenBytes)
d.w.Write(spaceBytes)
}
// Call Stringer/error interfaces if they exist and the handle methods flag
// is enabled
if !d.cs.DisableMethods {
if (kind != reflect.Invalid) && (kind != reflect.Interface) {
if handled := handleMethods(d.cs, d.w, v); handled {
return
}
}
}
switch kind {
case reflect.Invalid:
// Do nothing. We should never get here since invalid has already
// been handled above.
case reflect.Bool:
printBool(d.w, v.Bool())
case reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64, reflect.Int:
printInt(d.w, v.Int(), 10)
case reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uint:
printUint(d.w, v.Uint(), 10)
case reflect.Float32:
printFloat(d.w, v.Float(), 32)
case reflect.Float64:
printFloat(d.w, v.Float(), 64)
case reflect.Complex64:
printComplex(d.w, v.Complex(), 32)
case reflect.Complex128:
printComplex(d.w, v.Complex(), 64)
case reflect.Slice:
if v.IsNil() {
d.w.Write(nilAngleBytes)
break
}
fallthrough
case reflect.Array:
d.w.Write(openBraceNewlineBytes)
d.depth++
if (d.cs.MaxDepth != 0) && (d.depth > d.cs.MaxDepth) {
d.indent()
d.w.Write(maxNewlineBytes)
} else {
d.dumpSlice(v)
}
d.depth--
d.indent()
d.w.Write(closeBraceBytes)
case reflect.String:
d.w.Write([]byte(strconv.Quote(v.String())))
case reflect.Interface:
// The only time we should get here is for nil interfaces due to
// unpackValue calls.
if v.IsNil() {
d.w.Write(nilAngleBytes)
}
case reflect.Ptr:
// Do nothing. We should never get here since pointers have already
// been handled above.
case reflect.Map:
// nil maps should be indicated as different than empty maps
if v.IsNil() {
d.w.Write(nilAngleBytes)
break
}
d.w.Write(openBraceNewlineBytes)
d.depth++
if (d.cs.MaxDepth != 0) && (d.depth > d.cs.MaxDepth) {
d.indent()
d.w.Write(maxNewlineBytes)
} else {
numEntries := v.Len()
keys := v.MapKeys()
if d.cs.SortKeys {
sortValues(keys, d.cs)
}
for i, key := range keys {
d.dump(d.unpackValue(key))
d.w.Write(colonSpaceBytes)
d.ignoreNextIndent = true
d.dump(d.unpackValue(v.MapIndex(key)))
if i < (numEntries - 1) {
d.w.Write(commaNewlineBytes)
} else {
d.w.Write(newlineBytes)
}
}
}
d.depth--
d.indent()
d.w.Write(closeBraceBytes)
case reflect.Struct:
d.w.Write(openBraceNewlineBytes)
d.depth++
if (d.cs.MaxDepth != 0) && (d.depth > d.cs.MaxDepth) {
d.indent()
d.w.Write(maxNewlineBytes)
} else {
vt := v.Type()
numFields := v.NumField()
for i := 0; i < numFields; i++ {
d.indent()
vtf := vt.Field(i)
d.w.Write([]byte(vtf.Name))
d.w.Write(colonSpaceBytes)
d.ignoreNextIndent = true
d.dump(d.unpackValue(v.Field(i)))
if i < (numFields - 1) {
d.w.Write(commaNewlineBytes)
} else {
d.w.Write(newlineBytes)
}
}
}
d.depth--
d.indent()
d.w.Write(closeBraceBytes)
case reflect.Uintptr:
printHexPtr(d.w, uintptr(v.Uint()))
case reflect.UnsafePointer, reflect.Chan, reflect.Func:
printHexPtr(d.w, v.Pointer())
// There were not any other types at the time this code was written, but
// fall back to letting the default fmt package handle it in case any new
// types are added.
default:
if v.CanInterface() {
fmt.Fprintf(d.w, "%v", v.Interface())
} else {
fmt.Fprintf(d.w, "%v", v.String())
}
}
}
// fdump is a helper function to consolidate the logic from the various public
// methods which take varying writers and config states.
func fdump(cs *ConfigState, w io.Writer, a ...interface{}) {
for _, arg := range a {
if arg == nil {
w.Write(interfaceBytes)
w.Write(spaceBytes)
w.Write(nilAngleBytes)
w.Write(newlineBytes)
continue
}
d := dumpState{w: w, cs: cs}
d.pointers = make(map[uintptr]int)
d.dump(reflect.ValueOf(arg))
d.w.Write(newlineBytes)
}
}
// Fdump formats and displays the passed arguments to io.Writer w. It formats
// exactly the same as Dump.
func Fdump(w io.Writer, a ...interface{}) {
fdump(&Config, w, a...)
}
// Sdump returns a string with the passed arguments formatted exactly the same
// as Dump.
func Sdump(a ...interface{}) string {
var buf bytes.Buffer
fdump(&Config, &buf, a...)
return buf.String()
}
/*
Dump displays the passed parameters to standard out with newlines, customizable
indentation, and additional debug information such as complete types and all
pointer addresses used to indirect to the final value. It provides the
following features over the built-in printing facilities provided by the fmt
package:
* Pointers are dereferenced and followed
* Circular data structures are detected and handled properly
* Custom Stringer/error interfaces are optionally invoked, including
on unexported types
* Custom types which only implement the Stringer/error interfaces via
a pointer receiver are optionally invoked when passing non-pointer
variables
* Byte arrays and slices are dumped like the hexdump -C command which
includes offsets, byte values in hex, and ASCII output
The configuration options are controlled by an exported package global,
spew.Config. See ConfigState for options documentation.
See Fdump if you would prefer dumping to an arbitrary io.Writer or Sdump to
get the formatted result as a string.
*/
func Dump(a ...interface{}) {
fdump(&Config, os.Stdout, a...)
}

419
src/vendor/github.com/davecgh/go-spew/spew/format.go generated vendored Normal file
View File

@ -0,0 +1,419 @@
/*
* Copyright (c) 2013-2016 Dave Collins <dave@davec.name>
*
* Permission to use, copy, modify, and distribute this software for any
* purpose with or without fee is hereby granted, provided that the above
* copyright notice and this permission notice appear in all copies.
*
* THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
* WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
* MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
* ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
* WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
* ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
* OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
package spew
import (
"bytes"
"fmt"
"reflect"
"strconv"
"strings"
)
// supportedFlags is a list of all the character flags supported by fmt package.
const supportedFlags = "0-+# "
// formatState implements the fmt.Formatter interface and contains information
// about the state of a formatting operation. The NewFormatter function can
// be used to get a new Formatter which can be used directly as arguments
// in standard fmt package printing calls.
type formatState struct {
value interface{}
fs fmt.State
depth int
pointers map[uintptr]int
ignoreNextType bool
cs *ConfigState
}
// buildDefaultFormat recreates the original format string without precision
// and width information to pass in to fmt.Sprintf in the case of an
// unrecognized type. Unless new types are added to the language, this
// function won't ever be called.
func (f *formatState) buildDefaultFormat() (format string) {
buf := bytes.NewBuffer(percentBytes)
for _, flag := range supportedFlags {
if f.fs.Flag(int(flag)) {
buf.WriteRune(flag)
}
}
buf.WriteRune('v')
format = buf.String()
return format
}
// constructOrigFormat recreates the original format string including precision
// and width information to pass along to the standard fmt package. This allows
// automatic deferral of all format strings this package doesn't support.
func (f *formatState) constructOrigFormat(verb rune) (format string) {
buf := bytes.NewBuffer(percentBytes)
for _, flag := range supportedFlags {
if f.fs.Flag(int(flag)) {
buf.WriteRune(flag)
}
}
if width, ok := f.fs.Width(); ok {
buf.WriteString(strconv.Itoa(width))
}
if precision, ok := f.fs.Precision(); ok {
buf.Write(precisionBytes)
buf.WriteString(strconv.Itoa(precision))
}
buf.WriteRune(verb)
format = buf.String()
return format
}
// unpackValue returns values inside of non-nil interfaces when possible and
// ensures that types for values which have been unpacked from an interface
// are displayed when the show types flag is also set.
// This is useful for data types like structs, arrays, slices, and maps which
// can contain varying types packed inside an interface.
func (f *formatState) unpackValue(v reflect.Value) reflect.Value {
if v.Kind() == reflect.Interface {
f.ignoreNextType = false
if !v.IsNil() {
v = v.Elem()
}
}
return v
}
// formatPtr handles formatting of pointers by indirecting them as necessary.
func (f *formatState) formatPtr(v reflect.Value) {
// Display nil if top level pointer is nil.
showTypes := f.fs.Flag('#')
if v.IsNil() && (!showTypes || f.ignoreNextType) {
f.fs.Write(nilAngleBytes)
return
}
// Remove pointers at or below the current depth from map used to detect
// circular refs.
for k, depth := range f.pointers {
if depth >= f.depth {
delete(f.pointers, k)
}
}
// Keep list of all dereferenced pointers to possibly show later.
pointerChain := make([]uintptr, 0)
// Figure out how many levels of indirection there are by derferencing
// pointers and unpacking interfaces down the chain while detecting circular
// references.
nilFound := false
cycleFound := false
indirects := 0
ve := v
for ve.Kind() == reflect.Ptr {
if ve.IsNil() {
nilFound = true
break
}
indirects++
addr := ve.Pointer()
pointerChain = append(pointerChain, addr)
if pd, ok := f.pointers[addr]; ok && pd < f.depth {
cycleFound = true
indirects--
break
}
f.pointers[addr] = f.depth
ve = ve.Elem()
if ve.Kind() == reflect.Interface {
if ve.IsNil() {
nilFound = true
break
}
ve = ve.Elem()
}
}
// Display type or indirection level depending on flags.
if showTypes && !f.ignoreNextType {
f.fs.Write(openParenBytes)
f.fs.Write(bytes.Repeat(asteriskBytes, indirects))
f.fs.Write([]byte(ve.Type().String()))
f.fs.Write(closeParenBytes)
} else {
if nilFound || cycleFound {
indirects += strings.Count(ve.Type().String(), "*")
}
f.fs.Write(openAngleBytes)
f.fs.Write([]byte(strings.Repeat("*", indirects)))
f.fs.Write(closeAngleBytes)
}
// Display pointer information depending on flags.
if f.fs.Flag('+') && (len(pointerChain) > 0) {
f.fs.Write(openParenBytes)
for i, addr := range pointerChain {
if i > 0 {
f.fs.Write(pointerChainBytes)
}
printHexPtr(f.fs, addr)
}
f.fs.Write(closeParenBytes)
}
// Display dereferenced value.
switch {
case nilFound == true:
f.fs.Write(nilAngleBytes)
case cycleFound == true:
f.fs.Write(circularShortBytes)
default:
f.ignoreNextType = true
f.format(ve)
}
}
// format is the main workhorse for providing the Formatter interface. It
// uses the passed reflect value to figure out what kind of object we are
// dealing with and formats it appropriately. It is a recursive function,
// however circular data structures are detected and handled properly.
func (f *formatState) format(v reflect.Value) {
// Handle invalid reflect values immediately.
kind := v.Kind()
if kind == reflect.Invalid {
f.fs.Write(invalidAngleBytes)
return
}
// Handle pointers specially.
if kind == reflect.Ptr {
f.formatPtr(v)
return
}
// Print type information unless already handled elsewhere.
if !f.ignoreNextType && f.fs.Flag('#') {
f.fs.Write(openParenBytes)
f.fs.Write([]byte(v.Type().String()))
f.fs.Write(closeParenBytes)
}
f.ignoreNextType = false
// Call Stringer/error interfaces if they exist and the handle methods
// flag is enabled.
if !f.cs.DisableMethods {
if (kind != reflect.Invalid) && (kind != reflect.Interface) {
if handled := handleMethods(f.cs, f.fs, v); handled {
return
}
}
}
switch kind {
case reflect.Invalid:
// Do nothing. We should never get here since invalid has already
// been handled above.
case reflect.Bool:
printBool(f.fs, v.Bool())
case reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64, reflect.Int:
printInt(f.fs, v.Int(), 10)
case reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uint:
printUint(f.fs, v.Uint(), 10)
case reflect.Float32:
printFloat(f.fs, v.Float(), 32)
case reflect.Float64:
printFloat(f.fs, v.Float(), 64)
case reflect.Complex64:
printComplex(f.fs, v.Complex(), 32)
case reflect.Complex128:
printComplex(f.fs, v.Complex(), 64)
case reflect.Slice:
if v.IsNil() {
f.fs.Write(nilAngleBytes)
break
}
fallthrough
case reflect.Array:
f.fs.Write(openBracketBytes)
f.depth++
if (f.cs.MaxDepth != 0) && (f.depth > f.cs.MaxDepth) {
f.fs.Write(maxShortBytes)
} else {
numEntries := v.Len()
for i := 0; i < numEntries; i++ {
if i > 0 {
f.fs.Write(spaceBytes)
}
f.ignoreNextType = true
f.format(f.unpackValue(v.Index(i)))
}
}
f.depth--
f.fs.Write(closeBracketBytes)
case reflect.String:
f.fs.Write([]byte(v.String()))
case reflect.Interface:
// The only time we should get here is for nil interfaces due to
// unpackValue calls.
if v.IsNil() {
f.fs.Write(nilAngleBytes)
}
case reflect.Ptr:
// Do nothing. We should never get here since pointers have already
// been handled above.
case reflect.Map:
// nil maps should be indicated as different than empty maps
if v.IsNil() {
f.fs.Write(nilAngleBytes)
break
}
f.fs.Write(openMapBytes)
f.depth++
if (f.cs.MaxDepth != 0) && (f.depth > f.cs.MaxDepth) {
f.fs.Write(maxShortBytes)
} else {
keys := v.MapKeys()
if f.cs.SortKeys {
sortValues(keys, f.cs)
}
for i, key := range keys {
if i > 0 {
f.fs.Write(spaceBytes)
}
f.ignoreNextType = true
f.format(f.unpackValue(key))
f.fs.Write(colonBytes)
f.ignoreNextType = true
f.format(f.unpackValue(v.MapIndex(key)))
}
}
f.depth--
f.fs.Write(closeMapBytes)
case reflect.Struct:
numFields := v.NumField()
f.fs.Write(openBraceBytes)
f.depth++
if (f.cs.MaxDepth != 0) && (f.depth > f.cs.MaxDepth) {
f.fs.Write(maxShortBytes)
} else {
vt := v.Type()
for i := 0; i < numFields; i++ {
if i > 0 {
f.fs.Write(spaceBytes)
}
vtf := vt.Field(i)
if f.fs.Flag('+') || f.fs.Flag('#') {
f.fs.Write([]byte(vtf.Name))
f.fs.Write(colonBytes)
}
f.format(f.unpackValue(v.Field(i)))
}
}
f.depth--
f.fs.Write(closeBraceBytes)
case reflect.Uintptr:
printHexPtr(f.fs, uintptr(v.Uint()))
case reflect.UnsafePointer, reflect.Chan, reflect.Func:
printHexPtr(f.fs, v.Pointer())
// There were not any other types at the time this code was written, but
// fall back to letting the default fmt package handle it if any get added.
default:
format := f.buildDefaultFormat()
if v.CanInterface() {
fmt.Fprintf(f.fs, format, v.Interface())
} else {
fmt.Fprintf(f.fs, format, v.String())
}
}
}
// Format satisfies the fmt.Formatter interface. See NewFormatter for usage
// details.
func (f *formatState) Format(fs fmt.State, verb rune) {
f.fs = fs
// Use standard formatting for verbs that are not v.
if verb != 'v' {
format := f.constructOrigFormat(verb)
fmt.Fprintf(fs, format, f.value)
return
}
if f.value == nil {
if fs.Flag('#') {
fs.Write(interfaceBytes)
}
fs.Write(nilAngleBytes)
return
}
f.format(reflect.ValueOf(f.value))
}
// newFormatter is a helper function to consolidate the logic from the various
// public methods which take varying config states.
func newFormatter(cs *ConfigState, v interface{}) fmt.Formatter {
fs := &formatState{value: v, cs: cs}
fs.pointers = make(map[uintptr]int)
return fs
}
/*
NewFormatter returns a custom formatter that satisfies the fmt.Formatter
interface. As a result, it integrates cleanly with standard fmt package
printing functions. The formatter is useful for inline printing of smaller data
types similar to the standard %v format specifier.
The custom formatter only responds to the %v (most compact), %+v (adds pointer
addresses), %#v (adds types), or %#+v (adds types and pointer addresses) verb
combinations. Any other verbs such as %x and %q will be sent to the the
standard fmt package for formatting. In addition, the custom formatter ignores
the width and precision arguments (however they will still work on the format
specifiers not handled by the custom formatter).
Typically this function shouldn't be called directly. It is much easier to make
use of the custom formatter by calling one of the convenience functions such as
Printf, Println, or Fprintf.
*/
func NewFormatter(v interface{}) fmt.Formatter {
return newFormatter(&Config, v)
}

148
src/vendor/github.com/davecgh/go-spew/spew/spew.go generated vendored Normal file
View File

@ -0,0 +1,148 @@
/*
* Copyright (c) 2013-2016 Dave Collins <dave@davec.name>
*
* Permission to use, copy, modify, and distribute this software for any
* purpose with or without fee is hereby granted, provided that the above
* copyright notice and this permission notice appear in all copies.
*
* THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
* WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
* MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
* ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
* WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
* ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
* OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
package spew
import (
"fmt"
"io"
)
// Errorf is a wrapper for fmt.Errorf that treats each argument as if it were
// passed with a default Formatter interface returned by NewFormatter. It
// returns the formatted string as a value that satisfies error. See
// NewFormatter for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Errorf(format, spew.NewFormatter(a), spew.NewFormatter(b))
func Errorf(format string, a ...interface{}) (err error) {
return fmt.Errorf(format, convertArgs(a)...)
}
// Fprint is a wrapper for fmt.Fprint that treats each argument as if it were
// passed with a default Formatter interface returned by NewFormatter. It
// returns the number of bytes written and any write error encountered. See
// NewFormatter for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Fprint(w, spew.NewFormatter(a), spew.NewFormatter(b))
func Fprint(w io.Writer, a ...interface{}) (n int, err error) {
return fmt.Fprint(w, convertArgs(a)...)
}
// Fprintf is a wrapper for fmt.Fprintf that treats each argument as if it were
// passed with a default Formatter interface returned by NewFormatter. It
// returns the number of bytes written and any write error encountered. See
// NewFormatter for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Fprintf(w, format, spew.NewFormatter(a), spew.NewFormatter(b))
func Fprintf(w io.Writer, format string, a ...interface{}) (n int, err error) {
return fmt.Fprintf(w, format, convertArgs(a)...)
}
// Fprintln is a wrapper for fmt.Fprintln that treats each argument as if it
// passed with a default Formatter interface returned by NewFormatter. See
// NewFormatter for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Fprintln(w, spew.NewFormatter(a), spew.NewFormatter(b))
func Fprintln(w io.Writer, a ...interface{}) (n int, err error) {
return fmt.Fprintln(w, convertArgs(a)...)
}
// Print is a wrapper for fmt.Print that treats each argument as if it were
// passed with a default Formatter interface returned by NewFormatter. It
// returns the number of bytes written and any write error encountered. See
// NewFormatter for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Print(spew.NewFormatter(a), spew.NewFormatter(b))
func Print(a ...interface{}) (n int, err error) {
return fmt.Print(convertArgs(a)...)
}
// Printf is a wrapper for fmt.Printf that treats each argument as if it were
// passed with a default Formatter interface returned by NewFormatter. It
// returns the number of bytes written and any write error encountered. See
// NewFormatter for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Printf(format, spew.NewFormatter(a), spew.NewFormatter(b))
func Printf(format string, a ...interface{}) (n int, err error) {
return fmt.Printf(format, convertArgs(a)...)
}
// Println is a wrapper for fmt.Println that treats each argument as if it were
// passed with a default Formatter interface returned by NewFormatter. It
// returns the number of bytes written and any write error encountered. See
// NewFormatter for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Println(spew.NewFormatter(a), spew.NewFormatter(b))
func Println(a ...interface{}) (n int, err error) {
return fmt.Println(convertArgs(a)...)
}
// Sprint is a wrapper for fmt.Sprint that treats each argument as if it were
// passed with a default Formatter interface returned by NewFormatter. It
// returns the resulting string. See NewFormatter for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Sprint(spew.NewFormatter(a), spew.NewFormatter(b))
func Sprint(a ...interface{}) string {
return fmt.Sprint(convertArgs(a)...)
}
// Sprintf is a wrapper for fmt.Sprintf that treats each argument as if it were
// passed with a default Formatter interface returned by NewFormatter. It
// returns the resulting string. See NewFormatter for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Sprintf(format, spew.NewFormatter(a), spew.NewFormatter(b))
func Sprintf(format string, a ...interface{}) string {
return fmt.Sprintf(format, convertArgs(a)...)
}
// Sprintln is a wrapper for fmt.Sprintln that treats each argument as if it
// were passed with a default Formatter interface returned by NewFormatter. It
// returns the resulting string. See NewFormatter for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Sprintln(spew.NewFormatter(a), spew.NewFormatter(b))
func Sprintln(a ...interface{}) string {
return fmt.Sprintln(convertArgs(a)...)
}
// convertArgs accepts a slice of arguments and returns a slice of the same
// length with each argument converted to a default spew Formatter interface.
func convertArgs(args []interface{}) (formatters []interface{}) {
formatters = make([]interface{}, len(args))
for index, arg := range args {
formatters[index] = NewFormatter(arg)
}
return formatters
}

8
src/vendor/github.com/dgrijalva/jwt-go/LICENSE generated vendored Normal file
View File

@ -0,0 +1,8 @@
Copyright (c) 2012 Dave Grijalva
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

96
src/vendor/github.com/dgrijalva/jwt-go/MIGRATION_GUIDE.md generated vendored Normal file
View File

@ -0,0 +1,96 @@
## Migration Guide from v2 -> v3
Version 3 adds several new, frequently requested features. To do so, it introduces a few breaking changes. We've worked to keep these as minimal as possible. This guide explains the breaking changes and how you can quickly update your code.
### `Token.Claims` is now an interface type
The most requested feature from the 2.0 verison of this library was the ability to provide a custom type to the JSON parser for claims. This was implemented by introducing a new interface, `Claims`, to replace `map[string]interface{}`. We also included two concrete implementations of `Claims`: `MapClaims` and `StandardClaims`.
`MapClaims` is an alias for `map[string]interface{}` with built in validation behavior. It is the default claims type when using `Parse`. The usage is unchanged except you must type cast the claims property.
The old example for parsing a token looked like this..
```go
if token, err := jwt.Parse(tokenString, keyLookupFunc); err == nil {
fmt.Printf("Token for user %v expires %v", token.Claims["user"], token.Claims["exp"])
}
```
is now directly mapped to...
```go
if token, err := jwt.Parse(tokenString, keyLookupFunc); err == nil {
claims := token.Claims.(jwt.MapClaims)
fmt.Printf("Token for user %v expires %v", claims["user"], claims["exp"])
}
```
`StandardClaims` is designed to be embedded in your custom type. You can supply a custom claims type with the new `ParseWithClaims` function. Here's an example of using a custom claims type.
```go
type MyCustomClaims struct {
User string
*StandardClaims
}
if token, err := jwt.ParseWithClaims(tokenString, &MyCustomClaims{}, keyLookupFunc); err == nil {
claims := token.Claims.(*MyCustomClaims)
fmt.Printf("Token for user %v expires %v", claims.User, claims.StandardClaims.ExpiresAt)
}
```
### `ParseFromRequest` has been moved
To keep this library focused on the tokens without becoming overburdened with complex request processing logic, `ParseFromRequest` and its new companion `ParseFromRequestWithClaims` have been moved to a subpackage, `request`. The method signatues have also been augmented to receive a new argument: `Extractor`.
`Extractors` do the work of picking the token string out of a request. The interface is simple and composable.
This simple parsing example:
```go
if token, err := jwt.ParseFromRequest(tokenString, req, keyLookupFunc); err == nil {
fmt.Printf("Token for user %v expires %v", token.Claims["user"], token.Claims["exp"])
}
```
is directly mapped to:
```go
if token, err := request.ParseFromRequest(tokenString, request.OAuth2Extractor, req, keyLookupFunc); err == nil {
fmt.Printf("Token for user %v expires %v", token.Claims["user"], token.Claims["exp"])
}
```
There are several concrete `Extractor` types provided for your convenience:
* `HeaderExtractor` will search a list of headers until one contains content.
* `ArgumentExtractor` will search a list of keys in request query and form arguments until one contains content.
* `MultiExtractor` will try a list of `Extractors` in order until one returns content.
* `AuthorizationHeaderExtractor` will look in the `Authorization` header for a `Bearer` token.
* `OAuth2Extractor` searches the places an OAuth2 token would be specified (per the spec): `Authorization` header and `access_token` argument
* `PostExtractionFilter` wraps an `Extractor`, allowing you to process the content before it's parsed. A simple example is stripping the `Bearer ` text from a header
### RSA signing methods no longer accept `[]byte` keys
Due to a [critical vulnerability](https://auth0.com/blog/2015/03/31/critical-vulnerabilities-in-json-web-token-libraries/), we've decided the convenience of accepting `[]byte` instead of `rsa.PublicKey` or `rsa.PrivateKey` isn't worth the risk of misuse.
To replace this behavior, we've added two helper methods: `ParseRSAPrivateKeyFromPEM(key []byte) (*rsa.PrivateKey, error)` and `ParseRSAPublicKeyFromPEM(key []byte) (*rsa.PublicKey, error)`. These are just simple helpers for unpacking PEM encoded PKCS1 and PKCS8 keys. If your keys are encoded any other way, all you need to do is convert them to the `crypto/rsa` package's types.
```go
func keyLookupFunc(*Token) (interface{}, error) {
// Don't forget to validate the alg is what you expect:
if _, ok := token.Method.(*jwt.SigningMethodRSA); !ok {
return nil, fmt.Errorf("Unexpected signing method: %v", token.Header["alg"])
}
// Look up key
key, err := lookupPublicKey(token.Header["kid"])
if err != nil {
return nil, err
}
// Unpack key from PEM encoded PKCS8
return jwt.ParseRSAPublicKeyFromPEM(key)
}
```

85
src/vendor/github.com/dgrijalva/jwt-go/README.md generated vendored Normal file
View File

@ -0,0 +1,85 @@
A [go](http://www.golang.org) (or 'golang' for search engine friendliness) implementation of [JSON Web Tokens](http://self-issued.info/docs/draft-ietf-oauth-json-web-token.html)
[![Build Status](https://travis-ci.org/dgrijalva/jwt-go.svg?branch=master)](https://travis-ci.org/dgrijalva/jwt-go)
**BREAKING CHANGES:*** Version 3.0.0 is here. It includes _a lot_ of changes including a few that break the API. We've tried to break as few things as possible, so there should just be a few type signature changes. A full list of breaking changes is available in `VERSION_HISTORY.md`. See `MIGRATION_GUIDE.md` for more information on updating your code.
**NOTICE:** A vulnerability in JWT was [recently published](https://auth0.com/blog/2015/03/31/critical-vulnerabilities-in-json-web-token-libraries/). As this library doesn't force users to validate the `alg` is what they expected, it's possible your usage is effected. There will be an update soon to remedy this, and it will likey require backwards-incompatible changes to the API. In the short term, please make sure your implementation verifies the `alg` is what you expect.
## What the heck is a JWT?
JWT.io has [a great introduction](https://jwt.io/introduction) to JSON Web Tokens.
In short, it's a signed JSON object that does something useful (for example, authentication). It's commonly used for `Bearer` tokens in Oauth 2. A token is made of three parts, separated by `.`'s. The first two parts are JSON objects, that have been [base64url](http://tools.ietf.org/html/rfc4648) encoded. The last part is the signature, encoded the same way.
The first part is called the header. It contains the necessary information for verifying the last part, the signature. For example, which encryption method was used for signing and what key was used.
The part in the middle is the interesting bit. It's called the Claims and contains the actual stuff you care about. Refer to [the RFC](http://self-issued.info/docs/draft-jones-json-web-token.html) for information about reserved keys and the proper way to add your own.
## What's in the box?
This library supports the parsing and verification as well as the generation and signing of JWTs. Current supported signing algorithms are HMAC SHA, RSA, RSA-PSS, and ECDSA, though hooks are present for adding your own.
## Examples
See [the project documentation](https://godoc.org/github.com/dgrijalva/jwt-go) for examples of usage:
* [Simple example of parsing and validating a token](https://godoc.org/github.com/dgrijalva/jwt-go#example_Parse_hmac)
* [Simple example of building and signing a token](https://godoc.org/github.com/dgrijalva/jwt-go#example_New_hmac)
* [Directory of Examples](https://godoc.org/github.com/dgrijalva/jwt-go#pkg-examples)
## Extensions
This library publishes all the necessary components for adding your own signing methods. Simply implement the `SigningMethod` interface and register a factory method using `RegisterSigningMethod`.
Here's an example of an extension that integrates with the Google App Engine signing tools: https://github.com/someone1/gcp-jwt-go
## Compliance
This library was last reviewed to comply with [RTF 7519](http://www.rfc-editor.org/info/rfc7519) dated May 2015 with a few notable differences:
* In order to protect against accidental use of [Unsecured JWTs](http://self-issued.info/docs/draft-ietf-oauth-json-web-token.html#UnsecuredJWT), tokens using `alg=none` will only be accepted if the constant `jwt.UnsafeAllowNoneSignatureType` is provided as the key.
## Project Status & Versioning
This library is considered production ready. Feedback and feature requests are appreciated. The API should be considered stable. There should be very few backwards-incompatible changes outside of major version updates (and only with good reason).
This project uses [Semantic Versioning 2.0.0](http://semver.org). Accepted pull requests will land on `master`. Periodically, versions will be tagged from `master`. You can find all the releases on [the project releases page](https://github.com/dgrijalva/jwt-go/releases).
While we try to make it obvious when we make breaking changes, there isn't a great mechanism for pushing announcements out to users. You may want to use this alternative package include: `gopkg.in/dgrijalva/jwt-go.v2`. It will do the right thing WRT semantic versioning.
## Usage Tips
### Signing vs Encryption
A token is simply a JSON object that is signed by its author. this tells you exactly two things about the data:
* The author of the token was in the possession of the signing secret
* The data has not been modified since it was signed
It's important to know that JWT does not provide encryption, which means anyone who has access to the token can read its contents. If you need to protect (encrypt) the data, there is a companion spec, `JWE`, that provides this functionality. JWE is currently outside the scope of this library.
### Choosing a Signing Method
There are several signing methods available, and you should probably take the time to learn about the various options before choosing one. The principal design decision is most likely going to be symmetric vs asymmetric.
Symmetric signing methods, such as HSA, use only a single secret. This is probably the simplest signing method to use since any `[]byte` can be used as a valid secret. They are also slightly computationally faster to use, though this rarely is enough to matter. Symmetric signing methods work the best when both producers and consumers of tokens are trusted, or even the same system. Since the same secret is used to both sign and validate tokens, you can't easily distribute the key for validation.
Asymmetric signing methods, such as RSA, use different keys for signing and verifying tokens. This makes it possible to produce tokens with a private key, and allow any consumer to access the public key for verification.
### JWT and OAuth
It's worth mentioning that OAuth and JWT are not the same thing. A JWT token is simply a signed JSON object. It can be used anywhere such a thing is useful. There is some confusion, though, as JWT is the most common type of bearer token used in OAuth2 authentication.
Without going too far down the rabbit hole, here's a description of the interaction of these technologies:
* OAuth is a protocol for allowing an identity provider to be separate from the service a user is logging in to. For example, whenever you use Facebook to log into a different service (Yelp, Spotify, etc), you are using OAuth.
* OAuth defines several options for passing around authentication data. One popular method is called a "bearer token". A bearer token is simply a string that _should_ only be held by an authenticated user. Thus, simply presenting this token proves your identity. You can probably derive from here why a JWT might make a good bearer token.
* Because bearer tokens are used for authentication, it's important they're kept secret. This is why transactions that use bearer tokens typically happen over SSL.
## More
Documentation can be found [on godoc.org](http://godoc.org/github.com/dgrijalva/jwt-go).
The command line utility included in this project (cmd/jwt) provides a straightforward example of token creation and parsing as well as a useful tool for debugging your own integration. You'll also find several implementation examples in to documentation.

105
src/vendor/github.com/dgrijalva/jwt-go/VERSION_HISTORY.md generated vendored Normal file
View File

@ -0,0 +1,105 @@
## `jwt-go` Version History
#### 3.0.0
* **Compatibility Breaking Changes**: See MIGRATION_GUIDE.md for tips on updating your code
* Dropped support for `[]byte` keys when using RSA signing methods. This convenience feature could contribute to security vulnerabilities involving mismatched key types with signing methods.
* `ParseFromRequest` has been moved to `request` subpackage and usage has changed
* The `Claims` property on `Token` is now type `Claims` instead of `map[string]interface{}`. The default value is type `MapClaims`, which is an alias to `map[string]interface{}`. This makes it possible to use a custom type when decoding claims.
* Other Additions and Changes
* Added `Claims` interface type to allow users to decode the claims into a custom type
* Added `ParseWithClaims`, which takes a third argument of type `Claims`. Use this function instead of `Parse` if you have a custom type you'd like to decode into.
* Dramatically improved the functionality and flexibility of `ParseFromRequest`, which is now in the `request` subpackage
* Added `ParseFromRequestWithClaims` which is the `FromRequest` equivalent of `ParseWithClaims`
* Added new interface type `Extractor`, which is used for extracting JWT strings from http requests. Used with `ParseFromRequest` and `ParseFromRequestWithClaims`.
* Added several new, more specific, validation errors to error type bitmask
* Moved examples from README to executable example files
* Signing method registry is now thread safe
* Added new property to `ValidationError`, which contains the raw error returned by calls made by parse/verify (such as those returned by keyfunc or json parser)
#### 2.7.0
This will likely be the last backwards compatible release before 3.0.0, excluding essential bug fixes.
* Added new option `-show` to the `jwt` command that will just output the decoded token without verifying
* Error text for expired tokens includes how long it's been expired
* Fixed incorrect error returned from `ParseRSAPublicKeyFromPEM`
* Documentation updates
#### 2.6.0
* Exposed inner error within ValidationError
* Fixed validation errors when using UseJSONNumber flag
* Added several unit tests
#### 2.5.0
* Added support for signing method none. You shouldn't use this. The API tries to make this clear.
* Updated/fixed some documentation
* Added more helpful error message when trying to parse tokens that begin with `BEARER `
#### 2.4.0
* Added new type, Parser, to allow for configuration of various parsing parameters
* You can now specify a list of valid signing methods. Anything outside this set will be rejected.
* You can now opt to use the `json.Number` type instead of `float64` when parsing token JSON
* Added support for [Travis CI](https://travis-ci.org/dgrijalva/jwt-go)
* Fixed some bugs with ECDSA parsing
#### 2.3.0
* Added support for ECDSA signing methods
* Added support for RSA PSS signing methods (requires go v1.4)
#### 2.2.0
* Gracefully handle a `nil` `Keyfunc` being passed to `Parse`. Result will now be the parsed token and an error, instead of a panic.
#### 2.1.0
Backwards compatible API change that was missed in 2.0.0.
* The `SignedString` method on `Token` now takes `interface{}` instead of `[]byte`
#### 2.0.0
There were two major reasons for breaking backwards compatibility with this update. The first was a refactor required to expand the width of the RSA and HMAC-SHA signing implementations. There will likely be no required code changes to support this change.
The second update, while unfortunately requiring a small change in integration, is required to open up this library to other signing methods. Not all keys used for all signing methods have a single standard on-disk representation. Requiring `[]byte` as the type for all keys proved too limiting. Additionally, this implementation allows for pre-parsed tokens to be reused, which might matter in an application that parses a high volume of tokens with a small set of keys. Backwards compatibilty has been maintained for passing `[]byte` to the RSA signing methods, but they will also accept `*rsa.PublicKey` and `*rsa.PrivateKey`.
It is likely the only integration change required here will be to change `func(t *jwt.Token) ([]byte, error)` to `func(t *jwt.Token) (interface{}, error)` when calling `Parse`.
* **Compatibility Breaking Changes**
* `SigningMethodHS256` is now `*SigningMethodHMAC` instead of `type struct`
* `SigningMethodRS256` is now `*SigningMethodRSA` instead of `type struct`
* `KeyFunc` now returns `interface{}` instead of `[]byte`
* `SigningMethod.Sign` now takes `interface{}` instead of `[]byte` for the key
* `SigningMethod.Verify` now takes `interface{}` instead of `[]byte` for the key
* Renamed type `SigningMethodHS256` to `SigningMethodHMAC`. Specific sizes are now just instances of this type.
* Added public package global `SigningMethodHS256`
* Added public package global `SigningMethodHS384`
* Added public package global `SigningMethodHS512`
* Renamed type `SigningMethodRS256` to `SigningMethodRSA`. Specific sizes are now just instances of this type.
* Added public package global `SigningMethodRS256`
* Added public package global `SigningMethodRS384`
* Added public package global `SigningMethodRS512`
* Moved sample private key for HMAC tests from an inline value to a file on disk. Value is unchanged.
* Refactored the RSA implementation to be easier to read
* Exposed helper methods `ParseRSAPrivateKeyFromPEM` and `ParseRSAPublicKeyFromPEM`
#### 1.0.2
* Fixed bug in parsing public keys from certificates
* Added more tests around the parsing of keys for RS256
* Code refactoring in RS256 implementation. No functional changes
#### 1.0.1
* Fixed panic if RS256 signing method was passed an invalid key
#### 1.0.0
* First versioned release
* API stabilized
* Supports creating, signing, parsing, and validating JWT tokens
* Supports RS256 and HS256 signing methods

134
src/vendor/github.com/dgrijalva/jwt-go/claims.go generated vendored Normal file
View File

@ -0,0 +1,134 @@
package jwt
import (
"crypto/subtle"
"fmt"
"time"
)
// For a type to be a Claims object, it must just have a Valid method that determines
// if the token is invalid for any supported reason
type Claims interface {
Valid() error
}
// Structured version of Claims Section, as referenced at
// https://tools.ietf.org/html/rfc7519#section-4.1
// See examples for how to use this with your own claim types
type StandardClaims struct {
Audience string `json:"aud,omitempty"`
ExpiresAt int64 `json:"exp,omitempty"`
Id string `json:"jti,omitempty"`
IssuedAt int64 `json:"iat,omitempty"`
Issuer string `json:"iss,omitempty"`
NotBefore int64 `json:"nbf,omitempty"`
Subject string `json:"sub,omitempty"`
}
// Validates time based claims "exp, iat, nbf".
// There is no accounting for clock skew.
// As well, if any of the above claims are not in the token, it will still
// be considered a valid claim.
func (c StandardClaims) Valid() error {
vErr := new(ValidationError)
now := TimeFunc().Unix()
// The claims below are optional, by default, so if they are set to the
// default value in Go, let's not fail the verification for them.
if c.VerifyExpiresAt(now, false) == false {
delta := time.Unix(now, 0).Sub(time.Unix(c.ExpiresAt, 0))
vErr.Inner = fmt.Errorf("token is expired by %v", delta)
vErr.Errors |= ValidationErrorExpired
}
if c.VerifyIssuedAt(now, false) == false {
vErr.Inner = fmt.Errorf("Token used before issued")
vErr.Errors |= ValidationErrorIssuedAt
}
if c.VerifyNotBefore(now, false) == false {
vErr.Inner = fmt.Errorf("token is not valid yet")
vErr.Errors |= ValidationErrorNotValidYet
}
if vErr.valid() {
return nil
}
return vErr
}
// Compares the aud claim against cmp.
// If required is false, this method will return true if the value matches or is unset
func (c *StandardClaims) VerifyAudience(cmp string, req bool) bool {
return verifyAud(c.Audience, cmp, req)
}
// Compares the exp claim against cmp.
// If required is false, this method will return true if the value matches or is unset
func (c *StandardClaims) VerifyExpiresAt(cmp int64, req bool) bool {
return verifyExp(c.ExpiresAt, cmp, req)
}
// Compares the iat claim against cmp.
// If required is false, this method will return true if the value matches or is unset
func (c *StandardClaims) VerifyIssuedAt(cmp int64, req bool) bool {
return verifyIat(c.IssuedAt, cmp, req)
}
// Compares the iss claim against cmp.
// If required is false, this method will return true if the value matches or is unset
func (c *StandardClaims) VerifyIssuer(cmp string, req bool) bool {
return verifyIss(c.Issuer, cmp, req)
}
// Compares the nbf claim against cmp.
// If required is false, this method will return true if the value matches or is unset
func (c *StandardClaims) VerifyNotBefore(cmp int64, req bool) bool {
return verifyNbf(c.NotBefore, cmp, req)
}
// ----- helpers
func verifyAud(aud string, cmp string, required bool) bool {
if aud == "" {
return !required
}
if subtle.ConstantTimeCompare([]byte(aud), []byte(cmp)) != 0 {
return true
} else {
return false
}
}
func verifyExp(exp int64, now int64, required bool) bool {
if exp == 0 {
return !required
}
return now <= exp
}
func verifyIat(iat int64, now int64, required bool) bool {
if iat == 0 {
return !required
}
return now >= iat
}
func verifyIss(iss string, cmp string, required bool) bool {
if iss == "" {
return !required
}
if subtle.ConstantTimeCompare([]byte(iss), []byte(cmp)) != 0 {
return true
} else {
return false
}
}
func verifyNbf(nbf int64, now int64, required bool) bool {
if nbf == 0 {
return !required
}
return now >= nbf
}

4
src/vendor/github.com/dgrijalva/jwt-go/doc.go generated vendored Normal file
View File

@ -0,0 +1,4 @@
// Package jwt is a Go implementation of JSON Web Tokens: http://self-issued.info/docs/draft-jones-json-web-token.html
//
// See README.md for more info.
package jwt

147
src/vendor/github.com/dgrijalva/jwt-go/ecdsa.go generated vendored Normal file
View File

@ -0,0 +1,147 @@
package jwt
import (
"crypto"
"crypto/ecdsa"
"crypto/rand"
"errors"
"math/big"
)
var (
// Sadly this is missing from crypto/ecdsa compared to crypto/rsa
ErrECDSAVerification = errors.New("crypto/ecdsa: verification error")
)
// Implements the ECDSA family of signing methods signing methods
type SigningMethodECDSA struct {
Name string
Hash crypto.Hash
KeySize int
CurveBits int
}
// Specific instances for EC256 and company
var (
SigningMethodES256 *SigningMethodECDSA
SigningMethodES384 *SigningMethodECDSA
SigningMethodES512 *SigningMethodECDSA
)
func init() {
// ES256
SigningMethodES256 = &SigningMethodECDSA{"ES256", crypto.SHA256, 32, 256}
RegisterSigningMethod(SigningMethodES256.Alg(), func() SigningMethod {
return SigningMethodES256
})
// ES384
SigningMethodES384 = &SigningMethodECDSA{"ES384", crypto.SHA384, 48, 384}
RegisterSigningMethod(SigningMethodES384.Alg(), func() SigningMethod {
return SigningMethodES384
})
// ES512
SigningMethodES512 = &SigningMethodECDSA{"ES512", crypto.SHA512, 66, 521}
RegisterSigningMethod(SigningMethodES512.Alg(), func() SigningMethod {
return SigningMethodES512
})
}
func (m *SigningMethodECDSA) Alg() string {
return m.Name
}
// Implements the Verify method from SigningMethod
// For this verify method, key must be an ecdsa.PublicKey struct
func (m *SigningMethodECDSA) Verify(signingString, signature string, key interface{}) error {
var err error
// Decode the signature
var sig []byte
if sig, err = DecodeSegment(signature); err != nil {
return err
}
// Get the key
var ecdsaKey *ecdsa.PublicKey
switch k := key.(type) {
case *ecdsa.PublicKey:
ecdsaKey = k
default:
return ErrInvalidKeyType
}
if len(sig) != 2*m.KeySize {
return ErrECDSAVerification
}
r := big.NewInt(0).SetBytes(sig[:m.KeySize])
s := big.NewInt(0).SetBytes(sig[m.KeySize:])
// Create hasher
if !m.Hash.Available() {
return ErrHashUnavailable
}
hasher := m.Hash.New()
hasher.Write([]byte(signingString))
// Verify the signature
if verifystatus := ecdsa.Verify(ecdsaKey, hasher.Sum(nil), r, s); verifystatus == true {
return nil
} else {
return ErrECDSAVerification
}
}
// Implements the Sign method from SigningMethod
// For this signing method, key must be an ecdsa.PrivateKey struct
func (m *SigningMethodECDSA) Sign(signingString string, key interface{}) (string, error) {
// Get the key
var ecdsaKey *ecdsa.PrivateKey
switch k := key.(type) {
case *ecdsa.PrivateKey:
ecdsaKey = k
default:
return "", ErrInvalidKeyType
}
// Create the hasher
if !m.Hash.Available() {
return "", ErrHashUnavailable
}
hasher := m.Hash.New()
hasher.Write([]byte(signingString))
// Sign the string and return r, s
if r, s, err := ecdsa.Sign(rand.Reader, ecdsaKey, hasher.Sum(nil)); err == nil {
curveBits := ecdsaKey.Curve.Params().BitSize
if m.CurveBits != curveBits {
return "", ErrInvalidKey
}
keyBytes := curveBits / 8
if curveBits%8 > 0 {
keyBytes += 1
}
// We serialize the outpus (r and s) into big-endian byte arrays and pad
// them with zeros on the left to make sure the sizes work out. Both arrays
// must be keyBytes long, and the output must be 2*keyBytes long.
rBytes := r.Bytes()
rBytesPadded := make([]byte, keyBytes)
copy(rBytesPadded[keyBytes-len(rBytes):], rBytes)
sBytes := s.Bytes()
sBytesPadded := make([]byte, keyBytes)
copy(sBytesPadded[keyBytes-len(sBytes):], sBytes)
out := append(rBytesPadded, sBytesPadded...)
return EncodeSegment(out), nil
} else {
return "", err
}
}

67
src/vendor/github.com/dgrijalva/jwt-go/ecdsa_utils.go generated vendored Normal file
View File

@ -0,0 +1,67 @@
package jwt
import (
"crypto/ecdsa"
"crypto/x509"
"encoding/pem"
"errors"
)
var (
ErrNotECPublicKey = errors.New("Key is not a valid ECDSA public key")
ErrNotECPrivateKey = errors.New("Key is not a valid ECDSA private key")
)
// Parse PEM encoded Elliptic Curve Private Key Structure
func ParseECPrivateKeyFromPEM(key []byte) (*ecdsa.PrivateKey, error) {
var err error
// Parse PEM block
var block *pem.Block
if block, _ = pem.Decode(key); block == nil {
return nil, ErrKeyMustBePEMEncoded
}
// Parse the key
var parsedKey interface{}
if parsedKey, err = x509.ParseECPrivateKey(block.Bytes); err != nil {
return nil, err
}
var pkey *ecdsa.PrivateKey
var ok bool
if pkey, ok = parsedKey.(*ecdsa.PrivateKey); !ok {
return nil, ErrNotECPrivateKey
}
return pkey, nil
}
// Parse PEM encoded PKCS1 or PKCS8 public key
func ParseECPublicKeyFromPEM(key []byte) (*ecdsa.PublicKey, error) {
var err error
// Parse PEM block
var block *pem.Block
if block, _ = pem.Decode(key); block == nil {
return nil, ErrKeyMustBePEMEncoded
}
// Parse the key
var parsedKey interface{}
if parsedKey, err = x509.ParsePKIXPublicKey(block.Bytes); err != nil {
if cert, err := x509.ParseCertificate(block.Bytes); err == nil {
parsedKey = cert.PublicKey
} else {
return nil, err
}
}
var pkey *ecdsa.PublicKey
var ok bool
if pkey, ok = parsedKey.(*ecdsa.PublicKey); !ok {
return nil, ErrNotECPublicKey
}
return pkey, nil
}

63
src/vendor/github.com/dgrijalva/jwt-go/errors.go generated vendored Normal file
View File

@ -0,0 +1,63 @@
package jwt
import (
"errors"
)
// Error constants
var (
ErrInvalidKey = errors.New("key is invalid")
ErrInvalidKeyType = errors.New("key is of invalid type")
ErrHashUnavailable = errors.New("the requested hash function is unavailable")
)
// The errors that might occur when parsing and validating a token
const (
ValidationErrorMalformed uint32 = 1 << iota // Token is malformed
ValidationErrorUnverifiable // Token could not be verified because of signing problems
ValidationErrorSignatureInvalid // Signature validation failed
// Standard Claim validation errors
ValidationErrorAudience // AUD validation failed
ValidationErrorExpired // EXP validation failed
ValidationErrorIssuedAt // IAT validation failed
ValidationErrorIssuer // ISS validation failed
ValidationErrorNotValidYet // NBF validation failed
ValidationErrorId // JTI validation failed
ValidationErrorClaimsInvalid // Generic claims validation error
)
// Helper for constructing a ValidationError with a string error message
func NewValidationError(errorText string, errorFlags uint32) *ValidationError {
return &ValidationError{
text: errorText,
Errors: errorFlags,
}
}
// The error from Parse if token is not valid
type ValidationError struct {
Inner error // stores the error returned by external dependencies, i.e.: KeyFunc
Errors uint32 // bitfield. see ValidationError... constants
text string // errors that do not have a valid error just have text
}
// Validation error is an error type
func (e ValidationError) Error() string {
if e.Inner != nil {
return e.Inner.Error()
} else if e.text != "" {
return e.text
} else {
return "token is invalid"
}
return e.Inner.Error()
}
// No errors
func (e *ValidationError) valid() bool {
if e.Errors > 0 {
return false
}
return true
}

94
src/vendor/github.com/dgrijalva/jwt-go/hmac.go generated vendored Normal file
View File

@ -0,0 +1,94 @@
package jwt
import (
"crypto"
"crypto/hmac"
"errors"
)
// Implements the HMAC-SHA family of signing methods signing methods
type SigningMethodHMAC struct {
Name string
Hash crypto.Hash
}
// Specific instances for HS256 and company
var (
SigningMethodHS256 *SigningMethodHMAC
SigningMethodHS384 *SigningMethodHMAC
SigningMethodHS512 *SigningMethodHMAC
ErrSignatureInvalid = errors.New("signature is invalid")
)
func init() {
// HS256
SigningMethodHS256 = &SigningMethodHMAC{"HS256", crypto.SHA256}
RegisterSigningMethod(SigningMethodHS256.Alg(), func() SigningMethod {
return SigningMethodHS256
})
// HS384
SigningMethodHS384 = &SigningMethodHMAC{"HS384", crypto.SHA384}
RegisterSigningMethod(SigningMethodHS384.Alg(), func() SigningMethod {
return SigningMethodHS384
})
// HS512
SigningMethodHS512 = &SigningMethodHMAC{"HS512", crypto.SHA512}
RegisterSigningMethod(SigningMethodHS512.Alg(), func() SigningMethod {
return SigningMethodHS512
})
}
func (m *SigningMethodHMAC) Alg() string {
return m.Name
}
// Verify the signature of HSXXX tokens. Returns nil if the signature is valid.
func (m *SigningMethodHMAC) Verify(signingString, signature string, key interface{}) error {
// Verify the key is the right type
keyBytes, ok := key.([]byte)
if !ok {
return ErrInvalidKeyType
}
// Decode signature, for comparison
sig, err := DecodeSegment(signature)
if err != nil {
return err
}
// Can we use the specified hashing method?
if !m.Hash.Available() {
return ErrHashUnavailable
}
// This signing method is symmetric, so we validate the signature
// by reproducing the signature from the signing string and key, then
// comparing that against the provided signature.
hasher := hmac.New(m.Hash.New, keyBytes)
hasher.Write([]byte(signingString))
if !hmac.Equal(sig, hasher.Sum(nil)) {
return ErrSignatureInvalid
}
// No validation errors. Signature is good.
return nil
}
// Implements the Sign method from SigningMethod for this signing method.
// Key must be []byte
func (m *SigningMethodHMAC) Sign(signingString string, key interface{}) (string, error) {
if keyBytes, ok := key.([]byte); ok {
if !m.Hash.Available() {
return "", ErrHashUnavailable
}
hasher := hmac.New(m.Hash.New, keyBytes)
hasher.Write([]byte(signingString))
return EncodeSegment(hasher.Sum(nil)), nil
}
return "", ErrInvalidKey
}

94
src/vendor/github.com/dgrijalva/jwt-go/map_claims.go generated vendored Normal file
View File

@ -0,0 +1,94 @@
package jwt
import (
"encoding/json"
"errors"
// "fmt"
)
// Claims type that uses the map[string]interface{} for JSON decoding
// This is the default claims type if you don't supply one
type MapClaims map[string]interface{}
// Compares the aud claim against cmp.
// If required is false, this method will return true if the value matches or is unset
func (m MapClaims) VerifyAudience(cmp string, req bool) bool {
aud, _ := m["aud"].(string)
return verifyAud(aud, cmp, req)
}
// Compares the exp claim against cmp.
// If required is false, this method will return true if the value matches or is unset
func (m MapClaims) VerifyExpiresAt(cmp int64, req bool) bool {
switch exp := m["exp"].(type) {
case float64:
return verifyExp(int64(exp), cmp, req)
case json.Number:
v, _ := exp.Int64()
return verifyExp(v, cmp, req)
}
return req == false
}
// Compares the iat claim against cmp.
// If required is false, this method will return true if the value matches or is unset
func (m MapClaims) VerifyIssuedAt(cmp int64, req bool) bool {
switch iat := m["iat"].(type) {
case float64:
return verifyIat(int64(iat), cmp, req)
case json.Number:
v, _ := iat.Int64()
return verifyIat(v, cmp, req)
}
return req == false
}
// Compares the iss claim against cmp.
// If required is false, this method will return true if the value matches or is unset
func (m MapClaims) VerifyIssuer(cmp string, req bool) bool {
iss, _ := m["iss"].(string)
return verifyIss(iss, cmp, req)
}
// Compares the nbf claim against cmp.
// If required is false, this method will return true if the value matches or is unset
func (m MapClaims) VerifyNotBefore(cmp int64, req bool) bool {
switch nbf := m["nbf"].(type) {
case float64:
return verifyNbf(int64(nbf), cmp, req)
case json.Number:
v, _ := nbf.Int64()
return verifyNbf(v, cmp, req)
}
return req == false
}
// Validates time based claims "exp, iat, nbf".
// There is no accounting for clock skew.
// As well, if any of the above claims are not in the token, it will still
// be considered a valid claim.
func (m MapClaims) Valid() error {
vErr := new(ValidationError)
now := TimeFunc().Unix()
if m.VerifyExpiresAt(now, false) == false {
vErr.Inner = errors.New("Token is expired")
vErr.Errors |= ValidationErrorExpired
}
if m.VerifyIssuedAt(now, false) == false {
vErr.Inner = errors.New("Token used before issued")
vErr.Errors |= ValidationErrorIssuedAt
}
if m.VerifyNotBefore(now, false) == false {
vErr.Inner = errors.New("Token is not valid yet")
vErr.Errors |= ValidationErrorNotValidYet
}
if vErr.valid() {
return nil
}
return vErr
}

52
src/vendor/github.com/dgrijalva/jwt-go/none.go generated vendored Normal file
View File

@ -0,0 +1,52 @@
package jwt
// Implements the none signing method. This is required by the spec
// but you probably should never use it.
var SigningMethodNone *signingMethodNone
const UnsafeAllowNoneSignatureType unsafeNoneMagicConstant = "none signing method allowed"
var NoneSignatureTypeDisallowedError error
type signingMethodNone struct{}
type unsafeNoneMagicConstant string
func init() {
SigningMethodNone = &signingMethodNone{}
NoneSignatureTypeDisallowedError = NewValidationError("'none' signature type is not allowed", ValidationErrorSignatureInvalid)
RegisterSigningMethod(SigningMethodNone.Alg(), func() SigningMethod {
return SigningMethodNone
})
}
func (m *signingMethodNone) Alg() string {
return "none"
}
// Only allow 'none' alg type if UnsafeAllowNoneSignatureType is specified as the key
func (m *signingMethodNone) Verify(signingString, signature string, key interface{}) (err error) {
// Key must be UnsafeAllowNoneSignatureType to prevent accidentally
// accepting 'none' signing method
if _, ok := key.(unsafeNoneMagicConstant); !ok {
return NoneSignatureTypeDisallowedError
}
// If signing method is none, signature must be an empty string
if signature != "" {
return NewValidationError(
"'none' signing method with non-empty signature",
ValidationErrorSignatureInvalid,
)
}
// Accept 'none' signing method.
return nil
}
// Only allow 'none' signing if UnsafeAllowNoneSignatureType is specified as the key
func (m *signingMethodNone) Sign(signingString string, key interface{}) (string, error) {
if _, ok := key.(unsafeNoneMagicConstant); ok {
return "", nil
}
return "", NoneSignatureTypeDisallowedError
}

128
src/vendor/github.com/dgrijalva/jwt-go/parser.go generated vendored Normal file
View File

@ -0,0 +1,128 @@
package jwt
import (
"bytes"
"encoding/json"
"fmt"
"strings"
)
type Parser struct {
ValidMethods []string // If populated, only these methods will be considered valid
UseJSONNumber bool // Use JSON Number format in JSON decoder
}
// Parse, validate, and return a token.
// keyFunc will receive the parsed token and should return the key for validating.
// If everything is kosher, err will be nil
func (p *Parser) Parse(tokenString string, keyFunc Keyfunc) (*Token, error) {
return p.ParseWithClaims(tokenString, MapClaims{}, keyFunc)
}
func (p *Parser) ParseWithClaims(tokenString string, claims Claims, keyFunc Keyfunc) (*Token, error) {
parts := strings.Split(tokenString, ".")
if len(parts) != 3 {
return nil, NewValidationError("token contains an invalid number of segments", ValidationErrorMalformed)
}
var err error
token := &Token{Raw: tokenString}
// parse Header
var headerBytes []byte
if headerBytes, err = DecodeSegment(parts[0]); err != nil {
if strings.HasPrefix(strings.ToLower(tokenString), "bearer ") {
return token, NewValidationError("tokenstring should not contain 'bearer '", ValidationErrorMalformed)
}
return token, &ValidationError{Inner: err, Errors: ValidationErrorMalformed}
}
if err = json.Unmarshal(headerBytes, &token.Header); err != nil {
return token, &ValidationError{Inner: err, Errors: ValidationErrorMalformed}
}
// parse Claims
var claimBytes []byte
token.Claims = claims
if claimBytes, err = DecodeSegment(parts[1]); err != nil {
return token, &ValidationError{Inner: err, Errors: ValidationErrorMalformed}
}
dec := json.NewDecoder(bytes.NewBuffer(claimBytes))
if p.UseJSONNumber {
dec.UseNumber()
}
// JSON Decode. Special case for map type to avoid weird pointer behavior
if c, ok := token.Claims.(MapClaims); ok {
err = dec.Decode(&c)
} else {
err = dec.Decode(&claims)
}
// Handle decode error
if err != nil {
return token, &ValidationError{Inner: err, Errors: ValidationErrorMalformed}
}
// Lookup signature method
if method, ok := token.Header["alg"].(string); ok {
if token.Method = GetSigningMethod(method); token.Method == nil {
return token, NewValidationError("signing method (alg) is unavailable.", ValidationErrorUnverifiable)
}
} else {
return token, NewValidationError("signing method (alg) is unspecified.", ValidationErrorUnverifiable)
}
// Verify signing method is in the required set
if p.ValidMethods != nil {
var signingMethodValid = false
var alg = token.Method.Alg()
for _, m := range p.ValidMethods {
if m == alg {
signingMethodValid = true
break
}
}
if !signingMethodValid {
// signing method is not in the listed set
return token, NewValidationError(fmt.Sprintf("signing method %v is invalid", alg), ValidationErrorSignatureInvalid)
}
}
// Lookup key
var key interface{}
if keyFunc == nil {
// keyFunc was not provided. short circuiting validation
return token, NewValidationError("no Keyfunc was provided.", ValidationErrorUnverifiable)
}
if key, err = keyFunc(token); err != nil {
// keyFunc returned an error
return token, &ValidationError{Inner: err, Errors: ValidationErrorUnverifiable}
}
vErr := &ValidationError{}
// Validate Claims
if err := token.Claims.Valid(); err != nil {
// If the Claims Valid returned an error, check if it is a validation error,
// If it was another error type, create a ValidationError with a generic ClaimsInvalid flag set
if e, ok := err.(*ValidationError); !ok {
vErr = &ValidationError{Inner: err, Errors: ValidationErrorClaimsInvalid}
} else {
vErr = e
}
}
// Perform validation
token.Signature = parts[2]
if err = token.Method.Verify(strings.Join(parts[0:2], "."), token.Signature, key); err != nil {
vErr.Inner = err
vErr.Errors |= ValidationErrorSignatureInvalid
}
if vErr.valid() {
token.Valid = true
return token, nil
}
return token, vErr
}

100
src/vendor/github.com/dgrijalva/jwt-go/rsa.go generated vendored Normal file
View File

@ -0,0 +1,100 @@
package jwt
import (
"crypto"
"crypto/rand"
"crypto/rsa"
)
// Implements the RSA family of signing methods signing methods
type SigningMethodRSA struct {
Name string
Hash crypto.Hash
}
// Specific instances for RS256 and company
var (
SigningMethodRS256 *SigningMethodRSA
SigningMethodRS384 *SigningMethodRSA
SigningMethodRS512 *SigningMethodRSA
)
func init() {
// RS256
SigningMethodRS256 = &SigningMethodRSA{"RS256", crypto.SHA256}
RegisterSigningMethod(SigningMethodRS256.Alg(), func() SigningMethod {
return SigningMethodRS256
})
// RS384
SigningMethodRS384 = &SigningMethodRSA{"RS384", crypto.SHA384}
RegisterSigningMethod(SigningMethodRS384.Alg(), func() SigningMethod {
return SigningMethodRS384
})
// RS512
SigningMethodRS512 = &SigningMethodRSA{"RS512", crypto.SHA512}
RegisterSigningMethod(SigningMethodRS512.Alg(), func() SigningMethod {
return SigningMethodRS512
})
}
func (m *SigningMethodRSA) Alg() string {
return m.Name
}
// Implements the Verify method from SigningMethod
// For this signing method, must be an rsa.PublicKey structure.
func (m *SigningMethodRSA) Verify(signingString, signature string, key interface{}) error {
var err error
// Decode the signature
var sig []byte
if sig, err = DecodeSegment(signature); err != nil {
return err
}
var rsaKey *rsa.PublicKey
var ok bool
if rsaKey, ok = key.(*rsa.PublicKey); !ok {
return ErrInvalidKeyType
}
// Create hasher
if !m.Hash.Available() {
return ErrHashUnavailable
}
hasher := m.Hash.New()
hasher.Write([]byte(signingString))
// Verify the signature
return rsa.VerifyPKCS1v15(rsaKey, m.Hash, hasher.Sum(nil), sig)
}
// Implements the Sign method from SigningMethod
// For this signing method, must be an rsa.PrivateKey structure.
func (m *SigningMethodRSA) Sign(signingString string, key interface{}) (string, error) {
var rsaKey *rsa.PrivateKey
var ok bool
// Validate type of key
if rsaKey, ok = key.(*rsa.PrivateKey); !ok {
return "", ErrInvalidKey
}
// Create the hasher
if !m.Hash.Available() {
return "", ErrHashUnavailable
}
hasher := m.Hash.New()
hasher.Write([]byte(signingString))
// Sign the string and return the encoded bytes
if sigBytes, err := rsa.SignPKCS1v15(rand.Reader, rsaKey, m.Hash, hasher.Sum(nil)); err == nil {
return EncodeSegment(sigBytes), nil
} else {
return "", err
}
}

126
src/vendor/github.com/dgrijalva/jwt-go/rsa_pss.go generated vendored Normal file
View File

@ -0,0 +1,126 @@
// +build go1.4
package jwt
import (
"crypto"
"crypto/rand"
"crypto/rsa"
)
// Implements the RSAPSS family of signing methods signing methods
type SigningMethodRSAPSS struct {
*SigningMethodRSA
Options *rsa.PSSOptions
}
// Specific instances for RS/PS and company
var (
SigningMethodPS256 *SigningMethodRSAPSS
SigningMethodPS384 *SigningMethodRSAPSS
SigningMethodPS512 *SigningMethodRSAPSS
)
func init() {
// PS256
SigningMethodPS256 = &SigningMethodRSAPSS{
&SigningMethodRSA{
Name: "PS256",
Hash: crypto.SHA256,
},
&rsa.PSSOptions{
SaltLength: rsa.PSSSaltLengthAuto,
Hash: crypto.SHA256,
},
}
RegisterSigningMethod(SigningMethodPS256.Alg(), func() SigningMethod {
return SigningMethodPS256
})
// PS384
SigningMethodPS384 = &SigningMethodRSAPSS{
&SigningMethodRSA{
Name: "PS384",
Hash: crypto.SHA384,
},
&rsa.PSSOptions{
SaltLength: rsa.PSSSaltLengthAuto,
Hash: crypto.SHA384,
},
}
RegisterSigningMethod(SigningMethodPS384.Alg(), func() SigningMethod {
return SigningMethodPS384
})
// PS512
SigningMethodPS512 = &SigningMethodRSAPSS{
&SigningMethodRSA{
Name: "PS512",
Hash: crypto.SHA512,
},
&rsa.PSSOptions{
SaltLength: rsa.PSSSaltLengthAuto,
Hash: crypto.SHA512,
},
}
RegisterSigningMethod(SigningMethodPS512.Alg(), func() SigningMethod {
return SigningMethodPS512
})
}
// Implements the Verify method from SigningMethod
// For this verify method, key must be an rsa.PublicKey struct
func (m *SigningMethodRSAPSS) Verify(signingString, signature string, key interface{}) error {
var err error
// Decode the signature
var sig []byte
if sig, err = DecodeSegment(signature); err != nil {
return err
}
var rsaKey *rsa.PublicKey
switch k := key.(type) {
case *rsa.PublicKey:
rsaKey = k
default:
return ErrInvalidKey
}
// Create hasher
if !m.Hash.Available() {
return ErrHashUnavailable
}
hasher := m.Hash.New()
hasher.Write([]byte(signingString))
return rsa.VerifyPSS(rsaKey, m.Hash, hasher.Sum(nil), sig, m.Options)
}
// Implements the Sign method from SigningMethod
// For this signing method, key must be an rsa.PrivateKey struct
func (m *SigningMethodRSAPSS) Sign(signingString string, key interface{}) (string, error) {
var rsaKey *rsa.PrivateKey
switch k := key.(type) {
case *rsa.PrivateKey:
rsaKey = k
default:
return "", ErrInvalidKeyType
}
// Create the hasher
if !m.Hash.Available() {
return "", ErrHashUnavailable
}
hasher := m.Hash.New()
hasher.Write([]byte(signingString))
// Sign the string and return the encoded bytes
if sigBytes, err := rsa.SignPSS(rand.Reader, rsaKey, m.Hash, hasher.Sum(nil), m.Options); err == nil {
return EncodeSegment(sigBytes), nil
} else {
return "", err
}
}

69
src/vendor/github.com/dgrijalva/jwt-go/rsa_utils.go generated vendored Normal file
View File

@ -0,0 +1,69 @@
package jwt
import (
"crypto/rsa"
"crypto/x509"
"encoding/pem"
"errors"
)
var (
ErrKeyMustBePEMEncoded = errors.New("Invalid Key: Key must be PEM encoded PKCS1 or PKCS8 private key")
ErrNotRSAPrivateKey = errors.New("Key is not a valid RSA private key")
ErrNotRSAPublicKey = errors.New("Key is not a valid RSA public key")
)
// Parse PEM encoded PKCS1 or PKCS8 private key
func ParseRSAPrivateKeyFromPEM(key []byte) (*rsa.PrivateKey, error) {
var err error
// Parse PEM block
var block *pem.Block
if block, _ = pem.Decode(key); block == nil {
return nil, ErrKeyMustBePEMEncoded
}
var parsedKey interface{}
if parsedKey, err = x509.ParsePKCS1PrivateKey(block.Bytes); err != nil {
if parsedKey, err = x509.ParsePKCS8PrivateKey(block.Bytes); err != nil {
return nil, err
}
}
var pkey *rsa.PrivateKey
var ok bool
if pkey, ok = parsedKey.(*rsa.PrivateKey); !ok {
return nil, ErrNotRSAPrivateKey
}
return pkey, nil
}
// Parse PEM encoded PKCS1 or PKCS8 public key
func ParseRSAPublicKeyFromPEM(key []byte) (*rsa.PublicKey, error) {
var err error
// Parse PEM block
var block *pem.Block
if block, _ = pem.Decode(key); block == nil {
return nil, ErrKeyMustBePEMEncoded
}
// Parse the key
var parsedKey interface{}
if parsedKey, err = x509.ParsePKIXPublicKey(block.Bytes); err != nil {
if cert, err := x509.ParseCertificate(block.Bytes); err == nil {
parsedKey = cert.PublicKey
} else {
return nil, err
}
}
var pkey *rsa.PublicKey
var ok bool
if pkey, ok = parsedKey.(*rsa.PublicKey); !ok {
return nil, ErrNotRSAPublicKey
}
return pkey, nil
}

35
src/vendor/github.com/dgrijalva/jwt-go/signing_method.go generated vendored Normal file
View File

@ -0,0 +1,35 @@
package jwt
import (
"sync"
)
var signingMethods = map[string]func() SigningMethod{}
var signingMethodLock = new(sync.RWMutex)
// Implement SigningMethod to add new methods for signing or verifying tokens.
type SigningMethod interface {
Verify(signingString, signature string, key interface{}) error // Returns nil if signature is valid
Sign(signingString string, key interface{}) (string, error) // Returns encoded signature or error
Alg() string // returns the alg identifier for this method (example: 'HS256')
}
// Register the "alg" name and a factory function for signing method.
// This is typically done during init() in the method's implementation
func RegisterSigningMethod(alg string, f func() SigningMethod) {
signingMethodLock.Lock()
defer signingMethodLock.Unlock()
signingMethods[alg] = f
}
// Get a signing method from an "alg" string
func GetSigningMethod(alg string) (method SigningMethod) {
signingMethodLock.RLock()
defer signingMethodLock.RUnlock()
if methodF, ok := signingMethods[alg]; ok {
method = methodF()
}
return
}

108
src/vendor/github.com/dgrijalva/jwt-go/token.go generated vendored Normal file
View File

@ -0,0 +1,108 @@
package jwt
import (
"encoding/base64"
"encoding/json"
"strings"
"time"
)
// TimeFunc provides the current time when parsing token to validate "exp" claim (expiration time).
// You can override it to use another time value. This is useful for testing or if your
// server uses a different time zone than your tokens.
var TimeFunc = time.Now
// Parse methods use this callback function to supply
// the key for verification. The function receives the parsed,
// but unverified Token. This allows you to use properties in the
// Header of the token (such as `kid`) to identify which key to use.
type Keyfunc func(*Token) (interface{}, error)
// A JWT Token. Different fields will be used depending on whether you're
// creating or parsing/verifying a token.
type Token struct {
Raw string // The raw token. Populated when you Parse a token
Method SigningMethod // The signing method used or to be used
Header map[string]interface{} // The first segment of the token
Claims Claims // The second segment of the token
Signature string // The third segment of the token. Populated when you Parse a token
Valid bool // Is the token valid? Populated when you Parse/Verify a token
}
// Create a new Token. Takes a signing method
func New(method SigningMethod) *Token {
return NewWithClaims(method, MapClaims{})
}
func NewWithClaims(method SigningMethod, claims Claims) *Token {
return &Token{
Header: map[string]interface{}{
"typ": "JWT",
"alg": method.Alg(),
},
Claims: claims,
Method: method,
}
}
// Get the complete, signed token
func (t *Token) SignedString(key interface{}) (string, error) {
var sig, sstr string
var err error
if sstr, err = t.SigningString(); err != nil {
return "", err
}
if sig, err = t.Method.Sign(sstr, key); err != nil {
return "", err
}
return strings.Join([]string{sstr, sig}, "."), nil
}
// Generate the signing string. This is the
// most expensive part of the whole deal. Unless you
// need this for something special, just go straight for
// the SignedString.
func (t *Token) SigningString() (string, error) {
var err error
parts := make([]string, 2)
for i, _ := range parts {
var jsonValue []byte
if i == 0 {
if jsonValue, err = json.Marshal(t.Header); err != nil {
return "", err
}
} else {
if jsonValue, err = json.Marshal(t.Claims); err != nil {
return "", err
}
}
parts[i] = EncodeSegment(jsonValue)
}
return strings.Join(parts, "."), nil
}
// Parse, validate, and return a token.
// keyFunc will receive the parsed token and should return the key for validating.
// If everything is kosher, err will be nil
func Parse(tokenString string, keyFunc Keyfunc) (*Token, error) {
return new(Parser).Parse(tokenString, keyFunc)
}
func ParseWithClaims(tokenString string, claims Claims, keyFunc Keyfunc) (*Token, error) {
return new(Parser).ParseWithClaims(tokenString, claims, keyFunc)
}
// Encode JWT specific base64url encoding with padding stripped
func EncodeSegment(seg []byte) string {
return strings.TrimRight(base64.URLEncoding.EncodeToString(seg), "=")
}
// Decode JWT specific base64url encoding with padding stripped
func DecodeSegment(seg string) ([]byte, error) {
if l := len(seg) % 4; l > 0 {
seg += strings.Repeat("=", 4-l)
}
return base64.URLEncoding.DecodeString(seg)
}

27
src/vendor/github.com/pmezard/go-difflib/LICENSE generated vendored Normal file
View File

@ -0,0 +1,27 @@
Copyright (c) 2013, Patrick Mezard
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
The names of its contributors may not be used to endorse or promote
products derived from this software without specific prior written
permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

772
src/vendor/github.com/pmezard/go-difflib/difflib/difflib.go generated vendored Normal file
View File

@ -0,0 +1,772 @@
// Package difflib is a partial port of Python difflib module.
//
// It provides tools to compare sequences of strings and generate textual diffs.
//
// The following class and functions have been ported:
//
// - SequenceMatcher
//
// - unified_diff
//
// - context_diff
//
// Getting unified diffs was the main goal of the port. Keep in mind this code
// is mostly suitable to output text differences in a human friendly way, there
// are no guarantees generated diffs are consumable by patch(1).
package difflib
import (
"bufio"
"bytes"
"fmt"
"io"
"strings"
)
func min(a, b int) int {
if a < b {
return a
}
return b
}
func max(a, b int) int {
if a > b {
return a
}
return b
}
func calculateRatio(matches, length int) float64 {
if length > 0 {
return 2.0 * float64(matches) / float64(length)
}
return 1.0
}
type Match struct {
A int
B int
Size int
}
type OpCode struct {
Tag byte
I1 int
I2 int
J1 int
J2 int
}
// SequenceMatcher compares sequence of strings. The basic
// algorithm predates, and is a little fancier than, an algorithm
// published in the late 1980's by Ratcliff and Obershelp under the
// hyperbolic name "gestalt pattern matching". The basic idea is to find
// the longest contiguous matching subsequence that contains no "junk"
// elements (R-O doesn't address junk). The same idea is then applied
// recursively to the pieces of the sequences to the left and to the right
// of the matching subsequence. This does not yield minimal edit
// sequences, but does tend to yield matches that "look right" to people.
//
// SequenceMatcher tries to compute a "human-friendly diff" between two
// sequences. Unlike e.g. UNIX(tm) diff, the fundamental notion is the
// longest *contiguous* & junk-free matching subsequence. That's what
// catches peoples' eyes. The Windows(tm) windiff has another interesting
// notion, pairing up elements that appear uniquely in each sequence.
// That, and the method here, appear to yield more intuitive difference
// reports than does diff. This method appears to be the least vulnerable
// to synching up on blocks of "junk lines", though (like blank lines in
// ordinary text files, or maybe "<P>" lines in HTML files). That may be
// because this is the only method of the 3 that has a *concept* of
// "junk" <wink>.
//
// Timing: Basic R-O is cubic time worst case and quadratic time expected
// case. SequenceMatcher is quadratic time for the worst case and has
// expected-case behavior dependent in a complicated way on how many
// elements the sequences have in common; best case time is linear.
type SequenceMatcher struct {
a []string
b []string
b2j map[string][]int
IsJunk func(string) bool
autoJunk bool
bJunk map[string]struct{}
matchingBlocks []Match
fullBCount map[string]int
bPopular map[string]struct{}
opCodes []OpCode
}
func NewMatcher(a, b []string) *SequenceMatcher {
m := SequenceMatcher{autoJunk: true}
m.SetSeqs(a, b)
return &m
}
func NewMatcherWithJunk(a, b []string, autoJunk bool,
isJunk func(string) bool) *SequenceMatcher {
m := SequenceMatcher{IsJunk: isJunk, autoJunk: autoJunk}
m.SetSeqs(a, b)
return &m
}
// Set two sequences to be compared.
func (m *SequenceMatcher) SetSeqs(a, b []string) {
m.SetSeq1(a)
m.SetSeq2(b)
}
// Set the first sequence to be compared. The second sequence to be compared is
// not changed.
//
// SequenceMatcher computes and caches detailed information about the second
// sequence, so if you want to compare one sequence S against many sequences,
// use .SetSeq2(s) once and call .SetSeq1(x) repeatedly for each of the other
// sequences.
//
// See also SetSeqs() and SetSeq2().
func (m *SequenceMatcher) SetSeq1(a []string) {
if &a == &m.a {
return
}
m.a = a
m.matchingBlocks = nil
m.opCodes = nil
}
// Set the second sequence to be compared. The first sequence to be compared is
// not changed.
func (m *SequenceMatcher) SetSeq2(b []string) {
if &b == &m.b {
return
}
m.b = b
m.matchingBlocks = nil
m.opCodes = nil
m.fullBCount = nil
m.chainB()
}
func (m *SequenceMatcher) chainB() {
// Populate line -> index mapping
b2j := map[string][]int{}
for i, s := range m.b {
indices := b2j[s]
indices = append(indices, i)
b2j[s] = indices
}
// Purge junk elements
m.bJunk = map[string]struct{}{}
if m.IsJunk != nil {
junk := m.bJunk
for s, _ := range b2j {
if m.IsJunk(s) {
junk[s] = struct{}{}
}
}
for s, _ := range junk {
delete(b2j, s)
}
}
// Purge remaining popular elements
popular := map[string]struct{}{}
n := len(m.b)
if m.autoJunk && n >= 200 {
ntest := n/100 + 1
for s, indices := range b2j {
if len(indices) > ntest {
popular[s] = struct{}{}
}
}
for s, _ := range popular {
delete(b2j, s)
}
}
m.bPopular = popular
m.b2j = b2j
}
func (m *SequenceMatcher) isBJunk(s string) bool {
_, ok := m.bJunk[s]
return ok
}
// Find longest matching block in a[alo:ahi] and b[blo:bhi].
//
// If IsJunk is not defined:
//
// Return (i,j,k) such that a[i:i+k] is equal to b[j:j+k], where
// alo <= i <= i+k <= ahi
// blo <= j <= j+k <= bhi
// and for all (i',j',k') meeting those conditions,
// k >= k'
// i <= i'
// and if i == i', j <= j'
//
// In other words, of all maximal matching blocks, return one that
// starts earliest in a, and of all those maximal matching blocks that
// start earliest in a, return the one that starts earliest in b.
//
// If IsJunk is defined, first the longest matching block is
// determined as above, but with the additional restriction that no
// junk element appears in the block. Then that block is extended as
// far as possible by matching (only) junk elements on both sides. So
// the resulting block never matches on junk except as identical junk
// happens to be adjacent to an "interesting" match.
//
// If no blocks match, return (alo, blo, 0).
func (m *SequenceMatcher) findLongestMatch(alo, ahi, blo, bhi int) Match {
// CAUTION: stripping common prefix or suffix would be incorrect.
// E.g.,
// ab
// acab
// Longest matching block is "ab", but if common prefix is
// stripped, it's "a" (tied with "b"). UNIX(tm) diff does so
// strip, so ends up claiming that ab is changed to acab by
// inserting "ca" in the middle. That's minimal but unintuitive:
// "it's obvious" that someone inserted "ac" at the front.
// Windiff ends up at the same place as diff, but by pairing up
// the unique 'b's and then matching the first two 'a's.
besti, bestj, bestsize := alo, blo, 0
// find longest junk-free match
// during an iteration of the loop, j2len[j] = length of longest
// junk-free match ending with a[i-1] and b[j]
j2len := map[int]int{}
for i := alo; i != ahi; i++ {
// look at all instances of a[i] in b; note that because
// b2j has no junk keys, the loop is skipped if a[i] is junk
newj2len := map[int]int{}
for _, j := range m.b2j[m.a[i]] {
// a[i] matches b[j]
if j < blo {
continue
}
if j >= bhi {
break
}
k := j2len[j-1] + 1
newj2len[j] = k
if k > bestsize {
besti, bestj, bestsize = i-k+1, j-k+1, k
}
}
j2len = newj2len
}
// Extend the best by non-junk elements on each end. In particular,
// "popular" non-junk elements aren't in b2j, which greatly speeds
// the inner loop above, but also means "the best" match so far
// doesn't contain any junk *or* popular non-junk elements.
for besti > alo && bestj > blo && !m.isBJunk(m.b[bestj-1]) &&
m.a[besti-1] == m.b[bestj-1] {
besti, bestj, bestsize = besti-1, bestj-1, bestsize+1
}
for besti+bestsize < ahi && bestj+bestsize < bhi &&
!m.isBJunk(m.b[bestj+bestsize]) &&
m.a[besti+bestsize] == m.b[bestj+bestsize] {
bestsize += 1
}
// Now that we have a wholly interesting match (albeit possibly
// empty!), we may as well suck up the matching junk on each
// side of it too. Can't think of a good reason not to, and it
// saves post-processing the (possibly considerable) expense of
// figuring out what to do with it. In the case of an empty
// interesting match, this is clearly the right thing to do,
// because no other kind of match is possible in the regions.
for besti > alo && bestj > blo && m.isBJunk(m.b[bestj-1]) &&
m.a[besti-1] == m.b[bestj-1] {
besti, bestj, bestsize = besti-1, bestj-1, bestsize+1
}
for besti+bestsize < ahi && bestj+bestsize < bhi &&
m.isBJunk(m.b[bestj+bestsize]) &&
m.a[besti+bestsize] == m.b[bestj+bestsize] {
bestsize += 1
}
return Match{A: besti, B: bestj, Size: bestsize}
}
// Return list of triples describing matching subsequences.
//
// Each triple is of the form (i, j, n), and means that
// a[i:i+n] == b[j:j+n]. The triples are monotonically increasing in
// i and in j. It's also guaranteed that if (i, j, n) and (i', j', n') are
// adjacent triples in the list, and the second is not the last triple in the
// list, then i+n != i' or j+n != j'. IOW, adjacent triples never describe
// adjacent equal blocks.
//
// The last triple is a dummy, (len(a), len(b), 0), and is the only
// triple with n==0.
func (m *SequenceMatcher) GetMatchingBlocks() []Match {
if m.matchingBlocks != nil {
return m.matchingBlocks
}
var matchBlocks func(alo, ahi, blo, bhi int, matched []Match) []Match
matchBlocks = func(alo, ahi, blo, bhi int, matched []Match) []Match {
match := m.findLongestMatch(alo, ahi, blo, bhi)
i, j, k := match.A, match.B, match.Size
if match.Size > 0 {
if alo < i && blo < j {
matched = matchBlocks(alo, i, blo, j, matched)
}
matched = append(matched, match)
if i+k < ahi && j+k < bhi {
matched = matchBlocks(i+k, ahi, j+k, bhi, matched)
}
}
return matched
}
matched := matchBlocks(0, len(m.a), 0, len(m.b), nil)
// It's possible that we have adjacent equal blocks in the
// matching_blocks list now.
nonAdjacent := []Match{}
i1, j1, k1 := 0, 0, 0
for _, b := range matched {
// Is this block adjacent to i1, j1, k1?
i2, j2, k2 := b.A, b.B, b.Size
if i1+k1 == i2 && j1+k1 == j2 {
// Yes, so collapse them -- this just increases the length of
// the first block by the length of the second, and the first
// block so lengthened remains the block to compare against.
k1 += k2
} else {
// Not adjacent. Remember the first block (k1==0 means it's
// the dummy we started with), and make the second block the
// new block to compare against.
if k1 > 0 {
nonAdjacent = append(nonAdjacent, Match{i1, j1, k1})
}
i1, j1, k1 = i2, j2, k2
}
}
if k1 > 0 {
nonAdjacent = append(nonAdjacent, Match{i1, j1, k1})
}
nonAdjacent = append(nonAdjacent, Match{len(m.a), len(m.b), 0})
m.matchingBlocks = nonAdjacent
return m.matchingBlocks
}
// Return list of 5-tuples describing how to turn a into b.
//
// Each tuple is of the form (tag, i1, i2, j1, j2). The first tuple
// has i1 == j1 == 0, and remaining tuples have i1 == the i2 from the
// tuple preceding it, and likewise for j1 == the previous j2.
//
// The tags are characters, with these meanings:
//
// 'r' (replace): a[i1:i2] should be replaced by b[j1:j2]
//
// 'd' (delete): a[i1:i2] should be deleted, j1==j2 in this case.
//
// 'i' (insert): b[j1:j2] should be inserted at a[i1:i1], i1==i2 in this case.
//
// 'e' (equal): a[i1:i2] == b[j1:j2]
func (m *SequenceMatcher) GetOpCodes() []OpCode {
if m.opCodes != nil {
return m.opCodes
}
i, j := 0, 0
matching := m.GetMatchingBlocks()
opCodes := make([]OpCode, 0, len(matching))
for _, m := range matching {
// invariant: we've pumped out correct diffs to change
// a[:i] into b[:j], and the next matching block is
// a[ai:ai+size] == b[bj:bj+size]. So we need to pump
// out a diff to change a[i:ai] into b[j:bj], pump out
// the matching block, and move (i,j) beyond the match
ai, bj, size := m.A, m.B, m.Size
tag := byte(0)
if i < ai && j < bj {
tag = 'r'
} else if i < ai {
tag = 'd'
} else if j < bj {
tag = 'i'
}
if tag > 0 {
opCodes = append(opCodes, OpCode{tag, i, ai, j, bj})
}
i, j = ai+size, bj+size
// the list of matching blocks is terminated by a
// sentinel with size 0
if size > 0 {
opCodes = append(opCodes, OpCode{'e', ai, i, bj, j})
}
}
m.opCodes = opCodes
return m.opCodes
}
// Isolate change clusters by eliminating ranges with no changes.
//
// Return a generator of groups with up to n lines of context.
// Each group is in the same format as returned by GetOpCodes().
func (m *SequenceMatcher) GetGroupedOpCodes(n int) [][]OpCode {
if n < 0 {
n = 3
}
codes := m.GetOpCodes()
if len(codes) == 0 {
codes = []OpCode{OpCode{'e', 0, 1, 0, 1}}
}
// Fixup leading and trailing groups if they show no changes.
if codes[0].Tag == 'e' {
c := codes[0]
i1, i2, j1, j2 := c.I1, c.I2, c.J1, c.J2
codes[0] = OpCode{c.Tag, max(i1, i2-n), i2, max(j1, j2-n), j2}
}
if codes[len(codes)-1].Tag == 'e' {
c := codes[len(codes)-1]
i1, i2, j1, j2 := c.I1, c.I2, c.J1, c.J2
codes[len(codes)-1] = OpCode{c.Tag, i1, min(i2, i1+n), j1, min(j2, j1+n)}
}
nn := n + n
groups := [][]OpCode{}
group := []OpCode{}
for _, c := range codes {
i1, i2, j1, j2 := c.I1, c.I2, c.J1, c.J2
// End the current group and start a new one whenever
// there is a large range with no changes.
if c.Tag == 'e' && i2-i1 > nn {
group = append(group, OpCode{c.Tag, i1, min(i2, i1+n),
j1, min(j2, j1+n)})
groups = append(groups, group)
group = []OpCode{}
i1, j1 = max(i1, i2-n), max(j1, j2-n)
}
group = append(group, OpCode{c.Tag, i1, i2, j1, j2})
}
if len(group) > 0 && !(len(group) == 1 && group[0].Tag == 'e') {
groups = append(groups, group)
}
return groups
}
// Return a measure of the sequences' similarity (float in [0,1]).
//
// Where T is the total number of elements in both sequences, and
// M is the number of matches, this is 2.0*M / T.
// Note that this is 1 if the sequences are identical, and 0 if
// they have nothing in common.
//
// .Ratio() is expensive to compute if you haven't already computed
// .GetMatchingBlocks() or .GetOpCodes(), in which case you may
// want to try .QuickRatio() or .RealQuickRation() first to get an
// upper bound.
func (m *SequenceMatcher) Ratio() float64 {
matches := 0
for _, m := range m.GetMatchingBlocks() {
matches += m.Size
}
return calculateRatio(matches, len(m.a)+len(m.b))
}
// Return an upper bound on ratio() relatively quickly.
//
// This isn't defined beyond that it is an upper bound on .Ratio(), and
// is faster to compute.
func (m *SequenceMatcher) QuickRatio() float64 {
// viewing a and b as multisets, set matches to the cardinality
// of their intersection; this counts the number of matches
// without regard to order, so is clearly an upper bound
if m.fullBCount == nil {
m.fullBCount = map[string]int{}
for _, s := range m.b {
m.fullBCount[s] = m.fullBCount[s] + 1
}
}
// avail[x] is the number of times x appears in 'b' less the
// number of times we've seen it in 'a' so far ... kinda
avail := map[string]int{}
matches := 0
for _, s := range m.a {
n, ok := avail[s]
if !ok {
n = m.fullBCount[s]
}
avail[s] = n - 1
if n > 0 {
matches += 1
}
}
return calculateRatio(matches, len(m.a)+len(m.b))
}
// Return an upper bound on ratio() very quickly.
//
// This isn't defined beyond that it is an upper bound on .Ratio(), and
// is faster to compute than either .Ratio() or .QuickRatio().
func (m *SequenceMatcher) RealQuickRatio() float64 {
la, lb := len(m.a), len(m.b)
return calculateRatio(min(la, lb), la+lb)
}
// Convert range to the "ed" format
func formatRangeUnified(start, stop int) string {
// Per the diff spec at http://www.unix.org/single_unix_specification/
beginning := start + 1 // lines start numbering with one
length := stop - start
if length == 1 {
return fmt.Sprintf("%d", beginning)
}
if length == 0 {
beginning -= 1 // empty ranges begin at line just before the range
}
return fmt.Sprintf("%d,%d", beginning, length)
}
// Unified diff parameters
type UnifiedDiff struct {
A []string // First sequence lines
FromFile string // First file name
FromDate string // First file time
B []string // Second sequence lines
ToFile string // Second file name
ToDate string // Second file time
Eol string // Headers end of line, defaults to LF
Context int // Number of context lines
}
// Compare two sequences of lines; generate the delta as a unified diff.
//
// Unified diffs are a compact way of showing line changes and a few
// lines of context. The number of context lines is set by 'n' which
// defaults to three.
//
// By default, the diff control lines (those with ---, +++, or @@) are
// created with a trailing newline. This is helpful so that inputs
// created from file.readlines() result in diffs that are suitable for
// file.writelines() since both the inputs and outputs have trailing
// newlines.
//
// For inputs that do not have trailing newlines, set the lineterm
// argument to "" so that the output will be uniformly newline free.
//
// The unidiff format normally has a header for filenames and modification
// times. Any or all of these may be specified using strings for
// 'fromfile', 'tofile', 'fromfiledate', and 'tofiledate'.
// The modification times are normally expressed in the ISO 8601 format.
func WriteUnifiedDiff(writer io.Writer, diff UnifiedDiff) error {
buf := bufio.NewWriter(writer)
defer buf.Flush()
wf := func(format string, args ...interface{}) error {
_, err := buf.WriteString(fmt.Sprintf(format, args...))
return err
}
ws := func(s string) error {
_, err := buf.WriteString(s)
return err
}
if len(diff.Eol) == 0 {
diff.Eol = "\n"
}
started := false
m := NewMatcher(diff.A, diff.B)
for _, g := range m.GetGroupedOpCodes(diff.Context) {
if !started {
started = true
fromDate := ""
if len(diff.FromDate) > 0 {
fromDate = "\t" + diff.FromDate
}
toDate := ""
if len(diff.ToDate) > 0 {
toDate = "\t" + diff.ToDate
}
if diff.FromFile != "" || diff.ToFile != "" {
err := wf("--- %s%s%s", diff.FromFile, fromDate, diff.Eol)
if err != nil {
return err
}
err = wf("+++ %s%s%s", diff.ToFile, toDate, diff.Eol)
if err != nil {
return err
}
}
}
first, last := g[0], g[len(g)-1]
range1 := formatRangeUnified(first.I1, last.I2)
range2 := formatRangeUnified(first.J1, last.J2)
if err := wf("@@ -%s +%s @@%s", range1, range2, diff.Eol); err != nil {
return err
}
for _, c := range g {
i1, i2, j1, j2 := c.I1, c.I2, c.J1, c.J2
if c.Tag == 'e' {
for _, line := range diff.A[i1:i2] {
if err := ws(" " + line); err != nil {
return err
}
}
continue
}
if c.Tag == 'r' || c.Tag == 'd' {
for _, line := range diff.A[i1:i2] {
if err := ws("-" + line); err != nil {
return err
}
}
}
if c.Tag == 'r' || c.Tag == 'i' {
for _, line := range diff.B[j1:j2] {
if err := ws("+" + line); err != nil {
return err
}
}
}
}
}
return nil
}
// Like WriteUnifiedDiff but returns the diff a string.
func GetUnifiedDiffString(diff UnifiedDiff) (string, error) {
w := &bytes.Buffer{}
err := WriteUnifiedDiff(w, diff)
return string(w.Bytes()), err
}
// Convert range to the "ed" format.
func formatRangeContext(start, stop int) string {
// Per the diff spec at http://www.unix.org/single_unix_specification/
beginning := start + 1 // lines start numbering with one
length := stop - start
if length == 0 {
beginning -= 1 // empty ranges begin at line just before the range
}
if length <= 1 {
return fmt.Sprintf("%d", beginning)
}
return fmt.Sprintf("%d,%d", beginning, beginning+length-1)
}
type ContextDiff UnifiedDiff
// Compare two sequences of lines; generate the delta as a context diff.
//
// Context diffs are a compact way of showing line changes and a few
// lines of context. The number of context lines is set by diff.Context
// which defaults to three.
//
// By default, the diff control lines (those with *** or ---) are
// created with a trailing newline.
//
// For inputs that do not have trailing newlines, set the diff.Eol
// argument to "" so that the output will be uniformly newline free.
//
// The context diff format normally has a header for filenames and
// modification times. Any or all of these may be specified using
// strings for diff.FromFile, diff.ToFile, diff.FromDate, diff.ToDate.
// The modification times are normally expressed in the ISO 8601 format.
// If not specified, the strings default to blanks.
func WriteContextDiff(writer io.Writer, diff ContextDiff) error {
buf := bufio.NewWriter(writer)
defer buf.Flush()
var diffErr error
wf := func(format string, args ...interface{}) {
_, err := buf.WriteString(fmt.Sprintf(format, args...))
if diffErr == nil && err != nil {
diffErr = err
}
}
ws := func(s string) {
_, err := buf.WriteString(s)
if diffErr == nil && err != nil {
diffErr = err
}
}
if len(diff.Eol) == 0 {
diff.Eol = "\n"
}
prefix := map[byte]string{
'i': "+ ",
'd': "- ",
'r': "! ",
'e': " ",
}
started := false
m := NewMatcher(diff.A, diff.B)
for _, g := range m.GetGroupedOpCodes(diff.Context) {
if !started {
started = true
fromDate := ""
if len(diff.FromDate) > 0 {
fromDate = "\t" + diff.FromDate
}
toDate := ""
if len(diff.ToDate) > 0 {
toDate = "\t" + diff.ToDate
}
if diff.FromFile != "" || diff.ToFile != "" {
wf("*** %s%s%s", diff.FromFile, fromDate, diff.Eol)
wf("--- %s%s%s", diff.ToFile, toDate, diff.Eol)
}
}
first, last := g[0], g[len(g)-1]
ws("***************" + diff.Eol)
range1 := formatRangeContext(first.I1, last.I2)
wf("*** %s ****%s", range1, diff.Eol)
for _, c := range g {
if c.Tag == 'r' || c.Tag == 'd' {
for _, cc := range g {
if cc.Tag == 'i' {
continue
}
for _, line := range diff.A[cc.I1:cc.I2] {
ws(prefix[cc.Tag] + line)
}
}
break
}
}
range2 := formatRangeContext(first.J1, last.J2)
wf("--- %s ----%s", range2, diff.Eol)
for _, c := range g {
if c.Tag == 'r' || c.Tag == 'i' {
for _, cc := range g {
if cc.Tag == 'd' {
continue
}
for _, line := range diff.B[cc.J1:cc.J2] {
ws(prefix[cc.Tag] + line)
}
}
break
}
}
}
return diffErr
}
// Like WriteContextDiff but returns the diff a string.
func GetContextDiffString(diff ContextDiff) (string, error) {
w := &bytes.Buffer{}
err := WriteContextDiff(w, diff)
return string(w.Bytes()), err
}
// Split a string on "\n" while preserving them. The output can be used
// as input for UnifiedDiff and ContextDiff structures.
func SplitLines(s string) []string {
lines := strings.SplitAfter(s, "\n")
lines[len(lines)-1] += "\n"
return lines
}

23
src/vendor/github.com/stretchr/objx/LICENSE.md generated vendored Normal file
View File

@ -0,0 +1,23 @@
objx - by Mat Ryer and Tyler Bunnell
The MIT License (MIT)
Copyright (c) 2014 Stretchr, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

3
src/vendor/github.com/stretchr/objx/README.md generated vendored Normal file
View File

@ -0,0 +1,3 @@
# objx
* Jump into the [API Documentation](http://godoc.org/github.com/stretchr/objx)

Some files were not shown because too many files have changed in this diff Show More