10 KiB
Server Architecture
The Server is divided into a number of services. Each service is a Visual Studio project in the Server solution. These are:
- Admin
- Api
- Icons
- Identity
- Notifications
- SQL
Each service is built and run separately. The Bitwarden clients can use different servers for different services.
This means that you don't need to run all services locally for a development environment. You can run only those services that you intend to modify, and use Bitwarden.com or a self-hosted instance for all other services required.
By default some of the services depends on the Bitwarden licensed CommCore, however it can easily be disabled by including the /p:DefineConstants="OSS" as an argument to dotnet.
Local Development Environment Setup
This guide will show you how to set up the Api, Identity and SQL projects for development. These are the minimum projects for any development work. You may need to set up additional projects depending on the changes you want to make.
We recommend using Visual Studio, and PowerShell which is used for the helper scripts.
Docker containers
To simplify the setup process we provide a Docker Compose application model. This is split up into multiple service profiles to facilitate easily customization.
Some settings can be customized by modifying the dev/.env file, such as the MSSQL_PASSWORD which should be modified before starting the project.
# Copy the example environment file
cp ./.env.example ./.env
# We recommend running the following command when developing for self-hosted
docker compose --profile mssql --profile mail up
# We also provide a storage profile which uses Azurite to emulate some services used by the cloud instance
# Usually only needed by internal Bitwarden developers
docker compose --profile cloud --profile mail up
SQL Server
We recommend changing the MSSQL_PASSWORD variable in dev/.env to avoid exposing the sqlserver with a default password. Note: changing this after first running docker compose may require a re-creation of the storage volume. To do this, stop the running containers and run docker volume rm bitwardenserver_mssql_dev_data. (Warning: this will delete your development database.)
We provide a helper script which will create the development database vault_dev and also run all migrations. This command should be run after starting docker the first time, as well as after syncing against upstream and after creating a new migration.
.\dev\migrate.ps1
# You can also re-run the last migration using
.\dev\migrate.ps1 -r
Note: If all or many migrations are skipped even though this is a new database, make sure that there is not a last_migration file located in dev/.data/mssql. If there is, remove it and run the helper script again. This can happen if you create the database, run migrations, then delete it.
Azurite
Azurite is a emulator for Azure Storage API and supports Blob, Queues and Table storage. We use it to avoid a hard dependency on online services for cloud development.
To bootstrap the local Azurite instance please run the following command:
# This script requires the Az module, which can be installed using
Install-Module -Name Az -Scope CurrentUser -Repository PSGallery -Force
.\dev\setup_azurite.ps1
Mailcatcher
Since the server uses emails for many user interactions a working SMTP server is a requirement, we provide a pre-setup instance of MailCatcher which exposes a web interface at http://localhost:1080.
Certificates
In order to run Bitwarden, we require two certificates which for local development can be resolved by using self signed certificates.
Windows
We provide a helper script which will generate and add the certificates to the users Certificate Store. After running the script it will output the thumbprints needed for the next step. The certificates can later be accessed using certml.msc under Personal/Certificates.
.\create_certificates_windows.ps1
PSParentPath: Microsoft.PowerShell.Security\Certificate::CurrentUser\My
Thumbprint Subject
---------- -------
0BE8A0072214AB37C6928968752F698EEC3A68B5 CN=Bitwarden Identity Server Dev
C3A6CECAD3DB580F91A52FC9C767FE780300D8AB CN=Bitwarden Data Protection Dev
MacOS
We provide a helper script which will generate the certificates and add them to the keychain.
Note: You should update the Trust options for each certificate to always trust using Keychain Access.
./create_certificates_mac.sh
Certificate fingerprints:
Identity Server Dev: 0BE8A0072214AB37C6928968752F698EEC3A68B5
Data Protection Dev: C3A6CECAD3DB580F91A52FC9C767FE780300D8AB
User Secrets
User secrets are a method for managing application settings on a per-developer basis. They are stored outside of the local git repository so that they are not pushed to remote.
User secrets override the settings in appsettings.json of each project. Your user secrets file should match the structure of the appsettings.json file for the settings you intend to override.
For more information, see: Safe storage of app secrets in development in ASP.NET Core.
Automated Helper script
We provide a helper scripts which simplifies setting user secrets for all projects in the repository.
Start by copying the secret.json.example file to secret.json and modify the existing settings and add any other required setting. Afterwards run the following command which will add the settings to each project in the bitwarden repository.
.\setup_secrets.ps1
# The script also supports an optional flag which removes all existing settings before re-applying them
.\setup_secrets.ps1 -clear 1
Manually creating and modifying
It is also possible to manually create and modify the user secrets using either the dotnet CLI or Visual Studio on Windows. For more details see Appendix A.
Required User Secrets
selfhosted: It is highly recommended that you use the selfHosted: true setting when running a local development environment. This tells the system not to use cloud services, assuming that you are running your own local SQL instance.
sqlServer__connectionString: this provides the information required for the Server to connect to the SQL instance. See the example connection string in secrets.json.example. You may need to change the default password in the connection string.
licenseDirectory: this must be set to avoid errors, but it can be set to an arbitrary empty folder.
installation__key and installation__id: request your own private Installation Id and Installation Key for self-hosting: https://bitwarden.com/host/.
Running and Debugging
After you have completed the above steps, you should be ready to launch your development environment for the Api and Identity projects.
Visual Studio
To debug:
- On Windows, right-click on each project > click Debug > click Start New Instance
- On MacOS, right-click each project > click Start Debugging Project
To run without debugging, open a terminal and navigate to the location of the .csproj file for that project (usually in src/ProjectName). Start the project with:
dotnet run
NOTE: check the output of the running project to find the port it is listening on. If this is different to the default in appsettings.json, you may need to update your user secrets to override this (typically the Api user secrets for the Identity URL).
Rider
From within Rider, launch both the Api project and the Identity project by clicking the "play" button for each project separately.
Testing your deployment
- To test the deployment of each project, navigate to the following pages in your browser. You should see server output and no errors:
- Test the Api deployment: http://localhost:4000/alive
- Test the Identity deployment: http://localhost:33656/.well-known/openid-configuration
- If your test was successful, you can connect a GUI client to the dev environment by following the instructions here: Change your client application's environment. If you are following this guide, you should only set the API Server URL and Identity Server URL to localhost:port and leave all other fields blank.
- If you are using the CLI client, you will also need to set the Node environment variables for your self-signed certificates by following the instructions here: The Bitwarden command-line tool (CLI) > Self-signed certificates.
Troubleshooting
- If you get a 404 error, the projects may be listening on a non-default port. Check the output of your running projects to check the port they are listening on.
Appendix A (User Secrets)
Editing user secrets - Visual Studio on Windows
Right-click on the project in the Solution Explorer and click Manage User Secrets.
Editing user secrets - Visual Studio on macOS
Open a terminal and navigate to the project directory. Once there, initiate and create the blank user secrets file by running:
dotnet user-secrets init
Add a user secret by running:
dotnet user-secrets set "<key>" "<value>"
View currently set secrets by running:
dotnet user-secrets list
By default, user secret files are located in ~/.microsoft/usersecrets/<project name>/secrets.json. After the file has been created, you can edit this directly with VSCode, which is much easier than using the command line tool.
Editing user secrets - Rider
- Navigate to Preferences -> Plugins and Install .NET Core User Secrets
- Right click on the a project and click Tools > Open project user secrets