* Use StrictEmail validation for changing email
* Add trailing symbols to illegal chars in emails
* Add semicolon as always illegal
* Replace regex with MimeKit parsing, add unit test
* Add more unit tests
* Fix linting
* Share globalSettings hcaptcha public key with clients
* Require captcha valid only prior to two factor
users with two factor will have already solved captcha is necessary.
Users without two factor will have`TwoFactorVerified` set to false
* Do not require CaptchaResponse on two-factor requests
* Add option to always require captcha for testing purposes
* Allow for self-hosted instances if they want to use it
* Move refresh suggestion to correct error
* Expect lifetime in helper method
* Add captcha bypass token to successful captcha validations
* Remove twofactorValidated
* PR Feedback
* increasing the log levels on all of the services to prevent logging successes
* resetting the default logging and adding in constraints in logging in Production
* EF Database Support Init (#1221)
* scaffolding for ef support
* deleted old postgres repos
* added tables to oncreate
* updated all the things to .NET 5
* Addition to #1221: Migrated DockerFiles from dotnet/3.1 to 5.0 (#1223)
* Migrated DockerFiles from dotnet/3.1 to 5.0
* Migrated SSO/Dockerfile from dotnet 3.1 to 5.0
Co-authored-by: Daniel James Smith <djsmith85@users.noreply.github.com>
* EFDatabaseSupport: Updated links and description in README.md and SETUP.md (#1232)
* Updated requirements in README.md
* Updated link to documentation of app-secrets
* upgraded dotnet version to 5.0
* Ef database support implementation examples (#1265)
* mostly finished testing the user repo
* finished testing user repo
* finished org, user, ssoconfig, and ssouser ef implementations
* removed unused prop
* fixed a sql file
* fixed a spacing issue
* fixed a spacing issue
* removed extra database creation
* refactoring
* MsSql => SqlServer
* refactoring
* code review fixes
* build fix
* code review
* continued attempts to fix the the build
* skipped another test
* finished all create test
* initial pass at several repos
* continued building out repos
* initial pass at several repos
* initial pass at device repo
* initial pass at collection repo
* initial run of all Entity Framework implementations
* signup, signin, create/edit ciphers works
* sync working
* all web vault pages seem to load with 100% 200s
* bulkcopy, folders, and favorites
* group and collection management
* sso, groups, emergency access, send
* get basic creates matching on all repos
* got everything building again post merge
* removed some IDE config files
* cleanup
* no more notimplemented methods in the cipher repo
* no more not implementeds everywhere
* cleaned up schema/navigation properties and fixed tests
* removed a sql comment that was written in c# style
* fixed build issues from merge
* removed unsupported db providers
* formatting
* code review refactors
* naming cleanup for queries
* added provider methods
* cipher repo cleanup
* implemented several missing procedures from the EF implementation surround account revision dates, keys, and storage
* fixed the build
* added a null check
* consolidated some cipher repo methods
* formatting fix
* cleaned up indentation of queries
* removed .idea file
* generated postgres migrations
* added mysql migrations
* formatting
* Bug Fixes & Formatting
* Formatting
* fixed a bug with bulk import when using MySql
* code review fixes
* fixed the build
* implemented new methods
* formatting
* fixed the build
* cleaned up select statements in ef queries
* formatting
* formatting
* formatting
Co-authored-by: Daniel James Smith <djsmith85@users.noreply.github.com>
* Revert "Encode into b64 to avoid illegal xml encoding when sending to Azure (#1425)"
This reverts commit 2c9a5bb4ab.
* Azure queue to use base64 encoding universally
* Ensure byte size calc is using encoded byte count
* Remove message text extension from blockIP svc
* Remove unused using on blockIp hosted service
* Encode into b64 to avoid illegal xml encoding when sending to Azure
* Revert "Encode into b64 to avoid illegal xml encoding when sending to Azure"
This reverts commit d50de941da.
* HtmlEncode strings if they use multi-byte characters
* Add serializer to event processor
* Rename to used class
* Formatting
* PR feedback
* Creat TVP prior to opening sql connection
Data Table creation is slow. connection may be timing out while we create it.
* USe MARS-enabled connections to fix connection issue
https://github.com/dotnet/SqlClient/issues/54
* Added SsoConfigService tests
* Cleanup whitespace in SsoConfigServiceTests
* Work on PolicyServiceTests
* Refactor PolicyService to remove uneeded calls
* Implement Code Coverage
* Continued work on PolicyServiceTests
* Revert "Implement Code Coverage"
This reverts commit 4ada179ada.
* Fix PolicyServiceTests after rebasing
* Cleanup unused namespaces
* Added assertions that saving or logging of save aren't happening on exceptions
* Add support for bulk confirm
* Add missing sproc to migration
* Change ConfirmUserAsync to internally use ConfirmUsersAsync
* Refactor to be a bit more readable
* Change BulkReinvite and BulkRemove to return a list of errors/success
* Refactor
* Fix removing owner preventing removing non owners
* Add another unit test
* Use fixtures for OrganizationUser and Policies
* Fix spelling
* [Reset Password] Organization Keys APIs
* Updated details response to include private key and added more security checks for reset password methods
* Added org type and policy security checks to the enrollment api
* Updated based on PR feedback
* Added org user type permission checks
* Added TODO for email to user
* Removed unecessary policyRepository object
* Increase organization max seat size from 30k to 2b (#1274)
* Increase organization max seat size from 30k to 2b
* PR review. Do not modify unless state matches expected
* Organization sync simultaneous event reporting (#1275)
* Split up azure messages according to max size
* Allow simultaneous login of organization user events
* Early resolve small event lists
* Clarify logic
Co-authored-by: Chad Scharf <3904944+cscharf@users.noreply.github.com>
* Improve readability
This comes at the cost of multiple serializations, but the
improvement in wire-time should more than make up for this
on message where serialization time matters
Co-authored-by: Chad Scharf <3904944+cscharf@users.noreply.github.com>
* Queue emails (#1286)
* Extract common Azure queue methods
* Do not use internal entity framework namespace
* Prefer IEnumerable to IList unless needed
All of these implementations were just using `Count == 1`,
which is easily replicated. This will be used when abstracting Azure queues
* Add model for azure queue message
* Abstract Azure queue for reuse
* Creat service to enqueue mail messages for later processing
Azure queue mail service uses Azure queues.
Blocking just blocks until all the work is done -- This is
how emailing works today
* Provide mail queue service to DI
* Queue organization invite emails for later processing
All emails can later be added to this queue
* Create Admin hosted service to process enqueued mail messages
* Prefer constructors to static generators
* Mass delete organization users (#1287)
* Add delete many to Organization Users
* Correct formatting
* Remove erroneous migration
* Clarify parameter name
* Formatting fixes
* Simplify bump account revision sproc
* Formatting fixes
* Match file names to objects
* Indicate if large import is expected
* Early pull all existing users we were planning on inviting (#1290)
* Early pull all existing users we were planning on inviting
* Improve sproc name
* Batch upsert org users (#1289)
* Add UpsertMany sprocs to OrganizationUser
* Add method to create TVPs from any object.
Uses DbOrder attribute to generate.
Sproc will fail unless TVP column order matches that of the db type
* Combine migrations
* Correct formatting
* Include sql objects in sql project
* Keep consisten parameter names
* Batch deletes for performance
* Correct formatting
* consolidate migrations
* Use batch methods in OrganizationImport
* Declare @BatchSize
* Transaction names limited to 32 chars
Drop sproc before creating it if it exists
* Update import tests
* Allow for more users in org upgrades
* Fix formatting
* Improve class hierarchy structure
* Use name tuple types
* Fix formatting
* Front load all reflection
* Format constructor
* Simplify ToTvp as class-specific extension
Co-authored-by: Chad Scharf <3904944+cscharf@users.noreply.github.com>
* Add API for bulk removal of org users
* Refactor OrganizationService, extract some common code.
* Add tests for DeleteUserAsync
* Add tests for DeleteUsers
* Formating
* Update test/Core.Test/Services/OrganizationServiceTests.cs
added a space
Co-authored-by: Addison Beck <abeck@bitwarden.com>
* [Reset Password] Enterprise Policy
* Created UI for policy/edit policy // Updated TODOs for policy dependent checks
* Updated reset password data model field name to be more descriptive
* Update title to Master Password Reset
* Updated PoliciesModel, Policy Model spacing, and strings
* [Reset Password] Organization Key Pair
* Fixed type in Organization_ReadAbilites sproc
* Fixed broken unit test by making sure premium addon was false
* Updated PublicKey decorator and removed unecessary validation
* Fix password reprompt not working in org view
* Also fix Cipher_UpdateWithCollections and CipherDetails_CreateWithCollections. Rename migration script
* [Reset Password] Admin reset actions
* Updated thrown except for permission collision
* Updated GET/PUT password reset to use orgUser.Id for db operations
* Add Cipher attachment upload endpoints
* Add validation bool to attachment storage data
This bool is used to determine whether or not to renew upload links
* Add model to request a new attachment to be made for later upload
* Add model to respond with created attachment.
The two cipher properties represent the two different
cipher model types that can be returned. Cipher Response from
personal items and mini response from organizations
* Create Azure SAS-authorized upload links for both one-shot and block uploads
* Add service methods to handle delayed upload and file size validation
* Add emergency access method for downloading attachments direct from Azure
* Add new attachment storage methods to other services
* Update service interfaces
* Log event grid exceptions
* Limit Send and Attachment Size to 500MB
* capitalize Key property
* Add key validation to Azure Event Grid endpoint
* Delete blob for unexpected blob creation events
* Set Event Grid key at API startup
* Change renew attachment upload url request path to match Send
* Shore up attachment cleanup method.
As long as we have the required information, we should always delete
attachments from each the Repository, the cipher in memory, and the
file storage service to ensure they're all synched.
* Add send HideEmail to tables and models
* Respect HideEmail setting for Sends
* Recreate SendView to include new HideEmail column
* Enforce new Send policy
* Insert default value for new HideEmail column
* Delete c95d7598-71cc-4eab-8b08-aced0045198b.json
* Remove unrelated files
* Revert disableSendPolicy, add sendOptionsPolicy
* Minor style fixes
* Update SQL project with Send.HideEmail column
* unit test SendOptionsPolicy.DisableHideEmail
* Add SendOptionsPolicy to Portal
* Make HideEmail nullable, fix migrator script
* Remove NOT NULL constraint from HideEmail
* Fix style
* Make HideEmail nullable
* minor fixes to model and error message
* Move SendOptionsExemption banner
Co-authored-by: Chad Scharf <3904944+cscharf@users.noreply.github.com>
* Direct upload to azure
To validate file sizes in the event of a rogue client, Azure event webhooks
will be hooked up to AzureValidateFile.
Sends outside of a grace size will be deleted as non-compliant.
TODO: LocalSendFileStorageService direct upload method/endpoint.
* Quick respond to no-body event calls
These shouldn't happen, but might if some errant get requests occur
* Event Grid only POSTS to webhook
* Enable local storage direct file upload
* Increase file size difference leeway
* Upload through service
* Fix LocalFileSendStorage
It turns out that multipartHttpStreams do not have a length
until read. this causes all long files to be "invalid". We need to
write the entire stream, then validate length, just like Azure.
the difference is, We can return an exception to local storage
admonishing the client for lying
* Update src/Api/Utilities/ApiHelpers.cs
Co-authored-by: Chad Scharf <3904944+cscharf@users.noreply.github.com>
* Do not delete directory if it has files
* Allow large uploads for self hosted instances
* Fix formatting
* Re-verfiy access and increment access count on download of Send File
* Update src/Core/Services/Implementations/SendService.cs
Co-authored-by: Chad Scharf <3904944+cscharf@users.noreply.github.com>
* Add back in original Send upload
* Update size and mark as validated upon Send file validation
* Log azure file validation errors
* Lint fix
Co-authored-by: Chad Scharf <3904944+cscharf@users.noreply.github.com>
* Push syncs on Send Access
* Revert "Push syncs on Send Access"
This reverts commit 6a3eb7af4f.
* Push update of Send directly since we can't use SaveSendAsync method
* Fix single org policy when creating organization
Exclude owners and admins from policy when creating new org
* Fix single org and 2FA policy on accepting invite
Exclude owners and admins from policies
* Remove looped async calls
* Fix code style and formatting
* HttpStream must be read prior to knowing it length
We also need to create the send prior to saving the stream so we
have well defined save location. Solve chicken-and-egg problem by saving
the Send twice. This also allows for validation that the stream received
is the same length as that promissed by the content-length header
* Get encrypted file length from request
* Add sendId to path
Event Grid returns the blob path, which will be used to grab a Send and verify file size
* Re-validate access upon file download
Increment access count only when file is downloaded. File
name and size are leaked, but this is a good first step toward
solving the access-download race
It turns out Singleton DI of interfaces does not use the specified
instance's Singleton, but just creates its own. This fixes the bug
where classes expecting an IGlobalSettings were given an empty GlobaSettings
instance
* Remove Url from SendFileModel
Url is now generated on the fly with limited lifetime.
New model houses the download url generated
* Create API endpoint for getting Send file download url
* Generate limited-life Azure download urls
* Lint fix
* Get limited life attachment download URL
This change limits url download to a 1min lifetime.
This requires moving to a new container to allow for non-public blob
access.
Clients will have to call GetAttachmentData api function to receive the download
URL. For backwards compatibility, attachment URLs are still present, but will not
work for attachments stored in non-public access blobs.
* Make GlobalSettings interface for testing
* Test LocalAttachmentStorageService equivalence
* Remove comment
* Add missing globalSettings using
* Simplify default attachment container
* Default to attachments containe for existing methods
A new upload method will be made for uploading to attachments-v2.
For compatibility for clients which don't use these new methods, we need
to still use the old container. The new container will be used only for
new uploads
* Remove Default MetaData fixture.
* Keep attachments container blob-level security for all instances
* Close unclosed FileStream
* Favor default value for noop services
* Revert "Only return policy in TakeoverResponse if Owner"
This reverts commit b20e6f5e85.
* Revert "Return grantor policy info in TakeoverResponse"
This reverts commit 204217a5e0.
* Add endpoint to get grantor policies on takeover
* Delete sends belonging to user on user delete
* Update User_DeleteById.sql
* Clean up bad autoformats
Co-authored-by: Addison Beck <abeck@bitwarden.com>
Co-authored-by: Kyle Spearrin <kspearrin@users.noreply.github.com>
Co-authored-by: Addison Beck <abeck@bitwarden.com>
* Add Disable Send policy
* Test DisableSend policy
* PR Review
* Update tests for using CurrentContext
This required making an interface for CurrentContext and mocking out
the members used. The interface can be expanded as needed for tests.
I moved CurrentContext to a folder, which changes the namespace
and causes a lot of file touches, but most are just adding a reference
* Fix failing test
* Update exemption to include all exempt users
* Move all CurrentContext usages to ICurrentContext
* PR review. Match messaging with Web
* Initial commit of welcome email update
* Final revisions and updated hosted image urls
* added dynamic year // updated verbiage // fixed typo in text template
* Updated verbiage // changed mustache accessor // updated how year is generated
* Null checked org invite collections
* Null checked permissions on org invite
* Gave a static seat count to org invite fixture
* Null checked the right way
* Implemented Custom role and permissions
* Converted permissions columns to a json blob
* Code review fixes for Permissions
* sql build fix
* Update Permissions.cs
* formatting
* Update IOrganizationService.cs
* reworked a conditional
* built out tests for relevant organization service methods
* removed unused usings
* fixed a broken test and a bad empty string init
* removed 'Attribute' from some attribute instances
* Return revised ciphers on restore api call
* Return restored date from restore sproc
* Test Restore updates passed in ciphers
This is necessary for CipherController to appropriately return the
up-to-date ciphers without an extra db call to read them.
* Add missing SELECT
* Sorted usings
* Added CoreHelpersTests
* Added CloneObject test
* Added comments to SelfHostedAttributeTests
* Changed private variable declaration to be inline with the rest of the project
* Changed to SUT naming scheme
* Scaffolded AppleIapServiceTests
* Scaffolded I18nServiceTests.cs
* Scaffolded I18nViewLocalizerTests
* Scaffolded LocalSendStorageServiceTests
* Added tests to ReadableBytesSize_Success
* Add more CleanCertificateThumbprint tests
* Added more tests for the ExtendQuery method
* Removed AppleIapServiceTests
* Removed I18nServiceTests
* Removed I18nViewLocalizerTests
* Removed LocalSendStorageServiceTests
* Converted Batch_Success to theory with InlineData
* Simplified Batch_Success so there is less branching and more accuracy
* Switched to using size variable to format readable bytes for any number lower than 1024
* Fixed test data to work in all locales
* Initial commit of provisioning updates
* Updated strings
* removed extra BANG
* Separated orgUsers db lookup - prioritized existing user Id
* Updated create sso record method // Added sproc for org/email retrieval
* Implemented tax collection for subscriptions
* Cleanup for Sales Tax
* Cleanup for Sales Tax
* Changes a constraint to an index for checking purposes
* Added and implemented a ReadById method for TaxRate
* Code review fixes for Tax Rate implementation
* Code review fixes for Tax Rate implementation
* Made the SalesTax migration script rerunnable
* Add last updated validation to cipher replacements
* Add AutoFixture scaffolding.
AutoDataAttributes and ICustomizations are meant to automatically
produce valid test input. Examples are the Cipher customizations,
which enforce the model's mutual exclusivity of UserId and
OrganizationId.
FixtureExtensions create a fluent way to generate SUTs. We currently
use parameter injection to fascilitate service testing, which is nicely
handled by AutoNSubstitute. However, in order to gain access to the
substitutions, we need to Freeze them onto the Fixture. The For fluent
method allows specifying a Freeze to a specific type's constructor and
optionally to a parameter name in that constructor.
* Unit tests for single Cipher update version checks
* Fix test runner
Test runner requires Microsoft.NET.Test.Sdk
* Move to provider model for SUT generation
This model differs from previous in that you no longer need to specify
which dependencies you would like access to. Instead, all are
remembered and can be queried through the sutProvider.
* User cipher provided by Put method reads
Every put method already reads all relevant ciphers from database,
there's no need to re-read them.
JSON serialization of datetimes seems to leave truncate at second
precision. Verify last known date time is within one second rather than
exact.
* validate revision date for share many requests
* Update build script to use Github environment path
Co-authored-by: Matt Gibson <mdgibson@Matts-MBP.lan>
* added column ApiKey to dbo.User
* added dbo.User.ApiKey to User_Update
* added dbo.User.ApiKey to User_Create
* wrote migration script for implementing dbo.User.ApiKey
* Added ApiKey prop to the User table model
* Created AccountsController method for getting a user's API Key
* Created AccountsController method for rotating a user API key
* Added support to ApiClient for passed-through ClientSecrets when the request comes from the cli
* Added a new conditional to ClientStore to account for user API keys
* Wrote unit tests for new user API Key methods
* Added a refresh of dbo.UserView to new migration script for ApiKey
* Let client_credentials grants into the custom token logic
* Cleanup for ApiKey auth in the CLI feature
* Created user API key on registration
* Removed uneeded code for user API keys
* Changed a .Contains() to a .StartsWith() in ClientStore
* Changed index that an array is searched on
* Added more claims to the user apikey clients
* Moved some claim finding logic to a helper method
* Initial commit of require sso authentication policy enforcement
* Updated sproc to send UseSso flag // Updated base validator to send back error message // Added changes to EntityFramework (just so its there for the future
* Update policy name // adjusted conditional to demorgan's
* Updated sproc // Added migrator script
* Added .sql file extension to DeleteOrgUserWithOrg migrator script
* Added policy // edit // strings // validation to business portal
* Change requests from review // Added Owner & Admin exemption
* Updated repository function used to get org user's type
* Updated with requested changes
* added OnlyOrg to PolicyType enum
* blocked accepting new org invitations if OnlyOrg is relevant to the userOrg
* blocked creating new orgs if already in an org with OnlyOrg enabled
* created email alert for OnlyOrg policy
* removed users & sent alerts when appropriate for the OnlyOrg policy
* added method to noop mail service
* cleanup for OnlyOrg policy server logic
* blocked confirming new org users if they have violated the OnlyOrg policy since accepting
* added localization strings needed for the OnlyOrg policy
* allowed OnlyOrg policy configuration from the portal
* used correct localization key for onlyorg
* formatting and messaging changes for OnlyOrg
* formatting
* messaging change
* code review changes for onlyorg
* slimmed down a conditional
* optimized getting many orgUser records from many userIds
* removed a test file
* sql formatting
* weirdness
* trying to resolve git diff formatting issues
* Initial commit of accept user during set password flow
* changed new org user from accepted to invited // moved another check to token accept function
* Revised some white space // Moved business logic to UserService
* Fixed UserServiceTest
* Removed some white-space
* Removed more white-space
* Final white-space issues
* facilitate linking/unlinking existing users from an sso enabled org
* added user_identifier to identity methods for sso
* moved sso user delete method to account controller
* fixed a broken test
* Update AccountsController.cs
* facilitate linking/unlinking existing users from an sso enabled org
* added user_identifier to identity methods for sso
* moved sso user delete method to account controller
* fixed a broken test
* added a token to the existing user sso link flow
* added a token to the existing user sso link flow
* fixed a typo
* added an event log for unlink ssoUser records
* fixed a merge issue
* fixed a busted test
* fixed a busted test
* ran a formatter over everything & changed .vscode settings in .gitignore
* chagned a variable to use string interpolation
* removed a blank line
* Changed TokenPurpose enum to a static class of strings
* code review cleanups
* formatting fix
* Changed parameters & logging for delete sso user
* changed th method used to get organization user for deleting sso user records
Co-authored-by: Kyle Spearrin <kspearrin@users.noreply.github.com>
* [enum] Add Yandex to global equivalent domains list
Exhaustive list of Yandex domain names obtained by contacting their support.
* [Dictionary] Add Yandex to global equivalent domains list
Exhaustive list of Yandex domain names obtained by contacting their support.
After further research, amazon.co.nz has never been more than a redirect (in order: to amazon.com, amazon.co.uk then back to amazon.com). See PR for more information.
* [SSO] Added change password API (#836)
* Created API for updating password with no current comparison
* Changed name of method and request // Added user has password error flow
* Updated user service method name // Updated string null/empty check
* Replaced hardcoded sso domain hints with config loader (#850)
* Replaced hardcoded sso domain hints with config loader
* use async/await for sso config loader
* Update AccountsController.cs
Co-authored-by: Vincent Salucci <26154748+vincentsalucci@users.noreply.github.com>
Co-authored-by: Matt Portune <mportune@bitwarden.com>
Co-authored-by: Matt Portune <59324545+mportune-bw@users.noreply.github.com>
* Expanded the Plan model to make plan & product data a bit more dynamic
* Created a Product enum to track versioned instances of the same plan
* Created and API call and Response model for getting plan & product data from the server
* update welcome email download urls
Very cool service! Noticed the download urls in the welcome email didn't take me to the downloads. Hope this fixes it?
* Update Welcome.html.hbs
Co-authored-by: Kyle Spearrin <kspearrin@users.noreply.github.com>
* Sso user table, model and repo stubbed out
* switch to nullable org id, bigint id
* update GetBySsoUserAsync
* cleanup migrator file
* fix EF user repo
* fix pg repo
* is `IS NULL` checks
* unique indexes
* update migration scripts
* add another unique index
* remove old script
* do not follow local hosts or ip addresses
* remove cron from mssql
* migration script
* Use joins instead of temp tables
* update migration script with join changes
