1
0
mirror of https://github.com/bitwarden/server.git synced 2024-11-25 12:45:18 +01:00
Commit Graph

928 Commits

Author SHA1 Message Date
Thomas Avery
4c0f8d54f3
[PM-10560] Create notification database storage (#4688)
* Add new tables

* Add stored procedures

* Add core entities and models

* Setup EF

* Add repository interfaces

* Add dapper repos

* Add EF repos

* Add order by

* EF updates

* PM-10560: Notifications repository matching requirements.

* PM-10560: Notifications repository matching requirements.

* PM-10560: Migration scripts

* PM-10560: EF index optimizations

* PM-10560: Cleanup

* PM-10560: Priority in natural order, Repository, sql simplifications

* PM-10560: Title column update

* PM-10560: Incorrect EF migration removal

* PM-10560: EF migrations

* PM-10560: Added views, SP naming simplification

* PM-10560: Notification entity Title update, EF migrations

* PM-10560: Removing Notification_ReadByUserId

* PM-10560: Notification ReadByUserIdAndStatus fix

* PM-10560: Notification ReadByUserIdAndStatus fix to be in line with requirements and EF

---------

Co-authored-by: Maciej Zieniuk <mzieniuk@bitwarden.com>
Co-authored-by: Matt Bishop <mbishop@bitwarden.com>
2024-09-09 21:52:12 +02:00
Conner Turnbull
46ac2a9b3b
[AC-2568] Added invoices and transaction history endpoints. Added cursor paging for each (#4692)
* Added invoices and transaction history endpoints. Added cursor paging for each

* Removed try/catch since it's handled by middleware. Updated condition to use pattern matching

* Added unit tests for PaymentHistoryService

* Removed organizationId from account billing controller endpoints
2024-09-09 09:38:58 -04:00
Rui Tomé
f5caecc6d6
[AC-1722] Deprecate "Edit/Delete Assigned Collections" custom permissions (#4604)
* Add SQL script to migrate custom users with specific permissions to User type

Remove 'editAssignedCollections' and 'deleteAssignedCollections' properties from Permissions in OrganizationUser table. Migrate custom users who only have these permissions to the User type.

* Add MySQL migration to migrate custom users with specific permissions to User type

* Add Postgres migration to migrate custom users with specific permissions to User type

* Add Sqlite migration to migrate custom users with specific permissions to User type

* Update AutoFixture usage in tests to resolve creating ILogger mock instances

* Update EF integration tests database contexts to use each respective Migrations assembly. Configure Sqlite instance

* Add RunMigration method to BaseEntityFrameworkRepository

* Add FinalFlexibleCollectionsDataMigrationsTests

* Improve data migration efficiency by using OPENJSON instead of multiple JSON_EXTRACT

* Add batching to the sql data migrations

* Update DbMigrator to run a specific script based on its name

* Update DatabaseDataAttribute to be able to test a specific migration

* Add reference to the migration projects to Infrastructure.IntegrationTest

* Add integration test to test the migration FinalFlexibleCollectionsDataMigrations

* Remove EFIntegration tests and remove RunMigration method from BaseEntityFrameworkRepository

* Add IMigrationTesterService and implementations for SQL and EF migrations

* Add FinalFlexibleCollectionsDataMigrationsTests and remove test from OrganizationUserRepositoryTests

* Update sql data migration script based on performance feedback

* Bump date on EF migration scripts

* Add xmldoc comments to IMigrationTesterService and each implementation

* Bump up the date on the EF migration scripts

* Bump up dates on EF migrations

* Added tests to assert no unwanted changes are made to the permissions json. Refactor tests.

* Revert changes made to DbMigrator and refactor SqlMigrationTesterService to not use it.

* Add method description

* Fix test to assert no changes are made to custom user

* Remove unnecessary COALESCE and SELECT CASE

* Unident lines on SQL script

* Update DatabaseDataAttribute MigrationName property to be nullable

* Fix null reference checks

* Remove unnecessary COALESCE from Postgres script

* Bump dates on migration scripts

* Bump up dates on EF migrations

* Add migration tests for handling null

* Add test for non json values

* Fix test

* Remove migrations

* Recreate EF migrations

* Update Postgres data migration script to check for valid JSON in Permissions column

---------

Co-authored-by: Thomas Rittson <31796059+eliykat@users.noreply.github.com>
Co-authored-by: Thomas Rittson <trittson@bitwarden.com>
2024-09-02 11:04:55 +01:00
Thomas Rittson
0da62f9cee
[PM-10368] Drop Group.AccessAll (#4700)
- Add default constraint
- Update sprocs to remove column
- Drop column
2024-09-02 15:01:32 +10:00
Justin Baur
13ad872f51
Resolve Vault and DB Warnings (#4646)
* Resolve Vault Warnings

* Resolve DB Warnings
2024-08-27 19:05:34 -04:00
Thomas Rittson
22bd755b3c
[AC-10362] Remove OrganizationUser.AccessAll from code (#4622)
* Remove OrganizationUser.AccessAll from code

* Add shadow property

* Remove remaining reference

* dotnet format

* Fix tests

* Bump migration dates
2024-08-26 12:03:44 +01:00
Rui Tomé
15511df644
[PM-10589] Update database defaults for collection management settings (#4623)
* [PM-10589] Update database defaults for collection management settings

* Update default values for 'LimitCollectionCreationDeletion' and 'AllowAdminAccessToAllCollectionItems' in Organization table

* Add checks before column constraint drop and constraint creation
2024-08-21 13:18:21 +01:00
renovate[bot]
8253de350b
[deps] DbOps: Update EntityFrameworkCore to v8.0.8 (#4651)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-08-19 14:21:16 -04:00
Thomas Rittson
f04c3b8e54
[PM-10361] Remove Group.AccessAll from code (#4614)
* Remove Group.AccessAll from code

* Add shadow property config and migration
2024-08-13 08:54:03 +10:00
Thomas Rittson
916be50e66
[PM-10349] Drop Organization.FlexibleCollections column (#4583) 2024-08-12 08:28:13 +10:00
Thomas Rittson
374ef95656
Add OrganizationUser_UpdateDataForKeyRotation sproc (#4601) 2024-08-09 08:52:25 +10:00
Thomas Rittson
58a314d9f4
[PM-10360] Drop user cipher and collection details v2 functions (#4588) 2024-08-09 07:33:45 +10:00
Rui Tomé
8d69bb0aaa
[AC-1698] Check if a user has 2FA enabled more efficiently (#4524)
* feat: Add stored procedure for reading organization user details with premium access by organization ID

The code changes include:
- Addition of a new stored procedure [dbo].[OrganizationUserUserDetailsWithPremiumAccess_ReadByOrganizationId] to read organization user details with premium access by organization ID
- Modification of the IUserService interface to include an optional parameter for checking two-factor authentication with premium access
- Modification of the UserService class to handle the new optional parameter in the TwoFactorIsEnabledAsync method
- Addition of a new method GetManyDetailsWithPremiumAccessByOrganizationAsync in the IOrganizationUserRepository interface to retrieve organization user details with premium access by organization ID
- Addition of a new view [dbo].[OrganizationUserUserDetailsWithPremiumAccessView] to retrieve organization user details with premium access

* Add IUserRepository.SearchDetailsAsync that includes the field HasPremiumAccess

* Check the feature flag on Admin.UsersController to see if the optimization runs

* Modify PolicyService to run query optimization if the feature flag is enabled

* Refactor the parameter check on UserService.TwoFactorIsEnabledAsync

* Run query optimization on public MembersController if feature flag is enabled

* Restore refactor

* Reverted change used for development

* Add unit tests for OrganizationService.RestoreUser

* Separate new CheckPoliciesBeforeRestoreAsync optimization into new method

* Add more unit tests

* Apply refactor to bulk restore

* Add GetManyDetailsAsync method to IUserRepository. Add ConfirmUsersAsync_vNext method to IOrganizationService

* Add unit tests for ConfirmUser_vNext

* Refactor the optimization to use the new TwoFactorIsEnabledAsync method instead of changing the existing one

* Removed unused sql scripts and added migration script

* Remove unnecessary view

* chore: Remove unused SearchDetailsAsync method from IUserRepository and UserRepository

* refactor: Use UserDetails constructor in UserRepository

* Add summary to IUserRepository.GetManyDetailsAsync

* Add summary descriptions to IUserService.TwoFactorIsEnabledAsync

* Remove obsolete annotation from IUserRepository.UpdateUserKeyAndEncryptedDataAsync

* refactor: Rename UserDetails to UserWithCalculatedPremium across the codebase

* Extract IUserService.TwoFactorIsEnabledAsync into a new TwoFactorIsEnabledQuery class

* Add unit tests for TwoFactorIsEnabledQuery

* Update TwoFactorIsEnabledQueryTests to include additional provider types

* Refactor TwoFactorIsEnabledQuery

* Refactor TwoFactorIsEnabledQuery and update tests

* refactor: Update TwoFactorIsEnabledQueryTests to include test for null TwoFactorProviders

* refactor: Improve TwoFactorIsEnabledQuery and update tests

* refactor: Improve TwoFactorIsEnabledQuery and update tests

* Remove empty <returns> from summary

* Update User_ReadByIdsWithCalculatedPremium stored procedure to accept JSON array of IDs
2024-08-08 15:43:45 +01:00
Conner Turnbull
de79d57d6e
[AC-2820] Updated org edit form scripts to dynamically update expected values (#4439)
* Updated org edit form scripts to dynamically update expected values

* Added script to update null values on organization table

* Updated script to only add MaxStorageGb for premium tiers. Removed setting of seats since it's not a valid edge case

* Updated GetPlansHelper() to not use annonymous properties
2024-07-29 09:06:10 -04:00
Thomas Rittson
ffdc40b21c
[AC-2881] Remove Organization.FlexibleCollections from code (#4552)
* Remove Organization.FlexibleCollections from code

* Drop Organization.FlexibleCollections column in EF databases
(MSSQL column to be retained for 1 additional deployment to support rollback in cloud)
2024-07-26 15:07:33 +10:00
Justin Baur
1e0182008b
[PM-2943] Enable Nullable Repositories in Unowned Files (#4549)
* Enable Nullable In Unowned Repos

* Update More Tests

* Move to One If

* Fix Collections

* Format

* Add Migrations

* Move Pragma Annotation

* Add Better Assert Message
2024-07-24 09:48:09 -04:00
Thomas Rittson
2d762f8422
[AC-2877] Make OrganizationUser.AccessAll optional (#4521) 2024-07-24 14:29:45 +10:00
Thomas Rittson
1ac2f39623
[AC-2872] Make AccessAll optional in all group sprocs (#4551) 2024-07-24 11:10:12 +10:00
renovate[bot]
cca358c88c
[deps] DbOps: Update dbup-sqlserver to v5.0.41 (#4538)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-07-22 12:14:26 -04:00
Thomas Rittson
f57f98afe4
Drop CollectionCipher V2 sprocs (#4515)
These sprocs were used to remove AccessAll from
cipher access logic. Now the original sprocs have been
updated with the new logic, these v2 sprocs are unused
and are being dropped to complete the EDD cycle.
2024-07-18 08:24:34 +10:00
Thomas Rittson
b4e58ad942
chore: drop CipherRepository v2 sprocs (#4464)
These updated sprocs removed AccessAll from cipher access logic.
The non-versioned sprocs have been updated with the new logic and
these v2 copies are now unused. They are being dropped to complete
the EDD cycle.
2024-07-18 08:22:45 +10:00
Thomas Rittson
b0ea2a25f0
Drop v2 sprocs that added manage permission (#4463)
These sprocs have been copied back to non-versioned names and are no longer in use.
Now we are dropping the v2 sprocs to complete the EDD cycle.
2024-07-18 08:17:25 +10:00
Thomas Rittson
8f70dd98ba
[AC-2732] Remove AccessAll - Bump account revision date sprocs (#4490)
* Remove AccessAll logic from bump account revision date sprocs
and corresponding EF methods
2024-07-11 08:01:39 +10:00
Thomas Rittson
4ab608a636
[AC-2733] Remove AccessAll - misc sprocs (#4477)
Remove AccessAll logic from miscellaneous sprocs
and corresponding EF queries
2024-07-11 08:00:28 +10:00
renovate[bot]
41135c866d
[deps] DbOps: Update EntityFrameworkCore to v8.0.7 (#4484)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-07-09 12:11:35 -04:00
Thomas Rittson
1525c10bfb
[AC-2731] Remove AccessAll - Collection and Cipher functions (#4469)
* Remove AccessAll logic from UserCollectionDetails and UserCipherDetails
  and EF equivalents
2024-07-09 13:50:40 +10:00
Thomas Rittson
202dce3459
[AC-2730] Remove AccessAll - CollectionCipher (#4468)
Final removal of AccessAll logic in CollectionCipher sprocs. We had v2 sprocs already
containing this updated logic that were never used; copy the v2 logic back to the 
original sprocs so that we start using it. v2 sprocs will be dropped later.
2024-07-09 09:39:24 +10:00
Alex Morask
9c8a9f41fb
[AC-2804] Add client ID to provider client invoice report (#4458)
* Add client ID to provider client invoice report

* Run dotnet format
2024-07-05 10:12:03 -04:00
Justin Baur
8b5f65fc00
PM-2944] Make Entities Nullable In Admin Console (#4386)
* Enable `nullable` in `ISubscriber`

* Enable `nullable` in `Group`

* Enable `nullable` in `GroupUser`

* Enable `nullable` in `Organization`

* Enable `nullable` in `OrganizationUser`

* Enable `nullable` in `Policy`

* Enable `nullable` in `Provider`

* Enable `nullable` in `ProviderOrganization`

* Enable `nullable` in `ProviderUser`

* Update Tests

* Formatting

* Update TwoFactor Tests

* Fix Scim Tests

* Format

* Add Migrations

* Format
2024-07-04 21:14:37 -04:00
Thomas Rittson
7da37ee231
Drop unused CollectionRepository sprocs (#4455) 2024-07-05 09:43:31 +10:00
Justin Baur
8d1f6a9f66
[PM-2944] Update Null DB Constraints (#4459)
* Update Database Models

* Format
2024-07-03 16:21:25 -04:00
Kyle Spearrin
0d3a7b3dd5
[PM-5518] Sql-backed IDistributedCache (#3791)
* Sql-backed IDistributedCache

* sqlserver cache table

* remove unused using

* setup EF entity

* cache indexes

* add back cipher

* revert SetupEntityFramework change

* ef cache

* EntityFrameworkCache

* IServiceScopeFactory for db context

* implement EntityFrameworkCache

* move to _serviceScopeFactory

* move to config file

* ef migrations

* fixes

* datetime and error codes

* revert migrations

* migrations

* format

* static and namespace fix

* use time provider

* Move SQL migration and remove EF one for the moment

* Add clean migration of just the new table

* Formatting

* Test Custom `IDistributedCache` Implementation

* Add Back Logging

* Remove Double Logging

* Skip Test When Not EntityFrameworkCache

* Format

---------

Co-authored-by: Matt Bishop <mbishop@bitwarden.com>
Co-authored-by: Justin Baur <19896123+justindbaur@users.noreply.github.com>
2024-07-03 12:48:23 -04:00
Alex Morask
07d37b1b41
[AC-2805] Add AssignedSeats to ProviderOrganizationOrganizationDetailsView (#4446)
* Add 'AssignedSeats' to ProviderOrganizationOrganizationDetailsView

* Add newline

* Thomas' feedback
2024-07-03 10:33:37 -04:00
SmithThe4th
c390a6b589
[AC-2679] Adding a revoked, invited member with Can Manage access does not resolve unmanaged collections (#4397)
* Added check for revoked users

* removed check for users as any user status with can manage access should hide the add access badge

* updated comments
2024-07-02 11:19:59 -04:00
Matt Bishop
e8c5d73062
Sync EF migrations for .NET / EF Core 8 (#4448)
* Sync EF migrations for .NET 8

* Format

* Redo migrations with billing fix

* Forgot to format again
2024-07-02 10:08:34 -04:00
renovate[bot]
563adf54af
[deps] DbOps: Update EntityFrameworkCore to v8 (major) (#3744)
* [deps] DbOps: Update EntityFrameworkCore to v8

* Only Run EnsureDeleted If Factory Owns Connection

This only worked because of a bug in dotnet/efcore#33930 that was fixed in 8.0.

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Justin Baur <19896123+justindbaur@users.noreply.github.com>
2024-06-27 08:45:34 -04:00
Thomas Rittson
d9aa27d4cb
[AC-2204] Finalize sprocs that added the Manage permission (1 of 3) (#4204)
* Copy v2 sprocs and drop v2 suffix
2024-06-27 05:56:37 +10:00
Thomas Avery
8a1b64a21b
[SM-1075] Fix bulk remove organization users with Secrets Manager (#4197)
* Fix OrganizationUser_DeleteByIds procedure

* Add db migration
2024-06-21 17:29:36 -05:00
Alex Morask
83604cceb1
[AC-1943] Implement provider client invoice report (#4178)
* Update ProviderInvoiceItem SQL configuration

* Implement provider client invoice export

* Add tests

* Run dotnet format

* Fixed SPROC backwards compatibility issue
2024-06-14 12:26:49 -04:00
Conner Turnbull
fc1c488a78
[AC-2567] Billing Performance Improvements (#4143)
* Moved AccountsBilling controller to be owned by Billing

* Added org billing history endpoint

* Updated GetBillingInvoicesAsync to only retrieve paid, open, and uncollectible invoices, and added option to limit results

* Removed invoices and transactions from GetBillingAsync

* Limiting the number of invoices and transactions returned

* Moved Billing models to Billing namespace

* Split billing info and billing history objects

* Removed billing method GetBillingBalanceAndSourceAsync

* Removed unused using

* Cleaned up BillingInfo a bit

* Update migration scripts to use `CREATE OR ALTER` instead of checking for the `OBJECT_ID`

* Applying limit to aggregated invoices after they return from Stripe
2024-06-11 13:55:23 -04:00
Thomas Rittson
a60f70dde5
[AC-2300] Remove mssql utility migration record migrator (#4171)
* Remove mssql utility migration record migrator

* Remove old/unused files
2024-06-11 06:25:52 +10:00
Rui Tomé
308bd555a4
[AC-2286] Include the OrganizationUserId for each Organization in the user sync data (#4142)
* [AC-2286] Include the OrganizationUserId for each Organization in the user sync data

* Make OrganizationUserId property non-nullable
2024-06-07 13:32:09 -05:00
renovate[bot]
a1d609b208
[deps] DbOps: Update EntityFrameworkCore (#3981)
* [deps] DbOps: Update EntityFrameworkCore

* Update linq2db Package

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Justin Baur <19896123+justindbaur@users.noreply.github.com>
2024-06-07 09:55:59 -04:00
Alex Morask
725fc2eed3
[AC-1943] Add ProviderInvoiceItem table (#4163)
* Add ProviderInvoiceItem table

* Run dotnet format
2024-06-06 13:25:13 -04:00
Ike
97b3f3e7ee
[PM-5216] User and Organization Duo Request and Response Model refactor (#4126)
* inital changes

* add provider GatewayType migrations

* db provider migrations

* removed duo migrations added v2 metadata to duo response

* removed helper scripts

* remove signature from org duo

* added backward compatibility for Duo v2

* added tests for duo request + response models

* refactors to TwoFactorController

* updated test methods to be compartmentalized by usage

* fix organization add duo

* Assert.Empty() fix for validator
2024-06-05 11:42:02 -07:00
renovate[bot]
5d47adb0fa
[deps] DevOps: Update CommandDotNet to v7.0.4 (#4081)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-05-24 06:37:03 -06:00
Rui Tomé
5fabad35c7
[AC-2328] Add a Bulk OrganizationUsersController.GetResetPasswordDetails endpoint (#4079)
* Add new stored procedure for reading reset password details for multiple organization user IDs

* Add method IOrganizationUserRepository.GetManyResetPasswordDetailsByOrganizationUserAsync

* Add new API endpoint for getting reset password details for multiple organization users

* Add unit tests for bulk OrganizationUsersController.GetResetPasswordDetails

* Add alias to sql query result column

* Add constructor for automatic mapping

* Fix http method type for endpoint

* dotnet format

* Simplify the constructor in the OrganizationUserResetPasswordDetails

* Refactor stored procedure and repository method names for retrieving account recovery details

* Add integration tests for GetManyAccountRecoveryDetailsByOrganizationUserAsync

* Lock endpoint behind BulkDeviceApproval feature flag

* Update feature flag key value
2024-05-24 11:20:54 +01:00
Thomas Rittson
be41865b59
[AC-2522] Remove collection enhancements opt-in (#4110)
* Delete controller endpoint
* Delete command
* Drop sproc
2024-05-24 09:00:04 +10:00
Addison Beck
56c523f76f
Allow for bulk updating AuthRequest database objects (#4053)
* Declare a new repository interface method

To facilitate a new bulk device login request approval workflow in the
admin console we need to update `IAuthRequestRepisitory` (owned by Auth
team) to include an`UpdateManyAsync()` method. It should accept a list
of `AuthRequest` table objects, and implementations will do a very
simple 1:1 update of the passed in data.

This commit adds an `UpdateManyAsync()` method to the
`AuthRequestRepository` interface.

* Stub out method implementations to enable unit testing

This commit stubs out implementations of
`IAuthRequestRepository.UpdateManyAsync()` so the method signature can
be called in unit tests. At this stage the methods are not implemented.

* Assert a happy path integration test

* Establish a user defined SQL type for Auth Requests

To facilitate a bulk update operation for auth requests a new user
defined type will need to be written that can be used as a table input
to the stored procedure. This will follow a similar pattern to how the
`OragnizationSponsorshipType` works and is used by the stored procedure
`OrganizationSponsorship_UpdateMany`.

* Establish a new stored procedure

To facilitate the bulk updating of auth request table objects this
commit adds a new stored procedure to  update a collection of entities
on `AuthRequest` table by their primary key. It updates all properties,
for convention, but the endpoint created later will only change the
`Approved`, `ResponseDate`, `Key`, `MasterPasswordHash`, and
`AuthenticationDate` properties.

* Apply a SQL server migration script

This commit simply applies a migration script containing the new user
defined type and stored procedure comitted previously.

* Enable converting an `IEnumerable<AuthRequest>` to a `DataTable`

The current pattern in place for bulk update stored procedures is to
pass a `DataTable` through Dapper as an input for the update stored
procedure being run. In order to facilitate the new bulk update
procedure for the`AuthRequest` type we need a function added that can
convert an `IEnumerable<AuthRequest>` to a `DataTable`. This is commit
follows the convention of having a static class with a conversion method
in a `Helpers` folder: `AuthRequestHelpers.ToDataTable()`.

* Implement `Dapper/../AuthRequestRepository.UpdateMany()`

This commit implements `AuthRequestRepository.UpdateMany()` for the
Dapper implementation of `AuthRequestRepository`. It connects the stored
procedure, `DataTable` converter, and Dapper-focused unit test commits
written previously into one exposed method that can be referenced by
service callers.

* Implement `EntityFramework/../AuthRequestRepository.UpdateMany()`

This commit implements the new
`IAuthRequestRepository.UpdateManyAsync()`method in the Entity Framework
skew of the repository layer. It checks to make sure the passed in list
has auth requests, converts them all to an Entity Framework entity, and
then uses `UpdateRange` to apply the whole thing over in the database
context.

* Assert that  `UpdateManyAsync` can not create any new auth requests

* Use a json object as stored procedure input

* Fix the build

* Continuing to troubleshoot the build

* Move `AuthRequest_UpdateMany` to the Auth folder

* Remove extra comment

* Delete type that never got used

* intentionally break a test

* Unbreak it
2024-05-22 11:55:31 -05:00
Thomas Rittson
b679f98c19
[AC-2626] Re-run collection enhancements migration (#4111)
* Re-run collection enhancements migration

* Bump date

* Disambiguate name
2024-05-22 10:15:44 +01:00