rdagumampan/yuniql

yuniql (yuu-nee-kel) schema versioning and migration package

Work with databases in a much more stable and reliable way using db schema versioning and migration

yuniql (yuu-nee-kel) schema versioning and migration package

yuniql

yuniql (yuu-nee-kel). Free and open source schema versioning and database migration engine made natively with .NET Core. Use plain SQL scripts, bulk import CSV, integrate CI/CD pipelines, zero runtime dependencies and works with windows and linux. Supports SqlServer, PostgreSql, MySql, MariaDB, Snowflake, and Amazon Redshift.

Working with CLI

Manage local db versions and run database migrations from your CLI tool. Perform local migration run or verify with uncommitted runs to test your scripts. Install yuniql CLI with Chocolatey or use alternative ways listed here https://yuniql.io/docs/install-yuniql

choco install yuniql

Run migrations for SQL Server

docker run -d -e "ACCEPT_EULA=Y" -e "[email protected]!" -p 1400:1433 -d mcr.microsoft.com/mssql/server:2017-latest
SETX YUNIQL_CONNECTION_STRING "Server=localhost,1400;Database=yuniqldb;User Id=SA;[email protected]!"
SETX YUNIQL_WORKSPACE "c:\temp\yuniql-cli\samples\basic-sqlserver-sample"
git clone https://github.com/rdagumampan/yuniql.git c:\temp\yuniql-cli
cd c:\temp\yuniql-cli\samples\basic-sqlserver-sample

yuniql run -a --platform sqlserver
yuniql list --platform sqlserver

Running yuniql v1.0.1 for windows-x64
Copyright 2019 (C) Rodel E. Dagumampan. Apache License v2.0
Visit https://yuniql.io for documentation & more samples

+---------------+----------------------+------------+---------------+----------------------+--------------+
| SchemaVersion | AppliedOnUtc         | Status     | AppliedByUser | AppliedByTool        | Duration     |
+---------------+----------------------+------------+---------------+----------------------+--------------+
| v0.00         | 2021-02-04 06:06:46Z | Successful | sa            | yuniql-cli v1.1.55.0 | 164 ms / 0 s |
+---------------+----------------------+------------+---------------+----------------------+--------------+

Supported databases and platform tests

Amazon Aurora, Azure Synapse and Alibaba Aspara are being evaluated/developed/tested. For running migration from docker container, see instructions here.

Platforms Build Status Description Cloud Infrastructure Documentation
sqlserver Sql Server 2017, 2019, latest Azure, GCP, AWS Get started
postgresql PostgreSql v9.6, v12.1, latest Azure, GCP, AWS Get started
mysql MySql v5.7, v8.0, latest Azure, GCP, AWS Get started
mariadb MariaDb v10.2, latest Azure, GCP, AWS Get started
snowflake Preview in v1.1.55 (latest) Azure Get started
redshift Preview in v1.1.55 (latest) AWS Get started
synapse development Preview build ETA mar 27 2021 Azure

NOTE: Supported cloud platforms are based on limited testing and community feedbacks where users indicated the platform they were attempting to run against. yuniql primarily uses Amazon RDS as cloud provider for non-vendor specific platforms.

Distributions Build Status Description
Docker image linux-x64 docker pull yuniql/yuniql:linux-x64-latest
Docker imiage win-x64 docker pull yuniql/yuniql:win-x64-latest

Run migrations for PostgreSql, MySql and others

docker run -d -e POSTGRES_USER=sa -e [email protected]! -e POSTGRES_DB=yuniqldb -p 5432:5432 postgres
SETX YUNIQL_CONNECTION_STRING "Host=localhost;Port=5432;Username=sa;[email protected]!;Database=yuniqldb"
SETX YUNIQL_WORKSPACE "c:\temp\yuniql-cli\samples\basic-postgresql-sample"
git clone https://github.com/rdagumampan/yuniql.git c:\temp\yuniql-cli
cd c:\temp\yuniql-cli\samples\basic-postgresql-sample

yuniql run -a --platform postgresql
yuniql list --platform postgresql

Working with Azure DevOps Pipelines Tasks

Run your database migration from Azure DevOps Pipelines. The tasks downloads package and cache it for later execution just like how Use .NET Core or Use Node tasks works. Find Yuniql on Azure DevOps MarketPlace. Developer guide is available here https://yuniql.io/docs/migrate-via-azure-devops-pipelines.

Working with Docker Container

Run your database migration thru a Docker container. This is specially helpful on Linux environments and CI/CD pipelines running on Linux Agents as it facilitates your migration without having to worry any local installations or runtime dependencies. Developer guide is available here https://yuniql.io/docs/migrate-via-docker-container.

git clone https://github.com/rdagumampan/yuniql.git c:\temp\yuniql-docker
cd c:\temp\yuniql-docker\samples\basic-sqlserver-sample

docker build -t sqlserver-example .
docker run sqlserver-example -c "<your-connection-string>" -a --platform sqlserver

Working with ASP.NET Core

Run your database migration when your ASP.NET Core host service starts up. This ensures that database is always at latest compatible state before operating the service. Applies to Worker and WebApp projects. Developer guide is available here https://yuniql.io/docs/migrate-via-aspnetcore-application.

dotnet add package Yuniql.AspNetCore
using Yuniql.AspNetCore;
...
...

//1. deploy new sql server on docker or use existing instance
//$ docker run -dit --name yuniql-sqlserver  -e "ACCEPT_EULA=Y" -e "[email protected]!" -p 1400:1433 -d mcr.microsoft.com/mssql/server:2017-latest

//2. create custom trace message sinks, this can be your own logger framework
var traceService = new ConsoleTraceService { IsDebugEnabled = true };

//3. run migrations
app.UseYuniql(traceService, new Yuniql.AspNetCore.Configuration
{
	Platform = SUPPORTED_DATABASES.SQLSERVER,
	Workspace = Path.Combine(Environment.CurrentDirectory, "_db"),
	ConnectionString = "Server=localhost,1400;Database=helloyuniql;User Id=SA;[email protected]!",
	IsAutoCreateDatabase = true, IsDebug = true
});

Working with .NET Core Console Application

Run your database migration when Console App starts. Developer guide is available here https://yuniql.io/docs/migrate-via-netcore-console-application.

dotnet add package Yuniql.Core
using Yuniql.Core;
...
...

static void Main(string[] args)
{
	//1. deploy new sql server on docker or use existing instance
	//$ docker run -dit -e "ACCEPT_EULA=Y" -e "[email protected]!" -p 1400:1433 -d mcr.microsoft.com/mssql/server:2017-latest

	//2. create custom trace message sinks, this can be your own logger framework
	var traceService = new ConsoleTraceService { IsDebugEnabled = true };

	//3. configure your migration run
	var configuration = Configuration.Instance;
	configuration.Platform = SUPPORTED_DATABASES.SQLSERVER;
	configuration.Workspace = Path.Combine(Environment.CurrentDirectory, "_db");
	configuration.ConnectionString = "Server=localhost,1400;Database=helloyuniql;User Id=SA;[email protected]!";
	configuration.IsAutoCreateDatabase = true;

	//4. run migrations
	var migrationServiceFactory = new MigrationServiceFactory(traceService);
	var migrationService = migrationServiceFactory.Create();
	migrationService.Run();

Advanced use cases

  • How to bulk import CSV master data
  • How to replace tokens in script files
  • How to run environment-aware migrations
  • How to baseline your database
  • How yuniql works

Contributing & asking for help

Please submit ideas for improvement or report a bug by creating an issue.
Alternatively, tag #yuniql on Twitter or drop me a message rdagumampanATgmail.com.

If this is your first time to participate in an open source initiative, you may look at issues labeled as first timer friendly issues. If you found an interesting case, you can fork this repository, clone to your dev machine, create a local branch, and make Pull Requests (PR) so I can review and merge your changes.

To prepare your dev machine, please visit https://github.com/rdagumampan/yuniql/wiki/Setup-development-environment

License

Copyright (C) 2019 Rodel E. Dagumampan

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Credits

Yuniql relies on many open-source projects and we would like to thanks:

  • CommandlineParser for CLI commands
  • CsvTextFieldParser for CSV file parsing
  • Npgsql for PostgreSql drivers
  • Shouldly for unit tests
  • Moq for unit test mocks
  • Evolutionary database design by Martin Fowler and Pramod Sadalage
  • Microsoft, Oracle, for everything in dotnetcore seems open source now :)
  • All the free devops tools! GitHub, AppVeyor, Docker, Shields.io ++

Maintainers

Issues

Quick list of the latest Issues we found

Naimishah

Naimishah

Icon For Comments0

2022-09-06T10:46:21.9993631Z ##[section]Starting: Verify Yuniql 2022-09-06T10:46:22.0358555Z ============================================================================== 2022-09-06T10:46:22.0358905Z Task : Verify yuniql 2022-09-06T10:46:22.0359255Z Description : Runs an uncommitted database migrations with Yuniql CLI using version acquired from Use Yuniql task. 2022-09-06T10:46:22.0359610Z Version : 1.190.0 2022-09-06T10:46:22.0359837Z Author : Rodel E. Dagumampan 2022-09-06T10:46:22.0360131Z Help : https://yuniql.io/docs/migrate-via-azure-devops-pipelines/ 2022-09-06T10:46:22.0360493Z ============================================================================== 2022-09-06T10:46:22.7889326Z yuniql/input_version: latest 2022-09-06T10:46:22.7889950Z yuniql/input_workspacePath: _PBM_DataManagement_DB_CI/Database 2022-09-06T10:46:22.7890602Z yuniql/input_connectionString: Server=10.55.127.8;Port=5432;Database=tpm; User [email protected];Password=4UDTPPG!; 2022-09-06T10:46:22.7891255Z yuniql/input_targetPlatform: postgresql 2022-09-06T10:46:22.7891951Z yuniql/input_autoCreateDatabase: false 2022-09-06T10:46:22.7892440Z yuniql/input_targetVersion: latest 2022-09-06T10:46:22.7892886Z yuniql/input_tokenKeyValuePair: undefined 2022-09-06T10:46:22.7893323Z yuniql/input_delimiter: , 2022-09-06T10:46:22.7893739Z yuniql/input_additionalArguments: --debug 2022-09-06T10:46:22.7894166Z yuniql/var_osPlat: win32 2022-09-06T10:46:22.7894578Z yuniql/var_osArch: x64 2022-09-06T10:46:22.7898401Z Found tool in cache: yuniql 0.0.0 x64 2022-09-06T10:46:22.7898978Z yuniql/var_yuniqlBasePath: F:\Softwares\vsts-agent-win-x64-2.165.0_work_tool\yuniql\0.0.0\x64 2022-09-06T10:46:22.7899666Z yuniql/var_yuniqlExecFilePath: F:\Softwares\vsts-agent-win-x64-2.165.0_work_tool\yuniql\0.0.0\x64\yuniql.exe 2022-09-06T10:46:22.7901236Z yuniql/additionalArguments array is 2022-09-06T10:46:22.7901669Z arg#0: --debug 2022-09-06T10:46:22.7902526Z [command]F:\Softwares\vsts-agent-win-x64-2.165.0_work_tool\yuniql\0.0.0\x64\yuniql.exe verify -p _PBM_DataManagement_DB_CI/Database -c "Server=10.55.127.8;Port=5432;Database=tpm; User [email protected];Password=4UDTPPG!;" --platform postgresql --bulk-separator , --debug 2022-09-06T10:46:24.8716393Z Running yuniql v1.3.15 for windows-x64 2022-09-06T10:46:24.8756349Z Copyright 2019 (C) Rodel E. Dagumampan. Apache License v2.0 2022-09-06T10:46:24.8785438Z Visit https://yuniql.io for documentation and working samples 2022-09-06T10:46:24.8786565Z 2022-09-06T10:46:24.8793840Z WRN 2022-09-06 10:46:24Z Trace logs settings is set to silent (default) and no log files will be produced. To enable log file creation, pass parameter --trace-to-file or see our CLI command reference. 2022-09-06T10:46:25.1102849Z INF 2022-09-06 10:46:25Z Run configuration: 2022-09-06T10:46:25.1135532Z { 2022-09-06T10:46:25.1148728Z "isInitialized": true, 2022-09-06T10:46:25.1161427Z "workspace": "_PBM_DataManagement_DB_CI/Database", 2022-09-06T10:46:25.1165675Z "platform": "postgresql", 2022-09-06T10:46:25.1180094Z "connectionString": "", 2022-09-06T10:46:25.1180834Z "isAutoCreateDatabase": false, 2022-09-06T10:46:25.1181383Z "targetVersion": null, 2022-09-06T10:46:25.1181870Z "tokens": [ 2022-09-06T10:46:25.1198104Z {}, 2022-09-06T10:46:25.1200725Z {} 2022-09-06T10:46:25.1215472Z ], 2022-09-06T10:46:25.1218191Z "isVerifyOnly": true, 2022-09-06T10:46:25.1244687Z "bulkSeparator": ",", 2022-09-06T10:46:25.1245149Z "bulkBatchSize": 0, 2022-09-06T10:46:25.1245682Z "commandTimeout": 30, 2022-09-06T10:46:25.1246065Z "isDebug": true, 2022-09-06T10:46:25.1246439Z "appliedByTool": "yuniql-cli", 2022-09-06T10:46:25.1246888Z "appliedByToolVersion": "1.3.15.0", 2022-09-06T10:46:25.1247343Z "environment": null, 2022-09-06T10:46:25.1247717Z "metaSchemaName": null, 2022-09-06T10:46:25.1249488Z "metaTableName": null, 2022-09-06T10:46:25.1249977Z "isContinueAfterFailure": false, 2022-09-06T10:46:25.1250420Z "transactionMode": "session", 2022-09-06T10:46:25.1295066Z "isRequiredClearedDraft": false, 2022-09-06T10:46:25.1295528Z "isForced": false 2022-09-06T10:46:25.1295738Z } 2022-09-06T10:46:25.3018305Z INF 2022-09-06 10:46:25Z No explicit target version requested. We'll use latest available locally v0.01 on _PBM_DataManagement_DB_CI/Database. 2022-09-06T10:46:25.5313302Z DBG 2022-09-06 10:46:25Z Replaced token YUNIQL_TABLE_NAME with __yuniql_schema_version 2022-09-06T10:46:25.6235109Z DBG 2022-09-06 10:46:25Z Replaced token YUNIQL_SCHEMA_NAME with public 2022-09-06T10:46:25.6235670Z DBG 2022-09-06 10:46:25Z Executing statement BA15F9: 2022-09-06T10:46:25.6238274Z 2022-09-06T10:46:25.6258919Z SELECT 1 FROM pg_tables WHERE tablename = '__yuniql_schema_version' AND schemaname = 'public'; 2022-09-06T10:46:25.6266701Z
2022-09-06T10:46:26.4426448Z DBG 2022-09-06 10:46:26Z Statement BA15F9 executed in 891 ms 2022-09-06T10:46:26.4579709Z DBG 2022-09-06 10:46:26Z Executing statement E5CCD9: 2022-09-06T10:46:26.4583927Z 2022-09-06T10:46:26.4584433Z SELECT 'v1.1' FROM pg_tables WHERE tablename = '__yuniqldbversion'; 2022-09-06T10:46:26.4584784Z
2022-09-06T10:46:26.4639118Z DBG 2022-09-06 10:46:26Z Statement E5CCD9 executed in 6 ms 2022-09-06T10:46:26.4639997Z INF 2022-09-06 10:46:26Z The schema version tracking table is up to date for tpm on 10.55.127.8. 2022-09-06T10:46:26.4708801Z DBG 2022-09-06 10:46:26Z Replaced token YUNIQL_SCHEMA_NAME with public 2022-09-06T10:46:26.4709568Z DBG 2022-09-06 10:46:26Z Replaced token YUNIQL_TABLE_NAME with __yuniql_schema_version 2022-09-06T10:46:26.4710123Z DBG 2022-09-06 10:46:26Z Executing statement: 2022-09-06T10:46:26.4710410Z 2022-09-06T10:46:26.4711024Z SELECT sequence_id, version, applied_on_utc, applied_by_user, applied_by_tool, applied_by_tool_version, status, duration_ms, checksum, failed_script_path, failed_script_error, additional_artifacts 2022-09-06T10:46:26.4711683Z FROM public.__yuniql_schema_version ORDER BY version ASC; 2022-09-06T10:46:26.4712364Z
2022-09-06T10:46:26.4912457Z DBG 2022-09-06 10:46:26Z Replaced token YUNIQL_SCHEMA_NAME with public 2022-09-06T10:46:26.4913141Z DBG 2022-09-06 10:46:26Z Replaced token YUNIQL_TABLE_NAME with __yuniql_schema_version 2022-09-06T10:46:26.4913743Z DBG 2022-09-06 10:46:26Z Executing statement: 2022-09-06T10:46:26.4913930Z 2022-09-06T10:46:26.4915448Z SELECT sequence_id, version, applied_on_utc, applied_by_user, applied_by_tool, applied_by_tool_version, status, duration_ms, checksum, failed_script_path, failed_script_error, additional_artifacts 2022-09-06T10:46:26.4916140Z FROM public.__yuniql_schema_version ORDER BY version ASC; 2022-09-06T10:46:26.4916425Z
2022-09-06T10:46:26.4989304Z DBG 2022-09-06 10:46:26Z Replaced token YUNIQL_SCHEMA_NAME with public 2022-09-06T10:46:26.4990027Z DBG 2022-09-06 10:46:26Z Replaced token YUNIQL_TABLE_NAME with __yuniql_schema_version 2022-09-06T10:46:26.4990499Z DBG 2022-09-06 10:46:26Z Executing statement: 2022-09-06T10:46:26.4991060Z 2022-09-06T10:46:26.4991749Z SELECT sequence_id, version, applied_on_utc, applied_by_user, applied_by_tool, applied_by_tool_version, status, duration_ms, checksum, failed_script_path, failed_script_error, additional_artifacts 2022-09-06T10:46:26.4992651Z FROM public.__yuniql_schema_version ORDER BY version ASC; 2022-09-06T10:46:26.4993283Z
2022-09-06T10:46:26.5114790Z INF 2022-09-06 10:46:26Z Transaction created for current session. This migration run will be executed in a shared connection and transaction context. 2022-09-06T10:46:26.5248341Z INF 2022-09-06 10:46:26Z Found 0 script files on _PBM_DataManagement_DB_CI/Database_pre 2022-09-06T10:46:26.5263286Z INF 2022-09-06 10:46:26Z Found 0 bulk files on _PBM_DataManagement_DB_CI/Database_pre 2022-09-06T10:46:26.5263873Z INF 2022-09-06 10:46:26Z Executed script files on _PBM_DataManagement_DB_CI/Database_pre 2022-09-06T10:46:26.5390907Z INF 2022-09-06 10:46:26Z Found 1 script files on _PBM_DataManagement_DB_CI/Database\v0.01\34-20220905 2022-09-06T10:46:26.5392834Z + 4372_Add_New_fuel.sql 2022-09-06T10:46:26.5557832Z DBG 2022-09-06 10:46:26Z Executing sql statement as part of : _PBM_DataManagement_DB_CI/Database\v0.01\34-20220905\4372_Add_New_fuel.sql 2022-09-06T10:46:26.5567536Z DBG 2022-09-06 10:46:26Z Executing statement 3FBFA4: 2022-09-06T10:46:26.5567931Z CONNECT TP_DEV 2022-09-06T10:46:26.5568149Z 2022-09-06T10:46:26.5568228Z 2022-09-06T10:46:26.5568404Z DECLARE 2022-09-06T10:46:26.5568759Z id_max_p INTEGER; 2022-09-06T10:46:26.5568984Z id_Market INTEGER; 2022-09-06T10:46:26.5569366Z id_max_master_data_change INTEGER;
2022-09-06T10:46:26.5569779Z rowCnt INTEGER := 0; 2022-09-06T10:46:26.5570036Z id_fueltype INTEGER;
2022-09-06T10:46:26.5570253Z BEGIN 2022-09-06T10:46:26.5570518Z --**** Add New Fuel BIO1_MV3_CDS ***** 2022-09-06T10:46:26.5570923Z --SELECT nvl((SELECT 1 FROM fuel WHERE name='BIO1_MV3_CDS'),0) INTO rowCnt FROM dual;
2022-09-06T10:46:26.5571405Z SELECT count(1) INTO rowCnt FROM fuel WHERE name='BIO1_MV3_CDS' ; 2022-09-06T10:46:26.5571699Z
2022-09-06T10:46:26.5571902Z IF rowCnt = 0 THEN 2022-09-06T10:46:26.5572234Z SELECT ID INTO id_Market FROM market_area WHERE name = 'NL'; 2022-09-06T10:46:26.5572802Z SELECT ID INTO id_fueltype FROM Fuel_Type where Name='Biomass'; 2022-09-06T10:46:26.5573285Z SELECT MAX (ID) + 1 INTO id_max_p FROM fuel; 2022-09-06T10:46:26.5573684Z SELECT MAX(ID) + 1 INTO id_max_master_data_change FROM master_data_change_upload; 2022-09-06T10:46:26.5573993Z
2022-09-06T10:46:26.5574155Z
2022-09-06T10:46:26.5574674Z INSERT INTO fuel (id, name, fuel_type_id, fuel_level, fuel_tax, fuel_tax_uom, cds_price_curve, first_trading_hour,traded_unit_scaling_factor, fuel_hedged, reporting_uom,lagging_structure) 2022-09-06T10:46:26.5575609Z VALUES (id_max_p, 'BIO1_MV3_CDS', id_fueltype, 'Nothing', null, null, 'BIOMASS_MV3', 0, 1, 1, 't',null); 2022-09-06T10:46:26.5576000Z
2022-09-06T10:46:26.5576449Z INSERT INTO master_data_change_upload(ID, action, asset_type, asset_id,upload_name,comments,file_name,kid,upload_timestamp,market_area_id) 2022-09-06T10:46:26.5577106Z VALUES (id_max_master_data_change, 'Add', 'Fuel', id_max_p,NULL,'BIO1_MV3_CDS',NULL,NULL,SYSDATE,id_Market); 2022-09-06T10:46:26.5577832Z
2022-09-06T10:46:26.5578446Z INSERT INTO fuel_audit (id, name, fuel_type_id, fuel_level,fuel_tax,fuel_tax_uom, cds_price_curve, first_trading_hour, traded_unit_scaling_factor, fuel_hedged, reporting_uom,lagging_structure, master_data_change_upload_id) 2022-09-06T10:46:26.5579306Z VALUES (id_max_p,'BIO1_MV3_CDS',id_fueltype,'Nothing',null,null,'BIOMASS_MV3',0,1,1,'t', NULL,id_max_master_data_change);
2022-09-06T10:46:26.5579738Z -- COMMIT; 2022-09-06T10:46:26.5579941Z END IF; 2022-09-06T10:46:26.5580146Z End 2022-09-06T10:46:26.5580327Z / 2022-09-06T10:46:26.6949437Z ERR 2022-09-06 10:46:26Z Failed to execute verification function. Target database will be rolled back to its previous state. Migration of version v0.01 was running in transaction and has failed while attempting to execute script file _PBM_DataManagement_DB_CI/Database\v0.01\34-20220905\4372_Add_New_fuel.sql due to error "(0x80004005) Error 42601: syntax error at or near "CONNECT" 2022-09-06T10:46:26.6950344Z 2022-09-06T10:46:26.6951186Z POSITION: 1. Exception data: Severity: ERROR, InvariantSeverity: ERROR, SqlState: 42601, MessageText: syntax error at or near "CONNECT", Position: 1, File: scan.l, Line: 1135, Routine: scanner_yyerror". The target database will initiate a rollback attempt to revert all applicable DML and DDL changes. The database auto-created will be kept but would be empty. 2022-09-06T10:46:26.6952155Z Diagnostics stack trace captured a Yuniql.Core.YuniqlMigrationException: 2022-09-06T10:46:26.6952671Z ---> Npgsql.PostgresException (0x80004005): 42601: syntax error at or near "CONNECT" 2022-09-06T10:46:26.6953977Z 2022-09-06T10:46:26.6954239Z POSITION: 1 2022-09-06T10:46:26.6954861Z at Npgsql.Internal.NpgsqlConnector.g__ReadMessageLong|211_0(NpgsqlConnector connector, Boolean async, DataRowLoadingMode dataRowLoadingMode, Boolean readingNotifications, Boolean isReadingPrependedMessage) 2022-09-06T10:46:26.6955688Z at Npgsql.NpgsqlDataReader.NextResult(Boolean async, Boolean isConsuming, CancellationToken cancellationToken) 2022-09-06T10:46:26.6956160Z at Npgsql.NpgsqlDataReader.NextResult() 2022-09-06T10:46:26.6956657Z at Npgsql.NpgsqlCommand.ExecuteReader(CommandBehavior behavior, Boolean async, CancellationToken cancellationToken) 2022-09-06T10:46:26.6957273Z at Npgsql.NpgsqlCommand.ExecuteReader(CommandBehavior behavior, Boolean async, CancellationToken cancellationToken) 2022-09-06T10:46:26.6957859Z at Npgsql.NpgsqlCommand.ExecuteNonQuery(Boolean async, CancellationToken cancellationToken) 2022-09-06T10:46:26.6958294Z at Npgsql.NpgsqlCommand.ExecuteNonQuery() 2022-09-06T10:46:26.6958897Z at Yuniql.Core.ConnectionExtensions.ExecuteNonQuery(IDbConnection connection, String commandText, Nullable1 commandTimeout, IDbTransaction transaction, ITraceService traceService) 2022-09-06T10:46:26.6959755Z at Yuniql.Core.MetadataService.ExecuteSql(IDbConnection connection, String commandText, Nullable1 commandTimeout, IDbTransaction transaction, ITraceService traceService) 2022-09-06T10:46:26.6960451Z at Yuniql.Core.MigrationService.<>c__DisplayClass16_1.b__3(String sqlStatement) 2022-09-06T10:46:26.6960937Z at System.Collections.Generic.List1.ForEach(Action1 ) 2022-09-06T10:46:26.6961410Z at Yuniql.Core.MigrationService.<>c__DisplayClass16_0.b__1(String scriptFile) 2022-09-06T10:46:26.6961876Z at System.Collections.Generic.List1.ForEach(Action1 ) 2022-09-06T10:46:26.6962942Z at Yuniql.Core.MigrationService.RunVersionSqlScripts(IDbConnection connection, IDbTransaction transaction, TransactionContext transactionContext, Stopwatch stopwatch, String version, String workspace, String scriptDirectory, String metaSchemaName, String metaTableName, List1 tokens, Nullable1 commandTimeout, String environment, String appliedByTool, String appliedByToolVersion) 2022-09-06T10:46:26.6963881Z Exception data: 2022-09-06T10:46:26.6964115Z Severity: ERROR 2022-09-06T10:46:26.6964540Z SqlState: 42601 2022-09-06T10:46:26.6964848Z MessageText: syntax error at or near "CONNECT" 2022-09-06T10:46:26.6965125Z Position: 1 2022-09-06T10:46:26.6965343Z File: scan.l 2022-09-06T10:46:26.6965572Z Line: 1135 2022-09-06T10:46:26.6965850Z Routine: scanner_yyerror 2022-09-06T10:46:26.6966159Z --- End of inner exception stack trace --- 2022-09-06T10:46:26.6967138Z at Yuniql.Core.MigrationService.RunVersionSqlScripts(IDbConnection connection, IDbTransaction transaction, TransactionContext transactionContext, Stopwatch stopwatch, String version, String workspace, String scriptDirectory, String metaSchemaName, String metaTableName, List1 tokens, Nullable1 commandTimeout, String environment, String appliedByTool, String appliedByToolVersion) 2022-09-06T10:46:26.6968261Z at Yuniql.Core.MigrationService.<>c__DisplayClass12_2.b__11(String scriptSubDirectory) 2022-09-06T10:46:26.6968755Z at System.Collections.Generic.List1.ForEach(Action1 ) 2022-09-06T10:46:26.6969562Z at Yuniql.Core.MigrationService.<>c__DisplayClass12_0.g__RunVersionDirectoriesInternal|8(IDbConnection connection, IDbTransaction transaction, List1 scriptSubDirectories, String scriptDirectory, String versionDirectory, Stopwatch stopwatch) 2022-09-06T10:46:26.6970451Z at Yuniql.Core.MigrationService.<>c__DisplayClass12_0.<RunVersionDirectories>b__7(<>f__AnonymousType02 version) 2022-09-06T10:46:26.6970956Z at System.Collections.Generic.List1.ForEach(Action1 ) 2022-09-06T10:46:26.6972074Z at Yuniql.Core.MigrationService.RunVersionDirectories(IDbConnection connection, IDbTransaction transaction, List1 appliedVersions, String workspace, String targetVersion, TransactionContext transactionContext, List1 tokens, String bulkSeparator, String metaSchemaName, String metaTableName, Nullable1 commandTimeout, Nullable1 bulkBatchSize, String appliedByTool, String appliedByToolVersion, String environment, String transactionMode) 2022-09-06T10:46:26.6973691Z at Yuniql.Core.MigrationService.g__RunAllInternal|11_3(IDbConnection connection, IDbTransaction transaction, Boolean isRequiredClearedDraft, <>c__DisplayClass11_0& ) 2022-09-06T10:46:26.6975017Z at Yuniql.Core.MigrationService.Run(String workspace, String targetVersion, Nullable1 isAutoCreateDatabase, List1 tokens, Nullable1 isVerifyOnly, String bulkSeparator, String metaSchemaName, String metaTableName, Nullable1 commandTimeout, Nullable1 bulkBatchSize, String appliedByTool, String appliedByToolVersion, String environment, Nullable1 isContinueAfterFailure, String transactionMode, Boolean isRequiredClearedDraft) 2022-09-06T10:46:26.6976089Z at Yuniql.Core.MigrationService.Run() 2022-09-06T10:46:26.6976468Z at Yuniql.CLI.CommandLineService.RunVerifyOption(VerifyOption opts) 2022-09-06T10:46:26.6976967Z If you think this is a bug, please report an issue here https://github.com/rdagumampan/yuniql/issues. 2022-09-06T10:46:26.7150696Z 2022-09-06T10:46:26.7233107Z yuniql/error: The process 'F:\Softwares\vsts-agent-win-x64-2.165.0_work_tool\yuniql\0.0.0\x64\yuniql.exe' failed with exit code 1 2022-09-06T10:46:26.7272942Z ##[error]The process 'F:\Softwares\vsts-agent-win-x64-2.165.0_work_tool\yuniql\0.0.0\x64\yuniql.exe' failed with exit code 1

rdagumampan

rdagumampan

documentation
Icon For Comments0

@rioka Thanks for reaching out and apologies for the delay... Summer time in EU :). Its our backlog to write up a documentation for baselining existing database. Hmmm, I think what you can do is to create v0.00 with just simple script like init.sql and a --This is baseline database version statement. Execute this in your staging and production. This will establish the needed tracking tables in the database. Then you can add all the baseline script files in v0.00 and execute first time in dev and test. This enable you to hold all baseline scripts in v0.00 and get skipped in staging and production.

This is still hackish or work around. Sounds like yuniql rebase which is what we need here...

Originally posted by @rdagumampan in https://github.com/rdagumampan/yuniql/issues/280#issuecomment-1217053198

jan-hudec

jan-hudec

community-feedback
Icon For Comments1

I tested the Authentication=Active Directory Default authentication method as per https://docs.microsoft.com/en-us/sql/connect/ado-net/sql/azure-active-directory-authentication?view=azuresqldb-current and it logged in automatically by calling out to azure cli as expected.

Since all the Authentication methods in that list are configured with connection string and environment only, I am somewhat confident they will all work given the correct circumstances.

The only limitation then really is obtaining the authentication token externally (e.g. with az account get-access-token --resource=https://database.windows.net or the corresponding call of the identity library), because that cannot be passed via connection string but must be assigned to the SqlConnection.AccessToken separately.

rioka

rioka

documentation
Icon For Comments1

I'm not clear what's the best way to start using yuniql with existing databases...

Let's say I have my "v0.00" folder configured, with all the script required to create a database aligned to the current schema, and my staging or production database already in place and full of data; I cannot find a way to easily set "v0.00" as the current database version using yuniql itself.

As far as I can see, there's no clean and direct way to achieve that, the best I can think of is always include some manual hacks I'd rather avoid, also taking into account that it shuold then be replicated on potentially dozens of deployed databases.

What am I missing?

BTW kudos for this tool.

shaileshsolanki16

shaileshsolanki16

community-feedback
Icon For Comments1

Hello Rodel!

I added as string Schema= while setting up the environment variable SETX YUNIQL_CONNECTION_STRING="Host=localhost;Port=5432;Username=;Password=;Database=;Schema="

Below is the error and I think it is due to the addtional string Schema= in the connection string. Please let me know how do i mention the schema name.

ERR 2022-07-04 14:06:18Z Failed to execute run function. Couldn't set schema (Parameter 'schema') Diagnostics stack trace captured a System.ArgumentException: ---> System.Collections.Generic.KeyNotFoundException: The given key was not present in the dictionary. at Npgsql.NpgsqlConnectionStringBuilder.GeneratedSetter(String keyword, Object value) at Npgsql.NpgsqlConnectionStringBuilder.set_Item(String keyword, Object value) --- End of inner exception stack trace --- at Npgsql.NpgsqlConnectionStringBuilder.set_Item(String keyword, Object value) at System.Data.Common.DbConnectionStringBuilder.set_ConnectionString(String value) at Npgsql.NpgsqlConnectionStringBuilder..ctor(String connectionString) at Yuniql.PostgreSql.PostgreSqlDataService.GetConnectionInfo() at Yuniql.Core.MigrationService.Run(String workspace, String targetVersion, Nullable1 isAutoCreateDatabase, List1 tokens, Nullable1 isVerifyOnly, String bulkSeparator, String metaSchemaName, String metaTableName, Nullable1 commandTimeout, Nullable1 bulkBatchSize, String appliedByTool, String appliedByToolVersion, String environment, Nullable1 isContinueAfterFailure, String transactionMode, Boolean isRequiredClearedDraft) at Yuniql.Core.MigrationService.Run() at Yuniql.CLI.CommandLineService.RunRunOption(RunOption opts) If you think this is a bug, please report an issue here https://github.com/rdagumampan/yuniql/issues.

Thank you, Shailesh

nandlaly232

nandlaly232

az-devops
Icon For Comments2

Hi , I am trying to change some column in postgres database . I using Azure pipeline to run CI-CD part . Till last month it was working fine . Since this month the yuniql verify step has started to fail .

I suspect in new version of yuniql cli , default token are not passed . Even If I try to pass manually , it does not accept.

Here is log of last month when it was working .

MicrosoftTeams-image

Below is latest failed log

MicrosoftTeams-image (1)

Here yuniql cli version has been updated from 1.2.25 to 1.3.15

Thanks .

camillo-positano

camillo-positano

bug
Icon For Comments2

Hello, i noticed that if you delete the folder v.0.00 and execute vnext --major it goes in exception saying:

Diagnostics stack trace captured a Sequence contains no elements.

I decided to check where it could have generated and found out that at line 152 of WorkspaceService.cs in yuniql-core there is: localVersions.First().Major + 1

I think the first() is what generates the exception. It would be cool to have the ability to create a major without having to keep the v.0.00 file.

To give you some detail on how this happened:

I wanted to start a repo which began with a major. To do so i deleted the folder and wanted to create a major release from cli, which caused the exception.

The workaround was to keep v.0.00 and delete it afterwards.

Have a nice day, a fellow developer 😄

kamalsivalingam

kamalsivalingam

documentation
Icon For Comments2

Hi,

I was running into an issue with executing environment aware scripts using Azure DevOps Verify and Run Task. Looks like the argument for the environment is --environment "staging"

The documentation seems to be not in sync. Couple of things:

  • Environment name should be inside double quotes
  • Environment name is case sensitive

Hope this helps someone

coredat-abru

coredat-abru

new idea
Icon For Comments6

I have a setup where I'd like to apply migrations for multiple environments at the same time. I want to use this in order to be able to apply specific migrations for specific features only. As an example, say I'm developing feature "Foo" and have the environments "dev", "staging" and "prod", I'd setup my version folder the following way:

In this example _dev would run if dev was set dev_Foo would run if both dev and Foo are set Foo runs if Foo is set prod_Foo would run if both prod and Foo are set

From what I understand, the bulk of this change would be implemened https://github.com/rdagumampan/yuniql/blob/01465775b1d19604222d4a511dd9a13e0ae40f97/yuniql-core/DirectoryService.cs#L51 Is this a feature you would be willing to support? If yes, then I'd be happy to provide a PR

rdagumampan

rdagumampan

help wanted
Icon For Comments0

Hi friends of yuniql,

If you are using yuniql at work or personal projects, may I request for testimonials by commenting here or by private mail rdagumampan[AT]gmail.com. I have been maintaining this project for 2+ years now and I hope more people can discover its capabilities. Your testimonials could make a difference in lifting up its community profile. With more forks, more stars and more usage = more feedback, a better tool and more reliable yuniql.

P.S. If you have haven't Star this repo, please do. Thanks ;)

Sincerely, Rodel / Maintainer of yuniql project

avengerovsky

avengerovsky

state: investigating
Icon For Comments6

It looks like yuniql replaces non-ascii characters in my script with '�'. Example: select regexp_replace(asset_name, '[–—]', '-', 'g') as name from xxx becomes select regexp_replace(asset_name, '[��]', '-', 'g') as name from xxx

The same script executed with psql or dbeaver produces correct representation of the characters.

Thoughts?

rdagumampan

rdagumampan

test-improvement
Icon For Comments0

@absurdiumnoil I think yuniql is case insensitive in terms of directory names. Appreciate if you can test run and let me know if there was anything broken. I must admit I haven't spent much time lately testing the container image releases. If you can do with install the CLI via dotnet tool or choco, I would recommend this until we have more coverage on container images.

https://github.com/rdagumampan/yuniql/blob/01465775b1d19604222d4a511dd9a13e0ae40f97/yuniql-core/DirectoryService.cs#L76

Originally posted by @rdagumampan in https://github.com/rdagumampan/yuniql/issues/244#issuecomment-1026330274

rdagumampan

rdagumampan

refactoring
Icon For Comments0

@RobertWilc Thanks for looking out the code and @automada was right in the consistency/readability reasoning.

@automada In other implementations, we named this method GetDestinationSchema. A new connection is also initated here instead of passing existing connection. Can't remember why I did it this way but perhaps to isolate the connection if the bulk upload is happenign in streaming fashion, avoid uninintendted side effect.

I will akso mark this GetDestinationSchema and ParseCsv as something we can generalize to a base class, extension method or another class we can inject. It's duplicared several times already :)

Originally posted by @rdagumampan in https://github.com/rdagumampan/yuniql/pull/243#discussion_r796161992

absurdumnoil

absurdumnoil

community-feedback
Icon For Comments2

I'm using the latest yuniql Docker image for linux and I am wondering if it is possible to update the example commands of environment aware scripts in the docs to be all lowercase instead of uppercase. Because when using yuniql in said image it would not run --environment "TEST" for _test folder, but --environment "test" would work. I got the impression from the docks that the parameter should be uppercase, but I see even the unit tests in the source use lowercase. No idea if this behaves differently in the Windows image.

The locations I found so far in the docks are at:

yuniql-cli-command-reference/ -> yuniql run -> --environment, "DEV" => "dev"

environment-aware-scripts/ -> Run migrations, DEVELOPMENT => development, TEST => test, STAGING => staging, PRODUCTION => production

Versions

Quick list of the latest released versions

v1.3.15 - May 25, 2022

This is a hotfix to reported SNI-related errors when connecting to SqlServer

Error thrown when using installation from choco and downloaded exe file. This is because Microsoft.Data.SqlClient.SNI.dll is excluded when publishing self-contained application.

ERR 2022-05-24 20:55:24Z Failed to execute run function. The type initializer for 'Microsoft.Data.SqlClient.TdsParser' threw an exception. Diagnostics stack trace captured a System.TypeInitializationException: ---> System.TypeInitializationException: The type initializer for 'Microsoft.Data.SqlClient.SNILoadHandle' threw an exception. ---> System.DllNotFoundException: Dll was not found. at Microsoft.Data.SqlClient.SNINativeMethodWrapper.SNIInitialize(Boolean useSystemDefaultSecureProtocols, IntPtr pmo) at Microsoft.Data.SqlClient.SNINativeMethodWrapper.SNIInitialize()

v1.3.10 - May 23, 2022

This is release comes heavy on changes, the .NET 6 LTS upgrade and migration to Microsoft.Data.SqlClient driver for SqlServer. These changes promises better security and performance. And they are both contribution from our community! Enjoy this release. HTH.

Breaking changes

  • Due to migration to the new Microsoft driver for SQL Sever, the Encrypted connection is now enabled by default. An error described below will ne thrown in your next migration run targeting SqlServer. To resolve this, you may need to install necessary client certificate. Alternatively, you may use self-signed certificate by adding TrustServerCertificate=True in the connection string or ultimately set Encrypt=False as it is before.

ERR 2022-05-08 12:28:35Z Failed to execute run function. A connection was successfully established with the server, but then an error occurred during the login process. (provider: SSL Provider, error: 0 - The certificate chain was issued by an authority that is not trusted.) https://weblog.west-wind.com/posts/2021/Dec/07/Connection-Failures-with-MicrosoftDataSqlClient-4-and-later

Take control of script execution order with _sequence.ini

If you have hundreds of scripts in a directory and prefixed the files with a sequence number to control the sequence of execution, it will be very tedious to reorganize the scripts if you need to insert another script file in the middle. In this release, you can place _sequence.ini file in the version directories like v0.00, in the non-version directories like _erase or in the child directories of those directories like v0.00\setup-tables\. This enhancement works in any directories and including all subdirectories. See snipped below and details here #254:

Full featured container-based runner using docker run yuniql/cli

In this release, we also published new docker image yuniql/cli. This is a stand alone runner and you can run all known actions such as check, run, list, erase and all others. Using volume attachment, you don't need to build an image before you can run migration.

https://hub.docker.com/repository/docker/yuniql/cli https://hub.docker.com/repository/docker/yuniql/yuniql

image

New features and enhancements

  • Upgraded the solution to use .NET 6 LTS version from Net Core 3.1 #223
  • Migrated SqlServer driver to use Microsoft.Data.SqlClient #224
  • Supports integrated Azure AD authentication in SqlServer #229
  • Supports docker run -v <work:space> yuniql/cli:latest #234
  • One thing, currently script order is per File Name sort, sorting this way is not intuitive. #193
  • Script naming file to specify the order instead of numerical numbering #254
  • Display affected record count for every batch of sql statement #257
  • Supports --trace-to-file to support negation #260
  • Supports for SQL Server 2016 based on user feedback #265
  • Created pipelines for continuous testing of docker images #271

Bugs fixes

  • A fatal error was encountered. The library 'libhostpolicy.so' required to execute the application was not found in '/app/' #190
  • Failed connecting to Azure Sql database #199
  • Directory names like v0.00.1, v0.01.1, v0.02.1, v0.03.1 , It was working upto v0.02.1 but when I deployed v0.03.1 it was showing below error. #252
  • 'yuniql run' command executed through Azure Pipelines task exposes connection string if it contains an ampersand #259
  • A fatal error was encountered. The library 'libhostpolicy.so' required to execute the application was not found in '/app/'. #270

Credits

Special thanks to @pmelander for the migration to Microsoft.Data.SqlClient, @Jean-Paul76 for upgrading to .NET 6 LTS. These are significant contrubutions and certainly one hard to get through easily. Also for everyone who filed issue tickets and asked questions, thanks for all your feedback! HTH

v1.3.2 - Mar 06, 2022

v1.3.2

  • New feature from issue #254. Script naming file to specify the order instead of numerical numbering.
  • Hotfix for issue #252. Directory names like v0.00.1, v0.01.1, v0.02.1, v0.03.1 , It was working up to v0.02.1 but when I deployed v0.03.1 it was showing below error. If you are using {major}.{minor} version semantics, v1.2.25 should be fine for you.

v1.2.25 - Feb 20, 2022

Every release is special and this is not an exception. This pandemic period presents a rare opportunity to review more PRs, follow-up on reported issues, build prioritized features and fix major bugs. In fact, perhaps this is the release with the most bug-fixes! A testament of our growing usage and feedback from our community. We can only thank you all for your issue tickets and contributions. And here are the highlights:

Multi-tenant database support

The idea behind this popular request is to perform schema versioning on shared database where objects are segmented by schema. This would also enable developers and teams to get dedicated schema version and evolve independently while using single installation. This is more important when teams are not permitted to create and destroy databases during dev/test.

Support Oracle Enterprise & Autonomous Database

This release comes with previous support for Oracle. The platform tests has been executed using Oracle Enterprise 12C on docker container and Oracle Autonomous Database. You can explore this feature and get started here https://yuniql.io/docs/get-started-oracle/

Quick connectivity check with yuniql check

Can I resolve or ping the database server? Can I establish an Sql/Odbc connection to the master or catalog database? Will my connection string work if I run the migrations on this database server? Is my service account have permissions to establish a session? I hope this simple feature helps in diagnosing migration issues quickly.

Drop your dev/test database with yuniql destroy

Warning: Needless to say, don't use this in production. Just don't. This feature enables testers to rapidly deploy test databases and destroy when the test execution completes. If yuniql erase is not sufficient, yuniql destroy might suffice. It's quite simple, you prepare the destroy scripts yourself and do the necessary handling and sequence to drop the database.

Breaking changes

  • None

New features and enhancements

  • Support multi-tenant database
  • Support Oracle platform
  • yuniql check: Basic connectivity check to the target database.
  • yuniql destroy: Drops database using custom prepared scripts.
  • yuniql apply: A synonym for yuniql run to avoid conflict in docker run
  • trace-silent: Console window only log and no log files produced.
  • trace-sensitive-data: Print secrets and connection strings into console window and log files.
  • trace-directory: Set the directory where log files are created.
  • Many many bug fixes and code improvements

Bugs fixes

  • Directory names like v10.0.0 and higher versions are not taking as latest #249
  • Failed database updates should ideally fail with a non-zero exit code #198
  • MSSQL/New version: SUSER_SNAME() throw an exception: String or binary data would be truncated. #201
  • Providing table name for the schema version table cause error #202
  • I am implementing Erease Yuniql and I get the following error This NpgsqlTransaction has completed; it is no longer usable, the process deletes my tables in PostgreSQL (HyperScale) but flag that error. #204
  • Error when setting up multi-tenant on shared databases instance #207
  • Environment-aware migration not working. #208
  • -- environment argument does not work #214
  • CSV import towards SQLServer with a uniqueidentifier column #239
  • Fixed failing bulk import when destination tables uses Pascal case

Refactorings

Credits

Special thanks to @bobi591, @wahmedswl, @zwdOr20, @black-eagle17, @automada, @gfnavarro for your significatn contrubutions in this release. Also for everyone filing issue tickets and asking questions, thanks for all your feedback.

v1.1.55 - Feb 01, 2021

This release is very special as its packed with a back story of going through challenges of COVID-19 and support for two major cloud datawarehouse platforms Redshift and Snowflake. In this release, we're also able to support diverse transaction modes so we can run either in transaction for run session, for every version, or for every sql statement. The capability enables us to better track migration failures on databases that do not support full transactional DDL such as MySql and Snowflake.

Thank you and looking forward for continuous feedback and issue tickets. This tool only gets better with community of users sharing their issues and feature requests. Inside the box are…

Breaking changes

  • Renamed version tracking table into __yuniql_schema_version. Automatic backup and upgrade of __yuniqldbversion existing tables on your next run.

New features & enhancements

  • PREVIEW! Support for Snowflake Cloud DW
  • PREVIEW! Support for AWS Redshift Cloud DW
  • Sequence number in bulk CSV files ensure right order of CSV files
  • Diverse transaction modes per session, per version and per statement
  • Requires --require-cleared-draft option on yuniql run
  • Richer schema history infromation with duration and error information
  • Pretty print run configuration for better debugging
  • Dedicated yuniql/yuniql Dockerhub image repository
  • New CLI command yuniql platforms
  • Major refactoring, code cleanup and better test coverage
  • Many many bug fixes, prettier console and code improvements

Credits

Special thanks to @bymse, @TaurahSP, @wcontayon and @DavidHopkinsFbr for your significatn contrubutions in this release. Also for everyone filing issue tickets and asking questions, thanks for all your feedback.

Sneek Peek

image

image

Amazon Redshift rapid database deployment and schema versioning with yuniql - preview https://www.youtube.com/watch?v=TUJijZsXP7k

v1.1-0118221 - Jan 18, 2021

Latest nightly build for v1.1 from master

Key development:

  • Support for Snowflake DW (preview) *** NEW!
  • Support for AWS Redshift DW (preview) *** NEW!
  • Sequence number in bulk csv files to address FK constraints
  • Support transaction modes per session, per version, and per-statement
  • Requires --require-cleared-draft option on yuniql run
  • Push docker images to dedicated yuniql/yuniql Dockerhub account
  • New CLI command yuniql platforms
  • Major refactoring, code cleanup and better test coverage
  • Many many bug fixes and code improvements

See yuniql 2021 roadmap here https://github.com/rdagumampan/yuniql/projects/3 HTH, Rodel

v1.0.1 - Jun 28, 2020

v1.0.1

This is a PRODUCTION RELEASE. After 8 months of active development and continuous engament with our early adapters, yuniql is finally graduating into version 1.0. The release captured general use cases for schema versioning and migration for SQL Server, PostgreSql, MySql and MariaDB. Azure DevOps tasks are also made available in this release for both Windows and Linux hosted agents.

I am very excited with this release and looking forward for continuous user feedback and issues reported. This tool only gets better with community of users sharing their issues and feature requests.

image

Breaking changes

  • Renamed yuniql info into yuniql list #116
  • Required --force flag when executing yuniql erase
  • When using Azure DevOps Tasks, change all your tasks to use @1 . For example, user [email protected] instead [email protected]

New features & enhancements

  • Supports Linux Azure DevOps build agents #53
  • Supports MariaDB platform #114
  • Prettier display applied versions in console #82
  • Show the last failed script filename when migration failed #121
  • Reduced log trace messages when --debug is off
  • Supports --no-transaction flag to disable transactional ddl

image

Credits

Special thanks to @Jabestrada and @mguoth for significant contributions in this release. Their active contributions over several months drives many significant development in yuniql and other exciting features in vnext releases.

Also, this best build so far thanks for giving feedback and creating issue tickets:

  • @kuromukira
  • @djshaw01
  • @larsoleruben
  • @farazfaheem
  • @gasystarttask
  • @mikependon

Cheers, Rodel

v0.465.0 - May 27, 2020

v0.465.0

This is a BETA RELEASE. This is stable build and has so far captured and fixes many defects from early adopters of yuniql. Platform tests and unit tests coverage has been significantly increased in this build.

Breaking changes

  • Rename --delimiter parameter into --bulk-separator #84

New features & Enhancements

  • Support custom schema and table name of version tracking table #78
  • Support for labeling versions (to be closer to semantic versioning) #75
  • Remove global transaction for MySql platform #76
  • Build more robust sql batch parser #88
  • Support --continue-after-failure for MySql
  • Support --bulk-batch-size parameter #83

Bug fixes

  • Lack of correct exit code hangs Azure Pipelines when yuniql.exe fails #92
  • Bulk insert with CSV fails when word NULL is used instead of empty #77
  • Scripts _draft directory was skipped when db runs latest version #66
  • Arguments filed is not working as expected in Azure Pipeline Extension #73

Credits

Special thanks to @mguoth for significant improvement in yuniql's MySql support. Also, this best build so far thanks for giving feedback and creating issue tickets:

  • @kuromukira
  • @djshaw01
  • @larsoleruben
  • @farazfaheem
  • @gasystarttask
  • @mikependon

Cheers, Rodel

v0.355.0 - Mar 04, 2020

This PREVIEW release fixes some of reported bugs in the metadata table.

Enhancement:

#64 Add ToolVersion in metadata table #63 Support "Verify YUNIQL" Az DevOps Task #62 Support "Erase YUNIQL" Az DevOps Task

Bug fixes:

#65 Missing data in pgsql tracking table bug #60 "Incorrect syntax near '-'" when database name contains

v0.350.0 - Feb 11, 2020

v0.330.0 - Feb 02, 2020

v0.323.0 - Feb 02, 2020

Library Stats (Sep 18, 2022)

Subscribers: 8
Stars: 319
Forks: 47
Issues: 75

NET Standard wrapper for FFmpeg

It allows to process media without know how FFmpeg works, and can be used to pass customized arguments to FFmpeg from dotnet core application

NET Standard wrapper for FFmpeg

dotnet-sshdeploy

here, otherwise you are in the right place

dotnet-sshdeploy

✨ imgup [dotnet-imgup]

NET Core global CLI tool to upload images on published to nuget

✨ imgup [dotnet-imgup]

Dotnet client for Tarantool NoSql database

Some methods are not implemented yet because there are no direct analogs in IProto

Dotnet client for Tarantool NoSql database

dotnet-coverageconverter

coverage (binary format) files to

dotnet-coverageconverter
CLI

1.9K

Welcome to dotnet sdk

This repo contains core functionality needed to create

Welcome to dotnet sdk

dotnet-stellar-sdk Stellar API SDK for

Report Bug · Report Security Vulnerability

dotnet-stellar-sdk Stellar API SDK for

✨ namechk [dotnet-namechk]

NET Core global CLI tool to check for the availability of NuGet package names

✨ namechk [dotnet-namechk]

dotnet-jwk is a JSON Web Key manager for dotnet

It allow to generate, encrypt, decrypt, convert and check JWK

dotnet-jwk is a JSON Web Key manager for dotnet

community-id-dotnet-core

net core implementation of the Community ID flow hashing standard as described by

community-id-dotnet-core
JSON

148

A high-performance async/non-blocking redis client components for dotnet core,default support json and protobuf data format

A high-performance async/non-blocking redis client components for dotnet core,default support json and protobuf data format

A high-performance async/non-blocking redis client components for dotnet core,default support json and protobuf data format

Net Core Api blog application, its is not finished

Seed of all models with Bogus(the Faker nuget package for dotnet)

Net Core Api blog application, its is not finished