MSSQL-SCRIPTER - New SQL Server Command Line Tool for generating T-SQL Scripts

Microsoft recently introduced two new command line tools for SQL Server. 

• mssql-scripter tool enables developers, DBAs, and sysadmins to generate CREATE and INSERT T-SQL scripts for database objects in SQL Server, Azure SQL DB, and Azure SQL DW from the command line.
• The DBFS tool enables DBAs and sysadmins to monitor SQL Server more easily by exposing live data from SQL Server Dynamic Management Views (DMVs) as virtual files in a virtual directory on Linux operating systems.

In this blog post we will see how to use mssql-scripter tool. You can use mssql-scripter on Linux, macOS, and Windows to generate data definition language (DDL) and data manipulation language (DML) T-SQL scripts for database objects in SQL Server running anywhere, Azure SQL Database, and Azure SQL Data Warehouse. This tool is built using Python, so to get this tool installed first you need to install Python on your machine. To download Python click here. 

After installing python, you can try and check the help for PIP. PIP is the package management system used to install and manage packages written in Python.

PIP --help

Now we are good to install the mssql-scripter tool. 

    Pip install mssql-scripter

To check the existence of the package

Pip show mssql-scripter

Let’s have a look at the general usage guide of the mssql-scripter tool.

This tools gives you lot of options to customize your scripting. I have just listed few below

--data-only -> By default only the schema is scripted. if supplied, generate scripts that contains data only.
--schema-and-data -> By default only the schema is scripted. if supplied, generate scripts that contain schema and data.
--target-server-version -> {2005,2008,2008R2,2012,2014,2016,vNext,AzureDB,AzureDW}. Script only features compatible with the specified SQL Version.
--target-server-edition -> {Standard,PersonalExpress,Enterprise,Stretch}. Script only features compatible with the specified SQL
--extended-properties -> Script the extended properties for each object scripted.
--logins -> Script all logins available on the server, passwords will not be scripted.
--object-permissions -> Generate object-level permissions.
--change-tracking -> Script the change tracking information.
--data-compressions -> Script the data compression information.

First let’s try with just scripting schema alone for a particular table [sales.SalesOrderDetail] in AdventureWorks database and write the output to a .sql file.

mssql-scripter -S SQLInstance\vNext -d AdventureWorks2012 --object-schema --include-objects [sales.SalesOrderDetail] >> Sales.sql

--target-server-version and --target-server-edition option looks interesting. I have enabled compression on SalesOrderDetail table which is an enterprise only feature.

Now I am going to script out the schema with specifying target version and edition as 2005 and standard respectively. 

mssql-scripter -S SQLInstance\vNext -d AdventureWorks2012 --object-schema --include-objects sales.SalesOrderDetail --target-server-version 2005 --target-server-edition Standard >> sales2005.sql

I was expecting some sort of warning/error about compression as this feature is not supported in the target version and edition. But in the output it had the syntax for data compression as well.

I am not sure, if I am missing something here. But still it’s the first release of this tool, I am sure this tool is going to be getting better in the future. With this command line tool, moving a database from higher version to lower version can be done in easier way. Instead of going through the Generate SQL Server Script wizard, with this tools it’s just simple one line of code and we are done!

I will explore more and will update my findings here. Stay Tuned

You can download this tool from here.

Thanks for reading!

UPDATE 1 - 02-Jun-2017

Above issue is reported on Microsoft GitHub repository -> https://github.com/Microsoft/sql-xplat-cli/issues/89

Cheers,

Naveen 

SQL Server 2017 Hidden Gems – Part 2

In part-1, I discussed about tracking modified extent page counts.  Starting from SQL Server 2017 CTP 2.0 processor information is visible in sys.dm_os_sys_info dmv. This was another highly requested feature from the community. New columns which are added in sys.dm_os_sys_info dmv are socket_count, cores_per_socket, numa_node_count.

SELECT @@version as 'SQL Version', socket_count, cores_per_socket, numa_node_count FROM sys.dm_os_sys_info

Some of the other ways to get the processor information

Using PowerShell

Get-WmiObject -Class win32_processor | select DeviceID, SocketDesignation, NumberOfCores, NumberOfLogicalProcessors

Using System Information

Open run (Windows Key + R) -> Type msinfo32 -> Hit Enter

Hope this helps

Cheers,

Naveen

SQL Server 2017 Hidden Gems – Part 1

In this blog post series I will be covering the new features and enhancements brought in SQL Server 2017. With SQL Server 2017 CTP 2.0 release, new column modified_extent_page_count is introduced in sys.dm_db_file_space_usage to track differential changes in each database file of the database. This feature was requested by customers and community in this connect item.

With this, now you can configure your differential backup with an extra piece of code to determine the percentage of pages changed since last full backup. With large number of changes in the database, cost and time to complete differential backup is similar to that of full database backup so there is no real benefit of taking differential backup in this case but it can rather increase the restore time of database.

Let’s do a quick demo to see how it works

I am creating a new database and the running sys.dm_db_file_space_usage on the new database.

CREATE DATABASE vNext

GO

USE vNext

GO

SELECT modified_extent_page_count, * FROM SYS.dm_db_file_space_usage

As full backup will be the base for the differential backup, we are not seeing any pages to be tracked under modified extent page column. If you take a full backup and then run the same command, then you should see some value.

Now I am going to load some test data, take a full backup and then check the value in modified_extent_page_count column.

--Creating a test table and inserting 100 records

CREATE TABLE Test (ID INT IDENTITY(1,1), NAME CHAR(8000))

GO

INSERT INTO Test values ('This is a test data')

GO 100

--Perform a full backup of vNext database to create a base for differential backup

BACKUP DATABASE vNext to disk='NUL'

--Take a look at the modified_extent_page_count column

SELECT modified_extent_page_count, * FROM sys.dm_db_file_space_usage

Again I am inserting 500 more records to show that whatever changes we do on that file is being tracked

--Let's insert 500 more records

INSERT INTO Test values ('This is a test data')

GO 500

--Take a look at the modified_extent_page_count column

SELECT modified_extent_page_count, * FROM sys.dm_db_file_space_usage

Now let’s see what happens when you take a differential backup.

BACKUP DATABASE vNext TO DISK='vNext_Differential_1.bak' WITH DIFFERENTIAL

You can see 584 pages were processed in the data file with differential backup. This was the same number which was shown in modified_extent_page_count column.
Let’s do couple of backup test to see how the modified_extent_page_count column value changes. First, we’ll take a copy_only full backup

--value of modified_extent_page_count column
SELECT modified_extent_page_count AS 'Before full backup' FROM sys.dm_db_file_space_usage

--Copy only full backup

BACKUP DATABASE vNext TO DISK='NUL' WITH COPY_ONLY

--value of modified_extent_page_count column

SELECT modified_extent_page_count AS 'After COPY ONLY full backup' FROM sys.dm_db_file_space_usage

As expected, copy_only full backup will not make any change to the backup chain. Now let’s take a full backup of the database

--value of modified_extent_page_count column
SELECT modified_extent_page_count AS 'Before full backup' FROM sys.dm_db_file_space_usage

--Full backup
BACKUP DATABASE vNext TO DISK='NUL'

--value of modified_extent_page_count column
SELECT modified_extent_page_count AS 'After full backup' FROM sys.dm_db_file_space_usage

So once a fresh full backup is taken that will be the base for next differential backup. So modified_extent_page_count will have a new value.

Hope this helps. Please do share your thoughts in the comments below

Cheers,

Naveen

What's new with SQL Server 2017?

Microsoft announced the preview for next release of SQL Server at Microsoft Data Amp. It’s SQL Server 2017!!!

Community Technology Preview (CTP) 2.0 is the first production-quality preview of SQL Server 2017, and it is available on both Windows and Linux. Click here to download SQL Server 2017 CTP 2.0.

Here are some of the exciting new features added to database engine

• Adaptive Query Processing – Faster query processing with improved plan quality.
• Store and analyse non-hierarchical relationships with Graph data support using node/edge query.
• Resumable online index rebuild. Now you can pause and resume the online index rebuilds
• Cluster-less Availability Groups for scaling read operations. But with you scale up for read operation, you will not get high availability.

Syntax to create AG without cluster looks like

CREATE AVAILABILITY GROUP [AG-NAME] WITH (CLUSTER_TYPE=NONE)

What’s available on SQL Server on Linux?

Operational Features

• Support for RHEL, Ubuntu, Docker
• SQL Server Linux on Docker
• Package based installs, Docker images
• Support for Open shift, Docker Swarm
• Failover clustering through pacemaker 
• Availability Groups
• SQL Agent
• Replication
• SCOM management Pack
• Log shipping
• DMV's
• Full-Text Search
• Transparent Data Encryption (TDE)

Programming Features

• In memory OLTP
• Columnstore
• Python
• Compression
• Always Encrypted
• Row Level Security
• Dynamic Data Masking
• Service broker
• Change Data Capture (CDC)
• Partitioning
• Auditing
• CLR
• JSON
• XML

Security

• Active Directory authentication coming soon!

What is not available on Linux?

• DB Mail
• Alerts
• FileTables
• Stretch DB
• PolyBase
• XPs
• Mirroring
• VSS Snapshots

Other services which are not available on Linux

• SSAS
• SSRS
• SSIS
• R service
• MDS
• DQS

Windows based SQL Server tools like SSMS and Profiler works when connected to SQL Server on Linux. Native command line tools like SQLcmd, bcp and sqlpackage are also supported on Linux.

Microsoft is also planning to release a new cross platform database admin GUI tool for doing administrative tasks. WOO HOO! This tool will work on Windows, Linux and Mac OS. And this is actually an open source project which will be available on GitHub.

Go-ahead and get your copy of SQL Server 2017 CTP and start exploring these new features

Cheers,

Naveen

Always good to know your GUEST in SQL Server

Few days back came across an interesting issue, which was new for me. Application owner buzzed me and raised concerns about the SQL Server security setup.

App.Owner: We have some serious issue with our database security. I fear wrong people having elevated access to our data.
Me: Ok, Let me have quick look and get back to you.
App.Owner: Yes please. We need to fix this now.
Me: Did a quick check and everything seems to be fine. You’re ID with db_owner and other users with read access to the database.
App.Owner: I was also under the same impression. But today, I came to know there are other users who were also are also able to modify the data. That’s not at all good.
Me: hmm that’s strange. I will do a complete check to figure out the loophole.
App.Owner: Thanks. I will wait to hear from you.
Me: Okay. Ahh Found the suspect. All those privileges were getting transmitted from guest user. I could see someone had mistakenly provided extra permission to guest user, which is not at all required for this environment. Now things are back to normal. Please have a look from your end and let me know you if you have any questions
App.Owner: Great! Thanks for fixing this. Have a good day
Me: You too, have a good day!

In my case issue was with guest user permission. Let’s see the same in demo

-First we will create a new database named ‘TESTDB’. Then will check the status of guest account.

Note: If guest user is enabled in Model database, the same will get applied to newly created database

-Now we will create a SQL Login ‘UserA’, a new table ‘EPL’ and insert 3 records.

We have just created a SQL Login it’s not mapped to any of the database. At instance level also no permission is provided.

Now let’s try doing a simple select on table EPL using UserA.

As expected, ‘UserA’ is not able to fetch the data from TESTDB. Now let’s try enabling and providing read and write permission to ‘Guest’ user in TESTDB. You cannot enable/disable Guest user through GUI, it can be done only with T-SQL.

To Enable Guest Account:

USE [DATABASE_NAME]

GO

GRANT CONNECT TO GUEST

GO

To Disable Guest Account:

USE [DATABASE_NAME]

GO

REVOKE CONNECT TO GUEST

GO

Command completed successfully. Now let’s try running some commands.

So now UserA is able to read and insert records in TESTDB. The same task can be performed by all users associated with the instance. When Guest user is enabled, it will authorize access for logins which are not mapped to the database.

Things to know about Guest user

• It’s always recommended to disable guest user for user database, unless you have any specific/special requirement.
• Guest user cannot be dropped, but can be disabled/enabled.
• Guest user cannot be disabled for Master and Tempdb database.
• On msdb database guest user should not be disabled. If you have any plans to do so, please read this.

Do you use Guest user? On what scenarios you use them?

Please share your thoughts via comments below.

 

Cheers,

Naveen

Enabling trace flags in SQL Server vNext CTP 1.3 on Linux

In my previous post, I have shown you some of the trace flags which are enabled by default on SQL vNext CTP 1.3 Instance on Linux.

In this post we will see how to add trace flags to SQL Startup parameter on Linux.

In windows world, to add trace flag to start-up parameter all we had to do is open SQL configuration manager-> Right click on SQL Service and open its properties -> Navigate to startup parameters tab and add your trace flags.

In Linux world, everything is at file level. All configuration items are stored in a file. To make configurational changes to SQL Instance we will be using mssql-conf under /opt/mssql/bin/.

Now let’s see how to add trace flag to SQL Startup Parameters

If you are already in /opt/mssql/bin/ directory just run

./mssql-conf traceflag 1117 1118 1222 3226 on

Say if you are in /opt directory. Your command will be something like

./mssql/bin/mssql-conf traceflag 1117 1118 1222 3226 on



Now we have successfully added trace flag 1117, 1118, 1222 and 3226. We have to restart SQL service to apply these changes.

To check the status of SQL Service

Systemctl status mssql-server.service

To restart SQL Service

Systemctl restart mssql-server.service

Now let’s connect to the SQL instance and check if the trace flags are enabled

In the next blog post, I will show you how to change your SQL Instance collation on Linux.

Cheers,
Naveen

Trace Flags on SQL Server vNext CTP 1.3

Mid of February Microsoft announced the release of SQL Server vNext CTP 1.3. Below is the list of some of the key enhancements done on database engine

• Indirect checkpoint performance improvements.
• Cluster-less Availability Groups support added.
• Minimum Replica Commit Availability Groups setting added.
• Availability Groups can now work across Windows-Linux to enable cross-OS migrations and testing.
• Temporal Tables Retention Policy support added,
• Online non-clustered columnstore index built and rebuild support added

One thing which surprised me is by default you get a set of trace flags enabled globally. SQL Server vNext CTP 1.3 we have 7 trace flags enabled by default whereas in CTP 1.2 we'll get 6.

Click on the image for better readability

Have you used the below listed trace flags?

Can you share your thoughts on the comments section about these traces flags?

T835

T2563

T3105

T5504

T6529

T8020

T8415

In the next blog post we will see how to set trace flag on SQL startup parameter on Linux.

Cheers,

Naveen