Art of the DBA Rotating Header Image

Azure

Azure SQL Databases with PowerShell: Moving To The Cloud

cloud-computing-2It’s been a while, but I want to wrap up my series on Azure SQL Databases and Powershell. In my last post, I talked about how you can make point in time backups of Azure SQL Databases using the Start-AzureSqlDatabaseExport. This process allows you to make a backup of your Azure SQL Database and store it as an Azure blob. You can then restore these backups using Start-AzureSqlDatabaseImport. It’s all very handy to manage your data within the Azure environment.

Now let’s talk about what I alluded to at the end of that post: using this process to migrate data from your traditional SQL Servers to Azure SQL databases. As we have previously discussed, Azure SQL Database does not support traditional backups as we are used to with regular SQL Server. While there’s a debate on whether or not these should be supported, the reality is that we need another process to handle this. This is where the export/import process comes in.

BACPACing

SQL Server Data Tools(SSDT) have always had a process to extract your database. There are two types of extracts you can perform:

  • DACPAC – A binary file that contains the logical database schema and possibly the data. This file retains the platform version of the database (i.e. 2012, 2014, 2016).
  • BACPAC – A binary file that contains the logical database schema and the data as insert statements. This stores the platform version, but is not locked into it.

The DACPAC extract is useful for managing your database code. These objects can be source controlled and, using SSDT, can be used to compare to a current database state and deploy changes. I’m a big fan of using DACPACs to manage my database code and maintain code consistency. I don’t use the data extract portion, but it’s there if you need it (more on this in a later blog post).

BACPACs are a little more limited. They contain the schema and your data in a logical state, as insert VALUE statements to populate your database objects. The advantage is you get a flexible package that can deploy a database and its data to almost any SQL Server platform (with some caveats, which we will get to).  The disadvantages are that the BACPACs are not as efficient as native backups and are not transactionally consistent.

What this means is that BACPACs are not really suited for day to day backups of your database. They are intended to be a migration tool, especially if you are working with disparate versions of SQL Server. It is why they’re intended for moving databases to Azure SQL Database, because a BACPAC can move your database logically and you are not reliant on the underlying database architecture.

Which brings us to the caveat. While BACPACs allow us to move a database logically, it can not always take into account unsupported features. We can export a database from our SQL Server, but if we’re using something not supported in Azure SQL Database (such as CLR or Service Broker), then this migration will not work. If you are planning this sort of migration, you first need to review your databases for these issues.

Exporting Our Database

Enough about that, let’s get exporting. There are several ways to migrate your database, but we’re going to focus on Powershell and those cmdlets. We can use sqlpackage.exe to create the BACPAC, which is pretty easy (this is really one line, I’m separating it so it’s easier to read):

SqlPackage.exe 
/action:Export 
/SourceServerName:localhost 
/SourceDatabaseName:AdventureWorks2012 
/targetfile:'C:\temp\AW2012.bacpac'

Once we’ve got our .bacpac file, we need to get it up into Azure. For this we need a storage account, then it’s just a matter of copying the blob. For this, we will use the same storage container we used for the exports before. We will create a storage context for the storage account, then use Set-AzureStorageBlobContent to upload the .bacpac file.

$stctxt = (Get-AzureRmStorageAccount -ResourceGroupName test -Name msftest).Context
Set-AzureStorageBlobContent -File C:\TEMP\ADW2012.bacpac -Context $context -Container sqlexports

Liftoff to the Cloud

With our export loaded into Azure, we’re ready to import it into Azure SQL database. We need to first create a database to import the data into:

New-AzureRmSqlDatabase -ResourceGroupName 'IntroAzureSQL' -ServerName 'msf-sqldb' -Edition Basic -DatabaseName AWTEST

After this, it is just a matter of starting the import as we did previously. We need to create a connection context for our Azure SQL Database server, then start the import:

$cred = Get-Credential
$sqlctxt = New-AzureSqlDatabaseServerContext -ServerName msf-sqldb -Credential $cred
$request = Start-AzureSqlDatabaseImport -SqlConnectionContext $sqlctxt -StorageContext $context -StorageContainerName sqlexports -DatabaseName awtest -BlobName ADW2012.bacpac

Notice I save the output of  Start-AzureSqlDatabaseImport into a variable. This is an object requesting the import request, which I can use to check on the import. Remember that calling the cmdlet starts an asynchronous process that will import the schema and data of the .bacpac into your newly created database. We’ll want to check on the process to identify when it completes.

Double Check Your Work

The one last thing to do is to verify the migration, which requires two checks: the schema and the data. For the schema, we will go back to sqlpackage.exe to perform a schema comparison. This is a two step process:

  1. Extract the source schema as a .dacpac
  2. Generate a deploy report targeting the Azure SQL database. If no changes are shown, the schemas match.

Extracting the schema as a .dacpac is a similar operation to extracting the schema as a .bacpac:

SqlPackage.exe /action:Extract `
/SourceServerName:localhost `
/SourceDatabaseName:AdventureWorks2012 `
/p:ExtractAllTableData=True `
/targetFile:C:\Temp\AW2012.dacpac

Generating a deploy report is a similar command, though we have a couple gotchas. First, we’re connecting to our Azure SQL database, so we need our login and password passed to the command. Second, .dacpacs store the database platfom version and will halt if there’s a possible incompatibility. So we need to declare AllowIncompatiblePlatform=True in our sqlpackage call to ignore this. We’ll still get a flag that there could be incompatibilities, but it will at least run the comparison for us:

SqlPackage.exe /action:DeployReport `
/SourceFile:C:\Temp\AW2012.dacpac `
/targetServerName:msf-sqldb.database.windows.net `
/targetDatabaseName:AWTEST `
/targetUser:"$($cred.UserName)" `
/targetPassword:"$($cred.GetNetworkCredential().Password)" `
/p:AllowIncompatiblePlatform=True `
/OutputPath:C:\temp\AWMigration.xml

The resulting .xml file will tell us what changes the deployment would have to make to get everything lined up. Since the goal here is to verify that everything got deployed correctly, we want this report to not have any changes. The desired output would look a little like this:

2016-09-15_8-48-46

Since no changes are listed, there’s nothing to do and everything matches.

Checking your data can be a trickier proposition. For a basic “sniff test”, I just check the row counts of each table. For this, I have a query to check sys.partitions for heaps and clustered indexes (index_id is 0 or 1) and compares the source to the Azure SQL database to look for any differences:

$sql = "select
schema_name(schema_id) +'.'+ t.name as TableName
,p.Rows as Rows
from
sys.tables t
join (select * from sys.partitions where index_id in (0,1)) p on t.object_id = p.object_id
order by TableName"

$local = Invoke-Sqlcmd -ServerInstance localhost -Database AdventureWorks2012 -Query $sql
$azure = Invoke-Sqlcmd -ServerInstance msf-sqldb.database.windows.net -Database AWTEST -Query $sql -Username $cred.UserName -Password $cred.GetNetworkCredential().Password

$matches = @()
foreach($i in $local){
$matches += New-Object psobject -Property @{'TableName'=$i.TableName;'LocalRows'=$i.Rows;'AzureRows'=($azure | Where-Object {$_.TableName -eq $i.TableName}).Rows}
}
$matches | Where-Object {$_.LocalRows -ne $_.AzureRows}

Again, if everything worked correctly we shouldn’t get any output.

I should stress that these two checks do NOT replace thorough regression testing. These should give you a starting point to verify that your migration was successful, but you still will want to test your applications and processing.

The New Hotness

Now, while I would like for Microsoft to include some native cmdlets that accomplish these actions, it should be noted that sqlpackage.exe is a pretty solid tool. It can be used from the command line and wrapping PowerShell around it is still effective. The key is having something I can use from the command line so I can write repeatable scripts. This is important because Azure is driven by automation and, while we can use GUI tools and the portal, we won’t succeed long term in the cloud on these tools alone, so it’s important to understand all the options.

Creating Alerts for Azure SQL Database with Powershell

Last week, as I watched the Twitters, I saw an interesting blog post from Julie Koesmarno(@MsSQLGirl) about creating alerts for Azure SQL Database. These are cool because one of the many things I get asked about when talking about Azure SQL Database is “how do I monitor this darn thing?” With these alerts, you can monitor performance, capacity, and security settings, setting up either email notifications or webhook responses.

Now, me being me, I tweeted out that it would be cool to do this in PowerShell. I say this not just being a geek about the language, but also because if you want to live in the cloud you need to up your automation and scripting game. PowerShell is the way to manage your Azure resources, especially if you want to set up consistent alerts for all your Azure SQL Databases. Julie pointed me to some examples that, while not specific for Azure SQL Database, got me moving in the right direction. I’d like to share some of that learning here.

The cmdlets

To start, we need to know what cmdlets are at our disposal. You’re going to need the Azure and AzureRM modules first (I recommend using the Powershell gallery installation) and then we just need to use PowerShell’s command syntax to find what we’re looking for. Using Get-Command, just do a wildcard search around ‘Azure’ and ‘Alert’ as my search terms.

Get-Command *Azure*Alert*

2016-08-17_15-02-15

It’s not a long list, but we don’t need much to get our work done. It should be noted that these are generic and can be used across all Azure resources.

Because these cmdlets are generic, we have to identify the specific metrics we can monitor for Azure SQL Database. Fortunately, these are discoverable using the Get-AzureRmMetricDefinition, which is not listed but part of the AzureRM.Insights module (use Get-Command -Module AzureRM.Insights). To use it, we need to know our Azure resource ID, which we can derive from three pieces of information: the resource group, the server name, and the database name.

$ResourceGroup = 'IntroAzureSql'
$server = 'msf-sqldb'
$db = 'MSFADMIN'
$rid = (Get-AzureRmResource -ResourceGroupName $ResourceGroup -ResourceName "$server/$db").ResourceID
Get-AzureRmMetricDefinition -ResourceId $rid | Format-Table

2016-08-17_15-11-26

Most of these Name values match what Julie listed in her post, but the nice thing is if we ever get new ones, we can look them up with this method.

Creating an Alert

So let’s get down to brass tacks and actually create an alert. To do this, we need some info first:

  • The Resource Group we will create the alert in.
  • An Azure location where the alert will live.
  • An Azure SQL Database server and database we are creating the alert for.
  • What metric we will monitor and what is the threshold we will be checking.
  • (optional) An email to send an alert to.

With this, here’s the settings I’m going with:

Resource Group – IntroAzureSql
Location – West US
Server – msf-db
Database – MSFADMIN
Alert – dtu_consumption_percent greater than 90 (Flag if more than 90% DTU usage)

There are a couple other considerations for creating an alert. The first is a time window the alert will check against. This window can be anywhere from five minutes to a full day. The second is what aggregation of the metric we will check (i.e. total, average, maximum). You’ll want to use values and aggregations that make sense, but for the example alert, we’ll check the maximum value over a 5 minute window.

Once we have everything in place, we will call the New-AzureRmAlertRuleEmail cmdlet to create our email notification and Add-AzureRmMetricAlertRule cmdlet to create our alert:

$ResourceGroup = 'IntroAzureSql'
$location = 'West US'
$server = 'msf-sqldb'
$db = 'MSFADMIN'
$rid = (Get-AzureRmResource -ResourceGroupName $ResourceGroup -ResourceName "$server/$db").ResourceID
$email = New-AzureRmAlertRuleEmail -CustomEmails 'mfal@dummy.com' -SendToServiceOwners
Add-AzureRmMetricAlertRule -Name 'DTU90Check' `
-Location $location `
-ResourceGroup $ResourceGroup `
-TargetResourceId $rid `
-MetricName 'dtu_consumption_percent' `
-Operator GreaterThanOrEqual `
-Threshold 90 `
-WindowSize '00:05:00' `
-TimeAggregationOperator Maximum `
-Actions $email

2016-08-17_15-33-45

And now we can look at our shiny new alert using either the portal or Get-AzureRmAlertRule:

2016-08-17_15-37-18

2016-08-17_15-39-54

To get a rid of an alert is easy, too, just call Remove-AzureRmAlertRule:

Remove-AzureRmAlertRule -ResourceGroup $ResourceGroup -Name 'DTU90Check'

2016-08-17_15-42-50

Where To Now?

As you can see managing alerts in PowerShell isn’t all that difficult. The question is “why”? Hopefully you can already see the value, especially if you want to standardize your Azure SQL Database environment. By having all your alerts scripted out, you can apply them in a consistent fashion. This can be part of a larger automation process that helps scale out your environment as necessary. Regardless of how you manage it, using PowerShell can significantly decrease your management time for your Azure environment.

Thanks again to Julie for giving me the inspiration to figure this out!

Azure SQL Databases with #Powershell: Exporting and Importing

Previously we went over how Azure handles regular backups of your Azure SQL Databases. It is such a huge weight off an administrator’s shoulders when we only have to worry about restoring a database if something goes wrong. While many DBAs might struggle with letting go of the need to control and manage their backups, I think most of us will embrace the freedom from the tedium and worry of keeping an eye on this part of our disaster recovery.

There still is a need to take a backup of some sort. Over my career, while I have used backups for protecting my data, I have also used them for other tasks. Sometimes it is to snapshot the data at a point in time, such as before a code release or a major change. Other times a backup can serve as a great template for creating a new application database, especially if you have a federated database model. Whatever your use case, there are times we would need to snapshot a database so we can restore from it.

Exports

One approach would be to just mark down the time you want to use as your backup and restore from there, but this approach could be difficult to control and be tricky to automate. Azure offers us a better option: exporting the Azure SQL database in question to blob storage. We can restore (or, more precisely, import) this export to a new database.

To run an export is a simple call to the Start-AzureSqlDatabaseExport cmdlet. Just like the restore cmdlet, it will start the process in the Azure environment, running in the background while we do other work. To run an export, we need the following information:

  • Azure SQL database to export
  • The administrative login for the server hosting your Azure SQL Database (which we will define as a SQL Server storage context)
  • The storage container information

The only mildly frustrating thing with the export we need to use cmdlets from both the Azure module and the AzureRM module (assuming your storage blob is deployed using the resource manager model). Because of this, make sure you run Add-AzureAccount and Login-AzureRMAccount before you get started.

We first need to create a connection context for our Azure SQL Database instance, using a credential for our admin login and the server name/

$cred = Get-Credential
$sqlctxt = New-AzureSqlDatabaseServerContext -ServerName msfazuresql -Credential $cred

Once we have established our SQL connection context, we will then need to set our storage context using a combination of AzureRM and Azure cmdlets.

$key = (Get-AzureRmStorageAccountKey -ResourceGroupName Test -StorageAccountName msftest).Key1
$stctxt = New-AzureStorageContext -StorageAccountName msftest -StorageAccountKey $key

Now we can then start the export. Notice, we need a name for the export, used in the BlobName parameter.

$exp = Start-AzureSqlDatabaseExport -SqlConnectionContext $sqlctxt -StorageContext $stctxt -StorageContainerName sqlexports -DatabaseName awdb -BlobName awdb_export

Since this only starts the export, we need a way to check on the status. We can check using Get-AzureSqlDatabaseImportExportStatus. Oddly enough, the status cmdlet requires the username and password to be passed separately and does not take a credential object.

Get-AzureSqlDatabaseImportExportStatus -RequestId $exp.RequestGuid -ServerName msfsqldb -Username $cred.UserName -Password $cred.GetNetworkCredential().Password

And then there is our blob.

2016-05-14_11-23-48

There are two gotchas to keep in mind with both the export and the status. The first is you can not overwrite an existing blob, so make sure your blob name is unique (or get rid of the old one). Also, you can not check on the status of an export that has finished. If you get an error, chances are your export has completed.

Imports

Once we have our export, we now have a “backup file” we can create new databases from. All we need to do is run an import of our blob. Just as for our export, we need some information for our import, which we will (unsurprisingly) run with Start-AzureSqlDatabaseImport.

  • The storage container and blob that we will import from
  • A destination server and credentials for the server
  • The name for our database

Now, since we are creating a new Azure SQL Database with the import, the process needs to define a service objective. By default, it will import the database at standard 0 (S0), but you can defined a higher or lower edition if you want. To simplify things, we will go with the defaults and use the contexts from above, so all we really need to do to kick off the import is:

Start-AzureSqlDatabaseImport -SqlConnectionContext $sqlctxt -StorageContext $stctxt -StorageContainerName sqlexports -DatabaseName awdb_imp -BlobName awdb_export

Which, when completed, gives us a new Azure SQL Database created from our export blob:

2016-05-14_11-56-01

The Secret Sauce

What makes this black voodoo magic work? Is this some proprietary technique Microsoft has snuck in on us? Surprisingly, this is a bit of technology that have existed for sometime now as part of SQL Server Data Tools called BACPACs. A BACPAC is essentially a logical backup of a database, storing the schema and data as SQL statements.

This differs from a typical SQL Server backup, which stores your database pages directly in a binary format. Because of this, native backups are smaller and can be made/restored faster. However, they are more rigid, as you can only restore a native backup in specific scenarios. A logical backup, since it is a series of SQL statements, can be more flexible.

I don’t know why Microsoft went with BACPACs over some native format, but because they did,we can also migrate a database from on-premise SQL Server to Azure SQL database. This is a follow up to a common question I get: “How can I copy my database up to Azure SQL Database?” I want to look at this in my next post. Tune in next week, where we will create a BACPAC with regular SQL Server database and migrate it up to Azure SQL Database. 

Azure SQL Databases and #Powershell: Database Restores

logoAzureSqlA database administrator’s first priority is to ensure their datais protected in the event of a disaster. Typical scenarios range from someone forgetting a WHERE clause to completely losing the data center that houses our servers. We are charged with thinking through all the possible scenarios and making sure we can support our company’s Recovery Time and Recovery Point Objectives (RTO and RPO). This sort of responsibility is a lot why being DBA is a tough job.

Owning our servers gives us a lot of control of how our disaster recovery is constructed, but also a lot of responsibility. This can be a real challenge. We already know that cloud services, like Azure SQL Database, take that responsibility away by managing the infrastructure behind the scenes. These services also provide much of the disaster recovery protection, which is much more comprehensive and (in most cases) cost effective. In this post, we will review these options and how they can be managed using Powershell.

A Big investment

The most fundamental form of disaster recovery is database backups and restores. Typically setting up backups is a lot of work. DBAs need to make sure there’s enough storage available for backups, create schedules that accommodate business operations and support RTOs and RPOs, and implement jobs that execute backups according to those schedules. On top of that, there is all the work that has to be done when backups fail and making sure disk capacity is always large enough. There is a huge investment that must be made, but it is a necessary one, as losing a database can spell death for a company.

This is one of the HUGE strengths of Azure SQL Database. Since it a service offering, Microsoft has already built out the backup infrastructure for you. All that stuff we talked about in the previous paragraph? If you use Azure SQL Database, you do not have to do any of it. At all.

What DBAs still need to manage is being able to restore databases if something happens. This is where Powershell comes into play. While we can definitely perform these actions using the portal, it involves a lot of clicking and navigation. I would much rather run a single command to run my restore.

Executing a Restore

Before we get started, we need to first load up the classic Azure module. While our work so far has been done with the AzureRM module, there are many Azure SQL Databases tasks that can only be managed using the Azure module and database restores are one of them. Unfortunately, it also means we have to login to our Azure account a second time, as authentication is managed separately between the modules. Using Add-AzureAccount is just like Login-AzureRMAccount and will bring up a graphical dialog for you to sign in with.

Import-Module Azure
Add-AzureAccount

2016-05-07_12-25-46

To restore a database, first we just need three things to start: the server where the original database came from, the name of the database we want to restore, and a name to restore the database as. When you restore an Azure SQL Database, you have to create a database that does not already exist, so there is no WITH REPLACE option. Starting the restore is then just a matter of calling the right cmdlet:

Start-AzureSqlDatabaseRestore -SourceServerName msfazuresql -SourceDatabaseName msfsqldb -TargetDatabaseName msfsqldb-restore

2016-05-07_12-37-34

This only initiates the database restore operation and we have to wait for it to complete. To check on the status, we can use another Powershell cmdlet to display that information.

Get-AzureSqlDatabaseOperation -ServerName msfazuresql -DatabaseName msfsqldb-restore | Sort-Object LastModifyTime -Descending | Select-Object -First 1

2016-05-07_12-57-24

All the pipeline calls are there to filter out additional operations and only show the most recent, as the cmdlet returns all restore operations that have ever happened for that server and database. It is difficult to predict how long a restore will take, as this is all handled behind the scenes. I have not tested restore times, but my test databases usually take a few minutes to restore.

There are other options we can declare, depending on how we want to manage our restore, including:

  • Point in time within a minute of when we want to recover.
  • An optional destination server that we want to restore the database
  • A dropped database as our source

One other gotcha to be aware of, albeit a minor one, is that you can not perform a restore of an Azure SQL Database until 15 minutes after the creation of the database. This should not be a concern for most, but it can get in the way of rapidly spinning up databases for demo purposes (ask me how I know!).

Other Scenarios

Simple database restores is the entry point for most recovery scenarios. There are other options for disaster recovery as well as managing copies of your database. There are other options within the platform to support these scenarios. In the next post, I want to dive into a couple cmdlets that we can use to copy our Azure SQL Database as well as make a “backup file” to Azure blob storage that can be used to store a copy of our database in a particular state.

Azure SQL Databases with #Powershell: Managing your Databases

I meant to write this post a month ago, but then life happened. This is a continuation of my short series on Azure SQL Database and how you can manage it with Powershell. If you want, you can go back and read the previous two posts: Getting started and creating your database.

Administrative Access

Now I’d like to show you how to connect to the database, but since it’s been so long since I’ve looked at this demo, I’ve forgotten my administrative password! While it would be simple to blow away the Azure SQL server and database, there are many situations where this is not possible. Fortunately we have an easy way to reset the administrative password using the cmdlets.

$pw = ConvertTo-SecureString -AsPlainText -Force '********'
Set-AzureRmSqlServer -ResourceGroupName MSFAzureSQL -ServerName msfazuresql -SqlAdministratorPassword $pw

Walking through this, we just need to create a secure string for our password and then use the Set-AzureRmSqlServer cmdlet and pass the secure string to -SqlAdministratorPassword argument. Easy as that and we don’t even need to know what the previous password was. With this in mind, I also want to call out that you can only change the password and not the admin login name. While this is not such a big deal, be aware that once you have an admin login name, you are stuck with it.

Connecting and Querying

Armed with our reset password, now we can query the database. The easiest way for you to do this is to connect to your database with SQL Server Management studio:2016-04-30_10-20-41

Remember that your server name will be what you named it followed by “.database.windows.net”. You must have SQL Server Authentication selected, and then just enter your login and password. SSMS will then connect up to your Azure SQL Database and the rest is very much like managing your on premises databases.

What about using Powershell? Again, not all that different. Let’s run a basic query to get some information about our database using Invoke-SqlCmd:

$sql = @"
SELECT
DATABASEPROPERTYEX ( 'msfsqldb' , 'Edition' ) as Edition
,DATABASEPROPERTYEX ( 'msfsqldb' , 'ServiceObjective' ) as ServiceTier
,DATABASEPROPERTYEX ( 'msfsqldb' , 'Version' ) as Version
,convert(bigint,DATABASEPROPERTYEX ( 'msfsqldb' , 'MaxSizeInBytes' ))/1024/1024/1024 as MaxSizeGB

GO
"@

Invoke-Sqlcmd -ServerInstance msfazuresql.database.windows.net -Database msfsqldb -Username msf -Password '*********' -Query $sql

2016-04-30_10-39-00

As you can see, working with Azure SQL Database after it has been created and configured is not much different than your typical SQL Server installation. There is a lot of writing out there on features and use of Azure SQL Database, so I will skip that here. The best place to start is Grant Fritchey(@GFritchey) and his Azure blog posts.

Remaining Flexible

The last item I want to cover here is managing your database size. One of the promises of the cloud and Azure is the flexibility to change your resources as your needs demand and not be stuck on whatever hardware you purchased when you first built your data center. There are a couple items you should consider when first creating your Azure SQL Database, but the great thing is that you are not tied into many of those options permanently.

There are several service tiers available to you, with a mix between standard and premium offerings. Obviously, each service tier has a different price point, so how do you know what is the right choice for your database? What complicates this is you have a mystery metric for performance: Database Transaction Units. Ostensibly this is the number of transactions per second your database should be able to provide, but there’s more to it than that. My general recommendation is you should base your choice on features you need and user connections you expect. If you find, after that, you have a performance bottleneck, you can adapt upwards incrementally.

How do you make this change? It is just one line of Powershell:

Set-AzureRmSqlDatabase -ResourceGroupName MSFAzureSQL -ServerName msfazuresql -DatabaseName msfsqldb -RequestedServiceObjectiveName 'S1' 

Then it is off to the races. The actual operation is not instantaneous and Azure needs some time to allocate the resources and adjust the settings on its side. The great news here is this does not interfere with the operation of your database at all and it will be available while the resizing happens. Existing connections will be maintained and even queries that are running will continue to run. The end result will simply be:

2016-05-01_18-18-03

Ease of Use

As you can see, we can manage the access and scale of Azure SQL Database with a few straightforward commands. While these changes can also be managed through the web portal, I find the Powershell approach to not only be simpler, as we can skip all the browsing through blade windows and troublesome clicks. While these actions are not really ones we would automate, using the cmdlets can also ensure that our actions can be more precise and consistent.

In my next post, I want to cover a topic near and dear to the hearts of DBAs everywhere: database restores. While backups for Azure SQL Database are managed for us, we need the ability to restore our data to a point and time of our choosing. Fortunately this is fairly easy to accomplish with Powershell. Stay tuned and I promise you will not have to wait another month for me to demonstrate this.

Azure SQL Databases with #Powershell: Components

As with any project, before we can start, we need to know what we have to work with. Last post, we went over loading the Azure RM module and getting connected to our Azure account. Before I go over how you create the Azure SQL databases, I’d like to talk first about exactly what we will create first. It’s the old adage: measure twice, cut once.

The very first thing to create is an Azure Resource Group, as we are going to create our Azure SQL Database using the Azure Resource Manager deployment model. There’s nothing really fancy about this, just that it is a container that can house multiple resources. This resource group can house any Azure resource, not just our databases. Creating one using Powershell is simple, we just need to know what to call it and which Azure region to create it in:

New-AzureRmResourceGroup -Name MSFAzureSQL -Location ‘West US’

Once we have our resource group, we can start deploying our Azure SQL database components.

Parallels

From a logical standpoint, working with Azure SQL databases is not very different from setting up a SQL Server instance and database in your environment. There are 3 main components we will need to deploy:

  • A server
  • A database
  • A firewall (server or database)

3-20-2016 7-27-15 PM

Starting with the server, there are only a few things we need to know. The first is a server name so we can identify and connect to it. After that, we need to know what version of Azure SQL databases we want. There are two options here, 11 and 12. Yes, those are fairly abstract numbers, though version 12 is currently what is being updated and has many of the features that will be included in SQL 2016. Finally we need to create an administrative login, the equivalent of the sa login found in all SQL Servers. This all boils down to just two lines of Powershell:

$cred = Get-Credential
New-AzureRmSqlServer -ResourceGroupName MSFAzureSQL -Location ‘West US’ -ServerName msfazuresql -SqlAdministratorCredentials $cred -ServerVersion '12.0'

Running this, in my opinion, beats all the clicking and selecting you have to do in the portal.

Next up, we need to create our database. As with creating a server, we need a name for the database. We also need the resource group and server name we are creating the database in. We will then need to declare a service level objective. While there is a performance aspect to this selection, it more rests on what sort of connection load you will have, what kind of features you need, and what sort of disaster recovery support you require. The nice thing about selecting a service level is their easy to change once you have your database. Finally, we need to select an edition (again, relates to features). To deploy, our command looks like this:

New-AzureRmSqlDatabase -ResourceGroupName MSFAzureSQL -ServerName msfazuresql -Edition Standard -DatabaseName msfsqldb -RequestedServiceObjectiveName 'S0'

Security and Access

The final component to create is a firewall. By default, your Azure SQL Database has its access blocked to anything outside of it. Yes, this includes your access via management studio. To open this access, we need to create firewall rules that allow specific IP addresses to connect. There are two types of firewall rules we can use, server and database. These are what you would guess. The server rule allows access from that address to any database in your server, where the database rule only works for that database.

For our purposes, the server rule will work just fine. Creating a database rule is not really different, it just applies to a different scope. To create a rule, we need the resource group name, server name, rule name, and an IP address range. For this, we will use a trick I picked up from John Milner(@JFRMilner) to get our external IP and create a firewall rule for it:

$localIP = (Invoke-WebRequest ifconfig.me/ip).Content.trim()
New-AzureRmSqlServerFirewallRule -ResourceGroupName MSFAzureSQL -ServerName msfazuresql -FirewallRuleName 'msffirewall' -StartIpAddress $localIP -EndIpAddress $localIP

Now, there is also a special firewall rule we can use. This one is handy because it will make your database accessible to all Azure services. This saves you the headache of creating separate rules for websites, PowerBI, or any other service you want to use. Just call the firewall rule cmdlet with -AllowAllAzureIPs:

New-AzureRmSqlServerFirewallRule -ResourceGroupName MSFAzureSQL -ServerName msfazuresql -AllowAllAzureIPs

Fully Functional

With that, we now have a functioning Azure SQL database. To connect to it, you just need to use the servername along with database.windows.net as your connection string, along with the admin login you created. Put it into management studio and you are good to go:

3-21-2016 5-04-05 PM

Oh, you want the full script?

New-AzureRmResourceGroup -Name MSFAzureSQL -Location ‘West US’

$cred = Get-Credential
New-AzureRmSqlServer -ResourceGroupName MSFAzureSQL -Location ‘West US’ -ServerName msfazuresql -SqlAdministratorCredentials $cred -ServerVersion '12.0'

New-AzureRmSqlDatabase -ResourceGroupName MSFAzureSQL -ServerName msfazuresql -Edition Standard -DatabaseName msfsqldb -RequestedServiceObjectiveName 'S0'

$localIP = (Invoke-WebRequest ifconfig.me/ip).Content.trim()
New-AzureRmSqlServerFirewallRule -ResourceGroupName MSFAzureSQL -ServerName msfazuresql -FirewallRuleName 'msffirewall' -StartIpAddress $localIP -EndIpAddress $localIP
New-AzureRmSqlServerFirewallRule -ResourceGroupName MSFAzureSQL -ServerName msfazuresql -AllowAllAzureIPs

As you can see, deploying an Azure SQL database is a pretty simple process, it is just a matter of knowing what you need.  Next up, we will go over how you can manage and alter your Azure SQL Database. After all, the cloud promises us flexibility and using Powershell makes it easy.

Azure SQL Databases with #Powershell: Getting Started

I’m now a month into my new gig at UpSearch and loving it. The most interesting thing about moving from being a corporate world to being a consultant is the variety of work that I get to do. As tech folks, part of the reason we work in this field is because of all the things we get to learn.

One new area for me has been Azure SQL Databases. Sure, I’ve known about them for a while now, but it was never something that was needed by my corporate masters. Now, with several different clients to work for, it did not take long for Azure’s platform-as-a-service database offering to be a good fit for a project. Since this is me we are talking about here, I made a point of learning how to create and manage these databases in Powershell. I’d like to share some of those lessons here.

The Tools

Before we get started, we need the right cmdlets for working with Azure. With Windows Management Framework 5.0, it is actually really easy to install and get started with Azure and Powershell. The full process is documented up on Microsoft’s site. It takes about 30 minutes to get everything installed and configured.

If you have worked with the Azure cmdlets before, you might be surprised to see that the instructions references two different modules, Azure and AzureRM. Actually, if you do a lookup on any modules you will see a lot more:

Get-Module -ListAvailable  Azure* | Select-Object ModuleType,Version,Name

3-12-2016 11-48-54 AM

What is this all about? It took me a bit of digging, but what it boils down to is that Microsoft made a fundamental change to how things are managed within Azure. You will now find documentation on these two different deployment models: Classic Deployments and Resource Manager Deployments. These two different set of Powershell cmdlets reflect these different models, as anything for Classic Deployments are handled by cmdlets in the Azure and Azure.Storage modules. All the Resource Manager Deployment stuff is handled by the AzureRM* modules.

Choose Your Path

The reason I call this out is to address one major hangup with working with Azure: the documentation. Because Azure is changed so quickly, official documentation and other write ups (like this one) quickly go out of date. When I was looking for information on how to do this, I kept finding blog after blog that covered Classic Deployments. Since I am just starting with this, there is no reason for me to go down the Classic Deployments path. While these are still viable and perfectly valid, it makes more sense to me to focus on using Azure the way Microsoft intends it to be used.

To get started, we will first connect to our Azure RM account. I will be using my MSDN account. From my Powershell window, I’ll run the following commands:

Import-Module AzureRM
Login-AzureRMAccount

I am then greeted by a GUI login window, where I enter my credentials.

3-12-2016 12-35-48 PM

Hey, wait? A GUI? With Powershell? Yeah, this surprised me as well, because to me GUIs are an aberration when it comes to Powershell. You can not really automate them and that throws a wrench into the whole works. This is probably my biggest gripe about using these cmdlets. The nice thing is that once you login your session is maintained and you can make use of a session profile.

Thanks out to Adam Bertram(@adbertram) for helping me with this, but you have the ability to save off a profile file that will save a lot of this info for use by other Powershell sessions. This gives us the ability to connect and run commands without the need for this GUI nonsense. Accomplishing this is just a matter of using the right cmdlet:

Save-AzureRmProfile -Path C:\Users\Mike\mfalprofile.apf
Select-AzureRmProfile -Path C:\Users\Mike\mfalprofile.apf 

Note, the name of the file is pretty meaningless. I used the apf extension because of my own personal taste, but you can do whatever you want. The point is to have the file. Once you have this file, you can load it up in future/different Powershell sessions and avoid the login GUI completely. The real hurdle here is that, regardless of how you go about this, you need to login graphically at least once to get credentials.

First Steps

With our connection established, we can now start creating SQL databases. Before we do, however, we need to talk more about what we are going to create. While the beauty of working with public cloud providers such as Azure is it is really easy to spin up resources, it does not negate the importance of understanding how and what we are deploying to. It is just as important to “measure twice, cut once” when it comes to the cloud. My next post will focus on the different components that make up Azure SQL databases, deployment considerations, and how we end up creating SQL databases in the cloud.