Category Archives: Installation

SQL Hangout with Boris Hristov

I spoke at SQL Saturday in Slovenia just before Christmas and had a great time.  Whilst I was there I made a new SQL friend – Boris Hristov ( t | b ) a SQL Server expert from Bulgaria. He attended my session on SQL Server consolidation and joined in to an interesting discussion towards the end of the session.

After Christmas, Boris got in touch with me and asked if I would like to be a guest on his regular SQL Server Hangout “show” and talk with him about consolidating SQL Servers. I liked the sound of that, so we met up on G+ and had a pretty informal/unplanned chat about our experiences with consolidation and what we think needs special attention.

Boris said he will be posting more details about the Hangout on his blog, but in the mean time, here is the recording:

http://www.youtube.com/watch?v=LFbnOXPKLYI

Please make sure to check out the other Hangouts in the series, as Boris has managed to record some really interesting topics with experts from all over the world.

Windows Failover Cluster Patching using PowerShell

PowerShell is the new standard scripting and automation language for Microsoft products. In this post I will be describing how I used two new Powershell commands to help semi-automate the installation of Windows Updates to a Windows 2012 Failover Cluster.

I run a number of clusters on both Windows Server 2008 and Windows Server 2008R2 and I must say that although Windows Clustering was a new thing to me about 5 years ago, my overall experience has been pretty painless during that time. From what I hear/read Server 2008 was the first outing of a much updated Windows Failover Clustering (including PowerShell integration), so I managed to body-swerve the more painful versions before it.

Be that as it may, Windows Server 2008 was always lacking in terms of scripting/automation and didn’t get much better with Windows Server 2008R2. The previous incarnation of the automation tooling was “cluster.exe” which was a command line tool that allowed for quite detailed control of a cluster. This was a tool that has “grown” over time to cover many areas of Windows Clustering, but was not designed to be as flexible or programmable as PowerShell. The PowerShell command-set for Clustering covered just about everything that “cluster.exe” covered and then some more – a good overview can be found at the Windows File Server Team Blog.

As Windows has evolved, the PowerShell offerings for each feature within Windows has also evolved. I recall hearing that the with Windows Server 2012 you can control the entire O/S from within PowerShell and that the GUI basically constructs and runs these PowerShell commands in the background (my brain may be playing tricks on me though).

With this in mind, I am preparing a server migration/replacement which will move a cluster from Windows Server 2008 to Windows Server 2012.  I took another look at the PowerShell commands available to me on Server 2012 and was rather pleased to see two improvements in the Failover Clustering command-set. The major command addition that I discovered, and had to write about here, was the “Suspend-ClusterNode” command and its brother “Resume-ClusterNode”.

Suspend-ClusterNode allows you to temporarily “pause” a cluster node. This basically removes any currently running resources/services from the specified node and the cluster will not assign any workload to that node until it has been re-instated as an active cluster participant. Resume-ClusterNode brings a previously suspended node back online within the cluster.

You may ask “Why would you want to do this?” or be thinking “This was possible with Windows Server 2008R2”; well dear reader, let’s take a look at those two points.

“Why would you want to do this?”

The short answer: Server maintenance with minimal downtime.

The slightly longer answer: Imagine you have a 3 node cluster. It is patchday and you want to cleanly patch the nodes, one after the other, with a minimum of downtime. You can afford for one node of the three to be offline for maintenance at any one time. This means that you can suspend node 3 while nodes 1 and 2 remain online. The cluster then knows that node 3 is down, but it is not down not due to a failure (so will not start panicking). You can then patch the node, reboot as necessary (who am I kidding, this is a windows server, you’ll have to reboot it!) and then the node is ready to re-join the cluster as an active/available member. You are then able repeat this process on the remaining nodes to install updates across the whole cluster.

“This was possible with Windows Server 2008R2”

The short answer: Yes it was, but required some manual intervention.

The long answer: Windows Server 2008R2 offered the ability to suspend a cluster node, but without the ability to control how the cluster dealt with any resources on the node being suspended. This is where the real magic in the new PowerShell command “Suspend-ClusterNode” comes into play.

Let’s take a quick look at the syntax of both so we can compare:

Suspend-ClusterNode

Old Syntax

Suspend-ClusterNode [[-Name] ] [[-Cluster] ]

New Syntax

Suspend-ClusterNode [[-Name] ] [[-Cluster] ] [[-TargetNode] ] [-Drain] [-ForceDrain] [-Wait] [-Confirm] [-WhatIf]

As we can see, the new syntax offers quite a few extra parameters over the old; the main ones to note are [TargetNode] and [Drain]. [TargetNode] allows us to specify where any resources should be moved to and [Drain] initiates the move of the resources/services. This allows for a much finer control over resources within a cluster during maintenance operations. With the new command it is really easy to perform the 3 node cluster maintenance outlined earlier. We suspend one node after the other, moving any resources they should have to another node of our choosing and can then resume the node after the maintenance has completed. If we now take a look at Resum-ClusterNode, we will see another level of brilliance that becomes available to us that further eases node maintenance work:

Resume-ClusterNode

Let’s compare old vs. new:

Old Syntax

Resume-ClusterNode [[-Name] ] [-Cluster ]

New Syntax

Resume-ClusterNode [[-Name] ] [-Cluster ] [[-Failback] {Immediate | NoFailback | Policy}]

Again, we can see that there is more to decide upon with the new syntax. When you resume a suspended node, you can decide what happens to the resources that were previously running on that node.

The parameter [Failback] has three options:

  • “Immediate” is pretty obvious and will immediately take control of the resources that were previously running on that node before suspension.
  • “NoFailback” will resume the node but leave all resources where they currently are – this is a good idea if you have already failed over to an updated node and don’t want another service outage in this maintenance period.
  • Finally, “Policy” would postpone any failback until a pre-defined failover timeframe is reached.

Once we see how Suspend-ClusterNode and Resume-ClusterNode have been extended, we can understand how the extensions open up better scripting control of clusters and their resources.  I have prepared a script that can be run in our maintenance window that will suspend a node and push all resources to another node.  The suspended node can then receive all Windows Updates and be rebooted if necessary and finally a script can be run to bring the node back online.  The same set of scripts are then run against the second node, reversing the flow back to the node we have just patched.  Using Suspend-ClusterNode and Resume-ClusterNode only reduces the overall code in the PowerShell scripts by a few more lines, but the complexity is reduced drastically.  This makes code maintenance easier and the code is just easier on the eye.

After seeing the improvement to my local code, I am certain that the extension to the PowerShell command-set was internally driven at Microsoft. Imagine how many clusters they have running that needed this extension to improve their automation. As far as I can see it, this is an obvious knock-on effect of their drive to “The Cloud” and a very positive one at that!

As always, think before running any scripts and please don’t blindly setup scripts to suspend and resume nodes in a cluster. This is an aid to your daily work and not a robot you should build to run any/all updates onto a cluster 🙂

Have fun!

SQL Server Connection Encryption and .Net Framework 4.5

SQL Server has offered the ability to encrypt network connections for quite a while. This can be achieved using the self-generated certificate that SQL Server generates when installed, or using a certificate generated outside SQL Server (Your own Enterprise Certificate Authority in your Active Directory or a third-party Certificate Authority). These certificates, once registered to the server, can be loaded inside SQL Server and used to open up SSL encrypted connections between clients and the server.   In general you would use encryption as an additional layer of security when opening up a database server to the world (e.g. an internet facing DB-Server). However, there are cases when the network connection between a client and the server are required to be encrypted inside the confines of a corporate network.   SQL Server will check for the availability of a server certificate at start-up. If it finds a certificate issued by a certificate authority it will load and use it for encryption purposes. If, however, no certificate can be found, then a self-generated certificate will be used.   You can find out if you are using a self-generated certificate or a third-party certificate by reading the SQL Server error log and looking through the start-up messages at the beginning of the current log-file. A message about the encryption certificate is written after the initial information about start-up parameters and initialisation of the system DBs: A self-generated certificate generates the following log entry:

.

A third-party generated certificate on the other hand:

My Issue: I stumbled across an edge-case (for me at least) when working with encrypted connections to SQL Server using the .Net Framework v. 4.5 recently that I wrestled with for a few days before coming to a conclusion and thought it at least noteworthy.   As of v. 4.5 of the .Net Framework the authentication of a certificate is checked more thoroughly than in previous versions. As stated in the MSDN Library (http://msdn.microsoft.com/en-us/library/ex6y04yf.aspx), when the connection property “Encrypt” is set to true and the property “TrustServerCertificate” is set to false (default value), the server name (or Virtual Name for Clustered Instances) in the connection string must *exactly* match the common name (CN) specified in the certificate. If this is not the case, then the connection attempt will fail:   As you can see above, the principal name did not match so no connection could be established. This applies to the following scenarios:

  1. Not using fully qualified domain names in the connection string (FQDN):
    • You have a company domain (mydomain.com) You have a SQL Server (server1), FQDN = server1.mydomain.com
    • You have an SSL certificate generated for the FQDN (server1.mydomain.com) which is registered and loaded by SQL Server at startup
    • Your DB connections only use the server name, not the FQDN: “Server=server1; Initial Catalog=Northwind; Integrated Security=true; Encrypt=True”

In previous versions of the .Net Framework the connection would be opened and the servername “server1” would still be accepted when compared to the servername on the certificate, if the name was the same and the domain was not included, then the connection would still be accepted. As of v. 4.5 this certificate verification is basically doing what it should – identifying the name provided against the name in the certificate as strictly as expected.

  1. Using a DNS Alias to identify and connect to the SQL Server:
    • You have a company domain (mydomain.com) You have a SQL Server (server1), FQDN = server1.mydomain.com
    • You have an SSL certificate generated for the FQDN (server1. mydomain.com) which is registered and loaded by SQL Server at startup
    • You have a DNS Alias (mssql) created as an abstraction layer to your physical SQL Server (allows for easier infrastructure changes).
    • Your DB connections use the Alias name, not the Servername or FQDN: “Server=mssql; Initial Catalog=Northwind; Integrated Security=true; Encrypt=True”

As in scenario 1, previous .Net Framework versions were more than happy to honour the alias redirect to the actual server and accept encrypted connections   The same error occurs as in scenario 1 when trying to connect using .Net Framework 4.5, the reasons are the same – the alias name is not registered in the standard certificate so cannot be strictly identified with the name in the certificate. The on-going theme here is that name resolution between the connection string and the server name in the certificate has become much stricter and also made alias usage (scenario 2) impossible with a standard certificate requested through the certificate authority.   The solution is simple enough – either change your software to use the fully qualified name of the server (as is present in the certificate itself), or request a certificate that incorporates all possible “versions” of the server name and/or alias that you wish to use. These additional names are stored as part of the certificate in a portion of the certificate called the “Subject Alternative Name”.   As I want to continue to use aliases to keep our infrastructure “invisible” to any deployed applications, I went for door number two and created a certificate request that incorporates all name variations and aliases for the system that I plan on using. Solution:   Here are the steps for creating a certificate request within your domain that will fulfill the requirements by using the Subject Alternative Name field to store and use the alias when connecting to the SQL Server using SSL encryption (done on the server you are wanting to set up SSL on). This is an example using a certificate authority within a domain, the workflow may be different when using an external authority (I have never used them for this work before).   I have included a video showing how to create the custom certificate request using the Microsoft Management Console (mmc) and the certificate snap-in. The steps in the video should be simple to follow and will create an offline certificate request .txt file that will contain the certificate request information required at the certificate authority.   There are a few things to note from the video:

  • The friendly name and description are purely for reference.
  • The information entered is there to make your life easier when trying to identify all the different certificates that are installed on the machine.
  • The tab “Subject” is where the actual server information for the certificate will be entered.
  • The “Subject Name” needs to be the Fully Qualified Domain Name (FQDN) of the server you are creating this certificate for entered as the Common Name (CNAME) type.
  • The “Alternative Name” section is where all the alias names will be entered that you want to use to access the server. In this example I have entered a fully qualified alias name and also the short version (I am not sure if this is really necessary, I’m just hedging my bets with regard to the .Net 4.5 issues I was experiencing).

Notice that you can enter as many different aliases here as you feel like – This is the magic to allow you to offer up different aliases to your “customers” so they don’t necessarily know that they are all on one server. You could even use a wildcard here if you are happy for the certificate to be used by *any* server – I’m not too keen on this as *any* server can use the certificate then, so you lose a little control over the certificate usage (maybe this isn’t really an issue, but I prefer to assign single certificates instead). Once you have the request in a text file you then connect to the certificate authority site and send the contents of the text file to the authority system. The response is a certificate that is valid within your domain for the server named in the subject and also for the alternatives listed in the alternatives list. This process is listed below:   The text file created in the video should look something like this: First of all you need to call up the certificate authority site (ask your domain administrator team for the address). You will be confronted with a rather snazzy Microsoft website for Active Directory Certificate Services:

Your choice will be to request a certificate. The next page should offer two possibilities:

  1. “Create and submit a certificate request”
  2. “Submit a certificate request using a bas-64-encoded CMC or PKCS #10 file, or submit a renewal request by using a base64-encoded PKCS #7 file.”

Quite a mouthful (and written verbatim here for Google searches) but we want to take the second option as we have already created the certificate request and only want to submit it. You now open your previously created text file and copy the contents into the corresponding text box on the webpage:

There is a good possibility that your domain/certificate admins have created a template to include all other necessary settings (e.g. certificate expiry). These templates are offered up in the “Certificate Template” dropdown. Once this information has been entered you can submit the request. Assuming that the information is created correctly, your request will be successfully processed and a certificate will be available for download.   The certificate can be downloaded to the SQL server and “installed”. This is done by importing the certificate into the local computer certificate store (the MMC snap-in loaded at the beginning of this blog post). To import you need to navigate to personal certificate store, right-click and choose “Import”. A new wizard will prompt for the certificate file and then import this into the local certificate store. You will now have a new entry in the personal certificate folder with the friendly name used in the certificate request. As stated at the beginning of this blog post: At start-up, SQL Server will automatically look in the local machine certificate store to see if a valid SSL certificate is available. The certificate will be automatically loaded if the certificate subject exactly matches the server name (or the virtual name for a clustered server). You will be able to see this in the SQL Server log in the start-up messages:

You now have a SQL Server that can accept SSL encrypted connections, even if you use an alias. You can also then decide whether you want to force encryption of all SQL Server traffic, which is done through the SQL Server Configuration Manager under “SQL Server Network Configuration” and the properties of the Protocols:

This is also where you can see which certificate SQL Server will use for SSL encryption:

So that was how I solved my problem of wanting to use SSL with server aliases and .Net 4.5 on a SQL Server. I hope it could help you if you ever need to do the same in the future.

As always, comments/suggestions/improvements/corrections are more than welcome, just get in touch!

SQLRoadtrip, part the 6th

My #sqlroadtrip is now over (part 1, part 2, part 3, part 4 and part 5). I have had 2 1/2 weeks of offline time, where I ignored technology as far as I could and spent some much needed time with my family. During that time I had two more flights and a hotel stay, although a much more relaxed version (a week in Sardinia and not a server in sight!).

@Fatherjack (Web|twitter) requested a DBA-typical numbers breakdown of my trip, I have attempted to oblige him here:

38 days
20 flights
15 countries
11 servers
11 hotels
5 continents
4 power cuts
1 case of food poisoning

In that time, I racked up 37229 miles (according to www.gcmap.com) and reached the “Frequent Flyer” status with Lufthansa. I had a total flight time of approximately 80 hours (according to Lufthansa’s site), with the shortest flight being 150 miles and the longest 6526 miles.

I am happy to say, the rollout went well. The systems have been up an running since, and even survived a 5 day outage in one of the offices. The office came back online and synched up within a few hours, I was most surprised at that!

I am now easing myself back into my daily work, with an eye on my next trip to the PASS Summit in October – I hope to put my new Lufthansa status to use, maybe even upgrade this time round!

P.S. A final picture update is now online, with a few sites from Rio. Really nice city, if it wasn’t for all the smog!!

Be prepared!

While I was on my recent trans-global sojourn I came to the conclusion that the boy scouts really had a valid point with their motto “Be Prepared”. Of course, I am not meaning it quite in the semi-military sense that Robert Baden-Powell meant, but being prepared before the large system roll-out really saved a lot of time, nerves and money.

The roll-out was a set of reporting servers in a transactional replication setup, pushing operational data from a central server to the outlying branch offices around the world. I spent a great deal of time preparing these systems for the roll-out; standardising the installations, scripting out each step required to set up users, roles, permissions, DB objects, linked servers, jobs etc. This long preparation period was very tedious and I would often lose the motivation to keep at it.

The final payoff for this long drawn-out process has been the actual roll-out at each location. I have been able to arrive at each office, totally jet-lagged and tired from the entire travel regime and basically sit and watch the scripts run through and see the systems come to life. Had I not been prepared, I would have been in a world of pain, trying to remember what needed doing and when whilst fighting off sleep and headaches.

As a side note: A former colleague/mentor told me to save every script I ever run. If you don’t, you’ll need to repeat that script that took you an hour to write the very next day after you deleted it. This task has been made much easier to do thanks to the SSMS Toolspack provided by the more than awesome Mladen Prajdić. His SSMS extension has saved me hours of time when I have written code and accidentally thrown it away, or when SSMS has crashed. Use his tool and donate to the cause!

So, before you start any project, always keep the Scouts motto in mind – “Be prepared”. Hard disk space is so cheap as to be free, how cheap is your time?

The Home Stretch

I am now on the final third of my #sqlroadtrip (part 1, part 2, part 3 and part 4).

My last post saw me in New Zealand where I spent a great 36 hours taking in what I could of Auckland. Unfortunately I didn’t meet up with Dave Dustin, his work got in the way at short notice 🙁 – this didn’t stop me from checking out quite a bit of Auckland, including the SkyTower. I got a great view of the city and then walked around town taking in the sights and sounds.

The evening flight from Auckland to Los Angeles ended up being really comfortable. Air New Zealand’s Premium Economy is a major step up from standard economy, offering miles of legroom and the business class menu and winelist (for a snip at $150 on top). I didn’t sleep, but was definitely comfortable for the 12 hour flight. This was followed by a 90 minute layover at LAX, where I was given priority entry into the US so that I could get to my connecting flight on time.

Calgary offered up another uneventful installation, allowing me and a colleague to check out the Banff National Park. It has some great views, with some of the mountain ranges looking like the mountains from the start of my trip in Switzerland (decide for yourself by checking out my web-album – link at the bottom of the page).

As luck would have it, when I checked into my flight from Calgary to Houston, I was offered an upgrade to Continental First Class for the wallet-friendly price of $100. I took them up on the offer and had a very comfortable 4 hour flight down to Houston. I was picked up from the airport by my data-courier colleague who had landed in Houston a couple of hours before me. He had managed to get a nice rental car upgrade because the rental company had messed up the reservation and only had a big SUV left! 🙂 We took off and setup the server in Houston and were treated to a relaxing evening at the local IT manager’s house. We were able to cool off in his pool and enjoyed a few ice-cold beers. This was the first real chance of relaxation since starting the trip all those weeks ago – thank you Volker! After missing the chance to meet Dave Dustin in New Zealand, I got another opportunity to meet an online acquaintance from the ASKSSC forumOleg Netchaev – for lunch in Houston. It was great to put a name to a face.

I then flew down to Veracruz for the next server installation. This was a little touch-and-go, because the hardware had been stuck in customs for a few weeks (it had been sent before I started my trip!). This was sorted out whilst I was in Houston and the server was brought online just in time to get things setup before my flight down to Mexico.

I arrived and finished setting up the server in Veracruz, no problems, everything ran like clockwork. The local IT manager then took us for a quick drive round the city, showing us a few sights and letting us get an idea of what the “real” Mexico is like. We ended up eating some mexican food (although I skipped the pigs brain in tortillas!) and heading down to the Aquarium. Whilst there, we were offered the opportunity to get into a glass box and feed the sharks in the big shark tank. We jumped at the opportunity and spent a fascinating 15 minutes in the water with an oceanologist who speacialises in sharks. That was definitely an experience I will never forget.

From Veracruz I set off to Rio de Janeiro (via Houston – where I am right now) to setup the 11th and final server. This will be a short hop, arriving Thursday and leaving on Sunday, but at this point I really don’t mind. The trip has been really interesting, but extremely challenging, both physically and mentally, and I’m looking forward to going home.

Luckily, I will be arriving in Germany next Monday and taking a 3 week break from work (and possibly technology). I should be able to recharge my batteries and get back to some sort of normal daily routine. My son has been asking if I’m in the planes that fly over our house since the first week of my trip. He seems to be looking forward to me coming home, I was worried he wasn’t bothered or had even forgotten about me 🙂

So, time to sit back and relax before the second longest flight of this trip. The President’s Club is comfortable and quiet and I’ll be fine for the 4.5 hours I have till boarding.

TTFN!

P.S. I have recently updated my web-album with the pictures taken since my last blog-post, with views from Calgary, Houston and Veracruz.

Sqlroadtrip – Half Way Point

I am now at the half-way stage of my round the world trip (see part 1, part 2 and part 3).

After part 3 I went on from Kuala Lumpur to Perth. This time I flew with Singapore Airlines – an extremely good airline as far as I am concerned, the entire experience was positive. I had a short hop from Kuala Lumpur to Sinpagore and then a 5 hours flight to Perth.

There was a mix-up at my company regarding the data transport, so I arrived a full 24 hours too early in Perth. I finally caught up on some of the missing sleep – I slept a full 18 hours!

The data finally arrived and I got to work at the Perth office. The installation ran without any issues (we’ve got the rollout totally sorted by now) and even managed to do a little more testing of the performance improvements of running readonly reporting on the local server. Who would have thought it, a server in the LAN is faster than a server over WAN!!! 😀

I only saw the hotel (Esplanade in Fremantle), the office and a micro-brewery pub (The Mad Monk in Fremantle). This was the shortest stop of all so far, but I was more than happy to take that hit to allow me to have some “real” free time in New Zealand.

I arrived in Auckland this morning after an uneventful flight from Perth to Auckland. I spent the day exploring the city and adding to the photo collection. I will hopefully be meeting up with Dave Dustin (Twitter) for lunch tomorrow before jumping on my longest flight so far – Auckland to Los Angeles to Calgary. The first leg is over 10000 km and 12 hours, luckily in Premium Economy flying with Air New Zealand – a little extra legroom is always nice.

So, time for bed, the day starts early with me trying to take part in the PoSh webcast held by Aaron Nelson (Web|twitter) for the UK Sout West User Group. That’ll mean a 5am start, but my brain is scattered from timezone-hopping anyway.