Error Messages 8114 or 22122 When Performing Change Tracking Cleanup

 

You’re a data professional and you’re trying to keep up with patching a wide range of SQL Server versions and editions. How do you know what’s in the CU and whether you should apply it or not? My favorite way to read up on CUs is to go to SQLServerUpdates and click around there.  It will take you to the latest CU pages as well as give you a way to see previous CUs that are available.

While doing this recently, I discovered this for CU 26 on SQL Server 2017.

Known Issues SQL Server 2017 CU 26

How to check If Change Tracking Is Enabled

SQL Server 2017 CU 26 has a known issue with change tracking. Before you apply that CU, be sure to check that you’re not using Change Tracking somewhere. Well, how would you do that if you have a larger environment and don’t immediately know if you’re using Change Tracking somewhere?

You could connect to your Central Management Server in SQL Server Management Studio and then run the below to return the SQL Server instance and database where the Change Tracking Feature is enabled. In this case, we don’t care what tables are involved in the Change Tracking, so there is no need to look at sys.change_tracking_tables. We only need to know that Change Tracking is enabled. This is enough to know that we shouldn’t update to SQL Server 2017 CU26 for that SQL instance.

SELECT db_name(database_id) AS DBname
FROM sys.change_tracking_databases;

You might see this:

Msg 8114, Level 16, State 1, Procedure sp_add_ct_history, Line LineNumber

Error converting data type numeric to int.

Or this error:

“DateTime spidNum     Error: 22122, Severity: 16, State: 1.

DateTime spidNum     Change Tracking autocleanup failed on side table of “table_name”. If the failure persists, use sp_flush_CT_internal_table_on_demand to clean up expired records from its side table.

The solution is to either stay on CU25 or lower, or go up to CU 27 for SQL Server 2017. Microsoft specifically says on this page that CU27 has a fix. You could also uninstall CU26 for SQL Server 2017. You might need to do this if you’re already on this version and can’t update to CU27 for some reason. To do this you can go to Control Panel > Programs and Features > View Installed updates

If you’re on SQL Server 2016 SP3 and experiencing this problem, there is a hotfix here listed as KB5006943.

 

 

Thoughts On Pass Data Community Summit 2021

 

Pass Data Community Summit was November 10-12 and was hosted and sponsored by RedGate along with other supporting sponsors like Microsoft and smaller companies like MinionWare. Having been to last year’s virtual Summit, I think this one was done much better.

Pre-conference Sessions

There were a big collection of pre-conference sessions. I attended “AWS for SQL Server Professionals” because my current employer is entirely in AWS. Erin Stellato had an all day session on Query Store that I attended as well. For a shorter, one hour version of this presentation, click here to go to a YouTube presentation by Erin at the Saint Charles SQL Server Meetup. I learned a lot in both all day sessions and I plan on attending pre-conference sessions next time as well.

Thoughts on Pass Data Community Summit
Thoughts on Pass Data Community Summit

Things I liked about Pass Data Community Summit:

  • The main three day Summit was free.
  • Pre-conference sessions on Monday and Tuesday were accessible due to the low price for the sessions. Sessions were $200.00. I like that the speakers were getting paid for their sessions. This might always be the case. I don’t know because this is the first time I had paid for and scheduled any pre-conference sessions. Some speakers donated the money to a charitable organization.
  • Spatial chat app: This app was intriguing because it would drop you into a virtual room that had the appearance of an actual room with tables, chairs, a water cooler, etc. For people reading this who have played the Sims, the virtual lobby was very Sim-like. The application displayed a circular icon with the attendees face in it and video/audio could be turned on and off. Much like a scenario in a real building, the further your circle was from other people the lower the volume. If you couldn’t hear someone well, or just saw someone across the room that you wanted to chat with, you could move across the room by dragging your circle over next to someone.  There were a number of rooms available in addition to the lobby.
  • The session catalog: The catalog had a number of ways to find content. You could filter by day, day/time, type of session and even search for content by a speaker’s name.
  • Brent Ozar’s keynote on Friday morning. He gave a great presentation on 5 ways that the cloud impacts your career as a data professional.

Things that could have been better about Pass Data Community Summit:

  • There was widespread confusion about the 30 minute times on the schedule Wed-Fri and the need to watch pre-recorded content. As a solution, perhaps when someone is building their schedule, have a pop up on the session catalog screen that clearly tells attendees that they need to watch the pre-recorded video before the Q&A session they are scheduling.
  • Pre-recorded content that had to be watched ahead of time meant I had to spend extra time watching presentations prior to Q&A sessions. My schedule didn’t allow for a lot of that and so I opted for mostly live sessions. This also necessitated a lot of schedule changes in the My Schedule page.
  • In the Q&A sessions I did attend, which I believe I made it to 3 or 4, the attendees didn’t come with questions in hand so getting a conversation going was often slow to achieve. Maybe next year there could be a mechanism provided that would allow questions to be submitted ahead of time. These questions would go to presenters prior to their Q&A sessions.

 

Next Steps to Take

  1. Watch the RedGate website for additional announcements about the next Summit. They already announced that next year’s Summit will be a hybrid event.
  2. Start presenting. You don’t have to be an expert to explain how you solved a recent problem. You just describe your problem, solutions you tried and what ended up working. Someone has either had that problem and couldn’t solve it, or they will like your solution better, or someone may offer an enhancement to your solution to make it even better. Who knows, maybe we’ll see you presenting at a Pass Data Community Summit in the future!

 

 

6 Steps to Build a Training Plan

 

As an athlete there is a certain amount of preparation that goes into being selected to be on a team, competing for a starting role and then being competitive with the opponent. For tennis players to be competitive and go to the finals in Wimbledon, it takes hours and hours of training, practice, and discipline to eat well before ever stepping on the court in a tournament. To make a basketball team, takes hours of practice by yourself on a basketball court and competing with others and your team mates. To make the starting line up takes hard work, outside of game day. A certain hunger and desire to grow and be more than you currently are is important as an athlete.


Photo by Markus Spiske from Pexels

In his retirement speech in 1993, Michael Jordan had this to say, “When I lose the sense of motivation and the sense to prove something as a basketball player, it’s time for me to move away from the game of basketball…. I went through all the stages of getting myself prepared for the next year, and the desire wasn’t there.”

Tom Brady is arguably the best quarterback of all time. He has accomplished more than basically every other quarterback, but he’s still hungry for more accomplishments and he still wants to do the necessary work that makes it possible to compete at his best.

If a successful athlete stops training outside of game day, their game eventually begins to slip. You can’t perform at a high level on the tennis court, basketball court, football field, ice rink, etc. , if you aren’t putting in the preparation time outside of game day.

So, what does this has to do with you as an IT worker? You can no more perform well on “game day” at your job without adequate and continuous preparation, than great athletes can play well if they stop doing the hard work of game prep.

Take a Personal Assessment

Let me ask you some  questions.

  1. Are you still hungry to learn technology?
  2. Are you putting in the work necessary in order to be the best technologist you can be ?
  3. When was the last time you spent time during the week to learn something new?
  4. Are you regularly talking about how you’re not good at certain things, but then also not putting in effort to improve?
  5. Do tasks regularly take you extra time at work compared to others in your same role?

If your answers to the above questions indicate that you might not be putting in the work necessary to stay current in your area of technology, then maybe you need to ask yourself why. Maybe, you’re coasting and you have been for awhile. Why? Maybe you’ve had a very rough time since the beginning of 2020 because of the pandemic. Ok. I completely get it. What are some things getting in the way of building a training plan that you can execute?

Burn Out will stop “Game Day Prep”

Burnout Prevents You From Building a Training Plan
Burnout – pexels-pixabay-42230

Maybe you’re burned out and need to recharge. That is very common. The last 18 months or so have been especially hard on most people. I usually see at least 5-7 tweets a week about this topic. I have a post about this as related to myself. Burnout is real and it’s hard to go through and hard to recover from. It takes time to deal with burnout and may likely involve finding others who will support you. Also, realize that no one should expect you to be 100% on your game every day, all the time. That’s just not reality.

Athletes have a lot of people around them to help make them successful. They have coaches, trainers, medical personnel , executives, family members and teammates to help keep them going. There is no reason to think that you also don’t need a support network. Possibly find yourself a therapist to talk to and find people to encourage you and help you.  Try switching up your routine so you can take care of yourself better physically and mentally. Right the ship, so to speak. Then, get back to prepping for “game day.”

 

Long Term Coasting Will Stop “Game Day Prep”

Perhaps you’ve recently been in a period of extra after hours learning because an important project required new skills. Maybe you were gearing up to pass a certification. Good for you. After that you took a break. Ok. That’s reasonable. Be cautious though about your “little break” from game day prep turning into long term coasting where you don’t put in additional effort to learn and grow. If 3, 6, 9 months or more goes by and you’re still coasting, ask your self why and what, you might want to do about that.

 

Other Priorities Will Stop “Game Day Prep”

Everyone has priorities, whether we are aware of them or not. We all make choices and those choices prioritize our time. If you’re spending endless hours in front of the TV, the XBox or the PS3, then you’re prioritizing that activity over others. Perhaps you like to host social events with friends and family over good barbecue and a game of Frisbee in the back yard. Possibly you’re taking care of a sick family member. Certainly no one will fault you for a little down time via entertaining yourself with games or a movie. I certainly hope you are making time for friends and honoring your commitment to your family. I would just encourage you to take stock of whether other, unfruitful priorities are keeping you from being the technologist you could be.

 

6 Steps to Build a Training Plan and Execute On It

  1. Open up your favorite text editor and make a list of the top 3 things you want or need to learn in the next 6 months. You likely know what they are.
  2. Do an internet search for links that explain those 3 topics and add those links to their corresponding topics in the text editor. Be sure to have at least 3 links per topic you want to learn.
  3. Save the file to your desktop so you see it every day as a reminder to work on your skills.
  4. If you have a whiteboard, write your plan on the whiteboard so you see it every day.
  5. Work your personal training plan by reading one article a day from your training plan.
  6. Now, rinse and repeat. Start the process over with those same three topics.

As a bonus, hopefully at least some of those articles have demos. do the demos yourself on your own local install of SQL Server. Don’t just read the demos. After you’ve done these things, you will have read and worked through the content of 6 articles for each of your three topics that you wanted to learn. That’s a great start!

Next Steps to Take

  1. If you want to chat about any of this, hit me up on Twitter or post a comment here. I’m happy to discuss.
  2. Build a training plan using the steps above.
  3. If you’re struggling to find good resources, check out my post about how to find reliable resources for learning.
  4. Leave me a comment on this post to share your training plan, or message me on Twitter with your plan. Sharing your plan with others reinforces your commitment to the plan.

6 things to consider when presenting

 

Things to consider when presenting

I’ve presented a few times recently and discovered I like doing it. Presenting is a great way to give back to the I.T. community – in this case the SQL Server community. I’ve previously blogged about things I learned from giving my first presentation, and what it was like presenting at New Stars of Data. Here are six additional thoughts on presenting.

 

  • Be sure you show up early for your presentation

The day of New Stars of Data, I was up 90 minutes earlier than usual to use the link to the track I was in to join a panel discussion in progress. I just wanted to be sure that the link was going to work and catch a bit of that presentation. I also wanted to be sure I was wide awake when it was time for me to  present. Slumber does not mix well with presenting!

If you’re not presenting early in the morning like I was, be sure to leave yourself plenty of time to get to your computer to have it ready for presenting. You don’t want to turn it on 5 minutes before you’re supposed to present only to find that it won’t turn on for some reason or that you can no longer find your presentation materials.

  • Use stories whenever possible

In one case I not only used words to explain why data professionals should use a particular SQL Server feature, but I told a relevant store from my work experience. People connect to stories and they are likely to remember the story and the feature you discussed long after all the technical details.  Also, telling the story helped keep me engaged and excited about what I was saying. In turn, that helps the audience remember what was discussed.

 

  • Avoid making a bunch of changes to the slides or demos late in your preparation

Doing this may make you less comfortable when you get to the parts of the slide deck that you’ve recently modified. This is especially true if you’re a new speaker or if the slide(s) have undergone more than just cosmetic changes.

Also, once you’ve tested your demos and you know they are working, I would encourage you not to adjust them if your presentation date and time is coming up soon.

 

  • Practice frequently and record yourself giving the presentation

You need to practice so that your comfortable explaining the content of your presentation. The transitions between slides, to demos and back, also need to feel comfortable and as smooth as possible. Adequate practice will help you be feel at ease with the mouse or keyboard presses necessary to make those transitions.

You also want to listen to your tone of voice to ensure you seem engaged. This is a tough one because we all have different communication styles. Some people are naturally animated and excited when talking. Some people use their hands a lot. Some are more demure. You know yourself, so do some reflection about how you sound and make any adjustments that are needed and that you can comfortably make while still honoring your own personality.

Additionally, pay attention to your pace when speaking. You don’t want to sound rushed. Try to strike a balance with the speed of your speaking. Check out a post from Catherine Wilhelmsen on a tool she used recently that helps with presentation skills.

 

  • Be sure that your information is accurate

Please, please, please do your research as much as possible to ensure that what you’re presenting to people is the correct information. Everyone is learning as we go. Just make a commitment to yourself and to your audience that you will exercise due diligence to research your topic and present accurate information.

 

  • If possible, brush your teeth before presenting

Now, this seems a bit out of place, but I learned this recently from a practice session I was doing. I was practicing for New Stars of Data and had eaten food 1-2 hours before I was practicing. Early in the practice recording I realize that I have a piece of food that has now dislodged. Luckily this happened during a practice session and not while I was presenting live to people!

 

Next Steps to Take

1. Check out the speaker improvement library here.

2. If you would like help with anything in this post, or with something else related to SQL Server, reach out to me on Twitter, and I’ll be glad to offer assistance.

 

What is it like to be a presenter at the News Stars of Data virtual conference

 

Back in early July 2021, I presented at the SQL Server Meetup I co-host with Anthony Fortner. We had a speaker no show a couple of months prior and then for the July meeting a speaker had to cancel about a week before the meeting. I couldn’t let another month go by with no Meetup. So, I created a presentation from nothing, practiced it and gave it live for the first time in about a 7 day time span. It took about 15 hours of work and was my first SQL Server presentation to a public group. I learned a number of things from that experience. Then I gave a revamped version of the presentation to New Stars of Data in late October.

In late September this year I was on LinkedIn and engaged a conversation about presenting on SQL Server.  New Stars Of Data had been advertised recently and if I remember correctly, that came up in the discussion. I mentioned I might apply to be a speaker. Andy Yun chimed in and encouraged me to apply. I thought, “Why not! I’ll apply and see what happens.”

I applied and was accepted. Ben Weissman sent me an acceptance email with some introductory information about the conference and introduced me to Leslie Andrews, who would be my mentor.

I’ll admit that I had only heard the name. That’s it. I didn’t know anything else about Leslie, but I assumed that if she’s a mentor for New Stars of Data, she must be at least well-known. If you look at her Twitter profile and who is following her, etc. you can see that she is indeed well-known. I was excited and interested to work with her, and nervous. I mean, lots of people know her so she’s “SQL famous.” What will she think of me? Will I embarrass myself talking to her because I won’t know what to say? I was anxious for no reason. Leslie is very personable and easy to talk to.

3 Questions I asked my mentor about speaking at New Stars of Data

We set up a video meeting. As I prepped for the meeting with three questions:

  1. How many people are generally in the presentation sessions?
  2. Is there anything I should consider that’s different because of the number of people in the session?
  3. What advice do you have for me?

That first question was to set my expectations. I didn’t want to be overwhelmed by 50, 100, 200 people being in my session. That would have been fantastic, of course! However, I wanted to know what to expect because that would help me manage my emotions the day of the presentation. My theory was that the more people I was speaking to, the more nervous I might be, but if I at least knew that ahead of time, then I wouldn’t be as nervous.  Leslie told me that there might be as many as 50 people in the session.

My second question was “Is there anything I should consider that’s different because of the number of people in the session?” This was just a curiosity question since I had never given a presentation to more than just a handful of people at a SQL Server Meetup. Leslie told me that there was nothing really different to prepare for whether there were 5, 50, or 200 people. That was actually a bit of a relief.

Advice from Leslie Andrews: Round 1

My third question was, “What advice do you have for me?” Leslie gave me 3 main ideas to think about.

  • Write down what you want to say and memorize it

I didn’t end up doing this. the point of this advise was to get me comfortable with the presentation. I did practice the full presentation at least three times in the week or so before New Stars of Data. Additionally, I went over a few spots in the presentation where I was having difficulty with the slides and demos. By the time I presented at New Stars of Data, I had also presented the topic two previous times. I recorded each time I practiced it in it’s entirety so I could listen to it. I did the recordings of my practices because I wanted to hear how I sounded.  Was I too monotone? Did I sound engaged? Were there a lot of filler words? I was asking myself those questions as I watched the recordings.

  • Showing yourself on camera is not required

While not required, I did turn my camera on for the presentation. I had built this presentation about 6 weeks or so prior and had given it twice before to SQL Server Meetups. The second time I presented, I ended up with my camera off. I found that to be unnerving. It made me paranoid that I wasn’t actually talking to anyone. For that reason, I did turn on my camera when I presented for New Stars of Data

I was already familiar with this presentation as I had attended it at the last PASS Summit. I learned a lot about font sizes, colors, and other visibility issues related to presentations.

Advice from Leslie Andrews: Round 2

I provided Leslie with my existing slides and asked her for feedback. I also provided her with a link to the first time I gave the presentation. Here’s what she said.

  • Be sure to update font sizes in SSMS for results and execution plans.
  • Leslie felt that the slides were fairly wordy and commented that slides with a lots of words tempts people to read you slides rather than listen to what you’re saying.
  • She suggested that I add one or more slides to explain the concepts of High Availability and Disaster Recovery. She said I shouldn’t assume that everyone in the session will know what those ideas are about.
  • I needed to add at least three slides. One each for prompting for question at the end, one for thanking New Stars of Data and a duplicate of my contact info slide at the end to show when I was done.

I wasn’t surprised by bullet point #2. Wordiness in my writing has always been something that people point out. I’ve worked on it, but it’s a work in progress. Also, I want my slides to be a resource of sorts after the presentation. Nonetheless, I did tone down the amount of words on my slides.

I hadn’t realized I was making an assumption that people would know what HA and DR were about. Therefore, it was nice to have someone on the outside point that out so I could explain what I meant.

A Mistake I Made with the Slides

I made a bunch of changes to the slides based on the above. Then I had to practice for that second time I was giving this presentation to a live group prior to presenting at New Stars of Data. I made a significant mistake here. I didn’t have enough time to make a bunch of slide changes and practice enough with those changes before the second time I presented to a live group.

As a result, I felt that the second presentation wasn’t nearly as good as the first time I gave it. I just wasn’t comfortable yet with all the changes I had made. I made a few more tweaks to the slides before New Stars of Data, but by about two weeks out, I committed to stop messing with the slides so I could get comfortable with the presentation format.

 

What Happened the Day of New Stars of Data?

The day of New Stars of Data, I was up 90 minutes earlier than usual to use the link to the track I was in to join a panel discussion in progress. My presentation was at 8:45 AM Central time in the U.S. I just wanted to be sure that the link was going to work and catch a bit of that presentation. I also wanted to be sure I was wide awake when it was time for me to  present.

The speaker who was right before me had cancelled so there was a gap in the track. I was concerned that I might lose most of my audience, but I couldn’t control that. I connected early to my session track and Deborah Melkin was there to greet me. Andy Yun was also there. These people are “SQL Famous” , but they set me at ease.  I decided to take questions at the end because I didn’t want to interrupt my flow in the presentation once I had settled in. Deborah said that made complete sense and was happy to handle questions at the end.

The presentation went well overall. I had a few bobbles with my words and one time where my finger hit the scroll button on my mouse, causing my slides to scroll forward several slides. I felt good about the presentation though.

A few days later, Ben sent me  a pdf of my feedback. Everyone had good things to say, and that was encouraging. One person shared the observation that they felt my vocal tone could have been more varied so that I sounded more engaged and excited about my topic. I wasn’t shocked by that. I’m not a particularly animated person when I’m talking. I’m also a new speaker and I’m sure there were some nerves there. However, I also know from listening to my practice sessions that I had improved that part of the presentation already and it would get better as I settled in as a speaker.

 

Next Steps to Take

  1. Consider sharing what you know. If you’ve solved a problem in the I.T. realm, then you have something to share. You don’t have to be an “expert” to help others by sharing what you’ve learned.
  2. If you would like help with anything in this post, or with something else related to SQL Server, reach out to me on Twitter, and I’ll be glad to offer assistance.

 

Four Areas to Focus on When You Start A New DBA Role

 

You’ve just been hired into a DBA role at a new company, or you’ve been given the DBA keys at your current company. Maybe you’re a SysAdmin and your boss has informed you that you are now supposed to manage the SQL Servers as well as everything else on your plate. In any of these situations, you may have some confidence in your skills, but especially in the case of being a new hire, you have absolutely no true idea of what you’re walking into.

In these scenarios, where do you start? Start with these four areas.

  1. Backups
  2. SQL Agent Job Notifications
  3. Building an environment inventory
  4. Security

 

1. SQL Server backups: You have to be able to recover the data.

My experience has been that many companies aren’t doing an adequate job of managing the backup routines for their databases. I’ve been in companies where two or even three products were running backups on the same SQL Servers. I’ve been in situations where important databases weren’t backed up frequently enough and with the right set of backup types to recover appropriately, which would result in losing more data than the management was comfortable with. I’ve also seen scenarios where there were no backups at all.

So, what do you do to tackle the issue of SQL Server backups in your environment? First, connect to your SQL Servers and run the below query. This will give you the database name and the date of the last full backup if the last full backup was over 7 days ago . If the database has never been backed up, then you’ll get that information as well. Any results from this query should be investigated immediately. This query was adapted from sp_Blitz with some column names changed to make it more clear what it does.

SELECT  d.[name] AS DatabaseName ,
COALESCE(CAST(MAX(b.backup_finish_date) AS VARCHAR(25)),'No Backup Ever Made') AS LastFullBackup
FROM master.sys.databases AS D
LEFT OUTER JOIN msdb.dbo.backupset AS B ON D.name COLLATE SQL_Latin1_General_CP1_CI_AS = B.database_name COLLATE SQL_Latin1_General_CP1_CI_AS

AND B.type = 'D' /*Full backup*/
AND B.server_name = SERVERPROPERTY('ServerName') /*Backupset ran on server you're currently connected to. */

WHERE D.[name]<> 'tempdb'  /* Eliminate TempDB. No need to back that up */
AND D.state_desc NOT IN ('RESTORING', 'OFFLINE', 'OFFLINE_SECONDARY') /* Exclude databases that are offline or involved in log shipping, for example */
AND D.is_in_standby = 0 /* Exclude databases in stand-by state as part of log shipping*/
AND D.source_database_id IS NULL /* Excludes database snapshots */

GROUP BY d.name
HAVING MAX(B.backup_finish_date) <= GETDATE()-7 /*Full backup older than 7 days ago.*/
OR MAX(B.backup_finish_date) IS NULL;										    

 

This one will show you any databases in the Full recovery model that haven’t had a transaction log backup in over 24 hours. This query was adapted from sp_Blitz with some column names changed to make it more clear what it does.

SELECT  d.[name] AS DatabaseName ,
COALESCE(CAST(MAX(b.backup_finish_date) AS VARCHAR(25)),'No Backup Ever Made') AS LastLogBackup
FROM master.sys.databases AS D
LEFT OUTER JOIN msdb.dbo.backupset AS B ON D.name COLLATE SQL_Latin1_General_CP1_CI_AS = B.database_name COLLATE SQL_Latin1_General_CP1_CI_AS

AND B.type = 'L' /*Log backup*/
AND B.server_name = SERVERPROPERTY('ServerName') /*Backupset ran on server you're currnetly connected to. */

WHERE D.[name]<> 'tempdb'  /* Eliminate TempDB. No need to back that up */
AND D.state_desc NOT IN ('RESTORING', 'OFFLINE', 'OFFLINE_SECONDARY') /* Exclude databases that are offline or involved in log shipping, for example */
AND D.is_in_standby = 0 /* Database is read-only to restore a log backup as in Log Shipping. */
AND D.source_database_id IS NULL /* Excludes database snapshots */
AND D.recovery_model_desc = 'Full'
GROUP BY d.name
HAVING MAX(B.backup_finish_date) <= GETDATE()-1 /*Log backup older than 1 day ago.*/
OR MAX(B.backup_finish_date) IS NULL;										    

 

Review the SQL Server to see if there are any native backup methods in place, such as a Maintenance Plan or Ola Hallengren scripts.  Each of these backup methods will produce one or more SQL Server Agent jobs. Check to see if the jobs associated with those options are enabled.

If you don’t see either of those options set up, your company may be using a 3rd party backup software. These are typically, but not always, deployed by System Administrators. Check with them to see what 3rd party software is in place for backups and let them know your findings for the server in question. this could be a situation where this SQL Server was just not enrolled in the 3rd party software and so is not being backed up.

If none of these things turn up a backup solution, then get one in place ASAP. See this post for options on how to create backups.

2. SQL Agent Job Notifications: If a job is worth creating, it’s worth knowing if it failed.

This is in position #2 because it’s been an easy win in every job I’ve taken. In every new role I’ve had, I have discovered many SQL Agent jobs with no notifications or notifications set up to wrong people. If a job is worth creating, it’s worth knowing if it failed.

As you work through finding failed jobs that never notified anyone, you may find that you’re fixing important agent jobs that should have been working. This process fixes business and data related processes and also helps you find stake-holders for data processes. When jobs fail and you investigate, you often need to know who is impacted by the job failure. This further leads to identifying who should be notified if it fails. This process then, actually helps connect you to key players in the organization very early. Those people on the business side almost always appreciate your efforts to make sure that data processes that affect them are working properly. This gives you allies as you move forward in your role.

So how do you proceed if you find jobs that have no notification set up if they should fail?

  • First, locate jobs that are failing. I have a post that will show you two ways to do that. You can either look at the SQL Agent Job Activity Monitor on a SQL instance or query the msdb database.
  • Second, locate enabled jobs that have no email operators assigned. These will be jobs that no one will know whether they are working or not. This post will tell you about how to find these jobs.
  • Third, work through a process to make sure jobs with no notification set up will have an email operator assigned to them. This will ensure the right “someone” will know if the job fails.

3. Build an environment Inventory: You can’t manage what you don’t know about.

The process of gathering information about your environment tells you about the make up of versions and editions in your environment. This, in turn, leads to the answer to the question, “What SQL Server’s are out of support and likely need to be upgraded?”

“How many SQL Servers are there at company X?”

Sure, the hiring manager told you the company only has a dozen SQL Servers, but I’ve often found that hiring managers often don’t know the real answer to the question, “So, how many SQL Servers do you have at company x?” That’s a question you should always ask when interviewing by the way. When you hear the answer, I’d encourage you to add at least 50% to that answer. At one job, I was told there were about a dozen SQL Servers. Less than 6 months in I had located about 50 SQL Servers, and that was before I ran the MAP Toolkit.

If you are not familiar with the MAP Toolkit, when you run the utility it does a search of your network or Active Directory to find SQL Servers. You will want to ensure that your security team and/or System Administrators know you are going to run this utility. This is because it will likely set off security software or even be blocked, in some cases.

The tool produces an Excel format list of plenty of information for you to know and this is the simplest way to start creating a SQL Server inventory. This scan is likely to find a lot of SQL Servers that no one seems to remember existing. Managing all the SQL Servers within your purview is important. You can’t do that if you don’t know about them.

You will also probably find individual employees who have installed SQL Server Standard or even Enterprise Edition on their local PCs. That’s a licensing no-no. This situation gives you the chance to help your company avoid licensing entanglements with Microsoft by addressing these scenarios.

If you find that you aren’t allowed to use the MAP Toolkit, as you hear about other SQL Servers that exist in your environment, or as you discover them for yourself as you look through Active Directory Users and Computers, make a Central Management Server and add SQL Servers to it. Ensure you have access to those SQL Servers and then query them to find out all you can about them. Review the information available to you from SERVERPROPERTY and build your own queries to discover information about the company SQL Servers. Here is an example query from the MS Docs link above. As you discover information about the company SQL Servers, create your own spreadsheet of information, or create a database and tables to hold the information you find.

SELECT  
  SERVERPROPERTY('MachineName') AS ComputerName,
  SERVERPROPERTY('ServerName') AS InstanceName,  
  SERVERPROPERTY('Edition') AS Edition,
  SERVERPROPERTY('ProductVersion') AS ProductVersion,  
  SERVERPROPERTY('ProductLevel') AS ProductLevel;  
GO

 

4. Security: Who has access to SQL Server and what type of access do they have?

I generally focus on this one after the others because security is often a political subject within an organization. You don’t want to become embroiled in power struggles right away by trying to take away someone’s access, because you will lose. Additionally, if you’ve done the previous work mentioned, you have likely built some good business relationships and a few allies to help you have the discussion about SQL Server access and security.

As a DBA, you and the business need to know who has SysAdmin level access. The access that they possess allows their credentials to make any change they want to. In truth, it’s often not so much about trusting the person who has the access as it is realizing that every person who has SA permission is a person whom a hacker could exploit to gain access to the company’s data. You will want to make the business aware of how many credentials have this level of permission and whose credentials they are.

How do you find who has SysAdmin level access on your SQL Server? Here is a query for that from MS Docs. I have added a WHERE clause to exclude the commonly found SQL Server accounts that start with NT SERVICE.

 

SELECT	roles.principal_id AS RolePrincipalID,	
roles.name AS RolePrincipalName,	server_role_members.member_principal_id	AS MemberPrincipalID,	
members.name AS MemberPrincipalName

FROM sys.server_role_members AS server_role_members
INNER JOIN sys.server_principals AS roles
    ON server_role_members.role_principal_id = roles.principal_id
INNER JOIN sys.server_principals AS members 
    ON server_role_members.member_principal_id = members.principal_id
WHERE members.name NOT LIKE 'NT SERVICE%';

This query shows you what credentials have Server level SA access. You also will want to know what accounts have the db_owner role at the database level.

I would strongly encourage you to review the permissions scripts written and maintained by Kenneth Fisher. These will provide a wealth of information about your server level logins and your database level users.

 

Next Steps To Take

  1. Check out the First Responder Kit on Github.
  2. If you have a comment or question about this post specifically, leave a comment and I’ll get back to you.
  3. If you would like help with something else related to SQL Server, reach out to me on Twitter, and I’ll be glad to offer assistance.

 

Three Keys to SQL Server Performance

 

Everyone wants good performance from their database systems. Some even expect and need a high performing Ferrari all the time. How is this achieved though? What do you need to understand about SQL Server specifically in order to make your company’s applications hum along like a well tuned car? We will look at three keys to SQL Server performance.

 

SQL Server Speed
SQL Server Performance

Here’s the short list of performance keys.

1. SQL Server must be able to cache sufficient data in memory

2. SQL Server must be able to retrieve data from disk efficiently

3. You must write “good” T-SQL

 

SQL Server Memory

 

1. SQL Server must be able to cache sufficient data in memory

SQL Server does not use memory like most applications.  People unfamiliar with SQL Server’s use of memory are surprised to see SQL Server using 80% or more of a server’s RAM. “I have 128 GB  of RAM on this machine why is SQL Server set to consume 115 GB of that?!” Then they decide to change Max Server Memory down to like 64 GB or less. Suddenly they might find that disk IO shoots up. You will also likely see that execution plans are rapidly aging out of the plan cache, which will burn up CPU with new compiles for query plans and potentially lead to parameter sniffing issues. When all this comes into play people start complaining about applications not working as well. They start saying the magical phrase, “It’s slow.”

When a query is issued and the  data requested is not in memory, SQL Server must use CPU and disk IO to get the data into memory first. While it does this a PAGEIOLATCH_* wait is registered in SQL Server because the current query is waiting on the data to be retrieved. This wait causes the query to be put on hold.  Reading data off the disk is always, always going to be slower than working with data that’s already in memory.

So, how do you determine what is a sufficient amount of RAM for your SQL Server? The short answer is that I would encourage you to look at  my post here and use the DBATools cdmlet Set-DBAMaxMemory. The post will help you understand more about what the Max Server memory setting does and the PowerShell cmdlet will recommend a good starting place for setting Max Server Memory. Here are some examples from the documentation from DBATools and from the Help commands available in PowerShell.

<# If you have a Central Management Server for your SQL environment, consider using this command to loop through all the SQL Servers and set the Max Server Memory where it is set to something larger than the total amount of RAM assigned to the server. #> 

Get-DbaRegServer -SqlInstance sqlserver | Test-DbaMaxMemory | Where-Object { $_.MaxValue -gt $_.Total } | Set-DbaMaxMemory 

<# If you have a Central Management Server for your SQL environment, consider using this command to loop through all the SQL Servers and set the Max Server Memory to this accepted formula created by a SQL Server expert. #> 

Get-DbaRegServer -SqlInstance sqlserver | Test-DbaMaxMemory | Set-DbaMaxMemory 

<# If you don't have a registered server then just use the below #> 

Test-DbaMaxMemory -SQLinstance SQLServerInstanceNameHere | Set-DbaMaxMemory

The longer answer is this. Take a look at three metrics in the SQL Server:BufferManager category in Performance Monitor.  First, if you observe Page Life Expectancy over time and the value has long stretches where it plummets and stays low without recovering, then you have some heavy hitting queries running that could likely benefit from having more RAM available. This will likely be accompanied by SQL Server wait stats showing a high average wait for PAGEIOLATCH_* type waits. This is SQL Server reading pages from disk because the data pages it needed were not in memory.

Second, review the Perfmon counter Lazy Writes/Sec. If SQL Server is having to free up memory in between checkpoints, this value will be above zero. If it is regularly above zero, then SQL Server is regularly under memory pressure and is having to write data pages in memory back to the disk so that it can load different data pages to satisfy queries.

Third, look at Free list Stalls/sec. The value of this Performance Monitor counter indicates the number of requests per second that had to wait for a free data page in memory. If Page Life Expectancy is often low for long stretches and both Lazy Writes/sec and Free List Stalls/sec are greater than zero, then you need to either adjust Max Server Memory up (as long as you don’t go too high based on the above information), add memory or, take a hard look at your indexes and queries involved when these PerfMon metrics are out of balance.

SQL Server Indexing and Disk Performance

 

2. SQL Server Must Be Able to Retrieve Data From Disk Efficiently

Look at your SQL Server wait stats information and if you see PAGEIOLATCH_* very prominent then there could be a good chance that the indexes in your database need attention or those long IO waits could mean a problem with the disk subsystem.

PAGEIOLATCH_* type waits are all wait types related to reading 8KB data pages from the disk into memory. Inefficient indexes could be making these reads longer because SQL Server can’t quickly get the data it needs from the existing indexes. This can happen over time as query patterns and data patterns change. For example, your company might introduce a new category of products Suddenly people are querying that category and its associated products far more and older, mainstay products less. The distribution of that data may affect the execution plan generated.

The company may have re-factored an existing application into new tables and missed indexing the Category column. Now when people are searching for things like “Bike Accessories” there is no supporting index. This results in long table scans of millions of rows.

As a start to determine if there is inefficient indexing, run and save the output of sp_Blitzindex to examine your tables and indexes. Review its recommendations and make adjustments. Then re-measure index usage with sp_BlitzIndex. Some time after a restart of SQL Server re-run sp_BlitzIndex and compare the output to the previously saved run looking to see if SQL Server is using the adjusted indexes.

To review whether you have a disk IO subsystem issue, look at the DMV called sys.dm_io_virtual_file_stats. You can also review the SQL Server Error Log looking for messages indicating IO that took longer than 15 seconds. These could be an indication of an IO subsystem issue. Review this article and this article on these topics.

Both of these articles provide information on understanding the DMV and the error message. There is also information about Performance Monitor counters to use to measure potential problems.

If you are in the cloud, such as AWS, be sure to review settings for ebs and fsx storage to ensure that the IOPs and throughput are set up appropriately. Also, be sure to take into consideration how the AWS ec2 instance type might be throttling your IO and throughput capabilities. Just because your storage is set up with a certain IOPs and throughput doesn’t mean that the ec2 instance can support the storage settings.

T-SQL Anti-Patterns to Avoid

 

3. You Must Write “Good” T-SQL

Poorly written T-SQL can cause poor application and SQL Server performance. Here is a brief list of things to avoid.

A. Writing T-SQL that makes it non-SARGable. This will cause SQL Server to have to scan entire tables instead of using an existing index. For details and examples see this, this and this. there is no need for me to explain this in depth, as it has been written about quite frequently.

B. Overuse of cursors. SQL is a set based language so making it do operations row by row, is far less efficient than using set-based logic.  Take a look at this post and this post.

C. Overuse of SELECT *. Some tables are very wide and have millions or even billions of rows. If you don’t truly need every column, then do SQL Server and your application a favor and only return the columns that are actually needed!

D. Be careful with scalar user defined functions. This goes back to the idea of SQL being a set-based language. A scalar udf returns a single value for each value passed to it. when you use this sort of logic and pass into it large numbers of rows, then each row is processed one at a time inside the function to return a value. Also, SQL Server doesn’t do a good job of “seeing inside” Scalar UDFs and show you that one is present in a query plan. Asa result, you might not see this sort of thing if you’re looking at a query plan. Additionally, scalar UDFs kill SQL Server’s ability to go parallel. For more, take a look at this, this , and this.

This list could be much, much longer. the point here is that how you write your T-SQL often has a direct impact on performance. If you write T-SQL for applications, I strongly encourage you to look at the blogs of folks like Erik Darling and Kendra Little as well as sites that have a large number of entries on T-SQL like this one.

Next Steps To Take

If you would like help with anything in this post, or with something else related to SQL Server, reach out to me here, on Twitter, or by email at Leem@leemarkum.com and I’ll be glad to offer assistance.

 

How I Became a Database Administrator

I recently saw a Tweet from Kendra Little where she asked the simple question, “What got you into databases?”  I’m intrigued by people’s stories of how they got into their current careers so I looked through the thread. There were lots of interesting paths shared and even a blog post was shared by someone who had recently written about how they got into a career involving databases. Kendra’s question and all of the interesting responses inspired me to write a post about my story.

 

It was a Dark and Dreary Night

 

Actually, it was February 2008. For a number of years I had been a team Supervisor at a physical security company called Interface Security; think ADT only much smaller. I was good at leading the team of 8-12 people as we used a critical piece of software to interact with customer accounts. I understood a lot about this software and was often the “lucky” person to be selected to do application failovers. This involved using a custom GUI from the vendor to shut down all the services of the application and start them back up, pointed to a new server. I haven’t worked there in almost 7 years, but I can still see this GUI in my mind.  Anyway, the woman who was the liaison between our company and the software maker was leaving her job to go work for the software maker, Bold Technologies.

I had reached about the top of the pay scale in my position and, at that point, there really was nowhere else for me to go in the company. The next few positions above me were taken and those people weren’t going to leave any time soon. The employee who was leaving asked me to apply for her job. At first I was adamant that I didn’t want to do that and that I didn’t know enough. She insisted that I did know enough and that I should apply. I gave it some more thought and decided to throw my hat in the ring. After a conversation with my boss two levels above me (Dan Reynolds) and about a month later, I had the job and the title of Manitou System Administrator. Manitou was the name of the software.

Interface Security was in the planning phases of a major version upgrade for this software. I took the job thinking I was just going to manage software. However, we were also going to be doing hardware upgrades for the servers this application was running on. My desire to understand everything even tangentially connected to the software got the better of me. We completed the software upgrade later that year and the server hardware upgrade as well.

Lee, Meet SQL Server

As part of that application upgrade process, I learned that there was this thing called SQL Server installed on the servers we were going to replace. I dove in head first to learn all I could about SQL Server. I learned that the process I was controlling with this company’s custom GUI was actually transactional replication under the covers. I loved learning about it and it was an important process to understand for my company and for my role.

About 18 months after starting the new role I began learning T-SQL using whatever free tutorials I could find on the internet. I started experimenting with SQL Server Reporting Services. I began to augment the application’s reporting capabilities with SSRS, at first just building the reports and making them available via the Report Manager and then later by scheduling them to be delivered to various people. In Jan 2012 I was officially moved into the I.T. department.

During that time I discovered the Microsoft Certified Master video training that Kimberly Tripp and Paul Randall recorded. I watched all the videos, some of them multiple times. There was so much I didn’t understand, but as I kept watching them and searching for information on the internet, the more the concepts made sense. I was by no means on my way to being an MCM for SQL Server, but I was learning rapidly.

Somewhere in the following years I started experimenting with what at the time were brand new features in Excel to help with reporting – Power Query and Power Pivot. I used those new tools to pull in and visualize data so that I had both SSRS and these new Excel tools at my disposal.

I also found Brentozar.com during those years. At the time, it was Brent, Kendra Little, and Jeremiah Peschka. Later, as I continued to learn from all of them, they added Jes Borland. This post was one of my first forays into performance tuning. I found it after the I.T. Director asked me to look into the performance problems that the financial system was having. It was also on SQL Server.

Earning a Microsoft Certification

In May of 2012 I completed a certificate in database technology from the Continuing Education Center University of Missouri at Saint Louis. The course I took there were concentrated two day courses on various parts of SQL Server. They advanced my career greatly. Somewhere along the way, I bought a book called SQL Server 2008 R2 Unleashed. I read that book, watched the aforementioned MCM videos and all the blog posts I could from BrentOzar.com.  I bought a Microsoft practice test and then a test from another vendor as well. At night around 9pm, after my three kids and my wife went to bed, I would study for about 2 hours a night, 5 nights a week. I studied probably 4-8 hours on the weekends too. This went on for months.

In May 2013 I got my first certification. It was for Microsoft SQL Server 2008 Implementation and Maintenance. I took the test on the eighth anniversary of my dad’s passing. I took it for me to advance my career, but I took the test on that day to attempt to honor him as well. The test was challenging, but I passed and I was ecstatic! I knew he would have been so proud.

A Change of Title

I held the Manitou System Administrator title until March 2014 and continued to grow more and more as a data professional. In March 2014 I got a title change to Database Administrator. I was elated! I had achieved a remarkable career change from a Call Center Supervisor role to a Database Administrator!

 

Acknowledgments

Wife and Kids –

My wife, Dacia, missed time with me on the weekends when I was studying for my first certification to help me break into the world of database administration. She handled the kids while I was sequestered away in my basement office studying like crazy. My kids, of course, missed time with me because of that too. I missed being with them, but it was necessary to help all of us as I was trying to advance my career and support a family.

Dan Reynolds – Interface Security

My previous boss, Director and then later Vice President of Customer Operations, Dan Reynolds was instrumental in my career development and I’d be remiss if I didn’t mention him. He believed in me from years previous when I had been working under him in the Call Center. I continued to work closely with him even after I  officially went into I.T. He took me to the user conference for the software I was managing. It wasn’t cheap and I was grateful. I learned a lot at those conferences and met a lot of great people at Bold Technologies while I was at those conferences. I met a lot of people from other companies working in the same role I was in. Dan helped me pursue education for SQL Server at the Continuing Education and Training Center at the University of Missouri at Saint Louis.

Amy Spurgeon – Interface Security/Bold Technologies

Amy was the support person who encouraged me to apply for her job before she left. She pushed me to consider the possibilities.

Josh Tafoya – Bold Technologies

Josh was a person in the support department at Bold Technologies when I started my application support role. He knows the Manitou software very well. He helped me understand the software better and he taught me a lot about what it meant to be an I.T. professional.

Matthew Narowski – Bold Technologies

Matthew held a number of roles at Bold while I was at Interface and was eventually named either CEO or President at Bold. He was patient and taught me a lot about I.T. support. He explained a great many things to me about the world of I.T. in general.

I am sure there are others who made contributions to my transition into a new career. I thank all of you.

 

I hope my story helps and encourages you along your own journey.

 

 

How to Use the SQL Server CONCAT Function

SQL Server concatenation methods have been enhanced in modern versions of SQL Server. SQL Server 2012 introduced the CONCAT() function. In SQL Server 2017 we get CONCAT_WS().

A common usage of concatenation, or joining column values together in a string, is combining a FirstName and LastName column into a FullName column.  Another common usage might be for creating an address column that pulls together building number, street, city and zip code.

Below I have provided an example of the three common ways to concatenate strings to create an address. These all produce the same results because there are no NULL values. They are just different ways of doing the same thing.

The first example is the CONCAT() method from SQL Server 2012, followed by the CONCAT_WS() function, which is CONCAT With Separator. The third method is the traditional method of combining strings with the
+” sign.

USE WideWorldImporters;
GO

SELECT CONCAT(DeliveryAddressLine2, ' ', C.CityName, ' ',DeliveryPostalCode) AS NewConcatMethod,

CONCAT_WS(' ', DeliveryAddressLine2, C.CityName,DeliveryPostalCode) AS CONCAT_WSMethod,

(DeliveryAddressLine2 + ' ' + C.CityName + ' ' + DeliveryPostalCode) AS OlderConcatMethod

FROM [WideWorldImporters].[Sales].[Customers] AS CUST
INNER JOIN [Application].[Cities] AS C ON CUST.DeliveryCityID = C.CityID;

 

NULLs And Concatenation

Before SQL Server 2012 the “+” sign was used to combine strings together into whatever form you needed. One disadvantage to this method is that if a value in the string to be concactenated is NULL then the result set returns a NULL.

 

USE AdventureWorks2014;
GO

SELECT TOP 5
    Title, 
    FirstName, 
    MiddleName, 
    LastName,
    Title + ' ' + FirstName + ' ' + MiddleName + ' ' + LastName as MailingName
FROM Person.Person;

SELECT TOP 5
    Title, 
    FirstName, 
    MiddleName, 
    LastName,
    CONCAT(Title,' ',FirstName,' ',MiddleName,' ', LastName) as MailingName
FROM Person.Person;

 

In the top result set, you can see that whenever a NULL occurs the resulting MailingName column is also NULL. This makes some sense because NULL is “unknown” so you can’t really join an unknown value to other strings and get a predictable result.

However, with CONCAT(), columns with NULL values simply have those values ignored.

SQL Server CONCAT Function and NULL

 

What about concatenating numbers?

CREATE DATABASE TestDB;

CREATE TABLE IntValues
(Col1 INT);

INSERT INTO IntValues
VALUES(67),(45),(90)

SELECT CONCAT(67,45,90) AS Result
FROM dbo.IntAvlues;

SELECT CAST(67 as CHAR(2)) + CAST(45 as CHAR(2)) + CAST(90 as CHAR(2)) AS Result
FROM dbo.IntAvlues; 

Concatenating numbers works in the same way. The great thing here is that there is no need to do an explicit cast when using CONCAT().

 

SQL Server CONCAT_WS()

SQL Server 2017 brought us CONCAT_WS. With this new function, you specify the separator you want to use when building the string.  The value used as the separator goes first inside the parentheses followed by the column names that are to be joined together. These column names are separated by commas in a list.  Like CONCAT(), CONCAT_WS() eliminates NULLS from the result set, leaving you with the non-null values.

SELECT TOP 5 Title, FirstName, MiddleName, LastName, 
CONCAT_WS(' ',Title,FirstName,MiddleName,LastName) as MailingName 
FROM Person.Person;
SQL Server Concat_WS() Results

I think that the syntax with these new functions is easier to write and to read. These newer approaches reduce typing and having to switch between adding a “+” and ‘ ‘ in various places.  CONCAT_WS() reduces typing even more by only specifying the separator once at the beginning of the function.

Next Steps To Take

  1. Check here for the MS Docs on CONCAT() and here for CONCAT_WS().
  2. If you would like help with anything in this post, or with something else related to SQL Server, reach out to me here, or on Twitter, and I’ll be glad to offer assistance.

How to Configure SQL Server Temporal Tables Part 2

 

In the initial post, we talked about some very basic items to get you up and running with temporal tables. In part 2, I’ll cover a few more things to think about and implement when working with temporal tables.

In the first post, we looked at altering an existing table to be a temporal table. Now we will look at what is involved in making a table be a temporal table from the very beginning.

Designing a Temporal Table from Scratch

First of all, you can create a table to be a temporal table from the beginning, without naming the history table. But you get an ugly history table name in the following format:

[dbo].[MSSQL_TemporalHistoryFor_tableObjectId].

In my test case that turns out to be this:

[dbo].[MSSQL_TemporalHistoryFor_1913773875].

Also, notice that in this example below, the SysStartTime and SysEndTime required columns are not designated as hidden. This means they can be returned in queries, so be aware of this, particularly when using SELECT *.

This syntax also results in a clustered index being created on the history table
using the SysStartTime and SysEndTime columns.

Observe that I’m taking advantage of the inline index creation syntax that was new in SQL Server 2014. Look at examples S, T, and U to see more information on how to use this feature.

CREATE TABLE dbo.Employee
(
ID INT IDENTITY(1,1) NOT NULL,
LastName VARCHAR(100) NOT NULL,
FirstName VARCHAR(75) NOT NULL,
JobTitle VARCHAR(50) NOT NULL,
BirthDate DATE NOT NULL,
HireDate DATE NOT NULL,
SysStartTime DATETIME2 GENERATED ALWAYS AS ROW START NOT NULL,
SysEndTime DATETIME2 GENERATED ALWAYS AS ROW END NOT NULL,
 PERIOD FOR SYSTEM_TIME (SysStartTime,SysEndTime),
CONSTRAINT PK_Employees PRIMARY KEY CLUSTERED (ID),
INDEX IX_LastName_FirstName (LastName,FirstName),

)
WITH (SYSTEM_VERSIONING = ON);

ALTER TABLE dbo.Employee ADD CONSTRAINT DF_SysStart DEFAULT SYSUTCDATETIME() FOR SysStartTime;
ALTER TABLE dbo.Employee ADD CONSTRAINT DF_SysEndTime DEFAULT SYSUTCDATETIME() FOR SysEndTime;

This allows me to create the index on Lastname, FirstName directly in the CREATE TABLE statement and not have to do that afterward in a lengthier syntax.

INDEX IX_LastName_FirstName (LastName,FirstName)

To clean up this example and try a slightly different approach, run this code:

ALTER TABLE dbo.Employee SET(SYSTEM_VERSIONING = OFF)
DROP TABLE dbo.Employee

 

How To Name the History Table

You can create a temporal table and name the history table so it is more human friendly. This syntax also results in a clustered index being created on the history table using the SysStartTime and SysEndTime columns.

If you want to be able to create a clustered columnstore index, or a different clustered index then there are 2 choices:
1. Use this syntax and then drop the clustered index and make the new clustered index.
2. Create both the temporal table and the history table and when you make the history table, and specify the indexes you want to create.

 

CREATE TABLE dbo.Employee
(
ID INT IDENTITY(1,1) NOT NULL,
LastName VARCHAR(100) NOT NULL,
FirstName VARCHAR(75) NOT NULL,
JobTitle VARCHAR(50) NOT NULL,
BirthDate DATE NOT NULL,
HireDate DATE NOT NULL,
SysStartTime DATETIME2 GENERATED ALWAYS AS ROW START NOT NULL,
SysEndTime DATETIME2 GENERATED ALWAYS AS ROW END NOT NULL,
 PERIOD FOR SYSTEM_TIME (SysStartTime,SysEndTime),
CONSTRAINT PK_Employees PRIMARY KEY CLUSTERED (ID),
INDEX IX_LastName_FirstName (LastName,FirstName),

)
WITH (SYSTEM_VERSIONING = ON(HISTORY_TABLE = dbo.EmployeeHistory));

ALTER TABLE dbo.Employee ADD CONSTRAINT DF_SysStart DEFAULT SYSUTCDATETIME() FOR SysStartTime;
ALTER TABLE dbo.Employee ADD CONSTRAINT DF_SysEndTime DEFAULT SYSUTCDATETIME() FOR SysEndTime;

DROP INDEX IX_EmployeeHistory ON dbo.Employeehistory;

CREATE CLUSTERED COLUMNSTORE INDEX IX_CC ON dbo.EmployeeHistory;

 

Indexing Decisions for Temporal Tables

Per MS Docs, if you are creating this table as a temporal table for the purpose of auditing row changes, then a clustered rowstore index on the  SysStartTime, SysEndtime and the PK columns of the temporal table is a good choice. Otherwise, a clustered columnstore index is a good choice for analytic queries.

How to Handle Data Retention for Temporal Tables

The simplest option for managing data retention on a history table is to use
the HISTORY_RETENTION_PERIOD = syntax. The available time units for history retention are DAYS, WEEKS, MONTHS, YEARS. SQL Server then uses this information to manage the deletion of data from the history table for you. notice below I have added HISTORY_RETENTION_PERIOD = 6 MONTHS to the syntax.

CREATE TABLE dbo.Employee
(
ID INT IDENTITY(1,1) NOT NULL,
LastName VARCHAR(100) NOT NULL,
FirstName VARCHAR(75) NOT NULL,
JobTitle VARCHAR(50) NOT NULL,
BirthDate DATE NOT NULL,
HireDate DATE NOT NULL,
SysStartTime DATETIME2 GENERATED ALWAYS AS ROW START NOT NULL,
SysEndTime DATETIME2 GENERATED ALWAYS AS ROW END NOT NULL,
 PERIOD FOR SYSTEM_TIME (SysStartTime,SysEndTime),
CONSTRAINT PK_Employees PRIMARY KEY CLUSTERED (ID),
INDEX IX_LastName_FirstName (LastName,FirstName),

)
WITH (SYSTEM_VERSIONING = ON
    (HISTORY_TABLE = dbo.EmployeeHistory,
     HISTORY_RETENTION_PERIOD = 6 MONTHS)
    );

ALTER TABLE dbo.Employee ADD CONSTRAINT DF_SysStart DEFAULT SYSUTCDATETIME() FOR SysStartTime;
ALTER TABLE dbo.Employee ADD CONSTRAINT DF_SysEndTime DEFAULT SYSUTCDATETIME() FOR SysEndTime;

Other options for handling data retention include stretch tables, which place the entire history table, or part of it, in Azure. Partitioning the history table an writing your own custom cleanup script are also options.

Next Steps To Take

  1. Look here for a list of other considerations and limitations.
  2. Think about the tables in your environment where this feature could be useful and do some experimenting.
  3. Talk to your business personnel about this feature and ask them if they know of tables where knowing how the data has changed over time would be helpful. In some cases, you might find that the business is sometimes asked questions about how or when their data changed. This feature is perfect for answering that question.
  4. If you would like help with anything in this post, or with something else related to SQL Server, reach out to me here, or on Twitter, and I’ll be glad to offer assistance.