Monday, December 30, 2013

Happy New Year!!!

Hi all.

As you might have noticed this blog is a bit silent for the past few days. Why? Simply because I am enjoying a week of everything but System Center related issues, which is awesome.

From the 2nd of January I’ll be back and sharing some good things (I guess), so stay tuned.

For now I wish you all a Happy New Year!
Dragon
(This photo is taken by me on the 22nd of December at the entrance of this Chinese restaurant.)

Friday, December 20, 2013

Quick Fix: Register ASP.net 4.0 With IIS Windows Server 2012

Issue
Bumped into an issue where ASP.net 4.0 wasn’t registered with IIS running on Windows Server 2012. Before that Server OS the solution was an easy one: simply follow this posting of mine and all is fine again.

HOWEVER, Windows Server 2012 and later don’t support that anymore and the ONLY fix is removing IIS and reinstalling it with ASP.net 4.0. But that’s way too much and takes too much time, effort and resources.

Quick Fix
Thanks to Google I found two articles about how to fix this WITHOUT removing and reinstalling IIS:

    1. Open IIS Manager and select the webserver and select Modules (found under header IIS);
    2. Double click on it, so you open Modules, and remove the module ServiceModel;
    3. Go back to IIS Manager, select the webserver again in IIS, and select Handler Mappings (found under header IIS);
    4. Remove the handler svc-Integrated;.
    5. Restart IIS by using an elevated cmd prompt and issue this command: IISRESET <enter>;
    6. When IIS is running again add WCF by going to "Turn Windows Features On and Off" and enable .NET Framework 4.5 Features > WCF Services > HTTP Activation;
    7. Restart IIS by using an elevated cmd prompt and issue this command: IISRESET <enter>.

Now the SCOM 2012 Web Console will be fully functional WITHOUT reinstalling IIS Smile.


A big word of thanks to the authors of these two articles I used for this solution:

  1. http://stackoverflow.com/questions/15278027/cant-register-net-4-0-in-iis-on-windows-8
  2. http://www.codeproject.com/Articles/613812/Solve-IIS-8-Error-Could-not-load-type-System-Servi

OM12 Issue: SCOM Reports Not Updated & Not Working

Issue
Bumped into an issue where the latest MPs contained Reports which weren’t uploaded at all to SCOM Reporting. Took me a while to crack it but finally I found KB2771934 which was the starting point of the solution to this issue.

Causes & fixes
As it turned out there were multiple causes which joint forces Smile in order to frustrate the SCOM Reporting process in many kind of ways.

  1. The object Data Warehouse Synchronization Server was a goner. KB2771934 helped me in fixing that;
  2. The SCOM Data Warehouse Read account was lacking proper permissions. TechNet has some good articles about it, like this one and the online deployment guide Based on that information I set the permissions as required;
  3. There were also multiple time outs where the Management Servers couldn’t write their information to the Data Warehouse database or read from it (important for SCOM in order to know what Reports to upload to the SSRS instance). After some tweaking and tuning the SQL server got a bit more space to operate which reduced these errors enormously.

Afterwards the Reports were updated again: the old outdated Reports removed since their related MPs weren’t present anymore and the newest Reports, based on the MPs which were imported lately, showed up in SCOM Reporting.

To Do List When Adding Additional SCOM Management Server To Existing MG

When a SCOM Management Group (MG) is already in place for some time and an additional SCOM Management Server is added to it later on, their are quite a few steps one must do in order to get it working properly. When forgetting one of those steps it might result in a SCOM MG showing erratic behavior.

Mind you all these steps take place AFTER the new SCOM Management Server is installed. Also good to know, this To Do List is based on OM12 SP1.

A BIG word of thanks to Bob Cornellissen for updating this posting with many new items for this list.

  1. Antivirus exclusions
    Please make sure the new SCOM Management Server uses the same AV policy as the other SCOM Management Server. So the correct folders and processes are excluded from AV scans. Check KB975931 for more information.

  2. Certificates
    When using Gateway Servers and/or monitoring servers using certificates, make sure the new SCOM Management Server gets a valid certificate as well. And don’t forget to configure it properly.

  3. Firewall
    Make sure all the firewalls, either running on your Windows Server hosting the new SCOM Management Server role and the dedicated network firewalls, accept the traffic coming from the new SCOM Management Server. Also read this posting of my fellow MVP buddy Bob Cornelissen since it might prevent a lot of hassle.

  4. Resource Pools
    Make sure the new SCOM Management Server is added to the proper Resource Pools so it adheres to the original design.

  5. UNIX/Linux monitoring
    When monitoring UNIX/Linux systems and the new SCOM Management Server will become a member of that Resource Pool, make sure it has the proper certificates in place. Not only its own certificate but also the certificates of all the other Resource Pool members. Also the other Resource Pool members must get the certificate of the new SCOM Management Server as well. Kevin Holman wrote an excellent posting about it, to be found here. Look for the header Configure the Xplat certificates.

  6. Special MPs
    Sometimes special MPs are in place, requiring additional actions on the new SCOM Management Servers. Examples are the NetApp MP, SharePoint 2013 MP.

  7. Console extensions
    Some third party tools extend the SCOM Console, like Savision Live Maps. So install those Console extensions on the new SCOM Management Servers as well.

  8. Registry and/or config file modifications
    If you have implemented custom registry or config file settings on your management servers, don’t forget to implement those as well. Often it is advisable or required to have these settings the same on all management servers in the resource pool or management group.

  9. Run As Accounts & their related Run As Profiles
    It could be that certain runas accounts are set to more secure distribution and you had selected the initial Management Server(s). If so make sure you add the new Management Server as well to the distribution of such accounts.

  10. Custom scripts modifications
    If you are using custom scripts running on management servers for custom monitoring or command based notification channels, remember to copy those to the new management servers.

  11. Custom MP modifications
    You could have custom management packs, such as Backup Unsealed MP’s, which are set to backup to a directory on disk. In these kind of cases confirm that files and directories exist and that overrides which were set to target specific management servers are also applied to the new management servers if applicable. There could be some overrides you have made in management packs which target specific management servers. These need to be evaluated if those are needed on the new Management Server as well.

  12. Custom monitoring modifications
    Check other custom monitoring you have implemented that uses certain management servers as monitoring agents, such as web page checks. Of course only in case you want the new server to do the same kind of workflows, or if a new management server is eventually going to replace an existing one.

This covers it all and enables you to enroll successfully an additional SCOM Management Server to an existing MG without bumping into issues after it.

Tuesday, December 17, 2013

New Free eBook: Introducing System Center 2012 R2

Even though this book was announced to be released in January 2014 it’s already available
cover for Introducing Microsoft System Center 2012 R2

 

 

 

 

 

 

The book is downloadable as PDF, ePub and Mobi file. Want to know more and get if for free? Go here.

Thursday, December 12, 2013

Free eBooks All About System Center 2012

As we all know, Microsoft has released of FREE ebooks lately, all about the latest releases of their products. Many of those ebooks cover System Center 2012 and there are more to come.

What to think about these released ebooks all about System Center 2012 (R2):

Released Sept 2013
System Center Configuration Manager Field Experience
Released Oct 2013

Released Nov 2013

System Center Cloud Management with App Controller
Released Nov 2013

And there is even more in the pipeline:

  1. System Center: Optimizing Service Manager (coming in December 2013)
  2. System Center: Network Virtualization and Cloud Computing (coming in December 2013)
  3. System Center: Building a Virtualized Network Solution (coming in Jan 2014)
  4. System Center: Integrated Cloud Platform (coming in Jan 2014)
  5. Introducing Microsoft System Center 2012 R2 (coming in Jan 2014)

Remark:
This information is copied from this webpage.

SCOM & BizTalk: Some Good Resources

Today I had to roll out the BizTalk MPs for BizTalk 2010, 2009, 2006 R2 and 2006.

On itself this MP can be a bit of challenge to get it up and running. Also because BizTalk itself can be quite complicated on itself which makes monitoring it properly through SCOM a challenge. But when the MPs are in place and properly configured you’ll get a lot of information back from it. So it’s worth the effort.

It goes without saying that both guides for both BizTalk MPs require a lot of attention. These guides aid you in configuring them properly.

None the less, additional information is welcome and I am glad that I found two additional sources of information, all about monitoring BizTalk with SCOM. One source I found myself (thank you Google Smile) and another source was given to me by someone else.

Since these sources contain so much good information I want to share them with you on my blog:

  1. Microsoft Developer Network: Monitoring BizTalk Server
    Even though this site still talks about MOM 2005 or Operations Manager 2007 (duh!!!), this site still contains TONS of good relevant information. It also sheds good light on why certain components of BizTalk require additional attention.

  2. KB2449532: Common Issues and Resolutions with the BizTalk Server Administration console
    This KB article also takes monitoring with SCOM into account and goes very deep into detailed troubleshooting in order to get your BizTalk environment and SCOM working together as they’re supposed to be.

Personally I am happy with these two resources since they start where the two BizTalk MPs stop.

Updated MP: Windows Server 2012 R2 Message Queuing 6.3

Yesterday Microsoft released an updated version of the MSMQ MP, version 7.1.10100.0. This MP supports Microsoft Windows Server 2012 R2 Message Queuing 6.3.

MP can be downloaded from here.

Remark:
This MP works from SCOM 2012 RTM or higher!

Updated MP: Windows BranchCache MP for OM12

Yesterday Microsoft released the newest version of the Windows BranchCache MP for OM12, version 7.1.10100.0, which supports Windows Server 2012 R2.

MP can be downloaded from here.

Remark:
This MP works from SCOM 2012 SP1 or higher!

The Maze Of Update Rollup Packages For OM12 RTM & OM12 SP1

As we all have noticed, Microsoft is gaining more momentum per month resulting in new versions of the System Center 2012 product and the related components. On top of it all there are the Update Rollups which keep coming as well. Perhaps not on a the expected regular basis but still, there are many Update Rollups out there.

With all these different versions and their respective Update Rollups being available it’s easy to lose track of it all and even harder to get this simple question answered: What Update Rollup do I require for my current SCOM 2012 xyz environment?

Gladly Microsoft has published KB2906925, all about the available Update Rollup Packages for OM12 RTM and OM12 SP1. Per Update Rollup Package Microsoft tells what has changed (or not). So now you’re able to see on single webpage what Update Rollup your environment requires.

Remark!!!
Since we all know that the latest two to three monthly patch cycles contained foul updates/patches and that even some Update Rollups contained issues, I have taught myself a new Best Practice: To wait at least six weeks with applying the latest available Update Rollup so at least I have some certainty that this Update Rollup contains no ‘hidden features’…

Free Guide: SCOM 2012 R2 HA options with Server 2012 and SQL 2012

Fellow MVP Paul Keely has written an excellent guide all about the options available for deploying SCOM in a HA configuration using Microsoft technologies. He starts off with a base SCOM design, and then goes through clustering, SQL AlwaysOn, Stretched Clusters and HyperV replica.

For anyone involved with SCOM 2012 designs, deployments, stress testing and optimization this guide is a MUST HAVE. Simply because this guide is based on real life experiences and is totally free of marketing mumbo jumbo Smile.

A BIG word of thanks to Paul for sharing this kind of knowledge and experience. Awesome!!!

The guide can be downloaded from here.

Sunday, December 1, 2013

Updated MP: HP Storage MP

Some weeks ago I implemented the latest version of the HP Storage MP.

More then a year before that I had some serious issues with this MP since it contained too much noise, bad Discoveries and polluted the presentation in the SCOM Console. So I feared the worst.

HOWEVER… It turned out that HP had really listened to its customers since the MP has improved significantly. Examples: The noise is reduced, the MP is split up now, so you only get to see those items which are relevant to you and not – like before – presented with all the types of SANs HP delivers. Also the bad Discoveries are gone now.

So that’s a huge improvement and also the level of monitoring has become way much better. So compliments to HP for a job well done!

And now this MP is updated to the latest version in order to deliver full support for SCOM 2012 R2 and Windows Server 2012 R2 (mind you, I rolled out the previous version on both type of platforms and it worked without any issues at all but the documentation didn’t list it as supported environments Smile).

When you want to know more about this MP and what’s new, go here.

Again compliments for HP for delivering good MPs for monitoring their storage solutions with SCOM. And when you do have comments on their latest version of this MP know that they listen. Simply leave a good comment on the same website and they know.

Dell MP Suite Now Qualified For System Center 2012 R2

Taken directly from the website of Dell (thank you Aidan Finn Smile): “…We have qualified all the Management Pack suites listed below on the latest version of OpsMgr and updated the documentation for R2 support…”
  1. Dell Server MP Suite
    Discover, inventory and monitor Dell PowerEdge servers (agent-based option with Windows/OpenManage Server Administrator and agent-free option using WSMAN for 12th Generation of Dell PowerEdge Servers), Chassis Management Controllers using SNMP &  iDRACs using SNMP.  
    Download: version 5.1 , Documentation

  2. Dell Client MP
    Discover, inventory and monitor Dell Client PCs running Windows and OpenManage Client Instrumentation (OMCI)
    Download: version 5.0 , Documentation

  3. Dell Printer MP
    Discover, inventory and monitor Dell Printers using SNMP
    Download version 5.0, Documentation

  4. Dell MD Storage Array MP Suite
    Discover, inventory and monitor Dell PowerVault MD Storage arrays
    Download version 5.0, Documentation

  5. Dell EqualLogic MP Suite
    Inventory and monitor Dell EqualLogic storage arrays using SNMP
    Download version 5.0, Documentation

Please mind this:
The Chassis Modular Server Correlation feature of the Server MP Suite is not supported on R2
.

Free eBook: Cloud Management with App Controller

Some weeks ago Microsoft Press released a new FREE e-book, all about managing your cloud (s) with App Controller.
image

Content: ‘…App Controller is uniquely positioned as both an enabler and a self-service vehicle for connecting clouds and implementing the hybrid computing model. In Microsoft’s cloud computing solutions, both System Center and Windows Azure play critical roles…”

And: “…This book serves as an introduction to implementing and managing the hybrid computing solutions using App Controller. It describes the basic concepts, processes, and operations involved in connecting, consuming, and managing resources that are deployed both on and off premises...’

Book can be downloaded in various formats (PDF, ePub and MOBI) from here.

COMING SOON! Free eBook: Introducing Microsoft System Center 2012 R2

Expected in January 2014!

Microsoft Press will release a new FREE e-book all about Microsoft System Center 2012 R2.
image

Contents: ‘…Get a head start evaluating System Center 2012 R2 - with technical insights from a Microsoft MVP and members of the System Center product team. This guide introduces new features and capabilities, with scenario-based advice on how the platform can meet the needs of your business. Get the high-level overview you need to begin preparing your deployment now…’

The printed version can be bought at the same time the e-book becomes available through the normal channels like Amazon and O’Reilly for instance.

Wednesday, November 27, 2013

KB2775511 & Hotfix

As I blogged before KB2775511 causes issues on Windows Server 2008 R2 SP1 in conjunction with SCOM 2007/2012, whether it’s an Agent or the Management Server.

Some weeks ago Microsoft published a hotfix for this issue, also known as KB2878378. This KB provides a hotfix when you’re experiencing this issue.

Thursday, November 21, 2013

Troubleshooting EventID 33333

Prelude
Even though SCOM monitors itself, the Operations Manager event log on the SCOM Management Servers still tells a lot more. So periodically I go through those event logs on the SCOM Management Servers in order to check whether everything is okay.

Hello EventID 33333! Sad smile
This way I bumped into a SCOM 2012 R2 Management Server logging EventID 33333 way too many times:
image
I live by the credo: ‘…ONE event isn’t an event.’. or in other words, when a single event happens and the rest is okay, it was just a blurb and nothing more.

But this here tells me a different story, something isn’t going as planned. Time for some investigation.

Loving the event log
Seriously I do! Why? Because the events contain so much information. I have done a lot of troubleshooting and most of the times the Operations Manager event logs were the starting point of my investigations and also the clue to the solutions.

And in this case the event log helped me a lot since 99% of the EventID 33333 logged had the same sources in the description of the event:
image

As you can see the BaseManagedEntityId and MonitorId are logged, both with their GUIDs. Awesome! And yes, 99% of the events with EventID 33333 had the same GUIDs. So the cause was already pinned down to only ONE source and ONE monitor not functioning well. Awesome!

Sherlock Holmes or PowerShell?
Now it was time for some plain PowerShell commandlets in order to translate the GUIDs to understandable human language.

  1. In order to get a proper name for the GUID attached to BaseManagedEntityId I ran this PS cmdlet:
    Get-SCOMClassInstance -id: 'GUID' | ft DisplayName 
    (Replace GUID with the GUID for the BaseManagedEntityId shown in the event description.)

    This gave me the FQDN of the BaseManagedEntityId. It turned out to be a monitored Windows Server.

  2. In order to get a proper name for the GUID attached to MonitorID I ran this PS cmdlet:
    Get-SCOMMonitor -id: 'GUID' | ft DisplayName 
    (Replace GUID with the GUID for the MonitorID shown in the event description.)

    This gave me the name of the Monitor involved. In this case it was the Monitor System Center Management Health Service Memory Utilization.

Health Explorer
Time to open Health Explorer for that particular Windows server. Since it’s a Monitor targeted against the Agent, I opened the Health Explorer of the Agent of that Windows Server. And this is what I saw:
image

Yikes! Flip flopping! This Monitor is not doing well on this particular Windows Server. On all other monitored Windows Servers this Monitor runs just fine. I checked about 20 other servers in order to be sure, but on none of those servers this Monitor had issues. And the counter kept on growing….

So the culprit wasn’t the Monitor itself but the Windows Server.

The culprit
Time to start a RDP session with the Windows Server having issues with this Monitor. Also on this server I opened the Operations Manager event log. But all I got was this:
image

That’s not okay. But it could be the very same reason of flip flopping. Time to run a repair of the SCOM Agent running on this server:

  1. Go to Programs and Features > right click Microsoft Monitoring Agent > Change;
  2. Next > select the Repair option > Next;
  3. Now the Agent will be repaired > Finish.

After this repair job I could open the Operations Manager event log. And besides a few events it was empty and contained no errors.

On the Management Server side of things, the EventID 33333 stopped coming in from the moment the Agent on the Windows Server was repaired!

And in Health Explorer? The counter stopped. No more flip flopping!

Recap
Whenever you see an event (warning/critical) coming back in the Operations Manager event log on the Management Servers, changes are something is not okay.

Use those very same events as a starting point for your investigation and use PowerShell in order to get the understandable names of those GUIDs.

This way you obtain a lot of information within just a few minutes, aiding you in good old trouble shooting without ending up with a goose chase.

System Center 2012 R2 & Visio Stencils

My much respected Irish buddy Kevin Greene has a new hobby: creating Visio Stencils for many different components of the System Center 2012 R2 stack.

The score for now is already impressive since he has made Visio stencils for:

  1. DPM 2012 R2;
  2. SCOM 2012 R2 APM;
  3. SCOM 2012 R2 Infrastructure;
  4. SCOM 2012 R2 Network Monitoring;
  5. VMM 2012 R2.

And like the true community guy he is, he shares them with anyone who’s interested. So whenever you’re involved with one of these System Center 2012 R2 components, go here and GRAB those Visio stencils.

A BIG word of thanks to Kevin Greene for his effort AND willingness to share them with the public. Thanks man!|
image

Removing An OM12 Gateway Server With A Site Association

As I already blogged before, removing an OM12 Gateway Server with a Site association isn’t straight forward at all. It takes a lot of time and one has to go through non-supported steps in order to get things done.

Finally there’s some good news about it. Microsoft has fixed this issue with the release of SCOM 2012 R2. The tool Microsoft.EnterpriseManagement.GatewayApprovalTool.exe – included with the installation media of SCOM 2012 R2 – is rewritten for this purpose, so DON’T use previous versions of this tool.

On top of it, on the System Center: Operations Manager Engineering Blog a new article is posted, all about this topic and how to use this tool in different scenario’s.

Want to know more? Go here.

Tuesday, November 19, 2013

The Super Glue For Unleashed

Like all families there is always a single person who builds bridges and brings different minds, generations and opinions together. In many families this role is many times taken upon by the mother of the house. Somehow, somewhere she becomes the super glue bringing and keeping it all together, thus making it into a family.
image

When looking at the Unleashed series of books I many times see the same set of highly skilled and respected authors. Without them these books wouldn’t stand out that much like these books do now. They set the standard to such a height that other books are having a very hard time to get to the same level. And many times they simply don’t. None the less, these are still great books.

And yet, even in this kind of setup there is also a single person who brings all the different minds, opinions and levels of experience together in order to give you, the reader, the experience that the book is written by a single person/entity. So for those books she becomes the super glue, bringing it all together and creating a situation where 1 +1 isn’t 2 but becomes 2,5 or even 3!

For the Unleashed series of books that role is fulfilled by Kerrie Meyler. I know she isn’t the type of person to step up in the spotlights.  But IMHO she deserves a bit of extra attention. Because she’s the silent force behind many of the Unleashed books.

And – perhaps for one time only – she wrote a post about herself, all about how she got where she’s today. So for anyone who wants to gain a deeper understanding about the super glue of many of the books of the Unleashed series, this is THE posting to read.

Many times people ask me how I got so far in the community. One of the main reasons is that I learned a lot from people like Kerrie, Cameron, Pete, John, Anders, Marcus and so on. Compared to them I feel myself humble and am I just happy to know these people on a personal basis.

And you know what? Every single day I learn new stuff, and when having questions I many times find the answers in the Unleashed series, which closes the circle…

Free eBook: Introducing Windows Server 2012 R2 Technical Overview

Some days ago Microsoft published a free new eBook, titled: Introducing Windows Server 2012 R2 Technical Overview.
image

The eBook can be downloaded in various formats. The related download links can be found here.

Cross Post: Jonathan Almquist About Best Practices

In the SCOM community there are few BIG names which don’t need any introduction. IMHO Jonathan Almquist  is one of them. He knows a lot about SCOM and has deep knowledge and experience about the inner workings of SCOM and MP authoring.

Jonathan has started a whole new series of blog articles, all about Best Practices. He also explains why something is a Best Practice. Every single posting contains good information and should be read by anyone working with SCOM a daily basis.

Jonathan’s blog can be found here. Simply look for the postings starting with Best Practices. Thanks Jonathan for sharing. Awesome!
image

PKI Certificate Verification MP & Windows Server 2012 R2 & SCOM 2012 R2

There are some real pearls made by the community, and the PKI Certificate Verification MP is one of them. However, the last version (1.0.1.20) dates from March 20, 2012.

Since that date much has changed. We have seen new versions of Windows Server and SCOM 2012 as well. So how does this MP work with Windows Server 2012, Windows Server 2012 R2 and SCOM 2012 R2?

Based on my own experiences I can tell you this:

The latest version of the PKI Certificate Verification MP (version 1.0.1.20) works well with:

  • Windows Server 2012;
  • Windows Server 2012 R2;
  • SCOM 2012 R2.

The only trick you need is to set the proper overrides for the Discoveries you want to enable. These are the Discoveries:
image

Many times I only enable the Discovery related to the certificates residing in the personal computer store of the monitored servers, the Discovery of local computer's personal certificate store (registry).

In the ‘old days’ this Discovery was enabled by using the Objects Windows Server 2008 Computer or Windows Server 2003 Computer.

All you have to do now is to enable this Discovery against the Objects Windows Server 2012 Computer and/or Windows Server 2012 R2 Computer.
image

Soon the Discovered Certificate Stores will be shown in SCOM 2012 R2 and the related Certificates as well.

So I am glad to see this pearl of the community is still valid Smile.

Monday, November 18, 2013

New MP: SUSE Manager Management Pack

A few days ago SUSE launched something very special: the SUSE Manager Management Pack for SCOM, enabling Windows systems administrators to view server health information and perform both Windows and Linux patching duties via the same console.

This is really good news since it shows that finally SCOM is taken seriously by the much respected Linux community and software companies.

The SUSE Manager Management Pack for SCOM has some requirements:

  1. SUSE Manager version 1.2 or higher;

  2. SUSE Manager Management Pack;

  3. System Center Operations Manager 2007 R2 or later.

Want to know more? Go here.

Tuesday, November 12, 2013

SCOM 2012 Management Servers: Better One Or Two Too Many…

When the dino’s ruled the world
Back in the old days of SCOM 2007x one really had to consider how many SCOM 2007x Management Servers to roll out, mostly based on these two facts:
  1. Every single SCOM 2007x Management Server required a special SCOM license;
  2. Many environments were using physical hardware for their servers.

So any additional SCOM 2007x Management Server was an extra burden to the (many times already loaded) IT budgets. On top of these costs one had to consider the extra costs of the Server OS license as well.

In situations like these many times the minimum amount of required SCOM 2007x Management Servers was rolled out, never the optimum amount, resulting in an underperforming SCOM 2007x Management Group.

Back to the current situation
With the roll out of the System Center 2012 Product, Microsoft revamped the license model accordingly. This resulted in the related System Center 2012 Management Servers becoming free of System Center 2012 licenses since only the managed end points require a SC 2012 license.

So in the case of the SCOM 2012 Management Servers, these servers became free of the SC 2012 license (of course, when these servers are ‘touched’ by Orchestrator for instance, these servers do require a SC 2012 license…).

On top of it, virtualization of workloads had become default as well. So instead of rolling out physical hardware for servers, VMs were spawned as required. And when the underlying virtualization hosts are covered by a Data Center license for the Windows Server OS, the VMs running on top of those same virtualization hosts are covered as well by a Windows Server OS license. So no hidden costs!

Good to know, the SC 2012 license comes in two flavors: Standard and Data Center. Only difference is virtualization density. All components found in the System Center 2012 Product are covered by both licenses. Many times the SC 2012 Data Center license is the best solution since many virtualization hosts do run many VMs.

And now a second nice thing kicks in: with the Data Center flavor of the SC 2012 license, all VMs running on that same virtualization host are covered automatically with a SC 2012 license, no matter what SC 2012 based workload you run on them.

SQL for free?
And yes, SQL Server comes for free for the System Center 2012 Product when those SQL Servers only run SC 2012 based workloads AND the Standard edition of SQL Server is used.

So what?
The nice thing here is that you don’t have to be lean & mean anymore when rolling out SCOM 2012 Management Servers. So when your design tells you to roll out 3 of them, add an additional one. Yes, it will take resources like disk IO, CPU and RAM. But that goes for any other VM as well of which many new ones are deployed on a weekly basis.

But why? Because we can?
No. There is more to it. Now with SCOM 2012 you can monitor network devices way much better compared to SCOM 2007x. However, other MPs require SNMP to be present on at least one of the SCOM 2012 Management Servers.

For network monitoring SCOM 2012 uses a SNMP trap module of it’s own. And yes, the SNMP feature of Windows Server 2008 R2/2012 and that particular SCOM 2012 module don’t work well together.

In cases like these it’s better to use at least one dedicated SCOM 2012 Management Server, exclude it from network monitoring, and install the SNMP service on that server for those special MPs, like the SAN MP of HP for instance.

This way you know for sure these two components won’t bite each other, enabling you a more stable SCOM 2012 environment.

Recap
When designing a SCOM 2012 environment I use this new rule of thumb for the quantity of needed SCOM 2012 Management Servers:

(Required SCOM 2012 Management Servers based on quantity of monitored objects + future growth) + 1 additional SCOM 2012 Management Server = Total amount of SCOM 2012 Management Servers to be rolled out.

This way you know you have a SCOM 2012 environment in place which can be used in a smart manner with per Management Server a dedicated additional role, like communicating with the framework used by HP for monitoring their SAN solutions for instance.

For more information about the System Center 2012 Product license model, check out this series of postings I wrote about it.

Monday, November 11, 2013

Windows Event Log Monitoring: How To Get The Proper Event Source

First some background information
With SCOM it’s a straight forward process to create new Monitors/Rules which are triggered by certain Windows event log entries. However, you only want to trigger those Monitors/Rules on the correct type of event logged in the Windows event log since every false-positive Alert is one too many, breaking down the overall acceptance of SCOM and playing down the importance of the Alerts shown in the SCOM Console.

No NOISE please!
So the more filtering in that particular Monitor/Rule takes place, the better. This way the false-positives are skipped and only the Alerts which truly matter are triggered.

In the case of the Monitors/Rule which are targeted at a certain event in the Windows Event log you need to add additional filters on top of the most basic one which is the Event ID itself.

These additional filters can be used for instance:

  1. Event Source;
  2. EventDescription as described by Kevin Holman in this blog posting. Also take notion of his warning using this additional filter, stated in the same posting.

In SCOM these two additional Parameter Names (which are the filters) will be added and configured:
image

So far so good. But beware. The Parameter Name Event Source can be bit tricky. And when you don’t get it right from the start, NOT a single event will be captured by SCOM. Why? By default all these ‘filters’ (Parameter Names) are ANDed, so ALL of the filters must be met, or SCOM won’t pickup that event:
image

So when you get the Event Source wrong, not all filters in the AND group are met, thus causing SCOM to skip that particular event you want to be caught by SCOM in the first place…

What proper value to use
So what Value do you have to use for the Parameter Name Event Source in order to make it work as intended?

When you know it (duh!) it’s easy. First I want to show you what value NOT to use for Event Source:
image

For any given EventID you want SCOM to trigger an Alert or to capture it for Reporting purposes and you use the Event Source as well, you DON’T use the Source of that EventID as depicted in the screen dump shown above.

Even when you select the whole Source (which is in this example  Microsoft Windows security auditing., the dot included) SCOM won’t react at all.

Instead, open the EventID you want SCOM to act on and go to the second tab Details > select the option XML View > in the XML View go to Event > System > Provider Name:
image

The yellow highlighted entry is required for SCOM, which is in this example Microsoft-Windows-Security-Auditing. In SCOM the Value for Parameter Name Event Description will look like this:
image

And now we have the proper Event Source for SCOM which enables far more granular monitoring for certain EventIDs in the Windows Event Log.

Recap
When building Monitors/Rules in SCOM which are triggered by certain EventIDs, the more filtering is used the better. However keep Kevin Holman’s remarks in mind and when using the Event Source as one of the filers, make sure you use the proper Value for it. Otherwise that Monitor/Rule will fail to work.

Thursday, November 7, 2013

SCOM 2012 R2 Network Monitoring: Where Are The Alerts?!

First some background information
Okay, SCOM 2007 had some serious issues with network monitoring. So in SCOM 2012 this component got a complete overhaul and is rewritten from the ground up. And indeed, network monitoring in SCOM 2012 has improved compared to SCOM 2007. But to say it has really become top notch is a bit too much.

No, SCOM 2012 won’t replace the pure bred network monitoring tools. But guess what? Those tools will never replace SCOM 2012 as well. Ever. No matter what the marketing departments of those very same vendors want to make you to believe.

But when the network monitoring part of SCOM 2012 is put into perspective (SCOM 2012 monitors tons of work loads, whether it’s on-premise, cloud based, mobile units and from different angles, in- and outside) it’s okay. It’s has become an integrated part of the famous 360 degree monitoring. And for once I am on par with the marketing team of Microsoft because on this topic they tell the truth without any over estimation.

And now what?!
However, some things seem not to change and can still cause some strange issues. Suppose you have a brand new SCOM 2012 R2 RTM environment in place and everything is by the book. Many servers (Windows & Unix) are monitored and many different kind of workloads running on those very same servers. And yes, also many important network devices are being monitored.

And now one of those important monitored network devices goes down. In this case their were other monitoring solutions in place as well and they triggered the alarms. However, SCOM who’s monitoring that network device as well, stayed quiet. And now for a few minutes but for a long long time. And reported the network device to be HEALTHY!

Time to investigate
This really puzzled me so it was time for a deep dive into the way SCOM monitors network devices and alerts upon them. I agree, noise is bad but not Alerting when something is really amiss is even worse!

In Health Explorer of any given monitored network device you’ll find these two Unit Monitors:

  1. ICMP Ping
  2. SNMP Ping

These two Unit Monitors roll up to the Dependency Monitor Network Device Responsiveness, as seen in this screen dump:
image

So far so good. Both Unit Monitors are targeted against the Class Node, which is basically any monitored network device. However, per Unit Monitor there is an override in place which disables it.

The ICMP Ping Unit Monitor is disabled when the network device is covered by SNMP only, and the SNMP Ping Unit Monitor is disabled when the network device is covered by ICMP only. And this makes perfect sense.

But the configuration of those Unit Monitors really puzzled me.

Unit Monitor SNMP Ping
This Unit Monitor has some settings which I don’t fully understand. Let’s take a look at the Knowledge which describes this Unit Monitor in Health Explorer:
image

The options Interval and Number of Samples are most important here. First of all the Interval on this Unit Monitor isn’t 240 seconds in SCOM 2012 R2, but 300 seconds, which is 5 minutes. The Number of Samples is indeed set to three. Basically meaning any given monitored network device can be down for 15 minutes before SCOM 2012 R2 triggers an Alert!
image

Another thing which I am not happy with is the Health State when the network device doesn’t respond. It’s not set to Critical but to a Warning status:
image

However, when a network device goes down, I want it to be a Critical Alert, not a Warning. However, since this Unit Monitor (and the ICMP Ping Unit Monitor) roll up to a Dependency Monitor, which also triggers the Alert, this kind of modification shouldn’t be done on the Unit Monitor level.

So for the Unit Monitor SNMP Ping I set these two overrides:

  1. Interval: from 300 seconds to 30 seconds;
  2. Number of Samples: from 3 to 2.

So now this Unit Monitor will change State after a minute when a monitored Network Device is down:
image

Time to take a look at the second Unit Monitor, ICMP Ping.

Unit Monitor ICMP Ping
This Monitor is configured a bit differently compared to the SNMP Ping Unit Monitor. But still it needs some serious attention. This is what Health Explorer tells us:
image

So this Unit Monitor changes State after 6 minutes (Interval of 120 seconds x Number of Samples, 3) which is still too much. Also a Warning State is generated, not a Critical condition…

Time for some Overrides here as well. So now this Unit Monitor will change State after a minute when a monitored Network Device is down:
image

Time to move on to the Dependency Monitor, Network Device Responsiveness since I want a Critical Alert with Priority High (for the Notifications which sends out only New Alerts which are Critical and have Priority High).

These are the Overrides I set:
image

Time to test it
And now a new network device was added to SCOM to be monitored. This was a test network device. So when SCOM was monitoring it, the network cable was unplugged.

And YES! After a minute SCOM raised a Critical Alert with priority High. This Alert was neatly pushed out by the Notification Model as well. Awesome!

Recap
When you’re running SCOM 2012 R2 and are monitoring network devices, check the settings of the Monitors and make sure whether they match with the requirements of your organization. Changes are you have to make some modifications Smile.

Monday, November 4, 2013

Exchange Server 2010 MP: No Synthetic Transaction Tests Please

For many times I’ve imported and configured the Exchange Server 2010 MP. And now for the first time ever, for a particular use case, there are good reasons NOT to enable the Synthetic Transaction Tests, as described in the related MP guide on pages 14 and 15

However, even though the related MP guide is a big one, there is nowhere a description to be found about how to do that. Nor on the internet. So it was time for me to look for some solutions myself and soon I was disabling quite a few Rules and Monitors.

However, the Exchange 2010 MP has a whole different kind of operation so when disabling a Rule, the Monitor with the same name has to be disabled as well. When you don’t do that, changes are the related SCOM DB will get some serious issues. All thanks to the Correlation Engine, this marvelous wonder of code Smile

Even KB2592561 didn’t help at all. By the way, did you ever read that KB? It contains this sentence which makes my skin crawl: ‘…This is by far the largest MP to date from Microsoft, and provides a massive amount of visibility to Exchange issues.  However, there are just some things in the Management Pack that just don’t work…’
image

Still don’t know whether to laugh or to start crying here… However, I am wandering of now. Back to the topic of this blog posting now.

So here are the Rules and Monitors I disabled up to now. When there are more to come I’ll update this posting accordingly.

Rules: Test-OwaConnectivity

  • NonServiceImpacting: There was an Outlook Web App connectivity (External) transaction failure. The Test-OWAConnectivity cmdlet must be run on a Client Access server.
  • NonServiceImpacting: There was an Outlook Web App connectivity (Internal) transaction failure. The Test-OWAConnectivity cmdlet must be run on a Client Access server.
  • KHI: Exchange Control Panel connectivity (Internal) transaction failure - The test credentials can't be used to test the Exchange Control Panel.
  • KHI: Exchange Control Panel connectivity (External) transaction failure - The test credentials can't be used to test the Exchange Control Panel.
  • KHI: Failed to execute the Test-OWAConnectivity (Internal) diagnostic cmdlet.
  • KHI: Failed to execute the Test-OWAConnectivity (External) diagnostic cmdlet.
  • KHI: A directory error occurred while running the Test-MAPIConnectivity cmdlet.
  • KHI: An error occurred while executing the Test-OWAConnectivity (Internal) cmdlet.
  • KHI: An error occurred while running the Test-OWAConnectivity (External) cmdlet.
  • Script performance collection: Execute: Test-OwaConnectivity (External) diagnostic cmdlet. (Report Collection)
  • Script performance collection: Execute: Test-OwaConnectivity (Internal) diagnostic cmdlet. (Report Collection)

Rules: Test-ActiveSyncConnectivity

  • KHI: An error occurred while running the Test-ActiveSyncConnectivity (Internal) cmdlet.
  • KHI: Failed to execute the Test-ActiveSyncConnectivity (Internal) diagnostic cmdlet.
  • KHI: There was an Exchange ActiveSync connectivity (Internal) transaction failure. The Test-ActiveSyncConnectivity cmdlet must be run on a Client Access server.
  • Script performance collection: Execute: Test-ActiveSyncConnectivity (Internal) diagnostic cmdlet. (Report Collection)

Rules: Test-WebServicesConnectivity

  • KHI: An error occurred while running the Test-WebServicesConnectivity cmdlet in internal mode.
  • NonServiceImpacting: The internal Web Services connectivity transaction failed. The Test-WebServicesConnectivity cmdlet must be run on a Client Access server.
  • Script performance collection: Execute: Test-WebServicesConnectivity (Internal) diagnostic cmdlet

Rules: General

  • NonServiceImpacting: WebServices connectivity (Internal) transaction failure - The credentials can't be used to test Web Services.
  • WebServices connectivity (Internal) transaction failure - The credentials can't be used to test Web Services.
  • Some Client Access test cmdlets failed to run
  • KHI: An Outlook Web App connectivity (External) transaction failure occurred. The test credentials can't be used to test Outlook Web App.

Monitors: Test-OwaConnectivity

  • NonServiceImpacting: There was an Outlook Web App connectivity (External) transaction failure. The Test-OWAConnectivity cmdlet must be run on a Client Access server.
  • NonServiceImpacting: There was an Outlook Web App connectivity (Internal) transaction failure. The Test-OWAConnectivity cmdlet must be run on a Client Access server.
  • KHI: Exchange Control Panel connectivity (Internal) transaction failure - The test credentials can't be used to test the Exchange Control Panel.
  • KHI: Exchange Control Panel connectivity (External) transaction failure - The test credentials can't be used to test the Exchange Control Panel.
  • KHI: Failed to execute the Test-OWAConnectivity (Internal) diagnostic cmdlet.
  • KHI: Failed to execute the Test-OWAConnectivity (External) diagnostic cmdlet.
  • KHI: A directory error occurred while running the Test-MAPIConnectivity cmdlet.
  • KHI: An error occurred while executing the Test-OWAConnectivity (Internal) cmdlet.
  • KHI: An error occurred while running the Test-OWAConnectivity (External) cmdlet.

Monitors: Test-ActiveSyncConnectivity

  • KHI: An error occurred while running the Test-ActiveSyncConnectivity (Internal) cmdlet.
  • KHI: Failed to execute the Test-ActiveSyncConnectivity (Internal) diagnostic cmdlet.
  • KHI: There was an Exchange ActiveSync connectivity (Internal) transaction failure. The Test-ActiveSyncConnectivity cmdlet must be run on a Client Access server.

Monitors: Test-WebServicesConnectivity

  • KHI: An error occurred while running the Test-WebServicesConnectivity cmdlet in internal mode.
  • NonServiceImpacting: The internal Web Services connectivity transaction failed. The Test-WebServicesConnectivity cmdlet must be run on a Client Access server.

Monitors: General

  • NonServiceImpacting: WebServices connectivity (Internal) transaction failure - The credentials can't be used to test Web Services.
  • WebServices connectivity (Internal) transaction failure - The credentials can't be used to test Web Services.
  • Some Client Access test cmdlets failed to run
  • KHI: An Exchange ActiveSync connectivity (Internal) transaction failure occurred. The test credentials can't be used to test Exchange ActiveSync.
  • KHI: An Outlook Web App connectivity (External) transaction failure occurred. The test credentials can't be used to test Outlook Web App.
  • KHI: An Outlook Web App connectivity (Internal) transaction failure occurred. The test credentials can't be used to test Outlook Web App.

Yes, I know. Everybody is on Exchange Server 2013 by now or is using Office 365 Smile. But for the customers out there who’re still on Exchange Server 2010 (and that’s still a huge part…), this posting might come in handy when you don’t want to use the synthetic transactions.

HP MP For Blade Systems, Virtual Connect & Linux Systems: Where Are The Alerts?!

First of all, I want to compliment HP for the quality of their MPs. Seriously. The last few years HP has put a lot of effort into the overall quality of their MPs, the requirements and how they operate. And every new iteration showed progress and improvements.

In the last few weeks I have worked with the latest versions of the HP MPs for SCOM and I must really say, it has improved significantly. So that’s an awesome feat since we all know that overall quality of some other MPs delivered by other vendors isn’t that good at all which is a shame.

So this posting isn’t meant in any kind of way to bash HP. Instead I want to point out some challenges with the latest version of their MP targeted at monitoring ESX servers, Linux servers, Blade Systems, Virtual Connect and Agentless servers.

Challenges
The latest version of HP Insight Control 7.1 was in place, installed, imported and properly configured. Also the related Blade Systems and Linux servers were added. And soon enough these devices showed up in SCOM and got a status. Sweet!

So it was time for some tests. The system engineers went to the computer room and took out some hardware from the monitored Blade Systems and Linux Servers. And now something strange happened…

State Changes? Yes. Alerts? NO!
A bit late (?) SCOM started to show the related state changes. The time it took was far too long but nothing alarming. A properly configured override would take care of that issue. But what worried me was that no Alert what so ever showed up. Nothing. Zip. Nada! Time for some investigations.

No Noise please…
And this one really puzzled me. The related Monitor was set to generate an Alert, as this screen dump shows:
image

So why wasn’t the Alert being shown? SCOM itself was in an healthy state and Alerts for other monitored components, covered by other MPs still came in. So the cause was related to HP MP itself.

Time to check the overrides. And this one was a bit surprising. Since it turned out that ALL Monitors in the HP MP are set with an Override NOT to generate an Alert by default:
image

I don’t like noise for sure, but this kind of tuning is a bit too much when you ask me Smile. And no, none of the related guides for this MP tells you anything about this configuration…

Split brain scenario & Enforcing an Override
But this isn’t a nice situation at all since this MP has some configuration issues now which can be addressed but need some serious attention. Why? Well…

  1. The MP contains Monitors which by default generate Alerts;
  2. Out of the box these Monitors contain overrides which suppress this setting (Generate Alert: FALSE). And this Override is boxed in a Sealed MP, so it can’t be removed or edited directly;
  3. So an EXTRA Override is required (Generates Alert: TRUE).

However, with this option as described in Step 3 a new situation is born which is equivalent to the split brain scenario we had back in the days with the old failover clusters. There can only be one owner of the quorum any given time. But during disasters and their recoveries a situation can happen where two or even more nodes they think they’re the quorum owner. And this is even worse for your failover cluster.

With setting two Overrides on the same Parameter (Generates Alert), one time FALSE and the other time TRUE, SCOM doesn’t know what to do so it’s behavior becomes erratic. One time it will generate an Alert and the other time it won’t.

GLADLY, Microsoft had a very bright moment when they engineered SCOM 2007 RTM and from the beginning they added an extra option for setting Overrides: the ENFORCED option. Basically it means that for that particular Override, SCOM has to enforce it, no matter what other overrides for the same Monitor/Rule and Parameter of that very same Rule/Monitor are in place.

So when setting this Override I used the ENFORCED option like this:
image

While I was at it, I also changes the PeriodSeconds Parameter Name from 900 seconds (15 minutes) which is way too long, to 60 seconds, so this Alert would trigger an Alert far sooner. After these modifications the related Monitors looked like this:
image

And now the second test went far better: when the system engineers went out to pull out some disks or other hardware, SCOM showed a State Change within a minute AND the related Alert was also shown!

So for anyone having this MP in place, open the related hardware in the Health Explorer in SCOM and check one by one those Monitors. I’ll bet they have that Override in place, suppressing the Alerts. Now you know how to fix them, and when required also to make sure those very same Monitors run a bit more often…

Recap
Like I said before, HP has done a great job and delivers good MPs now. Still some additional tuning is required though, but when that’s in place, you have a good monitoring solution in place. And to be frank, I rather have MPs like this one (no noise) and the ability to tune them.

None the less, HP could do these two things:

  1. Document these Overrides so their consumers know about it;
  2. Put these Overrides in an additional Unsealed MP, so people can decide whether or not to import it.

And for the rest: RESPECT to HP!

Additional resources
There are some additional resources about this MP, how to import, configure and tune it:

  1. My respected fellow MVP Stanislav Zhelyazkov: https://cloudadministrator.wordpress.com/2012/08/12/configure-hp-bladesystem-management-pack-for-scom/
  2. And my own blog: http://thoughtsonopsmgr.blogspot.nl/2013/06/high-level-overview-hp-blade-monitoring.html 

Thursday, October 31, 2013

Hello Exchange Server 2010 MP. But WHERE Is The Correlation Engine?!

Had this strange issue.

The Exchange Server 2010 MP was installed and in the services snap-in for that particular account it showed up, titled Microsoft Exchange Monitoring Correlation. But when I logged on to the same server with my account and opened the same services snap-in, this service wasn’t to be found.

Logging on and off made no changes. The CE service wasn’t listed at all even though the process related to the services showed up in Task Manager. And YES, I was logged on to the proper server (triple checked it Smile).

So something else – and dumb I guessed – was at play here. Somehow the services snap-in for my account got a bit lost. As a last resort, before taking a deep dive, I tried this simple approach:

  1. Opened the services snap-in > File > Options and got this screen:
    image
  2. Hit the button Delete Files and got this warning:
    image
    > Yes, since it already looked like my profile was already damaged…
  3. > OK. Closed the services snap-in and opened it again. AND now the CE service was neatly shown:
    image

So whenever you bump into similar issues, simply clear the cache of the MMC and you’ll be fine. Almost sounds like clearing the cache of the SCOM Console, doesn’t it? SmileSmileSmile