ServiceNow – SCOM Connector – Overview

This is the first part of a series about ServiceNow Event Management with the SCOM Connector.

Part 2: SCOM Connector Setup

System Center Operations Manager is a great monitoring tool, but when you start to think about ITIL Event Management, then you realize that the second level event correlation is missing and the automatic integration into other ITIL processes like Incident, Request Fulfillment etc.

You probably think, but there is Service Manager, the ITSM tool from Microsoft, which can be integrated also with SCOM. Yes, but it only only has the direct event to ticket link and no additional event correlation.  Also you need a workflow engine like Orchestrator, SMA or Azure Automation to create the references, etc. A lot more separated components, which also need to be maintained.

ServiceNow is a cloud hosted service and one of the leaders in the ITSM area. It has an integrated workflow engine and the intention here is that you only need one console (ServiceNow).

Background

The Event Management application is part of the IT Operations Management area, which also covers CMDB Discovery, Service Mapping and Orchestration. With the Event Management application you connect one or more monitoring sources, which could also be emails sent to your ServiceNow instance, and create ServiceNow events out of it.

The SCOM Connector is only one option to connect external sources to the ServiceNow Event Management application. The SCOM connector takes the SCOM alerts and creates ServiceNow events out of them. The identifier is the SCOM Alert ID. With that it also can later identify which alert to close (bi-directional integration).

EventManagement Overview

You can see in the picture that ServiceNow takes the events and creates alerts out of them. The beauty of the alert is that it has a relation already to the Configuration Item, which is affected by this alert (Requirement: a filled and maintained CMDB).

ServiceNow has event rules to handle the incoming events:

  • Event rules can:
    • determine which events can be ignored
    • transform data from the events into fields from the alert
    • define how to map the correct Configuration Item for each alert

On top of the alerts there are multiple things that can happen:

1. Alert deduplication

Alerts with the same message key, will be correlated to one alert. This is really deduplication.

SCOM alerts have the SCOM Alert ID in the Message Key field of the ServiceNow alert. ServiceNow can automatically deduplicate (multiple events : one alert) on the same Message Key or the same Metric Name. The Message Key does not work or SCOM alerts, but the Metric Name works. If the events do not deduplicate correct to one alert, then check, if the Metric Name is filled. If not, then you can manipulate the Metric Name through an event rule (I would recommend to override it with the MonitoringObjectFullName).

2. Correlation through alert correlation rules

ServiceNow already tries to find alerts, which belong together based on machine learning, but you can create your own correlation rules. The correlation rule always defines, which one is the first alert, and which the second. Then you define what the relationship type is and in which timeframe the alerts should be correlated.

3. Alert flapping detection

Alert flapping detection is a general setting for all alerts, there are no rules, which can be defined. You configure the interval, frequency, quite interval and the minimum time in seconds to wait before an alert gets updated.

4. Alert action rules

With alert actions you can define if i.e. incidents should be created automatically for defined alerts. You can also link knowledge articles automatically or define recovery actions (this requires Orchestration).

I mentioned Configuration Items already, which shows that Event Management has a strong relation to Configuration Management and the CMDB. In order to leverage the full capabilities of Event Management, you will need to have a vital CMDB (minimum have all objects in your CMDB which you monitor in SCOM plus adding them to services => you will see why in the next section)

What do I get, when I connect SCOM to ServiceNow?

  1. One central alert console for all monitoring sources
    EventManagement AlertConsole
  2. ITIL automation
    – Central Knowledge repository
    – Automatic Incident creation and alert closure, when incident is closed (only with bi-directional integration checked)
  3. Service Dashboards
    EventManagement Dashboard
    You can define the services you have in your company. Through the relation between the alert and the CI you can see the affected services, where the CI belongs to.
    This dashboard only shows alerts, which are affecting the displayed services!
    EventManagement ServiceMap
    By double clicking one service, you get into the service map view
  4. Central Metric Dashboards, which visualizes the performance data, collected in SCOM (only available with Metric Collection, which needs to be licensed separately)

What is missing?

Alerts which are set into maintenance mode in SCOM are still transferred to ServiceNow, but they do not bring over the field “MonitoringObjectInMaintenanceMode”, therefore you cannot identify them.
I raised this problem with the Event Management product group and hope that they will fix that soon.

Important Notice
Do not try to use ServiceNow to improve your bad monitoring config! Try to avoid alerts in your SCOM environment and give the correct severity there.

 

 

 

Advertisements

SCOM:Veeam VMware Collector: VMware connection data processing errors

This is a short blog post to give you some alert troubleshooting details.

The alert VMware connection data processing errors from the Veeam VMware management pack (8.x) needs a collector server and a Run As Account with permissions in VCenter. Permissions are described in this article.

Alert details:
“The Veeam VMware Collector service was not able to process data it received from the VMware target. Some data could be missing in the VMware API response or it could be a parsing error. Some performance data may not be published. The exact error received from Collector: VP050 Error encountered while retrieving performance metrics for a cluster on system name. The VMware API error returned was: ‘One or more errors occurred.’.”

The alert details talk about performance metrics, which confused me.

The Veeam logs (you get them through the Veeam Enterprise Console) show these details:

[VP120] retrieveDatastores failed, A: No ‘datastore’ objects returned for VC,
Collector cannot access a datastore at the vCenter.
Then, [VP038] buildInventory failed [system name]
Cannot build topology always for system name

And after that you seeing [VP050] performanceDaemon One or more errors occurred.

Possible root causes:

1. system does not have a datastore at all
2. no permissions => re check permissions (see link above)

The permissions need to be set on the top level of the cluster not below!

To check, if the system has a datastore, you can browse the following webpage: http://system name/mob or https://system name/mob. Login with the Run As Account.

veeamcontent

Then click the content link.

veeamrootfolder

Click the link beside the rootFolder entry.

veeamchildentry

Click the link beside the childEntitiy entry.

veeamdatastore

If you see one or more datastores in the list, then the permissions are correct.

You can also click on one of the datastore links to verify.

veeamdatastoredetails

When both topics are resolved (permissions and datastore access) then the alert should resolve itself.

SCOM: Sample Maintenance Mode MP works on SCOM 2016

With all the great changes related to Maintenance Mode in SCOM 2016 you probably only miss the possibility to easily set Maintenance Mode on the agent without the need of knowing the PowerShell script details.

My old Sample Maintenance Mode management pack can help you with this also on SCOM 2016. I have imported it into my SCOM 2016 test environment and set a server into maintenance mode through it without any problem.

It was required in the past, that you deploy the files separately to the agents to have the Splash Screen available. Now I have added the files to the Visual Studio solution and deploy them to c:\it\mom\mm. The solution also adds a shortcut to the default user desktop and to the public startup folder on Windows Server 2012 and above (also applies to the corresponding client versions). I have used the examples from David Allen’s blog post.

I have posted the sample sealed mpb file on github including also the whole solution.

To adjust the solution to your needs I recommend to change the text in the file OpsMgrMM.ps1 which runs the Splash Screen.
Also you can change the target directory in the DeployableFile.ps1:

$TargetDirectory = “C:\IT\MOM\MM”

You will find both files in the Resources folder.

When you have done your adjustments, then build the solution (seal the mps) and import the mpb file into your SCOM environment. If you had a previous version installed, then you will need to remove that first.

Have fun with it!

 

OMS Management Packs

If you ever have to author a SCOM management pack, which references Microsoft OMS resources, then it would be good, if you have the management packs to reference.

I was recently searching for the Microsoft.IntelligencePacks.Performance.mp and Microsoft.IntelligencePacks.Types.mp, but could only find the xml here. Sealing this management pack did not help as I needed the original MP (with the signature from Microsoft) for my management pack.
But SystemCenterCore.com provided the information that the MP is part of the Microsoft.IntelligencePack.Core.mpb. And the last hint came from a MVP colleague (thanks Stefan!).

You can find all OMS (formerly Advisor) management pack bundles on the SCOM Management Server which is already connected to OMS. They all get downloaded to C:\Windows\Temp.

omsmbps

Don’t get confused by the names, you can use them as they are. So, for my reference, I selected the Microsoft.IntelligencePack.Core_635779185101086268.mpb in Visual Studio and all included Management Packs appeared – also the Microsoft.IntelligencePacks.Types.mp :-).

 

 

SCOM 2012: Monitor MSAs with the HP Storageworks MP 4.2.1

If you ever implemented the HP Storageworks Management Pack for System Center Operations Manager 2012 then you will find that it needs some improvements. Beside the missing monitor alerts and the view properties there is one other topic: it would require a better documentation about the prerequisites on the monitored systems.

I have recently implemented the HP Storageworks MSA Management Pack which is one part of this Storageworks solution and followed Chiyo’s blog to do the initial configuration. But I had problems to connect some MSAs to the Management Server (Error: “Unable to connect to the remote system” in the HP Storage Management Pack User Configuration Tool) and there was nothing in the Management Pack documentation about how to fix it.

Here is what I found out, what was missing:

  • The Firmware had to  be updated (Firmware TS240* or later is required for SMI-S)
  • SMI-S unencrypted had to be enabled (new setting in the newer Firmware)
  • The monitoring user (which you enter in the HP Storage Management Pack User Configuration Tool) needs SMI-S access permissions.

With that the error went away and SCOM started to monitor.

I hope this helps you too.

 

SCOM 2012: Updated SCUtils APC PDU MP V1.1

I already wrote about the SCUtils APC PDU Management Pack for SCOM 2012 in this post. That was about the version 1.0 which had problems with monitoring the PDUs from a Gateway Server.

SCUtils has now released the new Version 1.1, which enables the monitoring from Gateway Servers. I have tested it and it works. So download the latest version and implement it.

Here is what you need to do to get this working (directly taken out of the SCUtils APC PDU MP documentation):

1. Install the Operations Manager console on each gateway server that is a member of a resource pool for monitoring APC PDUs.

2. If the gateway server and management server are separated by a firewall, you have to adjust the firewall to open the following ports in both directions between the gateway server and its management server:

1. TCP 5723
2. TCP 5724

3. Prepare an account that is a member of Operations Manager Read-Only Operators group.

4. Copy SetAccountToRegistry.exe from the installation folder (the default path is ‘C:\Program Files\SCUtils\ SCUtils Management Pack for APC PDU’) to each gateway server that is a member of a resource pool for monitoring APC PDUs.

5. Run SetAccountToRegistry.exe using a local administrator account. Fill in all the required fields. And click on Test button.

Gateway.jpg

6. If the connectivity test has succeeded, click on Save button (the password will be encrypted before saving). Otherwise, reenter the information and try to test again.

7. Repeat the procedure on each gateway server that is a member of a resource pool for monitoring APC PDUs.

 

 

SCUtils APC PDU monitoring

I already wrote a blog post about the free SCUtils APC UPS monitoring pack a while ago. Now SCUtils provided the promised APC PDU Monitoring pack for System Center Operations Manager 2012 SP1/R2. The only disadvantage is that this one is with costs. You pay 716,14€ without VAT (852,21€ incl. VAT) per license which includes monitoring for 10 devices and you get one year support.

The management pack monitors two types of PDUs: PDU and PDU2, which are two different generations of APC PDUs. PDU objects are first generation devices of the AP7000 series. PDU2 objects are from the second generation AP8000 series, they have sensors included which are monitored also.

Here are some more details about the monitoring pack.

Included MPs:
SCUtils.AdminSettings.mp
SCUtils.APC.PDU.MP.mpb
SCUtils.APC.PDU.FirstDiscovery.Overrides.xml (only required to speed up the discoveries => changes frequency to 700 sec. You can remove that after all PDUs are discovered.)

The regular discoveries run ever 12 hours, that is ok. Most of the monitors run every 5 min, rules run every 10 min. All are enabled by default.

Monitors:
Monitors.JPG

Rules:Rules

The following folder/view structure gets created:
views

Diagram View:  (PDU Generation 1 => PDU2 Generation 2 would have additional objects)
diagram

You need to install the management pack on the management server and activate the license through a task. To do that go to the administration pane in the SCOM console and find the SCUtils Settings.
SCUtilsSettings

Select SCUtils Products Activation, then the Activation view is shown.
ActivationView

On the right side you have three Tasks:
ActivateTask
Check a license Task
Get unique ID for offline activation
If your console machine has internet access, then you can run the Activate task. Otherwise you can also use the Get Unique ID for an offline activation task to request the activation through email.

In the Activate task you override the LicenseKey and the CompanyName fields:ActivateTaskDetails

The management pack will work after activation.

I only tested PDU devices not PDU2.

This management pack closes the APC monitoring gap. So with both offered management packs (UPS/PDU) you can monitor your APC environment.

Attention! Version 1.0 of this MP only works if you monitor the devices through a management server not on a gateway server! A new version of this MP has been released, which also works on a Gateway Server.