Tag Archives: SCOM

SCOM 2016 – Agent Maintenance Mode

I was pleased to present a session about “Why we want you to migrate to SCOM 2016” with Bob Cornelissen at MMSMOA in Minneapolis. During this session I presented about SCOM 2016 Agent maintenance mode.

With System Center Operations Manager 2016 we finally have our scheduled maintenance mode and also the option to directly set the maintenance mode from the agent. We do not need custom management packs/solutions anymore to get this functionality. Everything is build in :-).

When I tried the Agent maintenance mode first, I followed this blog post. I want to share some details here:

  • Open PowerShell as Administrator, otherwise the module does not work. I tried to set permissions on the registry key, which is used, but that did not help. The module itself seems to check the permissions.
    2.AgentMaintenance-AdminError.PNG
  • You need to load the module with the full path, otherwise it is not working:
    import-module “C:\Program Files\Microsoft Monitoring Agent\Agent\MaintenanceMode.dll”
  • The Syntax to start maintenance mode is this:
    Start-SCOMAgentMaintenanceMode -Duration 10 -Reason ApplicationInstallation -Comment “Test”
    4.AgentMaintenance-SetMaint.PNG
    I recommend to use the PowerShell ISE as you then also get the entries for the reason.
  • The cmdlet writes the information into the registry.
    hklm:\Software\Microsoft\Microsoft Operations Manager\3.0\MaintenanceMode\Record
    8.AgentMaintenance-RegKey.PNG
  • When you try again to set the maintenance mode and the system is not in maintenance mode yet, then you get this feedback:
    5.AgentMaintenance-SetMaint-Pending
    Last Maintenance Mode Request pending. Try afterwards.
  • There is a rule running on the agent, which checks the registry key every 5 min.
    Agent Initiated Maintenance Mode Rule
    The rule runs a PowerShell script (ReadMaintenanceModeRegEntry,ps1), which writes a OperationsManager Event Log entry with the EventID 2222. After that appears, the maintenance mode should been set successfully.
    7.AgentMaintenance-Event2222
    After the maintenance mode is set, you get this message when you try to set it again during the maintenance mode:
    6.AgentMaintenance-ProbablyMaint
  • If you want to unset the maintenance mode and set a new one, then you can delete the registry key, unset the maintenance mode in the console and then set the maintenance mode again.
  • You can also easily read the key through PowerShell and give users who normally do not have registry access the possibility to check the current maintenance mode window.
    function Get-SCOMAgentMaintenanceMode {
    $mm=(Get-ItemProperty “hklm:\software\microsoft\microsoft operations manager\3.0\maintenancemode”).Record
    $split=$mm.split(“|”)
    write-host “Duration:”$split[0]”
    Reason:”$split[1]”
    User:”$split[2].split(“:”)[0]”
    Comment:”$split[2].split(“:”)[1]”
    StartTime:”$split[3]
    }
    Get-SCOMAgentMaintenanceMode

Have fun with it!

SCOM:Veeam VMware Collector: VMware connection data processing errors

This is a short blog post to give you some alert troubleshooting details.

The alert VMware connection data processing errors from the Veeam VMware management pack (8.x) needs a collector server and a Run As Account with permissions in VCenter. Permissions are described in this article.

Alert details:
“The Veeam VMware Collector service was not able to process data it received from the VMware target. Some data could be missing in the VMware API response or it could be a parsing error. Some performance data may not be published. The exact error received from Collector: VP050 Error encountered while retrieving performance metrics for a cluster on system name. The VMware API error returned was: ‘One or more errors occurred.’.”

The alert details talk about performance metrics, which confused me.

The Veeam logs (you get them through the Veeam Enterprise Console) show these details:

[VP120] retrieveDatastores failed, A: No ‘datastore’ objects returned for VC,
Collector cannot access a datastore at the vCenter.
Then, [VP038] buildInventory failed [system name]
Cannot build topology always for system name

And after that you seeing [VP050] performanceDaemon One or more errors occurred.

Possible root causes:

1. system does not have a datastore at all
2. no permissions => re check permissions (see link above)

The permissions need to be set on the top level of the cluster not below!

To check, if the system has a datastore, you can browse the following webpage: http://system name/mob or https://system name/mob. Login with the Run As Account.

veeamcontent

Then click the content link.

veeamrootfolder

Click the link beside the rootFolder entry.

veeamchildentry

Click the link beside the childEntitiy entry.

veeamdatastore

If you see one or more datastores in the list, then the permissions are correct.

You can also click on one of the datastore links to verify.

veeamdatastoredetails

When both topics are resolved (permissions and datastore access) then the alert should resolve itself.

SCOM: Sample Maintenance Mode MP works on SCOM 2016

With all the great changes related to Maintenance Mode in SCOM 2016 you probably only miss the possibility to easily set Maintenance Mode on the agent without the need of knowing the PowerShell script details.

My old Sample Maintenance Mode management pack can help you with this also on SCOM 2016. I have imported it into my SCOM 2016 test environment and set a server into maintenance mode through it without any problem.

It was required in the past, that you deploy the files separately to the agents to have the Splash Screen available. Now I have added the files to the Visual Studio solution and deploy them to c:\it\mom\mm. The solution also adds a shortcut to the default user desktop and to the public startup folder on Windows Server 2012 and above (also applies to the corresponding client versions). I have used the examples from David Allen’s blog post.

I have posted the sample sealed mpb file on github including also the whole solution.

To adjust the solution to your needs I recommend to change the text in the file OpsMgrMM.ps1 which runs the Splash Screen.
Also you can change the target directory in the DeployableFile.ps1:

$TargetDirectory = “C:\IT\MOM\MM”

You will find both files in the Resources folder.

When you have done your adjustments, then build the solution (seal the mps) and import the mpb file into your SCOM environment. If you had a previous version installed, then you will need to remove that first.

Have fun with it!

 

OMS Management Packs

If you ever have to author a SCOM management pack, which references Microsoft OMS resources, then it would be good, if you have the management packs to reference.

I was recently searching for the Microsoft.IntelligencePacks.Performance.mp and Microsoft.IntelligencePacks.Types.mp, but could only find the xml here. Sealing this management pack did not help as I needed the original MP (with the signature from Microsoft) for my management pack.
But SystemCenterCore.com provided the information that the MP is part of the Microsoft.IntelligencePack.Core.mpb. And the last hint came from a MVP colleague (thanks Stefan!).

You can find all OMS (formerly Advisor) management pack bundles on the SCOM Management Server which is already connected to OMS. They all get downloaded to C:\Windows\Temp.

omsmbps

Don’t get confused by the names, you can use them as they are. So, for my reference, I selected the Microsoft.IntelligencePack.Core_635779185101086268.mpb in Visual Studio and all included Management Packs appeared – also the Microsoft.IntelligencePacks.Types.mp :-).

 

 

Azure: Move OMS Resource Group to new Subscription

Sometimes you have to move Azure resources to a new subscription. One reason could be that your current subscription will end and you have a new one, which you want to use. Another reason could be that you need to migrate it to a subscription of someone else (demerger, etc.).

Anyhow if you do not want to loose the connection with your SCOM environment or your directly connected agents, how can I get this done? One way is always PowerShell, but I want to show you here, how you can do that within the Azure portal.

So open up http://portal.azure.com and Login.

Then go to Log Analytics and check which resource group you need to move.

oms

Now go to Resource Groups and select your resource group (here OMS).

In the Details view of the resource group you will have at the top the Move button.

OMSMove.JPG

You can now select all resources which belong to this resource group.

move

Select the new subscription and create a new resource group or select an existing one in the new subscription.

The problem which we get now is that the validation fails. Why? When you have Solutions selected in your OMS environment, which you normally have, then these cannot be moved. You need to remove them first and then the move works. The error gives the details:

Error: Resource move is not supported for resources that have plan. Resources are ‘Microsoft.OperationsManagement/solutions/ADAssessment(Lab),Microsoft.OperationsManagement/solutions/ADReplication(Lab),Microsoft.OperationsManagement/solutions/AgentHealthAssessment and tracking id is ‘xxx’. (Code: ResourceMoveFailed)

So open up OMS (http://oms.microsoft.com/oms) and remove your Solutions.

omsagenthealth

Click Remove.

When you check in the azure portal, then you will see that the resource is deleted after the solution is removed. Do that for all Solutions in your OMS resource.

At the end it should look like this, when you click Move again:

omsmove2

Do not miss to select the check box at the bottom.

Then click OK.

The old resource group will be empty when the move task is finished. So you can delete it afterwards.

You can then enable the OMS solutions again, now that all your OMS resources are in the new resource group.

The necessarity to remove the solutions from OMS can be a problem in productive environments, not for test environments. So be cautious there.

SCOM 2012: Monitor MSAs with the HP Storageworks MP 4.2.1

If you ever implemented the HP Storageworks Management Pack for System Center Operations Manager 2012 then you will find that it needs some improvements. Beside the missing monitor alerts and the view properties there is one other topic: it would require a better documentation about the prerequisites on the monitored systems.

I have recently implemented the HP Storageworks MSA Management Pack which is one part of this Storageworks solution and followed Chiyo’s blog to do the initial configuration. But I had problems to connect some MSAs to the Management Server (Error: “Unable to connect to the remote system” in the HP Storage Management Pack User Configuration Tool) and there was nothing in the Management Pack documentation about how to fix it.

Here is what I found out, what was missing:

  • The Firmware had to  be updated (Firmware TS240* or later is required for SMI-S)
  • SMI-S unencrypted had to be enabled (new setting in the newer Firmware)
  • The monitoring user (which you enter in the HP Storage Management Pack User Configuration Tool) needs SMI-S access permissions.

With that the error went away and SCOM started to monitor.

I hope this helps you too.

 

SCOM 2012: Updated SCUtils APC PDU MP V1.1

I already wrote about the SCUtils APC PDU Management Pack for SCOM 2012 in this post. That was about the version 1.0 which had problems with monitoring the PDUs from a Gateway Server.

SCUtils has now released the new Version 1.1, which enables the monitoring from Gateway Servers. I have tested it and it works. So download the latest version and implement it.

Here is what you need to do to get this working (directly taken out of the SCUtils APC PDU MP documentation):

1. Install the Operations Manager console on each gateway server that is a member of a resource pool for monitoring APC PDUs.

2. If the gateway server and management server are separated by a firewall, you have to adjust the firewall to open the following ports in both directions between the gateway server and its management server:

1. TCP 5723
2. TCP 5724

3. Prepare an account that is a member of Operations Manager Read-Only Operators group.

4. Copy SetAccountToRegistry.exe from the installation folder (the default path is ‘C:\Program Files\SCUtils\ SCUtils Management Pack for APC PDU’) to each gateway server that is a member of a resource pool for monitoring APC PDUs.

5. Run SetAccountToRegistry.exe using a local administrator account. Fill in all the required fields. And click on Test button.

Gateway.jpg

6. If the connectivity test has succeeded, click on Save button (the password will be encrypted before saving). Otherwise, reenter the information and try to test again.

7. Repeat the procedure on each gateway server that is a member of a resource pool for monitoring APC PDUs.

 

 

SCUtils APC PDU monitoring

I already wrote a blog post about the free SCUtils APC UPS monitoring pack a while ago. Now SCUtils provided the promised APC PDU Monitoring pack for System Center Operations Manager 2012 SP1/R2. The only disadvantage is that this one is with costs. You pay 716,14€ without VAT (852,21€ incl. VAT) per license which includes monitoring for 10 devices and you get one year support.

The management pack monitors two types of PDUs: PDU and PDU2, which are two different generations of APC PDUs. PDU objects are first generation devices of the AP7000 series. PDU2 objects are from the second generation AP8000 series, they have sensors included which are monitored also.

Here are some more details about the monitoring pack.

Included MPs:
SCUtils.AdminSettings.mp
SCUtils.APC.PDU.MP.mpb
SCUtils.APC.PDU.FirstDiscovery.Overrides.xml (only required to speed up the discoveries => changes frequency to 700 sec. You can remove that after all PDUs are discovered.)

The regular discoveries run ever 12 hours, that is ok. Most of the monitors run every 5 min, rules run every 10 min. All are enabled by default.

Monitors:
Monitors.JPG

Rules:Rules

The following folder/view structure gets created:
views

Diagram View:  (PDU Generation 1 => PDU2 Generation 2 would have additional objects)
diagram

You need to install the management pack on the management server and activate the license through a task. To do that go to the administration pane in the SCOM console and find the SCUtils Settings.
SCUtilsSettings

Select SCUtils Products Activation, then the Activation view is shown.
ActivationView

On the right side you have three Tasks:
ActivateTask
Check a license Task
Get unique ID for offline activation
If your console machine has internet access, then you can run the Activate task. Otherwise you can also use the Get Unique ID for an offline activation task to request the activation through email.

In the Activate task you override the LicenseKey and the CompanyName fields:ActivateTaskDetails

The management pack will work after activation.

I only tested PDU devices not PDU2.

This management pack closes the APC monitoring gap. So with both offered management packs (UPS/PDU) you can monitor your APC environment.

Attention! Version 1.0 of this MP only works if you monitor the devices through a management server not on a gateway server! A new version of this MP has been released, which also works on a Gateway Server.

SCOM 2012: Get Pool Member monitoring details

I recently had a problem that a custom rule was not running correctly, so I wanted to find out which of my SCOM 2012 Management Servers was running the All Management Servers Resource Pool instance, where the rule was targetted to.

I could not find a something which matched in the web so I contacted some of my great SCOM colleagues and got feedback from Kevin Holman with the correct solution. Thanks Kevin!

There are two tasks in SCOM already, which can give you more deatils about which Management Server takes care of which instance (class).

MSTasksI

I will show now where you find them and what you need to enter.

Both tasks you require the ID of the resource pool which handles the instances. In my case it is the ‘All Management Servers Resource Pool’.

To find that run the Operations Manager Shell and enter the command:

Get-SCOMResourcePool | FT Displayname, IDgetscomresourcepool

The output shows the resource pool names and the IDs. So copy the ID of the pool you need.

Then go to your Operations Manager Console.

Open the Management Servers State Dashboard view:
MSServerStateView

Select one of the Management Servers in the Management Server State section.

MSServerStateView2

Then run the task ‘Get the Pool Member Monitoring a Top Level Instance’.

Task1Details

Here you need to override the PoolId and the ManagedEntityID. In my case the IDs are both the same as I want to know which pool member of the All Management Servers Resource Pool manages the pool. In your case you perhaps want to know that for another class. You can find the ManagedEntityID of the class through the PowerShell command: (Get-SCOMClass -Displayname ‘xxx’).Id.

The output of the Task gives you the Management Server Name:

Task1Output

The second task has another approach. It gives you all top level instances which a Pool Member monitors.

So run the ‘Get Top Level Instances Monitored by a Pool Member’ task.

Task2Details

Here you only need to override the PoolId.

Task2Output

The output lists all classes monitored by the pool member you selected in the Management Server State view.

With that information you can now go on and troubleshoot the logs why things are not working correctly on that Management Server.

 

SCOM 2012: SCUtils APC Monitoring

It is already a while ago when I found out that there is a free management pack from SCUtils which monitors APC UPS devices, the SCUtils APC Monitoring Management Pack.
When I wanted to test it, I realized that it was only available for SCOM 2012 R2. So I contacted the support and asked if they can also provide a SP1 version for me.
And they really did it and were very responsive – a big plus!

So I was able to implement it in my test environment and checked it out.
Here are my findings.

The management pack is well designed. The bundle consists of two MPs:

SCUtils.APC.UPS.mpb
SCUtils.APC.UPS.Dashboard.xml

It monitors APC UPS devices and APC EMUs (environmental monitoring unit). APC PDUs are not covered yet, but the support promised, that this will be added in the near future.
All discoveries run on a 4 hour schedule, the rules every 5 min and the monitors between 5 and 15 min. That is ok.

It creates all necessary views, including a Diagram View:

APC Folder

UPS Diagram View

With the UPS Dashboard you get a good overview of your APC environment.

UPS Dashboard

Monitors:

APC Monitors

All monitors are enabled by default, but there are also overrides, which disable some EMU monitors:

APC Overrides

Rules:

APC Rules

Only one rule is disabled by default.

The MP has successfully detected the low battery runtime (8 min) and you can see that the Description, Path, Source is always very descriptive.APC Alert

They also added some nice reports:

APC Reports

So from what I see, it has all you need to monitor APC UPS devices. SCUtils promised to create a documentation for that MP bundle soon, but there is not really a lot you need to do to implement the management pack. The only thing is that you add the APC devices through the Network Monitoring to your environment and import the MPs. That’s it.
Very easy. And it is free at the moment.
I will only wait for the PDU monitoring to be added, then it will have all I want.

Information: I have created the Monitor, Rule and Report-Overview with MP Studio

Update: The APC PDU monitoring packs has been released. Here is my review.