SCOM 2016 – Agent Maintenance Mode

I was pleased to present a session about “Why we want you to migrate to SCOM 2016” with Bob Cornelissen at MMSMOA in Minneapolis. During this session I presented about SCOM 2016 Agent maintenance mode.

With System Center Operations Manager 2016 we finally have our scheduled maintenance mode and also the option to directly set the maintenance mode from the agent. We do not need custom management packs/solutions anymore to get this functionality. Everything is build in :-).

When I tried the Agent maintenance mode first, I followed this blog post. I want to share some details here:

  • Open PowerShell as Administrator, otherwise the module does not work. I tried to set permissions on the registry key, which is used, but that did not help. The module itself seems to check the permissions.
    2.AgentMaintenance-AdminError.PNG
  • You need to load the module with the full path, otherwise it is not working:
    import-module “C:\Program Files\Microsoft Monitoring Agent\Agent\MaintenanceMode.dll”
  • The Syntax to start maintenance mode is this:
    Start-SCOMAgentMaintenanceMode -Duration 10 -Reason ApplicationInstallation -Comment “Test”
    4.AgentMaintenance-SetMaint.PNG
    I recommend to use the PowerShell ISE as you then also get the entries for the reason.
  • The cmdlet writes the information into the registry.
    hklm:\Software\Microsoft\Microsoft Operations Manager\3.0\MaintenanceMode\Record
    8.AgentMaintenance-RegKey.PNG
  • When you try again to set the maintenance mode and the system is not in maintenance mode yet, then you get this feedback:
    5.AgentMaintenance-SetMaint-Pending
    Last Maintenance Mode Request pending. Try afterwards.
  • There is a rule running on the agent, which checks the registry key every 5 min.
    Agent Initiated Maintenance Mode Rule
    The rule runs a PowerShell script (ReadMaintenanceModeRegEntry,ps1), which writes a OperationsManager Event Log entry with the EventID 2222. After that appears, the maintenance mode should been set successfully.
    7.AgentMaintenance-Event2222
    After the maintenance mode is set, you get this message when you try to set it again during the maintenance mode:
    6.AgentMaintenance-ProbablyMaint
  • If you want to unset the maintenance mode and set a new one, then you can delete the registry key, unset the maintenance mode in the console and then set the maintenance mode again.
  • You can also easily read the key through PowerShell and give users who normally do not have registry access the possibility to check the current maintenance mode window.
    function Get-SCOMAgentMaintenanceMode {
    $mm=(Get-ItemProperty “hklm:\software\microsoft\microsoft operations manager\3.0\maintenancemode”).Record
    $split=$mm.split(“|”)
    write-host “Duration:”$split[0]”
    Reason:”$split[1]”
    User:”$split[2].split(“:”)[0]”
    Comment:”$split[2].split(“:”)[1]”
    StartTime:”$split[3]
    }
    Get-SCOMAgentMaintenanceMode

Have fun with it!

Advertisements

SCOM:Veeam VMware Collector: VMware connection data processing errors

This is a short blog post to give you some alert troubleshooting details.

The alert VMware connection data processing errors from the Veeam VMware management pack (8.x) needs a collector server and a Run As Account with permissions in VCenter. Permissions are described in this article.

Alert details:
“The Veeam VMware Collector service was not able to process data it received from the VMware target. Some data could be missing in the VMware API response or it could be a parsing error. Some performance data may not be published. The exact error received from Collector: VP050 Error encountered while retrieving performance metrics for a cluster on system name. The VMware API error returned was: ‘One or more errors occurred.’.”

The alert details talk about performance metrics, which confused me.

The Veeam logs (you get them through the Veeam Enterprise Console) show these details:

[VP120] retrieveDatastores failed, A: No ‘datastore’ objects returned for VC,
Collector cannot access a datastore at the vCenter.
Then, [VP038] buildInventory failed [system name]
Cannot build topology always for system name

And after that you seeing [VP050] performanceDaemon One or more errors occurred.

Possible root causes:

1. system does not have a datastore at all
2. no permissions => re check permissions (see link above)

The permissions need to be set on the top level of the cluster not below!

To check, if the system has a datastore, you can browse the following webpage: http://system name/mob or https://system name/mob. Login with the Run As Account.

veeamcontent

Then click the content link.

veeamrootfolder

Click the link beside the rootFolder entry.

veeamchildentry

Click the link beside the childEntitiy entry.

veeamdatastore

If you see one or more datastores in the list, then the permissions are correct.

You can also click on one of the datastore links to verify.

veeamdatastoredetails

When both topics are resolved (permissions and datastore access) then the alert should resolve itself.

SCOM: Sample Maintenance Mode MP works on SCOM 2016

With all the great changes related to Maintenance Mode in SCOM 2016 you probably only miss the possibility to easily set Maintenance Mode on the agent without the need of knowing the PowerShell script details.

My old Sample Maintenance Mode management pack can help you with this also on SCOM 2016. I have imported it into my SCOM 2016 test environment and set a server into maintenance mode through it without any problem.

It was required in the past, that you deploy the files separately to the agents to have the Splash Screen available. Now I have added the files to the Visual Studio solution and deploy them to c:\it\mom\mm. The solution also adds a shortcut to the default user desktop and to the public startup folder on Windows Server 2012 and above (also applies to the corresponding client versions). I have used the examples from David Allen’s blog post.

I have posted the sample sealed mpb file on github including also the whole solution.

To adjust the solution to your needs I recommend to change the text in the file OpsMgrMM.ps1 which runs the Splash Screen.
Also you can change the target directory in the DeployableFile.ps1:

$TargetDirectory = “C:\IT\MOM\MM”

You will find both files in the Resources folder.

When you have done your adjustments, then build the solution (seal the mps) and import the mpb file into your SCOM environment. If you had a previous version installed, then you will need to remove that first.

Have fun with it!

 

OMS Management Packs

If you ever have to author a SCOM management pack, which references Microsoft OMS resources, then it would be good, if you have the management packs to reference.

I was recently searching for the Microsoft.IntelligencePacks.Performance.mp and Microsoft.IntelligencePacks.Types.mp, but could only find the xml here. Sealing this management pack did not help as I needed the original MP (with the signature from Microsoft) for my management pack.
But SystemCenterCore.com provided the information that the MP is part of the Microsoft.IntelligencePack.Core.mpb. And the last hint came from a MVP colleague (thanks Stefan!).

You can find all OMS (formerly Advisor) management pack bundles on the SCOM Management Server which is already connected to OMS. They all get downloaded to C:\Windows\Temp.

omsmbps

Don’t get confused by the names, you can use them as they are. So, for my reference, I selected the Microsoft.IntelligencePack.Core_635779185101086268.mpb in Visual Studio and all included Management Packs appeared – also the Microsoft.IntelligencePacks.Types.mp :-).

 

 

Azure: Move OMS Resource Group to new Subscription

Sometimes you have to move Azure resources to a new subscription. One reason could be that your current subscription will end and you have a new one, which you want to use. Another reason could be that you need to migrate it to a subscription of someone else (demerger, etc.).

Anyhow if you do not want to loose the connection with your SCOM environment or your directly connected agents, how can I get this done? One way is always PowerShell, but I want to show you here, how you can do that within the Azure portal.

So open up http://portal.azure.com and Login.

Then go to Log Analytics and check which resource group you need to move.

oms

Now go to Resource Groups and select your resource group (here OMS).

In the Details view of the resource group you will have at the top the Move button.

OMSMove.JPG

You can now select all resources which belong to this resource group.

move

Select the new subscription and create a new resource group or select an existing one in the new subscription.

The problem which we get now is that the validation fails. Why? When you have Solutions selected in your OMS environment, which you normally have, then these cannot be moved. You need to remove them first and then the move works. The error gives the details:

Error: Resource move is not supported for resources that have plan. Resources are ‘Microsoft.OperationsManagement/solutions/ADAssessment(Lab),Microsoft.OperationsManagement/solutions/ADReplication(Lab),Microsoft.OperationsManagement/solutions/AgentHealthAssessment and tracking id is ‘xxx’. (Code: ResourceMoveFailed)

So open up OMS (http://oms.microsoft.com/oms) and remove your Solutions.

omsagenthealth

Click Remove.

When you check in the azure portal, then you will see that the resource is deleted after the solution is removed. Do that for all Solutions in your OMS resource.

At the end it should look like this, when you click Move again:

omsmove2

Do not miss to select the check box at the bottom.

Then click OK.

The old resource group will be empty when the move task is finished. So you can delete it afterwards.

You can then enable the OMS solutions again, now that all your OMS resources are in the new resource group.

The necessarity to remove the solutions from OMS can be a problem in productive environments, not for test environments. So be cautious there.

SCOM 2012: Monitor MSAs with the HP Storageworks MP 4.2.1

If you ever implemented the HP Storageworks Management Pack for System Center Operations Manager 2012 then you will find that it needs some improvements. Beside the missing monitor alerts and the view properties there is one other topic: it would require a better documentation about the prerequisites on the monitored systems.

I have recently implemented the HP Storageworks MSA Management Pack which is one part of this Storageworks solution and followed Chiyo’s blog to do the initial configuration. But I had problems to connect some MSAs to the Management Server (Error: “Unable to connect to the remote system” in the HP Storage Management Pack User Configuration Tool) and there was nothing in the Management Pack documentation about how to fix it.

Here is what I found out, what was missing:

  • The Firmware had to  be updated (Firmware TS240* or later is required for SMI-S)
  • SMI-S unencrypted had to be enabled (new setting in the newer Firmware)
  • The monitoring user (which you enter in the HP Storage Management Pack User Configuration Tool) needs SMI-S access permissions.

With that the error went away and SCOM started to monitor.

I hope this helps you too.

 

SCOM 2012: Updated SCUtils APC PDU MP V1.1

I already wrote about the SCUtils APC PDU Management Pack for SCOM 2012 in this post. That was about the version 1.0 which had problems with monitoring the PDUs from a Gateway Server.

SCUtils has now released the new Version 1.1, which enables the monitoring from Gateway Servers. I have tested it and it works. So download the latest version and implement it.

Here is what you need to do to get this working (directly taken out of the SCUtils APC PDU MP documentation):

1. Install the Operations Manager console on each gateway server that is a member of a resource pool for monitoring APC PDUs.

2. If the gateway server and management server are separated by a firewall, you have to adjust the firewall to open the following ports in both directions between the gateway server and its management server:

1. TCP 5723
2. TCP 5724

3. Prepare an account that is a member of Operations Manager Read-Only Operators group.

4. Copy SetAccountToRegistry.exe from the installation folder (the default path is ‘C:\Program Files\SCUtils\ SCUtils Management Pack for APC PDU’) to each gateway server that is a member of a resource pool for monitoring APC PDUs.

5. Run SetAccountToRegistry.exe using a local administrator account. Fill in all the required fields. And click on Test button.

Gateway.jpg

6. If the connectivity test has succeeded, click on Save button (the password will be encrypted before saving). Otherwise, reenter the information and try to test again.

7. Repeat the procedure on each gateway server that is a member of a resource pool for monitoring APC PDUs.