My SCOM 2007 R2 agents are stopped talking to the RMS server from one fine day. I don’t remember doing anything on SCOM servers like no new management packs. The following event ids were getting my nerves. Servers show up a “Not monitored” and with blank circle icon.
Event ID 21016
OpsMgr was unable to set up a communications channel to (RMS FQDN) and there are no failover hosts. Communication will resume when (RMS FQDN) is both available and allows communication from this computer.
Event ID 20070
The OpsMgr Connector connected to (RMS FQDN), but the connection was closed immediately after authentication occurred. The most likely cause of this error is that the agent is not authorized to communicate with the server, or the server has not received configuration. Check the event log on the server for the presence of 20000 events, indicating that agents which are not approved are attempting to connect.
Event ID 21023
OpsMgr has no configuration for management group (managementgroupname) and is requesting new configuration from the Configuration Service.
Event ID 21042
Operations Manager has discarded 1 items in management group (managementgroupname) , which came from $$ROOT$$. These items have been discarded because no valid route exists at this time. This can happen when new devices are added to the topology but the complete topology has not been distributed yet. The discarded items will be regenerated.
Event ID 29106
The request to synchronize state for OpsMgr Health Service identified by "f9bc56f5-d69b-fb52-0788-792a86aec09d" failed due to the following exception "Microsoft.EnterpriseManagement.Common.DataAccessLayerException: Invalid column name SizeNumeric_486ADDDB_2EB8_819A_FA24_8F6AB3E29543 for query MTV_SelectProperty_5de7b548-657d-7794-52b4-2a828da0cfd1.
at Microsoft.EnterpriseManagement.Mom.DataAccess.QueryDefinition.GetColumnDefinitionBySourceColumnName(String sourceColumnName, Int32 resultSetIndex)
at Microsoft.EnterpriseManagement.Mom.DataAccess.QueryDefinition.GetColumnDefinitionBySourceColumnName(String sourceColumnName)
at Microsoft.Mom.ConfigService.DataAccess.DatabaseAccessor.QueryInstanceProperties(ReadOnlyCollection`1 instances)
at Microsoft.Mom.ConfigService.Engine.ConfigurationEngine.CommunicationHelper.StateSyncRequestTask.ConfigurationItems.Instances.CollectPublicProperties(ReadOnlyCollection`1 identities, IConfigurationDataAccessor dataAccessor)
at Microsoft.Mom.ConfigService.Engine.ConfigurationEngine.CommunicationHelper.StateSyncRequestTask.ConfigurationItems.ConfigurationItemCollection`2.CollectPublicProperties(IConfigurationDataAccessor dataAccessor)
at Microsoft.Mom.ConfigService.Engine.ConfigurationEngine.CommunicationHelper.StateSyncRequestTask.ConfigurationItems..ctor(StateContext stateContext, IConfigurationDataAccessor dataAccessor)
at Microsoft.Mom.ConfigService.Engine.ConfigurationEngine.CommunicationHelper.StateSyncRequestTask.CreateResponse(Managers managers)
at Microsoft.Mom.ConfigService.Engine.ConfigurationEngine.Managers.Synchronize(OnDoSynchronizedWork onDoSynchronizedWork)
at Microsoft.Mom.ConfigService.Engine.ConfigurationEngine.CommunicationHelper.StateSyncRequestTask.Execute(Managers managers)
at Microsoft.Mom.ConfigService.Engine.ConfigurationEngine.CommunicationHelper.StateSyncRequestTask.Run(Guid source, String cookie, Managers managers, IConfigurationDataAccessor dataAccessor, Stream stream, IConnection connection)".
I searched the internet..seems everyone had same event ids as above for different reasons. None of their solutions didn’t apply to my situation. I have seen solutions like,
– Restore the SQL server database
– Restore the Key for SQL server database
– Some complicated SQL query to find out some incompatible management packs (supposedly given by Microsoft PSS)
– Stopping SCOM Agent service (Systems Center Management) and Delete all folders under C:Program FilesSystem Center Operations Manager 2007Health Service State and start the agent service
– Enable Read permission for “Authenticated Users” on all OUs in Active Directory
– Make SCOM database as “Unrestricted Growth”
– Check Free Disk space on the SCOM server and Database Server
Solution that worked for me: Update the management packs.
1. Open the SCOM Console
2. Select to open “Administration” section
3. Right click on “Management Packs” and select “Import Management Packs”
4. Click Add and select “Add from Catalog…”
5. On the “View” drop down box, select “Updates available for installed management packs” and click “Search” button
6. Click Ok to download and apply the updated management packs.
And Voila! Suddenly all my agents are started talking…I see lot more alerts that I supposed see.