Server 2012: Open Procedure for Service failed. Performance data for this service will not be available.

A Windows 2012 Server (my System Center Operations Manager server to be precise), was throwing the same 7 errors every 8 or so seconds, for as far back as I could see in the Application log.  The errors looked like this:

The Open Procedure for service ".NETFramework" in DLL "C:Windowssystem32mscoree.dll" failed. Performance data for this service will not be available. The first four bytes (DWORD) of the Data section contains the error code.

In addition to the .NETFramework the following services were listed in the other 6 error messages:

  • BITS
  • ESENT
  • HealthService
  • Lsa
  • MOMConnector
  • WmiApRpl

After much fruitless researching I came across the solution in this article.  It is important to note, that this command will not work in PowerShell, you must run it from the command prompt.

Click Start, type cmd right click cmd.exe, and select Run as administrator.
At the prompt, type lodctr /r and press ENTER. This will repair the pointers (those are stored in the registry).

If you do run it in PowerShell you get the following error message.

Error: Unable to rebuild performance counter setting from system backup store, error code is 2PS

Granting Permissions to Operations Manager 2012 Application Advisor Console

First, a disclaimer:  I wasn’t around when this environment was setup, I was hired several months aftewards and when I found out that Operations Manager wasn’t being utilized at all I took it upon myself to change that.  So, that being said, this may not be applicable to your environment, but this is what worked for me.

The first issue I ran into was that I was able to login to the Application Advisor console from the Operations Manager server, but if I tried from my desktop, it would not allow me to login.  The application log on the Operations Manager server recorded the following:

Event code: 3005 
Event message: An unhandled exception has occurred. 
Event time: 9/19/2013 1:47:31 PM 
Event time (UTC): 9/19/2013 6:47:31 PM 
Event ID: 7636b3787b134b22a227e21ed0492861 
Event sequence: 7 
Event occurrence: 1 
Event detail code: 0 
 
Application information: 
    Application domain: /LM/W3SVC/1/ROOT/AppAdvisor-1-130240900348697107 
    Trust level: Full 
    Application Virtual Path: /AppAdvisor 
    Application Path: D:Program FilesSystem Center 2012Operations ManagerWebConsoleAppDiagnosticsAppAdvisorWeb 
    Machine name: MDC-SC-OPSMGR01 
 
Process information: 
    Process ID: 9596 
    Process name: w3wp.exe 
    Account name: IIS APPPOOLOperationsManagerAppMonitoring 
 
Exception information: 
    Exception type: WebException 
    Exception message: The request failed with HTTP status 401: Unauthorized.
   at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall)
   at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
   at Avicode.Intercept.SEManager.Core.Services.ReportingServices.ReportingService.ListChildren(String Item, Boolean Recursive)
   at Avicode.Intercept.SEManager.Core.Services.ReportingServices.ReportServiceProvider.Avicode.Intercept.SEManager.Core.Services.ReportingServices.IReportPopulate.PopulateList(State state, SemCore semCore)
   at Avicode.Intercept.SEManager.Core.Services.ReportingServices.StateAdapter.GetItemsInternal(Func`2 match)
   at Avicode.Intercept.SEManager.Core.Services.ReportingServices.StateAdapter.GetListDisplayedItems()
   at Avicode.Intercept.SEManager.WebViewer.Modules.ReportServiceSelector.InitListOfReports()
   at Avicode.Intercept.SEManager.WebViewer.Modules.ReportServiceSelector.InitOrUpdateParameters()
   at Avicode.Intercept.SEManager.WebViewer.Modules.ReportServiceSelector.Page_Load(Object sender, EventArgs e)
   at System.EventHandler.Invoke(Object sender, EventArgs e)
   at Avicode.Intercept.SEManager.WebViewer.SemBase.SemPage.OnLoad(EventArgs e)
   at Avicode.Intercept.SEManager.WebViewer.Pages.ReportServices.ReportServicePage.OnLoad(EventArgs e)
   at System.Web.UI.Control.LoadRecursive()
   at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

 
 
Request information: 
    Request URL: http://mdc-sc-opsmgr01/AppAdvisor/Pages/ReportService/ReportServicePageImpl.aspx?_r=&_c=g&_pg=96245735-b221-4540-8205-37c4ff5927df&_s=4EFF0730 
    Request path: /AppAdvisor/Pages/ReportService/ReportServicePageImpl.aspx 
    User host address: 10.104.21.35 
    User: DOMAINjacob.benson 
    Is authenticated: True 
    Authentication Type: Forms 
    Thread account name: IIS APPPOOLOperationsManagerAppMonitoring 
 
Thread information: 
    Thread ID: 7 
    Thread account name: IIS APPPOOLOperationsManagerAppMonitoring 
    Is impersonating: False 
    Stack trace:    at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall)
   at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
   at Avicode.Intercept.SEManager.Core.Services.ReportingServices.ReportingService.ListChildren(String Item, Boolean Recursive)
   at Avicode.Intercept.SEManager.Core.Services.ReportingServices.ReportServiceProvider.Avicode.Intercept.SEManager.Core.Services.ReportingServices.IReportPopulate.PopulateList(State state, SemCore semCore)
   at Avicode.Intercept.SEManager.Core.Services.ReportingServices.StateAdapter.GetItemsInternal(Func`2 match)
   at Avicode.Intercept.SEManager.Core.Services.ReportingServices.StateAdapter.GetListDisplayedItems()
   at Avicode.Intercept.SEManager.WebViewer.Modules.ReportServiceSelector.InitListOfReports()
   at Avicode.Intercept.SEManager.WebViewer.Modules.ReportServiceSelector.InitOrUpdateParameters()
   at Avicode.Intercept.SEManager.WebViewer.Modules.ReportServiceSelector.Page_Load(Object sender, EventArgs e)
   at System.EventHandler.Invoke(Object sender, EventArgs e)
   at Avicode.Intercept.SEManager.WebViewer.SemBase.SemPage.OnLoad(EventArgs e)
   at Avicode.Intercept.SEManager.WebViewer.Pages.ReportServices.ReportServicePage.OnLoad(EventArgs e)
   at System.Web.UI.Control.LoadRecursive()
   at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

Which, didn’t really tell me a lot.  However Googling variations of this error message lead me to this article on TechNet, which in turn led me to this one.  The fix really was that simple.  I changed the authentication methods on the ApplicationAdvisor IIS website and was able to login from my desktop.

AppAdvisorIIS

 

Now, for the second problem.  Several developers have access to the ApplicationDiagnostics console, and have no problems viewing the application information they have rights to.  However, when they try to login to the Application Advisor, they get the error message below, stating that they are unauthorized.

Event code: 3005 
Event message: An unhandled exception has occurred. 
Event time: 9/20/2013 10:12:42 AM 
Event time (UTC): 9/20/2013 3:12:42 PM 
Event ID: a51b81250ffa47bbbfec997d9f4dd951 
Event sequence: 11 
Event occurrence: 1 
Event detail code: 0 
 
Application information: 
    Application domain: /LM/W3SVC/1/ROOT/AppAdvisor-1-130241635383154655 
    Trust level: Full 
    Application Virtual Path: /AppAdvisor 
    Application Path: D:Program FilesSystem Center 2012Operations ManagerWebConsoleAppDiagnosticsAppAdvisorWeb 
    Machine name: MDC-SC-OPSMGR01 
 
Process information: 
    Process ID: 8024 
    Process name: w3wp.exe 
    Account name: IIS APPPOOLOperationsManagerAppMonitoring 
 
Exception information: 
    Exception type: SoapException 
    Exception message: The permissions granted to user 'DOMAINfirst.last' are insufficient for performing this operation. ---> Microsoft.ReportingServices.Diagnostics.Utilities.AccessDeniedException: The permissions granted to user 'DOMAINfirst.last' are insufficient for performing this operation.
   at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall)
   at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
   at Avicode.Intercept.SEManager.Core.Services.ReportingServices.ReportingService.ListChildren(String Item, Boolean Recursive)
   at Avicode.Intercept.SEManager.Core.Services.ReportingServices.ReportServiceProvider.Avicode.Intercept.SEManager.Core.Services.ReportingServices.IReportPopulate.PopulateList(State state, SemCore semCore)
   at Avicode.Intercept.SEManager.Core.Services.ReportingServices.StateAdapter.GetItemsInternal(Func`2 match)
   at Avicode.Intercept.SEManager.Core.Services.ReportingServices.StateAdapter.GetListDisplayedItems()
   at Avicode.Intercept.SEManager.WebViewer.Modules.ReportServiceSelector.InitListOfReports()
   at Avicode.Intercept.SEManager.WebViewer.Modules.ReportServiceSelector.InitOrUpdateParameters()
   at Avicode.Intercept.SEManager.WebViewer.Modules.ReportServiceSelector.Page_Load(Object sender, EventArgs e)
   at System.EventHandler.Invoke(Object sender, EventArgs e)
   at Avicode.Intercept.SEManager.WebViewer.SemBase.SemPage.OnLoad(EventArgs e)
   at Avicode.Intercept.SEManager.WebViewer.Pages.ReportServices.ReportServicePage.OnLoad(EventArgs e)
   at System.Web.UI.Control.LoadRecursive()
   at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

 
 
Request information: 
    Request URL: http://mdc-sc-opsmgr01/AppAdvisor/Pages/ReportService/ReportServicePageImpl.aspx?_r=&_c=g&_pg=3906150e-aac6-412b-bbe0-0a3d3122762e&_s=52B48066 
    Request path: /AppAdvisor/Pages/ReportService/ReportServicePageImpl.aspx 
    User host address: 10.104.9.49 
    User: DOMAINfirst.last 
    Is authenticated: True 
    Authentication Type: Forms 
    Thread account name: IIS APPPOOLOperationsManagerAppMonitoring 
 
Thread information: 
    Thread ID: 8 
    Thread account name: IIS APPPOOLOperationsManagerAppMonitoring 
    Is impersonating: False 
    Stack trace:    at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall)
   at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
   at Avicode.Intercept.SEManager.Core.Services.ReportingServices.ReportingService.ListChildren(String Item, Boolean Recursive)
   at Avicode.Intercept.SEManager.Core.Services.ReportingServices.ReportServiceProvider.Avicode.Intercept.SEManager.Core.Services.ReportingServices.IReportPopulate.PopulateList(State state, SemCore semCore)
   at Avicode.Intercept.SEManager.Core.Services.ReportingServices.StateAdapter.GetItemsInternal(Func`2 match)
   at Avicode.Intercept.SEManager.Core.Services.ReportingServices.StateAdapter.GetListDisplayedItems()
   at Avicode.Intercept.SEManager.WebViewer.Modules.ReportServiceSelector.InitListOfReports()
   at Avicode.Intercept.SEManager.WebViewer.Modules.ReportServiceSelector.InitOrUpdateParameters()
   at Avicode.Intercept.SEManager.WebViewer.Modules.ReportServiceSelector.Page_Load(Object sender, EventArgs e)
   at System.EventHandler.Invoke(Object sender, EventArgs e)
   at Avicode.Intercept.SEManager.WebViewer.SemBase.SemPage.OnLoad(EventArgs e)
   at Avicode.Intercept.SEManager.WebViewer.Pages.ReportServices.ReportServicePage.OnLoad(EventArgs e)
   at System.Web.UI.Control.LoadRecursive()
   at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

Which is really strange considering, they can authenticate to the Application Diagnostic console, and can login to the Operations Manager Web Console and view their application.  After spending quite a bit of time looking into SQL Server permissions, and the Application Advisor application pool, I added both the developers to the Operations Manager Report Security Administrators group and that magically solved the issue.

Creating Dynamic Groups in Operations Manager

We recently implemented the Systems Center suite and I volunteered to become the Operations Manager person.  To say the least it has been quite the learning experience so far.  I was trying to create a group with a dynamic membership using a regular expression and wasn’t having any luck so I posted on the System Center Central forums here and was able to resolve the issue.

Since the issue ended up being something really simple that showed my inexperience with Operations Manager I wanted to share it here.

We have VMWare Virtual Machines that are used by our work at home users.  These VM’s are created as needed and destroyed when a user logs off.  I wanted to override a monitor for all these VM’s because Operations Manager didn’t think they were shutting down properly and was throwing errors that we didn’t need to worry about.

I first tried to create a regular expression like this:

( Object is Windows Computer AND ( DNS Name Matches regular expression oma-vdi[0123456789][0123456789] ) AND True )

But that didn’t return anything.   After an hour of googling I posted on the message boards and eventually got my answer as to what was wrong.

VDI

 

 

As this screenshot shows (and as Alexey pointed out in the forums) the VM’s were not being discovered as Windows Computers as I assumed they were.  If I looked under Windows Computers in SCOM they were not listed.  When I changed the rule to this, everything worked fine:

( Object is Veeam VMware AND ( Display Name Matches regular expression oma-vdi[0-9][0-9] ) AND True )

Important lesson learned!