Fixing Dead XenApp Servers automatically by script


My Citrix XenApp farm runs with XenApp 5 on Windows Server 2008 64bit servers. I found a strange issue on few XenApp Servers goes to Zombie mode. The server is pingable and operational. No users connected to this server. If you have specific application points to this server, Users get this error message:

It was not possible to launch the selected application.
Please contact your administrator
The Citrix client reported error: 12

If you run “QFarm /Load” on any one of the XenApp server in the farm, load results doesn’t list the zombie server.  ‘”Change Logon /Query” command result shows Logon is enabled. I read the available hot fixes and none of them fixes this issue.  smile_baringteeth

Easy fix is to restart “Citrix Independent Management architecture” aka IMA service on the problematic server. By restarting IMA service you are re-registering the server to the Farm. The load balancing algorithm starts accounting the server after IMA restart.

So I wrote the following script and run it by hour using Windows Scheduler on one of the XenApp server. This script sends emails details report to specified admins when it fixes the zombie server issue. Till I find a permanent fix from Citrix or Microsoft, This script is saving me from aches.

If you want to use this script, copy and paste this script to Notepad.  Look for “<<<< Modify the value here..” and change the appropriate values according to your company. Schedule it on one of your XenApp servers.

'======================================================================
' Name:      Fix-XenApp-Servers.vbs
' Purpose:   Detect if XenApp server is dropped from Load Balancing.
'            If Yes, Restart the Citrix IMA server and send email alert.
' Parameters: None
' Written By: Anand Venkatachalapathy
' Date Written: Feb 10th 2010
'====================================================================== 
 
Const IMAService = "Citrix Independent Management Architecture"
Dim WshShell, WshExec
Dim LoadData, emailSubject,emailTo,i
Public emailBody
Dim XenAppServers
 
XenAppServers = Array("CitrixSvr1","CitrixSvr2","CitrixSvr3")  '<<<<<<<<< Modify the value here
 
emailTo = "admin1@company.com;admin2@company.com"     '<<<<<<<<< Modify the value here
Set WshShell = WScript.CreateObject("WScript.Shell")
 
'Run qfarm /load command and get the results to LoadData variable
Set WshExec = WshShell.Exec("qfarm /load ") 
LoadData = LCase(WshExec.StdOut.ReadAll)
 
For i = 0 To UBound(XenAppServers)
    
    'Check if XENAPPxx server is missing
    If InStr(LoadData,LCase(XenAppServers(i))) = 0 Then
        WScript.Echo XenAppServers(i) & " is not responding"
        emailSubject = XenAppServers(i) & " was not responding, IMA service restarted"
        emailBody = XenAppServers(i) & " Server was not listed in QFarm /Load command as below" & vbCrLf
        emailBody = emailBody & LoadData & vbCrLf & vbCrLf & _
"Trying to fix by restarting IMA service to re-register to the farm" & vbCrLf & vbCrLf
        
        'Restart Citrix IMA service
        service_control XenAppServers(i),IMAService,"restart"
        
        Set WshExec = WshShell.Exec("qfarm /load ") 
        LoadData = LCase(WshExec.StdOut.ReadAll)
        
        emailBody = emailBody & "QFarm /Load results after IMA service restart" & vbCrLf
        emailBody = emailBody & LoadData
        emailBody = emailBody & vbCrLf & vbCrLf & "--------Written By: Anand Venkatachalapathy------------"
            
        'Send email to Citrix Admins
        Alert_Admins emailTo,emailSubject,emailBody
    End If
 
Next
 
'*** End of Script ***
 
 
 
 
 
 
'==============================================================================
' Name:      Service_Control
' Purpose:   Restart, Stop or Start a Service on a remote server 
' Input:     strComputer - Remote Server Name
'             sname - Service Display Name
'             purpose - has to be "stop" or "start" or "restart"
'==============================================================================
Sub Service_Control(strComputer,sname,purpose)
    'strComputer is the computer you want to connect to
    'sname is the service name
    'purpose is whether or not you want to start, stop or restart a service
    Dim delay, err_return
    delay = 20000    '20 seconds
    WScript.stdout.Write "*"
    Set objWMIService = GetObject("winmgmts:" _
        & "{impersonationLevel=impersonate}!\" & strComputer & "rootcimv2")
    Set colListOfServices = objWMIService.ExecQuery _
            ("Select * from Win32_Service where DisplayName = '" & sname & "'")
    For Each objService In colListOfServices
        'Just to double check we have the right service
        If objService.displayname = sname Then
            WScript.stdout.Write "*"
            If purpose = "stop" Or purpose = "restart" Then
                    emailBody = emailBody & "Trying to stop " & strComputer & "" & objService.name & vbCrLf
                    err_return = objService.stopservice
                    'service has dependencies, so we need to stop those first
                    If err_return = 3 Then
                        'GOTCHA - you have to use the service.name NOT service.displayname for this query!!
                        'even if you change Win32_Service.Name to Win32_Service.DisplayName
                        Set colServiceList2 = objWMIService.ExecQuery("Associators of " _
                           & "{Win32_Service.Name='" & objService.name & "'} Where " _
                                & "AssocClass=Win32_DependentService " & "Role=Antecedent" )
                        For Each objService2 in colServiceList2
                            emailBody = emailBody & "Trying to stop " & strComputer & "" & objService2.name & vbCrLf
                            objService2.StopService()
                            If Not objService2.state = "Stopped" Then
                                'you have to pause because the service wont start unless it is completely stopped
                                WScript.Sleep delay
                                service_control strComputer,sname,"stop"
                            Else
                                'WScript.Echo objService2.displayname & " has been stopped."
                                emailBody = emailBody & strComputer & "" & objService2.name & "has been stopped." & vbCrLf
                            End If
                        Next
                        'stop the original service that had dependencies
                        emailBody = emailBody & "Trying to stop " & strComputer & "" & objService.name & vbCrLf
                        objService.stopservice
                    Else
                        If Not objService.state = "Stopped" Then
                            'you have to pause because the service wont start unless it is completely stopped
                            WScript.Sleep delay
                            service_control strComputer,sname,"stop"
                        Else
                            'WScript.Echo objService.displayname & " has been stopped."
                            emailBody = emailBody & strComputer & "" & objService.displayname & " has been stopped." & vbCrLf & vbCrLf
                        End If
                    End If
                    'if restart was sent, start service after it has stopped
                    If purpose = "restart" Then service_control strComputer, sname, "start" End if
            Elseif purpose = "start" Then    
                    WScript.stdout.Write "*"
                    emailBody = emailBody & "Trying to start " & strComputer & "" & objService.name & vbCrLf
                    err_return = objService.startservice
                    If NOT err_return = 10 Then
                        'GOTCHA - you have to use the service.name NOT service.displayname for this query!!
                        'even if you change Win32_Service.Name to Win32_Service.DisplayName
                        Set colServiceList2 = objWMIService.ExecQuery("Associators of " _
                           & "{Win32_Service.Name='" & objService.name & "'} Where " _
                                & "AssocClass=Win32_DependentService " & "Role=Antecedent" )
                        For Each objService2 in colServiceList2
                            emailBody = emailBody & "Trying to start " & strComputer & "" & objService2.name & vbCrLf
                            objService2.StartService()
                            If Not objService2.state = "Running" Then
                                'you have to pause because the service wont start unless it is completely stopped
                                WScript.Sleep delay
                                service_control strComputer,sname,"start"
                            Else
                                WScript.Echo objService2.displayname & " has been started."
                                emailBody = emailBody & strComputer & "" & objService2.displayname & " has been started" & vbCrLf
                            End If
                        Next
                        
                        emailBody = emailBody & "Trying to start " &  strComputer & "" & objService.name & vbCrLf
                        err_return = objService.startservice
                        'service has dependencies, so we need to start those first
                        If err_return = 3 Then
                            'start the original service that had dependencies
                            emailBody = emailBody & "Trying to start " &  strComputer & "" & objService.name & vbCrLf
                            objService.startservice
                            'End If
                            If Not objService.state = "Running" then
                                'you have to pause because the service wont start unless it is completely stopped
                                WScript.Sleep delay
                                service_control strComputer,sname,"start"
                            Else
                                'WScript.Echo objService.displayname & " has been started."
                                emailBody = emailBody &  strComputer & "" & objService.displayname & " has been started." & vbCrLf
                            End If
                        End If
                    End If
                    'WScript.Echo objService.displayname & " has been started."
                    emailBody = emailBody &  strComputer & "" & objService.displayname & " has been started" & vbCrLf
            End If
        End If
    Next
 
End Sub 
'*** End of service_control sub ***
 
 
 
 
 
 
 
'======================================================================
' Name:      Alert_Admins
' Purpose:   Sends email 
' Input:     strTo - Who to send to, strSub - Subject of the message
'             strBody - Body of the message
' Output:    Sends the emails to the strTO recipients
'======================================================================
Sub Alert_Admins( strTo,strSub,strBody)
 
    'WScript.Echo strBody
 
    '* iMsg - holds CDO.Message object
    '* Flds - Enumeration for CDO SMTP object properties 
    '* iConf - holds CDO.Configuration
    Dim iMsg, Flds, iConf
 
    '* sSMTPServerName - SMTP Server Name
    Dim sSMTpServerName
 
    '* Assign corpml servers as SMTP server
    sSMTPServerName = "smtp.company.com"   '<<<<<<<<< Modify the value here
 
    '* Assign cdoSendUsingPort is set to 2, i.e., send using SMTP (25) port
    Const cdoSendUsingPort = 2
 
    '* Create CDO Objects and assign to variables
    Set iMsg = CreateObject("CDO.Message")
    Set iConf = CreateObject("CDO.Configuration")
    Set Flds = iConf.Fields
 
    '* Assign values to Flds class properties
 
    With Flds
        .Item("http://schemas.microsoft.com/cdo/configuration/sendusing") = cdoSendUsingPort
        .Item("http://schemas.microsoft.com/cdo/configuration/smtpserver") = sSMTPServerName
        .Item("http://schemas.microsoft.com/cdo/configuration/smtpconnectiontimeout") = 25  
        .Update
    End With
 
    '* Assign message properties and Send the mail
    With iMsg
        Set .Configuration = iConf
        .Fields("urn:schemas:httpmail:importance").Value = 2    'Setting Mail importance to High (2)
        .Fields.Update
        .To = strTo
 
        'Fake, but make-sense email FROM address
        .From = "XenApp-Fixer@company.com"               '<<<<<<<<< Modify the value here
 
        'Return Email address
        .Sender = "XenApp-Fixer@company.com"            '<<<<<<<<< Modify the value here
        .Subject = strSub
        .TextBody = strBody
    .Send
    End With
 
End Sub
'*** End of Alert_Admins Sub ***

Advertisements

4 thoughts on “Fixing Dead XenApp Servers automatically by script

  1. I just tried this on one of our XenApp 5 Win2K8 R1 x64 servers. It found a server that is not responding, however after that popup is shown, I get a Windows Script Host error (code 80070006) stating “The handle is invalid.” (Line 71 Char 5).

    Do you know what the issue may be?

  2. Not sure if this is what you meant, but I launched cmd via Run as administrator and this was the output (T3PTerm13 has logons disabled):

    D:\admin>xenapp-fixer.vbs
    Microsoft (R) Windows Script Host Version 5.7
    Copyright (C) Microsoft Corporation. All rights reserved.

    T3PTerm13 is not responding
    *D:\admin\XenApp-Fixer.vbs(72, 5) (null): 0x8004100E

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s