Server State using WLST in Oracle WebLogic

Hello! Dear Automation focused engineer we have this post about how to work simple server status list for a WebLogic domain. This logic can be build and executed for huge number of WebLogic or FMW Servers in Production helps to view all at a time hostnames, their corresponding states.   

Why we need WebLogic Server State? What all those states?

Knowing the status of the WebLogic Server  is one of the first troubleshoot steps performed by Middleware Administrator.  

Typical use cases include:

• Verifying server availability after maintenance activities
• Checking managed server status after deployments
• Monitoring cluster members during production incidents
• Validating startup and shutdown operations
• Automating health checks for large WebLogic environments
• Generating operational reports

This is the basic need when the all the servers are in bounced for production code move. This same script can be applicable for the pre-production or staging environment too. WLST provides the built-in methods, which gives the status of the Server instance or servers in a Cluster. Here we will deal with individual instance wise data.

WebLogic Server Life cycle state diagram

There are several ways to list out the servers. The simple way here you go with interactive way...

In the following example 

1. We are collecting all the list of servers present inside the WebLogic domain
2. The state function applied on each server as item  passed to it
3. repeat this step 2 until all server list ends

 

wls:/demodomain/serverConfig> x=ls('Servers',returnMap='true')
dr--   demoadm
dr--   demoms1
dr--   demoms2

wls:/demodomain/serverConfig> x
[demoadm, demoms1, demoms2]
wls:/demodomain/serverConfig> for i in x:
... state(i,'Server')
...
Current state of "demoadm" : RUNNING
Current state of "demoms1" : SHUTDOWN
Current state of "demoms2" : SHUTDOWN

Cluster listing

wls:/demodomain/serverConfig> c=ls('Clusters',returnMap='true')
dr--   clstr01

wls:/demodomain/serverConfig> c
[clstr01]
wls:/demodomain/serverConfig> state(c[0],'Cluster')

There are 2 server(s) in cluster: clstr01

States of the servers are
demoms1---SHUTDOWN
demoms2---SHUTDOWN

ServerLIfecycleRuntime Mbean tree

Using above shown MBean hierarchy we can fetch the all WebLogic domain server instance's states. If your production WebLogic domain consists of two digit (eg. 60 instances) or three digit number (eg. 120 instances) of managed server then, it is difficult to see all server’s state at once. Weblogic Administration console is unable to show all the servers in the domain on a single page. Navigating in between also a time eating process so think! think better way!! WLST has the solution.

To get the status of all servers in the domain can be obtained with the following steps

1. Connect to the WebLogic Admin Server
2. Fetch the Managed server list from the domainRuntime MBean
3. Iterate the loop and get the state of each Managed Server with ServerLifeCycle Runtime MBean
4. Repeat if required the step 3 as per the user input to Continue...
5. Finally if all desired output is visible then disconnect from the AdminServer and exit.

################################################## 
# This script is used to check the status of all WL instances including the admin
###########################################################

def conn():
    UCF='/path/.AdminScripts/userConfigFile.sec'
    UKF='/path/.AdminScripts/userKeyFile.sec'
    admurl = "t3://hostname:wlport"

    try:
        connect(userConfigFile=UCF, userKeyFile=UKF, url=admurl)
    except ConnectionException,e:
        print '\033[1;31m Unable to find admin server...\033[0m'
        exit()

def ServrState():
    print 'Fetching state of every WebLogic instance'
#Fetch the state of the every WebLogic instance
    for name in serverNames:
        cd("/ServerLifeCycleRuntimes/" + name.getName())
        serverState = cmo.getState()
        if serverState == "RUNNING":
            print 'Server ' + name.getName() + ' is :\033[1;32m' + serverState + '\033[0m'
        elif serverState == "STARTING":
            print 'Server ' + name.getName() + ' is :\033[1;33m' + serverState + '\033[0m'
        elif serverState == "UNKNOWN":
            print 'Server ' + name.getName() + ' is :\033[1;34m' + serverState + '\033[0m'
        else:
            print 'Server ' + name.getName() + ' is :\033[1;31m' + serverState + '\033[0m'
        quit()

def quit():
    print '\033[1;35mRe-Run the script HIT any key..\033[0m'
    Ans = raw_input("Are you sure Quit from WLST... (y/n)")
    if (Ans == 'y'):
        disconnect()
        stopRedirect()
        exit()
    else:
        ServrState()

if __name__== "main":
    redirect('./logs/Server.log', 'false')       
    conn()
    serverNames = cmo.getServers()
    domainRuntime()
    ServrState()

Smart Script

Recently I have online discussion with Dianyuan Wang, state of the Managed servers can be obtained with state() command. This function can be used in two ways: 

• To get individual managed server status you need to pass arguments as managed server name, type as 'Server'. 
• Other one is to get individual Cluster wise status. 

This can be achieved by passing two arguments cluster name and type as 'Cluster'. The following script will be illustrate the second option, which I found that shorten code that gives same script outcome as above script. It could be leverage your scripting thoughts it is like a plain vanilla form as shown below:

Note: Hope you follow the WLST Tricks & tips

try:
    connect(url = "t3://adminhostname:adminport")
except:
    print "Connection failed"
state('appclstr','Cluster')
state('web1clstr','Cluster')

...
state('webNclstr','Cluster')

-->

Extra Stroke of this new script is that prints how many servers available in each given cluster.

Server state with redirecting, re in WLST

Wang mailed me his situation clarity of explanation why he chosen state command. And how he resolved with Python script tricks here. Its a great learning for me so sharing with you when I saw same question in StackExchange today(28 April 2014) after 3 years!! "The reason I do not use (for now) domainConfig is because some how few Weblogic domains are not in a good state, and when I run the domainConfig command, it complains that it is not enabled. Hence the alternative way I've selected here is using state command. But it don't return the state. It always return None. But it prints out the state of the server. Here you go, Better way is capture that printing output to a file using WLST command redirect-stopRedirect and then, use the Python regular expression to extract the state of each server. The following is the Python snippet how I use the redirect:

 
# Fill your own connection details 
  serverlist=cmo.getServers()   
  for s in serverlist:   
    server_nm = s.getName()
    urldict[server_nm]='t3s://'+s.getListenAddress()+':'+str(s.getAdministrationPort())   
    #domainRuntime()
    #cd('ServerLifeCycleRuntimes/'+server_nm)
    fileName='/tmp/myserver_state.txt'
    redirect(fileName)
    state(server_nm,'Server')
    stopRedirect()
    f = open(fileName)
    try:
      for line in f.readlines():
        if re.search('Current state',line):
          status[server_nm]=line
    except:
      continue
 
  Ks = status.keys()
  for s in Ks:   
    if re.search('RUNNING',status[s]):
		try:
		connect(username,password,urldict[s])
		except:
		continue
		cd("/Servers/" + s)  


...

Troubleshooting Server State

Issue 1: Unable to Connect to Admin Server Error:

Connection refused

Verify: 

1.  Administration Server is running 
2. Hostname is reachable 
3. Port number is correct 
4. Firewall is not blocking connectivity 

Issue 2: Authentication Failure Error:

Authentication denied

Verify: 

1.  Username and password 
2. userConfigFile and userKeyFile 
3. Security realm configuration 

Issue 3: Server State Returns UNKNOWN 

 

Possible causes: Node Manager is not running Network communication issue Server startup failure Machine configuration mismatch

The WLST state command relies on Node Manager for state retrieval in many scenarios.

Best Practices 

Use encrypted userConfigFile and userKeyFile instead of hardcoded passwords.

Store WLST scripts in version control.

Schedule automated health checks using cron or enterprise schedulers.

Generate server-state reports before and after deployments.

Monitor cluster health in addition to individual server status.

Log output for auditing and troubleshooting purposes.

Conclusion

WLST provides a simple and powerful way to monitor WebLogic Server health and availability. Whether you need to check a single managed server, an entire cluster, or every server in a large domain, the WLST state() command offers a lightweight and automation-friendly solution. By combining WLST scripting with operational automation, WebLogic administrators can improve visibility, reduce manual effort, and accelerate troubleshooting across enterprise environments.

best regards! Dianyuan Wang

Here I request you please write back your experiencing with this posting looking ahead for your issues/ suggestions as comments.

Mail from WLST when abnormal state for Server

In most of production environments there could be possibility of a WebLogic server instances go on overload and crash or it could reach to a non RUNNING state due to many reasons. But if we know the state of that WebLogic server instance then we can act immediately for further harm to be predicated and confidently control that could happen  in that WebLogic domain.

Python network library smtplib in WLST

WLST supports multiple the network libraries such as

1. ftplib 
2. poplib 
3. imaplib 
4. smtplib 
5. telnetlib 

these are  network capabilities in their process, scripting we need to import the libraries and you can use the built-in functions. Before using this smtp libraries you need to check the SMTP service enabled on the machine.

Here I am trying to send the alert mail message when one of the WebLogic Managed Server goes to SHUTDOWN state or UNKNOWN state or some other state which is not RUNNING.

-->

Assuming that your machine have SMTP mail service must enabled. Check before creating this script on the box. Here in the following script you can replace the 'To' address value given as  pavanwla@MAILSERVER.com with mailing address of your supporting WebLogic Administrators (WLA) list by comma separation.

#======================================================
# Script File : StatusMail.py
# Author      : Pavan Devarakonda
# Updated on  : 29th April 2010
#======================================================
import smtplib
import time

From = "wla@WLSERVER.com"
To   =["pavanwla@MAILSERVER.com"]
Date = time.ctime(time.time())
URL='t3://'+get('ListenAddress')+':'+str(get('ListenPort'))

def getServerNames():
   domainConfig()
   return cmo.getServers()

def mailing(name, stat):
    serverRuntime()
    serverConfig()
    if stat == 'SHUTDOWN':
        Subject = ' major: '
    else:
        Subject= 'Critical:'
    Subject= Subject + 'The Weblogic server instance ' +name + ' is ' + stat

    Text='The Server ' +name +'   in the '+ stat+'  Listening at URL ' + URL
    Msg = ('From: %s\r\nTo: %s\r\nDate: \%s\r\nSubject: %s\r\n\r\n%s\r\n' %(From, To, Date, Subject, Text))
    s = smtplib.SMTP('YOURSMTP.DOMAIN.COM')
    rCode = s.sendmail(From, To, Msg.as_string())
    s.quit()

    if rCode:
        print 'Fail to send message...'

def serverStatus(server):
    cd('/ServerLifeCycleRuntimes/' +server)
    return cmo.getState()

def checkStatus():
    try:
        connect('username','******','t3://adminIP:AdminPort')
        serverNames= getServerNames()
        domainRuntime()
        for name in serverNames:
            print name        
            serverState = serverStatus(name)
            if serverState == "SHUTDOWN": 
                mailing(name, serverState)
            elif serverStat == 'UNKNOWN':
                mailing(name, serverState)
    except:
        mailing('AdminServer','Connection Fail')

if __name__== "main":
    redirect('./logs/status.log', 'false')
    checkStatus()
    print 'done'

To make you more comfortable here you need to update few lines 7, 8 10, 27, and 40. Hope you have idea what need to replace in these lines! for your environment.

Cronjob schedule

If you need this script need to run for every 30 mins you can schedule the following line into a shell script. map the shell script to crontab by giving the -e option. If got any trouble please write back to me :)

To test run this script in UNIX/Windows/MacOS/anyother...

prompt> java weblogic.WLST StatusMail.py

Keep posting your valuable comments and suggestions on this post.

References:

1. http://docs.python.org/library/smtplib.html
2. http://uthcode.sarovar.org/pyconindia2010/presentation.html

Cosmotising your WLST output

Here I am with new idea that colored output according to the state of the Server.

Hope most of you(WLA) knows WLST script for displaying Server status, my idea is when the server state is in SHUTDOWN state let it display in red color, STARTING state in YELLOW, and geen for RUNNING state and so on... colors you can specify as per your choice. 

I ran thru google for searching colors in python... it is simple logic print instruction in python putting UNIX shell code way. The color code need to be prefix in the line where color need to appear in the output.

My choice of WLST is that it should have a provision to rerun the script if user wish to run the script after a while. this makes less time to get output because re-initializing of WLST is not required 

CODE Snippet:

#Fetch the state of the every WebLogic instance

        for name in serverNames:

                cd("/ServerLifeCycleRuntimes/" + name.getName())

                serverState = cmo.getState()

                if serverState == "RUNNING":

                        print 'Server ' + name.getName() + ' is :\033[1;32m' + serverState + '\033[0m'

                elif serverState == "STARTING":

                        print 'Server ' + name.getName() + ' is :\033[1;33m' + serverState + '\033[0m'

                elif serverState == "UNKNOWN":

                        print 'Server ' + name.getName() + ' is :\033[1;34m' + serverState + '\033[0m'

                else:

                        print 'Server ' + name.getName() + ' is :\033[1;31m' + serverState + '\033[0m'