Search This Blog

Sunday, February 12, 2023

Server State using WLST in Oracle WebLogic

Hello! Dear Automation focused engineer we have this post about how to work simple server status list for a WebLogic domain. This logic can be build and executed for huge number of WebLogic or FMW Servers in Production helps to view all at a time hostnames, their corresponding states.   

Why we need WebLogic Server State? What all those states?

Knowing the status of the WebLogic Server  is one of the first troubleshoot steps performed by Middleware Administrator.  

Typical use cases include:

  • Verifying server availability after maintenance activities
  • Checking managed server status after deployments
  • Monitoring cluster members during production incidents
  • Validating startup and shutdown operations
  • Automating health checks for large WebLogic environments
  • Generating operational reports
This is the basic need when the all the servers are in bounced for production code move. This same script can be applicable for the pre-production or staging environment too. WLST provides the built-in methods, which gives the status of the Server instance or servers in a Cluster. Here we will deal with individual instance wise data.

Image of ServerLifeCycle in WLST
WebLogic Server Life cycle state diagram

There are several ways to list out the servers. The simple way here you go with interactive way...
In the following example 
  1. We are collecting all the list of servers present inside the WebLogic domain
  2. The state function applied on each server as item  passed to it
  3. repeat this step 2 until all server list ends
 
wls:/demodomain/serverConfig> x=ls('Servers',returnMap='true')
dr--   demoadm
dr--   demoms1
dr--   demoms2

wls:/demodomain/serverConfig> x
[demoadm, demoms1, demoms2]
wls:/demodomain/serverConfig> for i in x:
... state(i,'Server')
...
Current state of "demoadm" : RUNNING
Current state of "demoms1" : SHUTDOWN
Current state of "demoms2" : SHUTDOWN

Cluster listing
wls:/demodomain/serverConfig> c=ls('Clusters',returnMap='true')
dr--   clstr01

wls:/demodomain/serverConfig> c
[clstr01]
wls:/demodomain/serverConfig> state(c[0],'Cluster')

There are 2 server(s) in cluster: clstr01

States of the servers are
demoms1---SHUTDOWN
demoms2---SHUTDOWN

ServerLIfecycleRuntime Mbean tree


Using above shown MBean hierarchy we can fetch the all WebLogic domain server instance's states. If your production WebLogic domain consists of two digit (eg. 60 instances) or three digit number (eg. 120 instances) of managed server then, it is difficult to see all server’s state at once. Weblogic Administration console is unable to show all the servers in the domain on a single page. Navigating in between also a time eating process so think! think better way!! WLST has the solution.

To get the status of all servers in the domain can be obtained with the following steps
  1. Connect to the WebLogic Admin Server
  2. Fetch the Managed server list from the domainRuntime MBean
  3. Iterate the loop and get the state of each Managed Server with ServerLifeCycle Runtime MBean
  4. Repeat if required the step 3 as per the user input to Continue...
  5. Finally if all desired output is visible then disconnect from the AdminServer and exit.

################################################## 
# This script is used to check the status of all WL instances including the admin
###########################################################

def conn():
    UCF='/path/.AdminScripts/userConfigFile.sec'
    UKF='/path/.AdminScripts/userKeyFile.sec'
    admurl = "t3://hostname:wlport"

    try:
        connect(userConfigFile=UCF, userKeyFile=UKF, url=admurl)
    except ConnectionException,e:
        print '\033[1;31m Unable to find admin server...\033[0m'
        exit()

def ServrState():
    print 'Fetching state of every WebLogic instance'
#Fetch the state of the every WebLogic instance
    for name in serverNames:
        cd("/ServerLifeCycleRuntimes/" + name.getName())
        serverState = cmo.getState()
        if serverState == "RUNNING":
            print 'Server ' + name.getName() + ' is :\033[1;32m' + serverState + '\033[0m'
        elif serverState == "STARTING":
            print 'Server ' + name.getName() + ' is :\033[1;33m' + serverState + '\033[0m'
        elif serverState == "UNKNOWN":
            print 'Server ' + name.getName() + ' is :\033[1;34m' + serverState + '\033[0m'
        else:
            print 'Server ' + name.getName() + ' is :\033[1;31m' + serverState + '\033[0m'
        quit()

def quit():
    print '\033[1;35mRe-Run the script HIT any key..\033[0m'
    Ans = raw_input("Are you sure Quit from WLST... (y/n)")
    if (Ans == 'y'):
        disconnect()
        stopRedirect()
        exit()
    else:
        ServrState()

if __name__== "main":
    redirect('./logs/Server.log', 'false')       
    conn()
    serverNames = cmo.getServers()
    domainRuntime()
    ServrState()

Smart Script

Recently I have online discussion with Dianyuan Wang, state of the Managed servers can be obtained with state() command. This function can be used in two ways: 
  • To get individual managed server status you need to pass arguments as managed server name, type as 'Server'. 
  • Other one is to get individual Cluster wise status. 
This can be achieved by passing two arguments cluster name and type as 'Cluster'. The following script will be illustrate the second option, which I found that shorten code that gives same script outcome as above script. It could be leverage your scripting thoughts it is like a plain vanilla form as shown below:

Note: Hope you follow the WLST Tricks & tips
try:
    connect(url = "t3://adminhostname:adminport")
except:
    print "Connection failed"
state('appclstr','Cluster')
state('web1clstr','Cluster')

...
state('webNclstr','Cluster')
-->

Extra Stroke of this new script is that prints how many servers available in each given cluster.

Server state with redirecting, re in WLST

Wang mailed me his situation clarity of explanation why he chosen state command. And how he resolved with Python script tricks here. Its a great learning for me so sharing with you when I saw same question in StackExchange today(28 April 2014) after 3 years!! "The reason I do not use (for now) domainConfig is because some how few Weblogic domains are not in a good state, and when I run the domainConfig command, it complains that it is not enabled. Hence the alternative way I've selected here is using state command. But it don't return the state. It always return None. But it prints out the state of the server. Here you go, Better way is capture that printing output to a file using WLST command redirect-stopRedirect and then, use the Python regular expression to extract the state of each server. The following is the Python snippet how I use the redirect:
 
# Fill your own connection details 
  serverlist=cmo.getServers()   
  for s in serverlist:   
    server_nm = s.getName()
    urldict[server_nm]='t3s://'+s.getListenAddress()+':'+str(s.getAdministrationPort())   
    #domainRuntime()
    #cd('ServerLifeCycleRuntimes/'+server_nm)
    fileName='/tmp/myserver_state.txt'
    redirect(fileName)
    state(server_nm,'Server')
    stopRedirect()
    f = open(fileName)
    try:
      for line in f.readlines():
        if re.search('Current state',line):
          status[server_nm]=line
    except:
      continue
 
  Ks = status.keys()
  for s in Ks:   
    if re.search('RUNNING',status[s]):
		try:
		connect(username,password,urldict[s])
		except:
		continue
		cd("/Servers/" + s)  


...

Troubleshooting Server State

Issue 1: Unable to Connect to Admin Server Error:
Connection refused
Verify: 
  1.  Administration Server is running 
  2. Hostname is reachable 
  3. Port number is correct 
  4. Firewall is not blocking connectivity 
Issue 2: Authentication Failure Error:
Authentication denied
Verify: 
  1.  Username and password 
  2. userConfigFile and userKeyFile 
  3. Security realm configuration 
Issue 3: Server State Returns UNKNOWN 
 
Possible causes: Node Manager is not running Network communication issue Server startup failure Machine configuration mismatch

The WLST state command relies on Node Manager for state retrieval in many scenarios.

Best Practices 

  • Use encrypted userConfigFile and userKeyFile instead of hardcoded passwords.
  • Store WLST scripts in version control.
  • Schedule automated health checks using cron or enterprise schedulers.
  • Generate server-state reports before and after deployments.
  • Monitor cluster health in addition to individual server status.
  • Log output for auditing and troubleshooting purposes.
  • Conclusion

    WLST provides a simple and powerful way to monitor WebLogic Server health and availability. Whether you need to check a single managed server, an entire cluster, or every server in a large domain, the WLST state() command offers a lightweight and automation-friendly solution. By combining WLST scripting with operational automation, WebLogic administrators can improve visibility, reduce manual effort, and accelerate troubleshooting across enterprise environments.
    best regards! Dianyuan Wang

    Here I request you please write back your experiencing with this posting looking ahead for your issues/ suggestions as comments.

    31 comments:

    1. Hi Pavan,

      Thank you so much for these examples you put online. These are immense help in learning WLST.

      I have a quick question: I want to modify the server state monitoring script you have to allow monitoring multiple admin and their managed servers (I have 6 different admin servers with 6 managed servers each. They are not clustered).

      I am trying to find out if WLST has the capability of reading from a txt file where I can put the admin server URL, username, pass.

      Could you please give me some hints on how I can try to implement this?

      ReplyDelete
    2. use loadProperties instead of file reading.

      ReplyDelete
    3. Hello Pavan,

      I have modiefied a bit the script like this:

      def ServerConn():
      username = 'weblogic'
      password = 'wlsfusion01'
      url = 't3://muciipd01:7011'
      connect(username, password, url)


      def ServrState():
      print 'Fetching state of every WebLogic instance'
      #Fetch the state of the every WebLogic instance
      for name in serverNames:
      cd("/ServerLifeCycleRuntimes/" + name.getName())
      serverState = cmo.getState()
      if serverState == "RUNNING":
      print 'Server ' + name.getName() + ' is :' + serverState


      print 'End of script ...'


      when I run it I get:

      [oracle11@muciipd01 ~]$ java weblogic.WLST state.py

      Initializing WebLogic Scripting Tool (WLST) ...

      Welcome to WebLogic Server Administration Scripting Shell

      Type help() for help on available commands

      End of script ...



      I guess I should explicity define the domainRuntime() in the beginning because I am not sure that after logging the cmo is the domainRuntime().

      Thanks in advance.

      Loukas

      ReplyDelete
    4. Hey Loukas,

      You can run the above script as is by updating your credentials. If you don't want colors in ur output then you can goahead with your way of script, Yes you required domainRuntime tree to fetch the Managed servers state. By default it will be on serverConfig tree when you connect the admin server. Define your module on top it will be easy to access. Remember to use python indentation blocks (4 spaces or a tab) while writing the blocks of code. Hope this will help you.

      Please comment if anything missing...

      ReplyDelete
    5. Hey Pavan,

      unfortunately I cannot replay the same exact script as your since I have not configured yet sec config files for the credential. But it doesn't matter. Basically i get your script and I am trying to do something very simple but still it doesn't work.
      I am writting my script on the Eclipse (Oracle Enterprise Pack for Eclipse (OEPE))
      By the way what is the editor you are using and it is shown in the screenshot above? Hiw can I tag my code so that you can see my script with the indentation blocks?

      I tried to even more make it more simple like following:

      print 'starting the script ....'

      def getServerNames():
      domainConfig()
      return cmo.getServers()

      def serverStatus(server):
      cd('/ServerLifeCycleRuntimes/' +server)
      return cmo.getState()

      def printStatus():
      print 'Fetching state of every WebLogic instance'
      connect('weblogic','welcome1','t3://localhost:7001')
      serverNames = getServerNames()
      domainRuntime()
      for name in serverNames:
      serverState = serverStatus(name.getName())
      print serverState


      print 'End of script ...'

      and it is still failing with a funny output:


      Initializing WebLogic Scripting Tool (WLST) ...

      Welcome to WebLogic Server Administration Scripting Shell

      Type help() for help on available commands

      starting the script ....
      End of script ...

      Am I possibly missing some libraries in the Eclipse? The Eclipse is throwing many warning for "Undefined variables" which are by default int he WLST like:
      cmo
      domainConfig()
      connect

      etc,

      Kind Regards,

      Loukas

      ReplyDelete
    6. Hi Lokus,

      With your comment I started using OEPE first time. Initially I had seen similar issue when I directly tried out the sample WLST script. To find the mystery I build the script from the scratch. newWLST.py run as WLST Run. Then after finding SUCCESS print in the output then started editing the script. It works many times unstoppable then onwards :)

      I had refer http://biemond.blogspot.com/2010/08/wlst-scripting-with-oracle-enterprise.html

      Now I am pasting the same code which you tried I got the output:


      #Conditionally import wlstModule only when script is executed with jython

      def serverStatus(server):
      cd('/ServerLifeCycleRuntimes/' +server)
      return cmo.getState()

      if __name__ == '__main__':
      from wlstModule import *#@UnusedWildImport

      print 'starting the script ....'
      username = 'weblogic'
      password = 'weblogic1033'
      url='t3://localhost:7001'

      connect(username,password,url)


      try:

      print "script returns SUCCESS"

      domainConfig()

      serverNames = cmo.getServers()
      domainRuntime()
      for name in serverNames:
      print name.getName()
      print serverStatus(name.getName())



      except Exception, e:
      print e
      print "Error while trying to save and/or activate!!!"
      dumpStack()
      raise

      ReplyDelete
    7. I've been looking for just such functionality since I discovered WLST. I've got the script running BUT it only reports on the status of the first of about a dozen Server instances that I have configured. What could be going wrong?

      - CDM

      ReplyDelete
    8. Hey CDM,

      The script which I had posted that is working for me more than 70 servers without any hassle. I m not clear about your problem with number of servers. Are you able to get the Exception?
      What does it tells you?

      ReplyDelete
    9. There's no exception being thrown. The script appears to work fine but it's only returning results for the first server in the list:

      Here are all the servers that I have:

      wls:/testEnvironment/serverConfig> ls('Servers')
      dr-- AdminServer
      dr-- alfrescocloneServer
      dr-- alfrescoclonesandboxServer
      dr-- boweb1Server
      dr-- boweb2Server
      dr-- ctrac1Server
      dr-- ctrac2Server
      dr-- ctrac3Server
      dr-- ctrac4Server
      dr-- esp1Server
      dr-- esp2Server
      dr-- qualitycentreServer

      This is what the script is returning:


      Fetching state of every WebLogic instance
      Server AdminServer is :RUNNING
      Re-Run the script HIT any key..
      Are you sure Quit from WLST... (y/n)y

      ReplyDelete
    10. after 20 line print servers
      give me reply

      ReplyDelete
    11. Sorry but I'm not sure what you mean by that?

      - CDM

      ReplyDelete
    12. From which file it wil get the status of the servers.
      I want to manually check the status of the admin server not from console. is there any other ways to find the status........

      ReplyDelete
    13. @Chinni, you have two options. you can redirect the smart script as posted above and read it. Other option is on the first script you can print the state of managed server to a file instead of Standard output. You need to use Jython File operation.

      write back your updates... :)

      ReplyDelete
    14. Hi Pavan,

      I am running this script as is (modifying the credentials and admin URL) and it is only printing the admin server state (managed are running fine as I see from the console).

      I put a print statement after line 20 and its output is:

      array([[MBeanServerInvocationHandler]com.bea:Name=CAMAdminServer,Type=Server, [MBeanServerInvocationHandler]com.bea:Name=CAMServer1,Type=Server, [MBeanServerInvocationHandler]com.bea:Name=CAMServer2,Type=Server, [MBeanServerInvocationHandler]com.bea:Name=CAMServer3,Type=Server, [MBeanServerInvocationHandler]com.bea:Name=CAMServer4,Type=Server], weblogic.management.configuration.ServerMBean)

      I see that it is returning all the servers, but not printing the status for any of the managed servers. It only prints the first server in the list (the admin):

      Server CAMAdminServer is :RUNNING
      Re-Run the script HIT any key..
      Are you sure Quit from WLST... (y/n)

      Any ideas why it would be doing that? and how can i get it to display state of all the managed servers as well?

      ReplyDelete
    15. the script was made for WebLogic 9 version. may be that is the reason.
      20 is looking for server names. you might get it by get(Name)

      ReplyDelete
    16. For 10.3 use:

      domainRuntime()
      for server in cmo.getServerLifeCycleRuntimes():
      serverName = server.getName()
      serverState = server.getState()

      ReplyDelete
    17. This comment has been removed by the author.

      ReplyDelete
    18. This comment has been removed by a blog administrator.

      ReplyDelete
    19. Thanks for sharing, well done. But the guy saying the script only shows admin server is right. you need to put the quit() call out of the loop. Moving at the end of the main made the trick for me.

      ReplyDelete
    20. I really liked ur blog..It has helped me a lot.

      Keep posting

      Regards,
      Fabian

      ReplyDelete
    21. 't3://'+str(s.getListenAddress())+':'+str(s.getAdministrationPort()) , getAdministrationPort() seems a wrong method giving syntax error

      ReplyDelete
    22. Hi Pavan,

      Thanks for the write up, it is useful indeed, however, there is one more issue that I am facing, a few of my servers health shows up as "Not Reachable" even when the State is RUNNING. Is there a script to find out the Health as displayed on the Admin Console.

      ReplyDelete
    23. This is realy a Nice blog post read on of my blogs It is really helpful article please read it too my blog FACEBOOK IMAGES NOT LOADING.

      ReplyDelete
    24. hey . is there any way we can get instance health using shell script. my production machine doesnot have python installed so any suggestion is appreciated

      ReplyDelete
    25. Hi all, is anyone have script, such as it will shows all servers status including Nodemanager & after that has option to start & stop individually

      ReplyDelete
    26. Hi Pavan Garu,
      Can you please share the wlst script to monitor server health not state.

      ReplyDelete
    27. This comment has been removed by the author.

      ReplyDelete
    28. Hi, nice article. Linux VPS hosting have been pondering concerning this matter, so thanks for sharing.

      ReplyDelete
    29. If you are looking for flats for rent in greater kailash 1, Delhi then connecting with Reallworld can be the best suit for you, as our time will assist you in renting or even buying property in the greater kailash, Delhi location. Feel free to connect to Reallworld today, as we are the leading housing, property renting & real estate company in Delhi.

      ReplyDelete
    30. While renting the 3 bhk flats in Gk 1 from a private individual, ensure that you have a stranger’s social security number and bank account details, and they conduct the credit check for you.

      ReplyDelete

    Please write your comment here

    Popular Posts