Search This Blog

Sunday, June 10, 2012

JDBC Monitoring

JDBC Monitoring with LWLST script
One fine morning we (WLA support Team) got an assignment, The summary of the assignment was to find "How the WebLogic server instance performing for a DataSource?". WebLogic 9.x onwards a DataSource is associated with a ConnectionPool (a pool of connections to the DB). If we monitor ConnectionPool, inturn it is nothing but monitoring a DataSource.

Of-course the task is not that easy, I have gone through various WebLogic forums to find a appropriate solution for this task.

Oracle WebLogic provides two ways to Monitor a Datasource
1. Monitoring Datasource server wise
2. Testing the Connection Pool

Here I am publishing the same script which Madan Noru/Satya prepared in the old bea fourms or ObjectMix forums. Only one thing is difference is that displaying pattern, I had created a separate header, so that output looks good to see in a table form. To make this possible I have used C-Style print command from Python Language. This format you can change as per your screen display size.

The script will retrieve the JDBC Connection Pool MBean using adminHome, which is deprecated object in WLST. The output of the script will gives you the values of the following attributes:
  • DataSource Name
  • Maximum Capacity of the Connection Pool at Run-time
  • Active Connections Current Count
  • Active Connections High Count
  • Wait Seconds High Count
  • Waiting For Connection Current Count
  • State of the Connection Pool

#=======================================================
# This script will monitor the JDBC CONNECTION POOL
# more details on this script contact: Pavan Devarkonda
#=======================================================
connect("username","passwd","t3://AdminIP:AdminPort")
try:
 poolrtlist=adminHome.getMBeansByType('JDBCConnectionPoolRuntime')
 print ' '
 print ' '
 print 'JDBC CONNECTION POOLS'
 print ' '
 print 'Name Max Active Active WaitSecs Waiting State'
 print ' capacity Current HighCnt HighCnt Count'

 for poolRT in poolrtlist:
  pname = poolRT.getName()
  pmaxcapacity = poolRT.getAttribute("MaxCapacity")
  paccc = poolRT.getAttribute("ActiveConnectionsCurrentCount")
  pachc = poolRT.getAttribute("ActiveConnectionsHighCount")
  pwshc = poolRT.getAttribute("WaitSecondsHighCount")
  pwfccc = poolRT.getAttribute("WaitingForConnectionCurrentCount")
  pstate = poolRT.getAttribute("State")
  print '%10s %7d %7d %7d %7d %7d %10s' % (pname,pmaxcapacity,paccc,pachc, pwshc,pwfccc,pstate)
  print ' '
except:
  print 'Error:'
  dumpStack()
  pass
disconnect()

In Year 2012 revisting the same script

We have trouble in the produciton environment with JDBC Connection pool overloaded. All the ActiveConnectionCount reaching the MaxCapacity of the Connection pool. As a temprory workaround we need to reset the Connection pool for that movement. Permanent cure is tuning the Connection pool. For both solutions we need active monitoring the JDBC Connection pool.
To monitor this we have revisited the same script. Now adding more saus to the script we need to find that ActiveConnectionCount reaching 85% as threshold limit. Once it reaches 85 or greater then monitoring tool like HP OV/some other tool will reads log file generated by the script and then sends warns message to supporting Middleware Admin when threshold crossed. If you have sendmail service on your machine you can send message fromt he script itself.
To do this we had manupulated the above script according to the requirements
  1. We have specified DataSource that are causing this trouble
  2. Used separate properties file for user credentials.
  3. The output of script is redirected to a file
  4. Managed Server name opposite to the monitor values it is tricky but resolved The new script is

#=======================================================
# This script will monitor the JDBC CONNECTION POOL
# Date :  18 Apr 2012
#=======================================================
import sys
import time
try:
        loadProperties('./DBcheck.properties')
        fp=open('ActiveConn.log','a+')
        Date = time.ctime(time.time())
        admurl='t3://'+admAdrs+':'+admPort
        connect(userConfigFile=UCF, userKeyFile=UKEY, url=admurl)
        poolrtlist=adminHome.getMBeansByType('JDBCConnectionPoolRuntime')
        print ' '
        print 'JDBC CONNECTION POOLS'
        print>>fp, '================================================ '

 for poolRT in poolrtlist:
                pname = poolRT.getName()
                pmaxcapacity = poolRT.getAttribute("MaxCapacity")
                paccc = poolRT.getAttribute("ActiveConnectionsCurrentCount")
        
  if pname == 'myDataSource1' or pname == 'myDS2'' or pname == 'myDS3':
                        server= str(poolRT).split(',')[2].split('=')[1]
                        p=(paccc /(pmaxcapacity * 1.0)) * 100 
                        if p >= 85:
                                print >>fp, 'WARNING: The Active connections are Greater than Threshold 85%'
                        print>>fp, '%24s %15s %18s %7d %7d' % (Date,server,pname,pmaxcapacity,paccc)

except Exception, e:
        sys.stderr.write('ERROR: %s\n' % str(e))
        print 'Error:', e
        dumpStack()
        pass
fp.close()
disconnect()


-->

Post script actions

  1. Prepare a Shellscript that will let you know howmuch time it will counsume to run py script.
    clear
    date
    $JAVA_HOME/bin/java weblogic.WLST /path/urscript/jdbcmon.py
    date
    
  2. Schedule a scheduler to run this script autosys or crontab that invokes above shell script.
  3. Configure a monitoring tool frequently lookup the logs and send alert messages such as HP OVO or smpt mailing also fine.
What do you think about this new version of script? Write back your comments and suggestions to deliver better. Refernce Object Mix discussion:
http://objectmix.com/weblogic/549153-weblogic-monitoring-script-wlst-2.html -->

Thursday, October 27, 2011

WLST Errors and Exceptions

When first time you started using WLST you might get many of these Python based WLST Errors. Here I had collected few of them which are very simple to handle them with care. Only thing you need to understand when what kind of errors raises, and what need to do to handle them. When there is error on your flow of WLST Script or prompt don't be panic, relax for a moment then after a while take a deep breath and focus on your error and map with one of the following and do the required workaround.


In Jython we have issubclass() to check superclass, subclass relation we can verify class relationship. You can find parent-child relationship with it. As per my understanding the Error hierarchy can be defined in WLST(Jython) as follows:

This WLST(Jython) Error tree I prepared and posted for your reference, so that you can make your script in perfect manner, here you can find what is going wrong why it is happen while working out your script.
try:
 # WLST code Block 
 # perform some tasks that may throw an exception
except ExceptionType, ExceptionVar:
 # WLST code or
 # perform some exception handling
finally:
    # perform tasks that must always be completed (Will be performed before the exception is # raised.)
else:
 # execute code that must always be invoked

The pass statement in WLST

While writing WLST scripts, there are some situations where you need ‘do nothing’ statement syntactically. That is provided by the pass statement. When you start working on Exception handling this statement will be the first experimenting statement for you.

WLST raise statement

In WLST raise is used to generate or invoke an exception condition. The syntax of this statement allows three comma separated expressions which are optional. If no expression is present, WLST attempt to re-raise the last exception that was raised. This raise statement we can use when we need exception handling with robust scripts. When you pass the expressions to the raise statement, the first two expressions are evaluated to get the objects. These objects are then used to determine the type and value of the exception. Omitted expressions are treated as None. The third expression could be used for traceback objects.

wls:/offline> try:
...     raise Exception('SituationalResponse')
...except Exception, e:
...     print e
...
java.lang.Exception: SituationalResponse

SyntaxError

This you cannot be handled with the try-except block, because it will be thrown when your syntax is not in properly arranged, that is in the statement missing indentation or improper arguments. SyntaxError could be raised when the script lines are given for parsing, it will do token by token parsing wherever the improper syntax given WLST Shell points with cap char under the token.


Now let us see the sample indentation issue that raises the Syntax Error.
wls:/offline>; try:
...connect('system','weblogic103','t3://adminhost:adminport')
Traceback (innermost last):
(no code object)
at line 0
File
"", line 2
connect('system','weblogic103','t3://adminhost:adminport')
^
SyntaxError: invalid syntax

Another option for Syntax Error
This Error I have observed when I tried to migrate the WebLogic domain with createDomain() command.

wls:/offline/wdomain>createDomain('/home/backup/olddomain.jar', '/home/otherusr/domains/newdomain',’system', 'weblogic103')
Traceback (innermost last):
  (no code object) at line 0
  File "", line 1
        createDomain('/home/backup/olddomain.jar', '/home/otherusr/domains/newdomain',?system', 'weblogic103')
                                                                                                          ^
SyntaxError: Lexical error at line 1, column 99.  Encountered: "\ufffd" (65533), after : ""

This "Lexical error" got due to the copying from the webpage which has single quotes in different unicode value. When I retype the single quotes it was resolved.

Here in the above sample after issuing try block starting we must use a tab or 4 spaces before connect() command. When it found that is not
having proper indentation it raised the SyntaxError.

NameError


When the name used to do something like print or use in some other expression without assigning the value before it was defined then WLST will raises NameError. When first time scripting most of the time user encounters this unknowingly.

The following example might give you an idea how to resolve your issue.

wls:/offline> var1=100
wls:/offline> var3=var1+var2
Traceback (innermost last):
  File "", line 1, in ?

You can handle this kind of error with our try-except block
wls:/offline> try: var3=var1+var2
...except NameError, e:
...     print "Please check there is: ", sys.exc_info()[0], sys.exc_info()[1]
...
Please check there is:  exceptions.NameError var2

The beauty of handling your Exception/Error more transparent and easy to understand with sys.exc_info() list.

KeyError


This error can be raised by the WLST while using the dictionary objects or map objects accessed with non-matching key.
wls:/offline> urls['b']
Traceback (innermost last):
File
"", line 1, in ?
KeyError: b

ValueError

The ValueError is raised by the WLST shell when there is a inappropriate element is accessed in a list or a variable, that is such as the value specified for searching in the list with index() method. Removing the element which is not really exists in the list.
wls:/offline> L.index('web2')
Traceback (innermost last):
File
"<console>", line 1, in ?
ValueError: list.index(x): x not in list
I was working on thread and JVM monitoring script, encountered with the ValueError in different way. After storing the ThreadPool values, JVM values into local variables I was using C type of formatting to display the data in a row of Table. Some of the attribute values are Long integers, some of them plain integers some of them are strings.
 cd('/ServerRuntimes/'+ svr +'/ThreadPoolRuntime/ThreadPoolRuntime')
 thtot=`get('ExecuteThreadTotalCount')`
 thid= `get('ExecuteThreadIdleCount')`
 hog= `get('HoggingThreadCount')`
 sbth= `get('StandbyThreadCount')`
 cr =`get('CompletedRequestCount')`
 pr =`get('PendingUserRequestCount')`
 ql =`get('QueueLength')`
 th= `get('Throughput')`
 
 cd('/ServerRuntimes/'+svr+'/JVMRuntime/'+svr)
        freejvm = long(get('HeapFreeCurrent'))/(1024*1024)
        totaljvm = long(get('HeapSizeCurrent'))/(1024*1024)
        usedjvm = (totaljvm - freejvm)
      

When I ran with all numbered values with format as %5d it was shouted as follows:
ValueError: unsupported format character ' ' (0x20) at index 23
Don't know what attribute requires which format ... Initially to resolve this display without any format for all attributes values from the MBean.
print svr, thtot, thid, hog, sbth, cr, pr, ql, th, hs, totaljvm, freejvm, usedjvm
But still ValueError was exists, when I updated with formatter it was stuck with CompletedRequestCount that was not integer type, it is actually Long integer type and that was causing the Error. So, changed the format for that attribute it resolved one issue. Now the issue with different index number came... I have an idea that, if I found all the attributes and their data types then it will be easy to fix the right format for each. I tried the following way
print type(thtot),type(thid), type(hog), type(sbth), type(cr), type(pr), type(ql), type(th),type(freejvm), type(totaljvm), type(usedjvm)
formatted accordingly the ValueError is resolved.
print '%14s %10s %5s %5s %5s %5s %8s %5s %5s %8s %5dMB %5dMB %5dMB' %  (svr, hs, thtot, thid, hog, sbth, cr, pr, ql, th, totaljvm, freejvm, usedjvm) 
So now you can take care of Values of variables for your WLST code before use them for any operation!!

AttributeError


You might be on MBean tree where there is no such attribute defined and you tried to access it then WLST Shell raises AttributeError. Let us see an Example you can easily understand.

wls:/demodom/serverConfig> cmo.State()
Traceback (innermost last):
  File "", line 1, in ?
AttributeError: State
 
 

IndexError

The IndexError will be raised by the WLST shell, when thelist object is accessed with a out of range index value.

Let us see the example a list is defined with 5 elements

wls:/offline> L['app1', 'app2', 'app3', 'app4', 'web1']
when it is accessed with out of range index value say 7 then you get the IndexError.
wls:/offline> L[7]
Traceback (innermost last):
File
"<console>", line 1, in ?
IndexError: index out of range: 7

TypeError

The basic python object types int, str assignments or expressions or print statements with
concatenation does not allows you, raises the TypeError.

wls:/offline> print 'Number of servers:', 5
Number of servers: 5
 
wls:/offline>print 'Number of servers:'+ 5
Traceback (innermost last):
File
"<console>", line 1, in ?
TypeError: cannot concatenate 'str' and 'int' objects


Thanks for reading this post, Give your feedback in comments. cheers!!

Good References:
  1. http://www.doughellmann.com/articles/how-tos/python-exception-handling/
  2. http://www.jython.org/jythonbook/en/1.0/ExceptionHandlingDebug.html
  3. http://www.tutorialspoint.com/python/python_strings.htm

Popular Posts