Search This Blog

Wednesday, June 26, 2013

JMS Bridges using WLST



My new project where I found many new things to learn about JMS. I was searching for a Configure JMS bridge using Python, but there very few blog posts found and that too, their scope limited to target to AdminServer and they are limited for single bridge configuration only. We have analyzed and developed this where you can take complete advantage.

Oracle WebLogic JMS- MQ Series



WebLogic JMS Bridge to MQ Series broker

Oracle WebLogic JMS-JMS



The JMS bridge configuration is dependency on Local and remote Queues or Topics. The bridge can be established between source and targeted destinations need to be created with the following parameters :
  •  JMS bridge destination
  •  Connection URL
  • JMS ConnectionFactory JNDI Name
  •  JMS destination JNDIName

Once the source and target is configured you can configure the message bridges. The bridge configuration requires the following:
  • Bridge name
  •  Source destination
  • Target destination
  • JMS Bridge deployment target Cluster or server

Usually there could be different messaging systems can communicate with JMS Bridges. Bridge can be JMS-JMS communication or external systems can be integrated with messages. This messages can be plain Text or xml files or Java objects. To handle them JEE applications must use Message driven Beans MDB.

#========================================
#  JMS Bridge Configuration script.
#  FileName:  jmsBridges.py
#=========================================
from java.util import *
from java.io import FileInputStream
def createdestination(JMSBridgeDestination,ConnectionURL,ConnectionFactoryJNDIName,DestinationJNDIName):
        cmo.createJMSBridgeDestination(JMSBridgeDestination)
        JMSBridgeDestination = cmo.lookupJMSBridgeDestination(JMSBridgeDestination)
        JMSBridgeDestination.setClasspath('')
        JMSBridgeDestination.setConnectionURL(ConnectionURL)
        JMSBridgeDestination.setAdapterJNDIName('eis.jms.WLSConnectionFactoryJNDINoTX')
        JMSBridgeDestination.setConnectionFactoryJNDIName(ConnectionFactoryJNDIName)
        JMSBridgeDestination.setDestinationJNDIName(DestinationJNDIName)
        return JMSBridgeDestination
 
def create_bridge(MessagingBridge,Cluster,srcbdest,TJMSBridgeDestination,qos):
        cmo.createMessagingBridge(MessagingBridge)
        bridge = cmo.lookupMessagingBridge(MessagingBridge)
        cluster = cmo.lookupCluster(Cluster)
        targets = bridge.getTargets()
        targets.append(cluster)
        bridge.setTargets(targets)
        bridge.setSourceDestination(srcbdest)
        bridge.setTargetDestination(TJMSBridgeDestination)
        bridge.setStarted(true)
        bridge.setSelector('')
        bridge.setQualityOfService(qos)

 
def getp(x):
        return configProps.get(""+x+"")
 
envproperty=""
if (len(sys.argv) > 1):
  envproperty=sys.argv[1]
else:
    print "Environment Property file not specified"
    sys.exit(2)
 
propInputStream=FileInputStream(envproperty)
configProps=Properties()
configProps.load(propInputStream)
 
USER=configProps.get("USER")
PASSWD=configProps.get("PASSWD")
ADMNURL=configProps.get("ADMNURL")
 
print 'CONNECT TO ADMIN SERVER'
connect(USER, PASSWD, ADMNURL)
 
print 'START EDIT MODE'
n=int(getp("num_of_bridges")) # number of Brdiges
edit()
startEdit()
try:
        print 'CREATE SOURCE JMS BRIDGE DESTINATION'
        for i in range(1,n+1):
                # if Bridge already exists skip
                MessagingBridge=getp("MessagingBridge"+str(i))
                print "checking ... ", MessagingBridge
                ref = getMBean("/MessagingBridges/"+MessagingBridge)
                if(ref == None):
                        S_Dest=getp("S_Dest"+str(i))
                        S_ConnURL=getp("S_ConnURL"+str(i))
                        S_ConnFJNDI=getp("S_ConnFJNDI"+str(i))
                        S_DestJNDI=getp("S_DestJNDI"+str(i))
                        src=createdestination(S_Dest,S_ConnURL,S_ConnFJNDI,S_DestJNDI)

                        print 'CREATE TARGET JMS BRIDGE DESTINATION'+str(i)

                        T_Dest=getp("T_Dest"+str(i))
                        T_ConnURL=getp("T_ConnURL"+str(i))
                        T_ConnFJNDI=getp("T_ConnFJNDI"+str(i))
                        T_DestJNDI=getp("T_DestJNDI"+str(i))

                        target=createdestination(T_Dest,T_ConnURL,T_ConnFJNDI,T_DestJNDI)
                        print 'CREATE MESSAGING BRIDGE'
                        cluster=getp("jms_mod_target1")
                        qos=getp("QualityOfService"+str(i))
                        create_bridge(MessagingBridge,cluster,src,target,qos)
                else:
                        pass


Flexibility in defining Bridges Passed through many patterns and learnings, I have developed this as reusable, generic as possible. Scope for further extendable.

I believe "Do it One at Once", To make work simple, Here I had used JMS Bridges with non-transactional. You can change if required. Correspondingly there is the implication with "Atmost-once" option. The "Exactly-Once" option is best suitable choice for non-transnational messages QoS.

WLST Bridge Configuration Properties



You need to customize it according to your domains. This script allows you to create as many bridges as you wish. Just one thing you need to change in the properties is that num_of_bridges value.
Here is the Sample properties file where you can replace the values according to your requirements. The WebLogic destination can be identified with t3 protocol, whereas MQ series uses file:// protocol which requires the full path of the path so it looks like three slashes (file:///)

In the sample CF are connectionFactories that participate in the bridge communication. The name with Dest indicates the Queues/Topics which are involved in the message holders.
#JMS BRIDGES CONFIGURATION FOR MQ

numBridges=1

MessagingBridge1=com.my.TestBridge
QualityOfService1=Atmost-once

S_ConnFJNDI=mywlsCF
S_ConnURL=t3://hostname:port
T_ConnURL=file:///mqlocal/jndi/bindings
T_ConnFJNDI=MQCF

S_Dest1=com.mq.bridge.my.Qsrc
S_DestJNDI1=test/outgoing/request
T_Dest1=com.mq.bridge.my.Qdest
T_DestJNDI1=QSRC_TO_QDEST

To execute the above Python script for JMS Bridge configuration can be done as follows:
java weblogic.WLST JmsBridge.py bridge.properties

Scalability for Bridge

After a while working in the project there is new requirement came in to add few more bridges to the existing  Domain. Now my task is that, need to skip those bridges which are existing bridges list and need to create the new bridge only when it is NOT in the list.  To do so I have solution that using Bridge Runtime MBean finding in the run-time domain, pass is keyword to skip them. So need to update the numBrige variable in properies file and append the new bridge details at the end.

The key logic is here...

              # if Bridge already exists skip
                MessagingBridge=getp("MessagingBridge"+str(i))
                print "checking ... ", MessagingBridge
                ref = getMBean("/MessagingBridges/"+MessagingBridge)
                if(ref == None):
                       #create the bridge
                else:
                       pass # skipping

Monitoring Bridges
In WebLogic 11g (10.3.x version) we can only able to fetch the monitoring information about Bridges configured on the domain using weblogic.Admin. The KSH script developed for bridge monitoring status.

Sunday, June 23, 2013

Socket module using WLST

Network Socket with Python/WLST

Internet protocol libraries for Python can be used in WLST. hostname you can use to make most generic for your automation scripts. Use this idea in build and deployment process.

wls:/offline> import socket
wls:/offline> print(socket.gethostname())
pavanb.wlst.by.examples.com

There are many socket related modules and built-in functions available to check we can run dir command on each to know wha is inside that package.
 
wls:/offline> import smtplib
wls:/offline> dir(smtplib)
['CRLF', 'OLDSTYLE_AUTH', 'SMTP', 'SMTPAuthenticationError', 'SMTPConnectError',
 'SMTPDataError', 'SMTPException', 'SMTPHeloError', 'SMTPRecipientsRefused', 'SM
TPResponseException', 'SMTPSenderRefused', 'SMTPServerDisconnected', 'SMTP_PORT'
, 'SSLFakeFile', 'SSLFakeSocket', '__all__', '__doc__', '__file__', '__name__',
'base64', 'encode_base64', 'hmac', 'quoteaddr', 'quotedata', 're', 'rfc822', 'so
cket', 'types']

FTP library in WLST

wls:/offline> import ftplib
wls:/offline> dir(ftplib)
['CRLF', 'Error', 'FTP', 'FTP_PORT', 'MSG_OOB', 'Netrc', '_150_re', '_227_re', '
__all__', '__doc__', '__file__', '__name__', 'all_errors', 'error_perm', 'error_
proto', 'error_reply', 'error_temp', 'ftpcp', 'os', 'parse150', 'parse227', 'par
se229', 'parse257', 'print_line', 'socket', 'string', 'sys', 'test']

Did you ever thought of using FTP from WLST?

IMAPLIB in WLST

wls:/offline> import imaplib
wls:/offline> dir(imaplib)
['AllowedVersions', 'CRLF', 'Commands', 'Continuation', 'Debug', 'Flags', 'IMAP4
', 'IMAP4_PORT', 'Int2AP', 'InternalDate', 'Internaldate2tuple', 'Literal', 'Mon
2num', 'ParseFlags', 'Response_code', 'Time2Internaldate', 'Untagged_response',
'Untagged_status', '_Authenticator', '__all__', '__doc__', '__file__', '__name__
', '__version__', '_cmd_log', '_cmd_log_len', '_dump_ur', '_log', '_mesg', 'bina
scii', 'print_log', 'random', 're', 'socket', 'sys', 'time']
This is implib

telnet Library in WLST

Just for connectivity before create your database connection pools.
wls:/offline> import telnetlib
wls:/offline> dir(telnetlib)
['AO', 'AUTHENTICATION', 'AYT', 'BINARY', 'BM', 'BRK', 'CHARSET', 'COM_PORT_OPTI
ON', 'DEBUGLEVEL', 'DET', 'DM', 'DO', 'DONT', 'EC', 'ECHO', 'EL', 'ENCRYPT', 'EO
R', 'EXOPL', 'FORWARD_X', 'GA', 'IAC', 'IP', 'KERMIT', 'LFLOW', 'LINEMODE', 'LOG
OUT', 'NAMS', 'NAOCRD', 'NAOFFD', 'NAOHTD', 'NAOHTS', 'NAOL', 'NAOLFD', 'NAOP',
'NAOVTD', 'NAOVTS', 'NAWS', 'NEW_ENVIRON', 'NOOPT', 'NOP', 'OLD_ENVIRON', 'OUTMR
K', 'PRAGMA_HEARTBEAT', 'PRAGMA_LOGON', 'RCP', 'RCTE', 'RSP', 'SB', 'SE', 'SEND_
URL', 'SGA', 'SNDLOC', 'SSPI_LOGON', 'STATUS', 'SUPDUP', 'SUPDUPOUTPUT', 'SUPPRE
SS_LOCAL_ECHO', 'TELNET_PORT', 'TLS', 'TM', 'TN3270E', 'TSPEED', 'TTYLOC', 'TTYP
E', 'TUID', 'Telnet', 'VT3270REGIME', 'WILL', 'WONT', 'X3PAD', 'XASCII', 'XAUTH'
, 'XDISPLOC', '__all__', '__doc__', '__file__', '__name__', 'select', 'socket',
'sys', 'test', 'theNULL']
This is having more functions and variables check it...

httplib in WLST for Sanity check

You can use 'httplib' for checking sanity of an application after deployment.
wls:/offline> import httplib
wls:/offline> dir(httplib)
['BadStatusLine', 'CannotSendHeader', 'CannotSendRequest', 'FakeSocket', 'HTTP',
 'HTTPConnection', 'HTTPException', 'HTTPMessage', 'HTTPResponse', 'HTTPS', 'HTTPSConnection', 'HTTPS_PORT', 'HTTP_PORT', 'ImproperConnectionState', 'IncompleteRead', 'InvalidURL', 'LineAndFileWrapper', 'NotConnected', 'ResponseNotReady', 'SSLFile', 'SharedSocket', 'SharedSocketClient', 'StringIO', 'UnimplementedFileMode', 'UnknownProtocol', 'UnknownTransferEncoding', '_CS_IDLE', '_CS_REQ_SENT', '_CS_REQ_STARTED', '_UNKNOWN', '__all__', '__doc__', '__file__', '__name__', 'errno', 'error', 'mimetools', 'socket', 'test', 'urlsplit']
Here, I fond the HTTPLIB example given by Valdmir blog post->Python: HTTP Basic authentication with httplib. Where in the Python script used to do the Sanity test for a provided application URL, which uses the authentication provided using base64 module.

The xmlrpclib library for WLST

This is xmrpclib can be used while interacting with config.xml file. have a look and decide :)
wls:/offline> import xmlrpclib
wls:/offline> dir(xmlrpclib)

['ArrayType', 'Binary', 'Boolean', 'BuiltinFunctionType', 'BuiltinMethodType', 'ClassType', 'CodeType', 'ComplexType', 'DateTime', 'DictProxyType', 'DictType','DictionaryType', 'EllipsisType', 'Error', 'ExpatParser', 'False', 'FastParser',
 'FastUnmarshaller', 'Fault', 'FileType', 'FloatType', 'FrameType', 'FunctionType', 'GeneratorType', 'InstanceType', 'IntType', 'LambdaType', 'ListType', 'LongType', 'MAXINT', 'MININT', 'Marshaller', 'MethodType', 'ModuleType', 'NoneType',
'ObjectType', 'ProtocolError', 'ResponseError', 'SafeTransport', 'Server', 'ServerProxy', 'SgmlopParser', 'SliceType', 'SlowParser', 'StringType', 'StringTypes', 'TracebackType', 'Transport', 'True', 'TupleType', 'TypeType', 'UnboundMethodT
ype', 'UnicodeType', 'Unmarshaller', 'WRAPPERS', 'XRangeType', '_Method', '__doc__', '__file__', '__name__', '__version__', '_decode', '_stringify', 'binary', 'boolean', 'classDictInit', 'datetime', 'dumps', 'escape', 'getparser', 'loads','operator', 're', 'string', 'time']

Saturday, June 15, 2013

SOA & ADF Bounce script


We were working on SOA doamin for automation with WLST. After woring on the retiring and activation  of  the composites of SOA_Infra we are ready for shutdown the managed servers. Now the task is simple we have break-down the task into two simple functions.

  1. stop the cluster
  2. start the cluster
  3. Main program
Here the main program uses the logic of fetching the cluster list from the admin server. The functions are made in such a way that each cluster control operation can be tracked with state command, that will tell about all the managed servers in the cluster state. Once we did a trial found that there could be some time required for shutdown the managed servers. Double checked the state with Console as well. Re-run the same script with the menu option to start the Cluster.

Here NodeManager is independently running on each machines where the cluster spread across the  managed servers are running. So, this script only controls the clustered managed servers.

You can extend the same script to turn down the Admin server as well. after cluster down you can have that action. The command is shutdown('AdminServer', 'Server')



If you want more stories on the automation go thru the recommended book!
  
#====================================
# 
# Description: This script objective is to provide the choices to perform operation
#    1 is for stop the cluster
#    2 is for start the cluster
#    other is for exit from this script
#  this uses two functions startClstr and stopClstr which takes parameter as 'cluster name'.
#
#====================================
# Stop all instances of a Cluster 
#====================================
def stopClstr(clstrName):
 try:
  shutdown(clstrName,"Cluster")
  java.lang.Thread.sleep(25000)
  state(clstrName,"Cluster")
 except Exception, e:
  print 'Error while shutting down cluster ',e
 
#====================================
# Startp all instances of a Cluster 
#====================================
def startClstr(clstrName):
 try:
  start(clstrName,"Cluster")
  state(clstrName,"Cluster")
 except Exception, e:
  print 'Error while shutting down cluster ',e
 

if __name__== "main":
 connect('weblogic','welcome1','t3://localhost:7001')
 print "1.To Stop Instances"
 print "2.To Start Instances"
 print "3.Exit from Menu"
 ch=input('Enter Your Choice: ')
 cd("Clusters")
 clstrList=ls(returnMap='true')
 if ch== 1 :
  for clstr in clstrList:
   stopClstr(clstr)
 elif ch == 2 :
  for clstr in clstrList:
   startClstr(clstr)
 else:
  exit()

SOA Retire Activate Composites


Why we need retire and activate composites?

Before you go for bouncing the SOA Suite Domain which contains SOA, ADF Clusters with multiple managed servers, where each Weblogic server hosted all the sessions which are in execution state on them must be persisted to a restore further when Servers back to RUNNING state. To make this possible we need to use the retire composites and then after RUNNING servers bring them back to activate state for service.

Here is a sample trail script where the SOA application composites management. Here you need to navigate to the wlst.sh or cmd path. To access SCA function you must start WLST from /common/bin. When trying to run the SOA WLST command from regular 'java weblogic.WLST' cannot execute the SCA functions. When you try to execute the sca_retireComposite() it could throw the 'NameError'. So the cause is started WLST from the wrong location, to avoid that our PATH is to change and run the script from the following path: $SOA_ORACLE_HOME/common/bin/wlst.sh will work on Unix/Linux environments. Similarly you choose for Windows environment instead of using executing the shell script you need to use 'call' in the batch script.

Here the major task we have proceeding with the following two functions.

  1. Retire is to retire the composites .
  2. Activate is to activate the composites.
loadProperties('/user/test/scripts/composites.properties')
def retireComposites(Soahost,port, username,password,scacompositename, ver):
        sca_retireComposite(Soahost, port, username, password, scacompositename, revision=ver, partition='default') 
def activateComposites(Soahost,port, username,password,scacompositename, ver):
        sca_activateComposite(Soahost, port, username, password, scacompositename, revision=ver, partition='default')
if __name__== "main":
        print "1.To retire composites"
        print "2.To activate composites"
        print "3.Exit from Menu"
        ch=input('Enter Your Choice: ')
        f=open('/user/test/scripts/composites.txt','r')
        if ch== 1 :
                for c in f:
                        c=c.rstrip('\n')
                        retireComposites(Soahost,port, username,password,c, ver)
        elif ch == 2 :
                for c in f:
                        c=c.rstrip('\n')
                        activateComposites(Soahost,port, username,password,c, ver)
        else:
                exit()
        f.close()

Here is the composite.txt sample, where you can specify your composites configured in the SOA partition. Which are usually visible on enterprise manager(em) console. Oracle must given a easy module for displaying the deployed SCA composites list. Anyway we have stored in a separate file as shown below. You have flexibility of changing the order when you use this composite.txt file.

B2B_BPEL_TEST_Sub_reccive
B2B_BPEL_TIBCO_PUB_invoke

The actual trouble started when we tried to use a separate composite.txt file. each line can be read and the value always having at the end an EOL ( \n ). To supress that escape sequance we have used python scring function rtrip function.

The regular properties file which is loaded on the fist line can be created as follows:


Soahost=localhost
port=8001
username=weblogic
password=welcome1
ver=1.0



Please share this with your friends and collegues, comment if you already tried or successfully implemented on your SOA environment.

Video References:

Iris Li demonstrates how to deploy a SOA composite application using the Oracle Enterprise Manager 11g

SOA Composites references :

Popular Posts