Search This Blog

Oracle Fusion Middleware Administration Course

Oracle Fusion Middleware Administration Course
OFMW Admin Course

Wednesday, December 31, 2014

Regular Expressions in WLST

Python based WLST tutorial is to present a detailed and descriptive introduction into regular expressions. Every scripting language powered up with this regular expressions.


Regular expressions examples in WLST
Regular Expressions in WLST

In scripting, Very complex problems can be resolved with simple regular expressions. If you are not aware of each expressions meaning then it would be Greek &  Latin. You need to understand how they can be applicable then you can construct your WLST script in such a way that it could give a fantastic outcomes. A regular expression is that a set of possible input raw strings (includes alphanumerical, whitespaces, symbols).

Regular expressions descends from fundament concept in Computer Science called finite automata theory. A regular expression can match a string in more than one place.

Meta Characters in WLST

The special meaning for each symbol that we use in the regular expression. We have the following meta characters allowed in WLST.
WLST using meta chars


Dot . in expression – any char

The . dot or period regular expression can be used to matches any character. Remember this don’t matches new line ‘\n’.



The Character classes

Character classes [] can be used to match any specific set of characters.
Here is the example to understand the expression that uses character class. The expression second char can be one of the three chars given in the char class [eor]. So the pattern matches three words from the given text.


Negate character class

Character class can be negated with ^ symbol inside the character class. That is [^].
[aeiou] will match any of the characters a, e, i, o, or u
[yY]es will match Yes or yes
Negate Character class in re


Ranges can also be specified in character classes
[1-9] is the same as [123456789]
[abcde] is equivalent to [a-e]


You can also combine multiple ranges in the pattern
[abcde123456789] is equivalent to [a-e1-9]

Note that the hyphen - character has a special meaning in a character class but only if it is used within a range, if you use hypen at beginning its meaning changes it doesn't meant for range.
[-123] would match the characters -, 1, 2, or 3

Commonly used character classes can be referred to by name (alpha, lower, upper, alnum, digit, punct, cntrl)
Syntax [:name:]
[a-zA-Z]       [[:alpha:]]
[a-zA-Z0-9]    [[:alnum:]]
[45a-z]      [45[:lower:]]
Important for portability across languages
compile - search, match patterns on data 

import re

t="Demoadmin, demo_ms1, demo_ms2, my_clustr1"
p=re.compile('(d\w+)',re.I)
pos=0
while 1:
        m=p.search(t, pos)
        if m:
                print "Now search start position :", pos
                print m.group()
                pos=m.end()
        else:
                break


Regular expression Compile pattern on search method

Regular expression FLAGS


re.I == re.IGNORECASE Ignore case
re.L == re.LOCALE Make \w, \b, and \s locale dependent
re.M == re.MULTILINE Multiline
re.S == re.DOTALL Dot matches all (including newline)
re.U == re.UNICODE Make \w, \b, \d, and \s unicode dependent
re.X == re.VERBOSE Verbose (unescaped whitespace in pattern
is ignored, and '#' marks comment lines)


5 important re functions on WLST

There are few regular expression functions defined in the re module. Lets experiment with each one and see how they work.
wls:/offline> dir(re)
['DOTALL', 'I', 'IGNORECASE', 'L', 'LOCALE', 'M', 'MULTILINE', 'S', 'U', 'UNICODE', 'VERBOSE', 'X', '__all__', '__doc__', '__file__', '__name__', 'compile', 'error', 'escape', 'findall', 'match', 'module', 'name', 'purge', 'search', 'split', 'sre', 'sub', 'subn', 'sys', 'template']
The re module methods and their functionality

Lets begin here...

  1. The re.match() method
  2. The match() method works is that it will only find matches if they occur at the start of the string being searched.
    wls:/offline> import re
    wls:/offline> str="WLST WebLogic wsadmin WebSphere"
    wls:/offline> mo=re.match(r'WLST',str)
    wls:/offline> mo.group(0)
    'WLST'
    

  3. The re.split() method
  4. The re.split() method accepts a pattern argument. This pattern specifies the delimiter(s) with it, we can use any text that matches a pattern as the delimiter to separate text data. This is powerful to get the desired data from the text data.

     
    clstrAddress="machine01.vybhava.com:8901,machine02.vybhava.com:8902"
    wls:/offline>  s=re.split(r"[:,]\d{4}",clstrAddress)
    wls:/offline> s
    ['machine01.vybhava.com', ',machine02.vybhava.com', '']
    

    Now lets experiment with simple script where various forms of split works.

     
    Split method samples in WLST
    import re
    
    servers="admin, ms1, ms2, ms3, cluster1"
    print re.split('\W+', servers)
    
    print re.split('(\W+)', servers)
    
    print re.split('\W+', servers, 2)
    
    print re.split('\W+', servers, 3)
    
    # The split mehtod works with multiple delimiters
    e=os.environ['PATH']
    for p in re.split(r':', e):
        print p
    
    


  1. The re.search() method 
  2. The following examples will illustrate how to search function with re module flags without it.
    #=====================================================================
    # Created by : Manish K
    # Updated by : Pavan Devarakonda
    # Date : 20 Dec 2014
    #=====================================================================
    import re
    
    s1 = "India is a country!"
    s2 = "Delhi is the capital of India"
    str = s1 + "\n" + s2
    
    ###### To Search first char in multilne
    mo = re.search(r"^D[\w]*", str, re.M)
    print "searched word with first char D:",mo.group()
    
    ###### To Search last char in multilne
    
    mo1 = re.search(r"I[\w]*$", str, re.M)
    print "Searched India:", mo1.group()
    
    ##### To match ignoring case
    
    mo2 = re.search(r"india", str, re.I)
    print "Ignore case:", mo2.group()
    
    if re.search(r'[!@#$%^&*()]', str):
     print "Some special Char found"
    else:
     print "Nothing special found"
    
    print "Original str:", str
    splitstr = re.split("\n", str)
    print "split string with newline:",splitstr
    
    ## Optional Items
    optSearch = "Find date Feb 2015, 12"
    mo3 = re.search(r"Feb(ruary)? 2015", optSearch)
    print "Optional search:", mo3.group()
    
    ## using +
    mo4 = re.search(r"(Feb(ruary)?) ([0-9]+)", optSearch)
    print "Search with + : ", mo4.group()
    
    
    ## unsing *
    mo5 = re.search(r"[0-9].*", optSearch)
    print "Using *", mo5.group()
    
    ### Grouping by the use of ()
    
    gprStr = "Customer number: 232454, Date: February 12, 2015"
    mo6 = re.search("([0-9]+).*: (.*)", gprStr)
    print "Grouping with ():"
    print mo6.group()
    print mo6.group(1)
    print mo6.group(2)
    print mo6.groups()
    
    
    Lets execute the above experiment on search function.
    $ wlst search_re.py
    
    Initializing WebLogic Scripting Tool (WLST) ...
    
    Welcome to WebLogic Server Administration Scripting Shell
    
    Type help() for help on available commands
    
    searched word with first char D: Delhi
    Searched India: India
    Ignore case: India
    Some special Char found
    Original str: India is a country!
    Delhi is the capital of India
    split string with newline: ['India is a country!', 'Delhi is the capital of India']
    Optional search: Feb 2015
    Search with + :  Feb 2015
    Using * 2015, 12
    Grouping with ():
    232454, Date: February 12, 2015
    232454
    February 12, 2015
    ('232454', 'February 12, 2015')
    
    

  3. The re.findall() method

  4. When there is the need to find the multiple occurrences of a pattern in a text then findall() is the best function to use. This function returns a list of resulted strings.

    Example on findall method in re module.
    import re
    line="WebLogic Automation course helps WebLogic Admins fish best opportunities"
    
    print "words starts with A"
    print re.findall(r"\bA[\w]*",line)
    
    print "Find all five characthers long words"
    print re.findall(r"\b\w{5}\b", line)
    
    # Find all four, six characthers long words
    print "4, 6 char long words"
    print re.findall(r"\b\w{4,6}\b", line)
    
    # Find all words which are at least 13 characters long
    print "13 char"
    print re.findall(r"\b\w{13,}\b", line)
    
    Execution of the above example gives the output as follows:
    $ wlst findallre.py
    
    Initializing WebLogic Scripting Tool (WLST) ...
    
    Welcome to WebLogic Server Administration Scripting Shell
    
    Type help() for help on available commands
    
    words starts with A
    ['Automation', 'Admins']
    Find all five characthers long words
    ['helps']
    4, 6 char long words
    ['course', 'helps', 'Admins', 'fish', 'best']
    13 char
    ['opportunities']
    
    


  5. The re.sub() method in WLST

  6. One of the most important function in the re module is sub(). It works like 'find and replace'.
    sub(pattern, repl, string [, count=0])
    
    

    This method replaces all occurrences of the re pattern in the given string with repl, substituting all occurrences unless max count provided. This method would return modified string as output.
    #!/usr/bin/python
    import re
    
    DOB = "28-11-2003# This is DOB "
    
    # Delete Python-style comments
    num = re.sub(r'#.*$', "", DOB)
    print "DOB Num : ", num
    
    # Remove anything other than digits
    x = re.sub(r'\D', "", DOB)
    print "DOB without - : ", x
    
    # Substituting other symbol in anything other than digits
    FDOB = re.sub(r'\D', "/", num)
    print "DOB in new format : ", FDOB
    
    This find and replacement experiment output
    DOB Num :  28-11-2003
    DOB without - :  28112003
    DOB in new format :  28/11/2003
    

    Hope this has given you some basic idea on re module usage in WLST shell. This can help you when you write monitoring script and that will be given to report or graph designs on UI.
Write back to us with your Feedback, queries on regular expressions usage in WLST.

Thursday, December 25, 2014

File IO in WLST: How to store output of WLST/Jython?

Hey WLST scripting enthos Guys!!

I was prepared some of industry most reffered WLST monitoring scripts... they are running well and good... Now the need is to store the result of this monitoring script into a FILE. How to make this? how to redirect this output... normally output is coming from print command in the jython/python. I was searching in google for this output storing to a file from WLST not really good outcomes... then I thought better I choose file style as in C programming allows Standard Output to a files with file pointers. This found a jython tutorial which is already available in online...

WLST File input output operations

-->

Standard Out and Error


Standard output and standard error (commonly abbreviated stdout and stderr) are pipes that are built into every unix-like platform. The stdout (fileObject) is defined in the Python standard sys module, and it is a stream object. Calling its write() function which will print out whatever string you pass as argument to it, then return the length of the output. In fact, this is what the print function really does; it adds a carriage return to the end of the string that you’re printing, and calls sys.stdout.write function.
##############################################
# This program is to illustrate sys fileojects
# Standard Output stdout
# Standard Error stderr
##############################################
print "Normal print..."
for i in range(3):
        print "WLST is awesome!!!"

import sys
print "sys.stdout...."
for i in range(5):
        l=sys.stdout.write('WLST By Examples\n')

print "sys.stderr...."
for i in range(4):
        errline=sys.stderr.write('WLST in 12.1.3 Wonderful!! ')

Here it shows the difference of print, write function with/without carriage return example.
$ wlst fileObj_sys.py

Initializing WebLogic Scripting Tool (WLST) ...

Welcome to WebLogic Server Administration Scripting Shell

Type help() for help on available commands

Normal print...
WLST is awesome!!!
WLST is awesome!!!
WLST is awesome!!!
sys.stdout....
WLST By Examples
WLST By Examples
WLST By Examples
WLST By Examples
WLST By Examples
sys.stderr....
WLST in 12.1.3 Wonderful!! WLST in 12.1.3 Wonderful!! WLST in 12.1.3 Wonderful!!

In the simplest case, sys.stdout and sys.stderr send their output to the same place. In Python, standard output, standard error does not add carriage returns for you. If you want carriage returns. you’ll need to write manually include the carriage return characters. sys.stdout and sys.stderr are stream objects, but they are write-only. Here important fact is that when you attempting to call their read() method will always raise an IOError.

Redirecting output with file descriptor is fine worked good... 
now one more thing... need a timestamp for everytime when you run the monitoring script...
Solution again available in the same tutorial...
jython is going to support Java internally... Java's Date utility gives you the facility to have a time stamp redirect this print to the same file descriptor ... your job is done!!
fileObj = open('test.txt', 'w')
writeList=["I am an Indian", "I love my India", \
"I want to go to USA", "I want to earn dollars", \
"I want to die in India"]

for i in writeList:
 print >>fileObj, i

fileObj.close()

# Default file mode is read (r)
fileObj = open('test.txt')

# We can move fileObj position here it is 0
fileObj.seek(0)

for lines in fileObj.readlines():
 print lines

fileObj.close()
Creates test.txt with lines in list and then will read lines from same file.
wls:/offline> execfile('filereadWrite.py')
I am an Indian

I love my India

I want to go to USA

I want to earn dollars

I want to die in India


The file attributes

Following table have the list of the file object attributes. Which we can directly access them public.
AttributeDescription
file.closedReturns true if file is closed, false otherwise.
file.modeReturns access mode with which file was opened.
file.nameReturns name of the file.
# This program illustrates the file attributes

f=open('fileattr.py')
print "mode:",f.mode
print "name:",f.name

if f.closed:
        print f.name, " is closed"
else:
        print f.name," is not closed"

Its execution gives the output as follows
wls:/offline> execfile('fileattr.py')
mode: r
name: fileattr.py
fileattr.py  is not closed

Testing

with-as operation on Files

Following example shows how to use with

How do I test a simple Jython script at Jython prompt?

You can use the following lines to invoke Jython Shell on your WebLogic environment. This is possible because WLST is based on Jython Shell, obviously you can get a Jython prompt in the WebLogic environment.
Normally you can get with
$ java  org.python.util.jython

Otherwise use the following way to skip caching:
$ java -Dpython.cachedir.skip=true org.python.util.jython

Remember that, WebLogic 9.2 supports Jython 2.1 featured shell, The new Oracle Fusion middleware, WebLogic 11g supports Jython 2.2 features.

In many capacity planning analysis you need the statistcs of day or hour by hour etc., These statistics can be for JVM monitoring or for JMS or JDBC DataSource or Thread Count of a WebLogic domain.
Simply printing to standard output you can redirect them to your desired filenames which you specify in the Jython script. To do print to a file use:
f = open("JVMStat.txt", "w")
# JVM monitoasring Script loop
       print >>f, "JVM Status Line here"
f.close()
And to add to the end of a existing file use:
f = open("JVMStat.txt", "a")
       print >>f, "Added lines comes here"
f.close()

Hope this will give you hint for your need in WLST and file usage. For any issue keep writing back to me.

Enjoy with Jython!!! in WLST !!!

Friday, December 19, 2014

Object Oriented Programming in WLST

I taught Object orientation programming with C++ and Java previously. With that experiance I am  trying to explore more in the Pythonic OOP in WLST.

class definition in WLST Script


In WLST every thing is object as its base scripting language is Python suppose the same. If you create normal variable assignment uses their base objects like int, float, str etc. Each object will be created on JVM with ID, Type (object type) and value. In this post you will get to know about Object orientation for WLST scripts. The four pillars of Object orientation are supported in WLST as well.

  • Encapsulation - How you hide within object
  • Abstraction - How the object make interact with the world
  • Inheritance -Reusing functionalities of a object
  • Polymorphism - Dynamism to your methods

Why should I use OOP in WLST?

The best features identified and designed for object oriented programming mainly following:
  • Object are natural easy to understand and write
  • Objects are more reusable
  • Objects are more maintainable
  • Objects can be extended

What is class in WLST?

A class is a special user defined data type, which defines how to build a certain kind of objects. These classes also stores some common data itms that are shared by all its instances of the class. Instances are objects that are created which follow the definition given inside of the class.

In simple words class is a blue print for constructing the objects. 
Initializer method __init__
The functions that you define within the class are called methods. for all methods must use the first argument as self.

What is 'self' do in WLST?

The Self is object pointer like in C++, Java we have this pointer that refers to the newly created runtime object attributes values.In __init__, self refers to the object currently being created. In WLST classes self used more often than Java. You don't need to give a value for self. When you call the method VM automatically provides the value for self.

What is Instance?

An instance can be created in the main program that is after completing the class block. To create we need the className followed by perantesis.

i=className()

calling the method can be done with instance period followed by method signature with required parameters. 

Magical methods
There are some methods which are executed without calling them! Example initializer method, printing the object into string type using __repr__, and __str__ .When you call the instance in print command it will automaticall invoke these magical methods.

Best practics for class definitons
  • While you defining a class the class name must begin with Capital letter 
  • Explicit inheriting object type give more power to your class
  • Define a method like a function. must have a first argument as self which provide access to current instance attribute values
  • Better to have a DocString for each method you define in the class 
class level methods
static methods
Inheritance 

Multiple Inheritance

# This illustrates the subclassing
# Empty class with pass 
# usage of issubclass function

class Animal(object): pass
class Snake(Animal):
 def __init__(self):
  print "Ssss n a k e!!"
class Python(Snake):
 "I like Python class"
 atr=0
 def __init__(self):
  print "Busssshhhh!!!"
  Snake.__init__(self)
  self.atr+=1
 def __repr__(self):
  return 'This is Python object'

print "issubclass(Animal,object:)",issubclass(Animal,object)
print "issubclass(Snake,Animal:)",issubclass(Snake,Animal)
print "issubclass(Python, Snake:)",issubclass(Python, Snake)

p=Python()
print "Instance printing:", p
print "docString:", p.__doc__
print "p is from class:", p.__class__

print "p type:",type(p)
print "p id:", id(p)
print "p attribute:", p.atr
 

Private variables in class

In Python classes by default everything is public access. to provide encapsulation for the object double underscore represents the private variables(__var1). 

  Exercise for Learners

  1. Create a class for configuring a simple domain with initializer method that will load all properties that required to configure. [managed servers, machines, cluster]
  2. Create a singleton class that would allow single connection object that can be preserved and used for monitoring JVM, Threads etc.

Tuesday, December 2, 2014

I will Protect WebLogic Environments : storeUserConfig

There are many monitoring scripts that requires secure connection to admin server. Creation of username, password in encrypted and stored into a file UserConfigFile you can name it as per your domain requirements. That encryption would be done with private secure key that can be stored in another file User Key File.

Prerequisites

To run the storeUserConfig command you must be in online mode. Before you execute you need to connect to the admin server. You can create secure files for admin user or nodemanager user. nm = Optional. Boolean value specifying whether to store the username and password for Node Manager or WebLogic Server. If set to true, the Node Manager username and password is stored. This argument default to false.

WLST storeUserConfig usage

Execution of storeUserConfig command on WLST

You can execute without any parameters it will store the user secure files with the Operating systtem username and it will be stored in the domain path. storeUserConfig() You are allow to specifying the path where to store the secure files.
wls:/demodomain/serverConfig>  storeUserConfig('/home/wldomains/demodomain/demodomain.config', '/home/wldomains/demodomain/demodomain.key')

Once you create this storeUserConfig files you can use these in two scenarios where it invokes connect command on offline WLST to online WLST:

  • Interactive WLST mode
  • Script mode
  • Useful for weblogic.Admin utility
  • Useful for weblogic.Deployer utility

Once you created this secure config files for a new domain where you can apply for the following:

  • Reconfiguration new system resources like datasources, JMS 
  • Application deployments, undeployment, and redeployments
  • Monitoring WebLogic runtime mbeans.


Note: Creating the key file can reduce the security of your system if it is not kept in a secured location after it is created.

Generic script for protect your domain scripts

The following script is ready use script it is generic script it will interactive with you to connect to a domain and generates user config file, key files in the specified location. It will also validates the newly created the secure files working as expected or not.
"""
DescriptionT :  This script will create user config file, key file
Author   :   Manish Khatwani
Date   :  2/12/2014
"""

def getConnect(user, password, AdminURL):
 connect(user, password, AdminURL)

def testValues(userconf, keyconf, AdminURL):
 print "Connecting to Admin with userConfigFiles"
 connect(userConfigFile=userconf, userKeyFile=keyconf, url=AdminURL)

def createUCFUKF():
 user = raw_input("Enter user Name: ")
 password = raw_input("Enter Password: ")
 AdminURL = raw_input("Admin URL: ")
 configPath = raw_input("Enter Path for storing config files : ")
 userPath = configPath + '/YOURDOMAIN.ucf'
 keyPath = configPath + '/YOURDOMAIN.ukf'
 print "Connecting to Admin"
 getConnect(user, password, AdminURL)
 storeUserConfig( userPath, keyPath)
 disconnect()
 # Reconnecting with newly created secure files
 print "disconnected from Admin, Lets Validate..."
 testValues(userPath, keyPath, AdminURL)
 ls ()
 disconnect()

# =================== MAIN PROGRAM ================================
if __name__=='main':
 redirect('/dev/null','false')
 createUCFUKF()
 redirect('/dev/null','true')
exit()

Execution Procedure
The sample execution look like this:
java weblogic.WLST createUCFUKF.py
enter user name: weblogic
enter password: Webl0gic
enter url: t3://192.168.1.105:7001
Enter Path for stroring config files : /home/wldomains/demodomain
Connecting to Admin
Connecting to t3://192.168.1.105:7001 with userid weblogic ...
Successfully connected to Admin Server 'AdminServer' that belongs to domain 'demodomain'.
 
Warning: An insecure protocol was used to connect to the
server. To ensure on-the-wire security, the SSL port or
Admin port should be used instead.
 
Creating the key file can reduce the security of your system if it is not kept in a secured location after it is created. Do you want to create the key file? y or ny
...

Applicability to WLST
UCF='/home/wlsadmin/wldomains/demodomain/demodomain.ucf'
UKF='/home/wlsadmin/wldomains/demodomain/demodomain.ukf'
connect(UCF, UKF, admurl)

Applicability for Deployer tool

java weblogic.Deployer -userConfigFile /home/wlsadmin/wldomains/demodomain/demodomain.ucf -userKeyFile /home/wlsadmin/wldomains/demodomain/demodomain.ukf -deploy benefits.war -targets democlstr

Applicability of Admin utility

For your monitoring server state purpose you can use weblogic.Admin utility tool use GETSTATE.
#######################################################
# FileName         :           serverstate.sh
# Date             :           03/12/2014
#######################################################
clear
ucf=/home/wlsadmin/wldomains/demodomain/demodomain.ucf
ukf=/home/wlsadmin/wldomains/demodomain/demodomain.ukf
admurl=t3://192.168.1.105:7001

java weblogic.Admin -url $admurl -userconfigfile $ucf -userkeyfile $ukf GETSTATE demoMan1
java weblogic.Admin -url $admurl -userconfigfile $ucf -userkeyfile $ukf GETSTATE demoMan2

GetSate using weblogic.Admin utility

Current state of "demoMan1" : RUNNING


Current state of "demoMan2" : RUNNING

Note this out is executes multiple times JVM creations. Better way is using WLST :)
Thats all for now...
Cheers guys,


share your thought on this post!!

Popular Posts

Oracle WebLogic Administration Course

Oracle WebLogic Administration Course
WebLogic Training

Followers