Scripted AIA installation and JDeveloper

To install Oracle AIA 11.1.1.5.0 with SOA Suite 11.1.1.5.0 you need a JDeveloper installation.

Apparently it's a very bad idea to install JDeveloper in the same MIDDLEWARE_HOME as the Middleware Products as it causes conflicts with Java Faces

So if your AIA installation fails because it can't restart the servers during the installation take a look in your server log files. If you see errors where it complains that it can't find javax/faces/application/ProjectStage make sure your JDeveloper installation isn't in the same MIDDLEWARE_HOME.

Adventures in Eggdrop TCL Scripting - Part 2

This is the second part of my "Adventures in Eggdrop TCL Scripting" posts. In the first post I wrote down the solutions to some simple TCL specific issues I had. In this post I will explain how I solved some eggdrop related issues I faced.

Eggdrop IRC interaction
To make you eggdrop bot useful you want to be able to give commands that eggdrop will catch and process. This is done by using eggdrop binds like this:
bind msg - "start" start;

This will bind run the start method every time someone messaged the Eggdrop bot directly typing: start

Sometimes you want the bot to listen for commands in the channel where the Eggdrop bot is active. This is done with the following bind command:
bind pub - "!start" start;

The above example is a bit dumb but notice how it says "bind pub" instead of "bind msg". This bind will listen in all channels where the bot is active and run the start method whenever someone writes !start in the channel.

When you've issued a command to your Eggdrop bot, you would most likely like it to give you some sort of response. If you just want to respond to a specific channel where it resides you could do the following:
channel "#somechannel"
putserv "PRIVMSG $channel : Response String";

But sometimes you don't want it to respond to a channel but to you in a private message. To do so you would do something very similar like this:
nick "" putserv "PRIVMSG $nick : Response String";

Notice how it's almost identical to the above, but instead of a channel name it uses an IRC nick.

Eggdrop Timers
After building simple request/response kind of functionality it was time to try a new thing. We wanted some functionality to announce once new things happened in our guild. That required Eggdrop to periodically check for news and then announce those news.

For that purpose I utilized Eggdrop's timer functionality.
proc run_periodically {} {
  #DO SOME CODE HERE

  timer 10 run_periodically;
  return 1;
}

The above code will run some code (checking for news and output to channel) and the call the timer functionality. The above example will then wait 10 minutes and execute the same function again.

There are a few caveats to this though. You have to be careful how you call the run_periodically method the first time. If you just do it on load time of the script it will get called if you rehash eggdrop WITHOUT cancelling the previous timers. For this purpose I created some start and stop methods to control the execution of the periodically needed code.
proc hasTimers {} {
  set timerList [timers];
  return [llength $timerList];
}

proc start {nick uhost handle text} {
  if {[hasTimers] == 0} {
    #No current timer is active, meaning its safe to start running periodically
    run_periodically
  } else {
    #Do nothing as a timer is already active, meaning it is already running periodically
  }
}

proc stop {nick uhost handle text} {
  set timerList [timers];
  foreach timer $timerList {
    killtimer [lindex $timer 2];
  }
}

In the above code there is a few things to note. First is the hasTimers method which is a utility method for checking if we have any currently running timers. Notice the nick, uhost, handle and text parameters in the start and stop methods. Those parameters are needed by eggdrop as the start and stop methods are called using Eggdrop bind commands explained above. Also notice the killtimer command to kill a currently running timer. Having these methods allows the user to control whether or not the timed methods should be run or not.

Adventures in Eggdrop TCL Scripting - Part 1

During the last couple of weeks my spare time has gone into writing a few scripts for Eggdrop for use in my WoW guilds IRC channel. The idea was to add some functionality like looking up character information on the armory and announcing to channel when someone achieves something. We wanted some internal functionality for displaying upcoming events and signups for those events as well.

In this post I will write about some of the experiences I've made during development. I've never done any TCL scripting before so it was all new to me. So some of the solutions described below could most likely be done in better ways.

TCL HTTP Requests
The first thing I needed to figure out was how to create a HTTP request and do something sensible with the response. I ended up using the solution below:


proc armory::requestCharacter {characterName realm} {
  set url "http://someurl.com"

  if { [catch {set token [::http::geturl "$url" -timeout 6000]}] } {
    puts "Error making http request!"
  }

  if {([info exists token]) && ([::http::status $token] == "ok")} {
    set response [::http::data $token]
    return $response
  } else {
    return ""
  }


}


The code seems to be a fairly standard way of making a HTTP request and it works fine at least for my purpose. A slightly more advanced method could be to add the -useragent parameter using ::http::config

TCL URL Encoding
Once I started doing HTTP request I quickly found the need to create a method that could do URL encoding for me. I ended up using the below method:


proc urlencode {str} {
  set uStr [encoding convertto utf-8 $str]
  set chRE {[^-A-Za-z0-9._~\n]}; # Newline is special case!
  set replacement {%[format "%02X" [scan "\\\0" "%c"]]}
  return [string map {"\n" "%0A"} [subst [regsub -all $chRE $uStr $replacement]]]
}


The above code is a ripoff from something I found on the net somewhere (unfortunately I can't seem to find the reference now)


TCL Regular Expressions
As a big part of the script I built relies on data from either an HTML og JSON response I needed to fetch the relevant data from the responses. For that I rely heavily on TCL regex functionality. Basically I use a variation of the below to fetch lines from a result response. This fetches all lines with the below href tag in it.

set newsItems [regexp -all -inline {SEARCHSTRING.*?\.} $fileString]; 


The important thing to notice here is the regexp method. -all means get all hits and return them as a list of strings.

As I'm really poor at writing good regular expressions I opted to get some values using the good old fashioned way of getting substrings in the following manor:

set length [string length $search];
set begin [string first $search $response 0];
set end [string first "," $response $begin];
return [string range $response [expr int($begin+$length+2)] [expr int($end-1)]];


The idea here is to get the first occurence of the $search string. Then look for the next ',' sign after that occurence. I then do some calculations from those values to fetch the substring between those two pointers that I'm after. I know this seems stupid, but I thought I would mention it anyway. The dream is of course to someday rewrite this to use regex all the way, but that is something for another day.


So far I've only covered TCL specific things. None of this actually interacts with Eggdrop. Stay tuned for part two of this post where I will give some examples on how I used Eggdrops timer functionality to periodically execute some code. And how eggdrop interacts with IRC.

WLST Offline - Creating a Cluster


This is a post in the "WLST Offline" series. This series of posts will be small tidbits of creating various objects in WLST Offline. Mostly this is useful for easily remembering how to do these things when creating your own WLST domain creation scripts in offline mode.

Creating a Cluster
Creating a cluster in the domain using WLST Offline is done by doing something similar to the following:

cd('/')
create(CLUSTER_NAME, 'Cluster')

In the above, please replace CLUSTERNAME with proper values. After creating the Managed Servers they can be assigned to the cluster like this:

cd('/')
assign('Server',SERVERNAME,'Cluster',CLUSTER_NAME)


Of course you should replace SERVERNAME and CLUSTERNAME with the proper values.

WLST Offline - Creating a Managed Server


This is a post in the "WLST Offline" series. This series of posts will be small tidbits of creating various objects in WLST Offline. Mostly this is useful for easily remembering how to do these things when creating your own WLST domain creation scripts in offline mode.

Creating Managed Server
Creating managed servers in the domain using WLST Offline is done by doing something similar to the following:

cd('/')

create(SERVERNAME, 'Server')
cd('Servers/'+SERVERNAME)
set('ListenAddress',HOST)
set('ListenPort',PORTNUMBER)
set('Machine', HOST)


In the above, please replace SERVERNAME, HOST and PORTNUMBER with proper values.

WLST Offline - Creating a machine

This is a post in the "WLST Offline" series. This series of posts will be small tidbits of creating various objects in WLST Offline. Mostly this is useful for easily remembering how to do these things when creating your own WLST domain creation scripts in offline mode.

Creating Machines
Creating machines in the domain using WLST Offline is done by doing something similar to the following:

cd('/')
create(MACHINENAME, 'Machine')
cd('/Machine/'+MACHINENAME)
create(MACHINENAME, 'NodeManager')
cd('NodeManager/'+MACHINENAME)
set('ListenAddress',HOST)
set('NMType', 'plain')

In the above, please replace MACHINENAME and HOST with proper values.

Run ODI configuration silently with Oracle RCU

I had to run the Oracle RCU (Repository Creation Utility) silently to create a Master and Work repository for Oracle Data Integrator recently. That proved quite the annoyance. I kept getting the following error:

Value of custom variable of type NUMBER can not be null.
RCU-6091:Component name/schema prefix validation failed.



After fighting with that for a couple of hours and exploring various ways of setting custom variables when running RCU silently, I finally came up with the solution. I'm a little annoyed it took me so long to figure out, but the solution was to use the same method as when supplying passwords for the tool.

When running the RCU silently you can supply the needed database and schema passwords using a simple text file like this:


./rcu -silent -createRepository -connectString -dbUser sys -dbRole sysdba -schemaPrefix DEV -component ODI -f < passwords.txt


Basically each line the the passwords.txt file corresponds to the answer of each promted question from the RCU tool. So a normal passwords.txt file would look like this:

DB_PASSWORD
SCHEMA_PASSWORD


But in the case of running the ODI Master and Work Repository creation you need to supply additional values, so the result looks similar to this:

DB_PASSWORD
SCHEMA_PASSWORD
001
SUPERVISOR_PASSWORD
D
001
WORKREPO
WORK_REPO_PASSWORD


Please note that in the above examples DB_PASSWORD, SCHEMA_PASSWORD, SUPERVISOR_PASSWORD and WORK_REPO_PASSWORD should be replaced by the corresponding values for your environment.

In the above the numbers 001 and 001 is the ID values of the MASTER_REPO and WORK_REPO.

Scripted OSB installation

When doing a scripted Oracle Service Bus installation there is a few things to consider as there is a bug in the way that the template builder works for OSB. The following is a known issue but can fairly easily be worked around.

If you experience errors like this
The current login role is not authorized to use the console action: RANDOM_URI
when logging into /sbconsole then you have to apply the following workaround.

When creating your template file, make sure to copy the following files:

$DOMAIN_HOME/security/DefaultAuthorizerInit.ldift
$DOMAIN_HOME/security/XACMLAuthorizerInit.ldift
somewhere safe.

After building the domain using your generated template it is important that you copy those two files into $DOMAIN_HOME/security/ BEFORE trying to start the domain. That should fix the above errors. Of course this mean a bit of post domain creation work in your scripting but it is easily done.

AIA Installation Learnings - Part II

This is the second part of the AIA Installation Learnings series (The first part can be found here). Today I learned another valuable lesson when it comes to the installation of AIA Foundation Pack on top of the Oracle SOA Suite.

Remember to set Listen Address
Apparently the AIA installation scripts uses information from the configuration of SOA Suite without being very intelligent about it. Long story short: Don't forget to set the Listen Address of the Managed servers in your SOA cluster.

If you are using WLST to generate your domain, you will want to add something similar to this in your script:

cd('/Servers/soa_server1')
set('ListenAddress', HOSTNAME)


Where HOSTNAME is the hostname of the interface where soa_server1 is listening. This should be done for all the managed servers in your SOA Suite cluster. Otherwise you might get an MalformedURLException during the installation.

About 2 Months with Gnome Shell 3.2

I've been an avid Ubuntu user for several years now. I even liked the Unity desktop and have been using that for a long time as well. But after the update to 11.10 there was just to many things wrong with Unity. They made it impossible hard to remove Gwibber and other if their indicator entries. And I couldn't keep ignoring the entries in top of the other bugs and missing features I just to just work around.

So I decided to give Gnome Shell a try, and after using it for about two months I decided it was time to write down some of my impressions with it. My initial impression is that I quite like it. However there are a few things I would like to change. Forgive me if these things have already been reported as bugs and fixed in later versions.

Notifications

  • I don't understand why some notifications stays persistent on the screen. For instance when a portable drive (usb stick) is inserted it shows a notification that the drive is ready. That notification doesn't disappear until you click either one of the action buttons or the black edge. I would also like to see the possibility to turn of those notifications as they appear every time I boot my laptop at home (as it mounts nfs drives at home)
  • Chat notifications are very handy. I like that you can keep your conversations in the notifications, but for some reason you can't keep them persistent and one wrong click and it opens the empathy window instead. I would like the ability to keep my conversations in those notifications.
Chat
  • I would love to see even deeper integration between gnome shell and empathy (chat). For instance you can search for contacts from empathy in the Activities, but you can't initiate a chat with that person... You can only see their contact details (even though you can see their online status) That just seems inconsistent.
  • I would also love to see the ability with keeping chats in the notifications described above.
Activities
  • It would be nice with better navigation between searched items in the Activities view. Once you searched for something partly and a few suggestions remain, it's not always obvious that only the up/down arrow keys can be used to navigate horizontally between suggestions.
Google
  • As I'm a Google fanboy, I would like to see even deeper integration with google products such as documents, gmail and calendar. READ: Please get rid of the dependency on evolution to be able to sync google calendars to the calendar widget.
Having listed some of the things I would really love to see in Gnome Shell, I have to say I'm quite impressed with how well it works and how stable it is. It's pleasant to use and doesn't have all the small annoyances in Unity. Ok, to be fair, it has a few things that gets a bit annoying (see above) but overall it just feels more finished than Unity. 

AIA Installation Learnings - Part I

By doing a scripted installation of Oracle AIA on top of an Oracle SOA Suite, I've learned a few things I thought I'd share.

Start SOA suite with NodeManager
There is a requirement from AIA that the SOA Suite domain has to be running while doing the installation. Not only does it have to be running. All servers MUST be started using NodeManager or the installation is not able to restart the servers during the installation. Please Note that this also includes the Administration Server.

Starting servers in a Cluster
Another thing is that to start all the servers in the cluster it can be necessary to enroll a specific nodemanager in the domain by doing a nmEnroll before starting the servers. I build my startup WLST script like this:

connect(ADMIN_USERNAME, ADMIN_PASSWORD, "t3://"+ADMIN_HOST+":"+ADMIN_PORT)
if ADMIN_HOST != CURRENT_HOST:
nmEnroll(DOMAIN_HOME, NM_HOME)

start(SERVER_NAME)
disconnect()



In the above the variables in Upper case should either be initialized or replaced with the appropriate values. But this should give you an idea on what needs to happen on servers not running the Administration Server

4 weeks of Google+

I've used Google+ for about 4 weeks, and my initial impressions are good. I find it so good that I've deactivated my facebook account to see if I can live with only using Google+ for my personal social networking.

The Pros:

  • Very easy to share the stuff you want, with only the people you want.
  • Intuitive (at least for me) to use
  • Good integration with other Google products (and getting better by the week)
  • Good Android application
The Cons:
  • Of course there is a lot of my contacts from facebook missing still.
  • Poor integration to other services (Boxee.tv etc.)
  • Integration to Blogger could be better.
So far so good, lets see if Google can keep it up and keep evolving Google+ to be the perfect social networking site.

Using sed to replace properties

I found myself wanting to replace some values in a property file using a bash shell script. The easiest way of doing that is by utilizing sed. I ended up doing the following to replace a value in the properties file (well actually it was the AIA installation response file, but the idea is exactly the same).

sed -ie s/^PROPERTYNAME=.*/PROPERTYNAME=VALUE/ PROPERTY_FILE

Below is an example:

sed -ie s/^hostname=.*/hostname=techblog.moerks.dk/ server.properties

Constructing replacements this way is a nice way of replacing property values as it is not dependent or special tags or anything like that.

Fixing SOA Suite coherence clustering

During an investigation into another issue I investigated the SOA Suite 11g coherence configuration. This led me to configuring for unicast instead of the default multicast. The configuration is actually quite simple. All that is needed is to change a few parameters in the setDomainEnv.sh file.

The line EXTRA_JAVA_PROPERTIES line in the setDomainEnv.sh file should be changed to:
EXTRA_JAVA_PROPERTIES="${EXTRA_JAVA_PROPERTIES} -Dsoa.archives.dir=${SOA_ORACLE_HOME}/soa -Dsoa.oracle.home=${SOA_ORACLE_HOME} -Dsoa.instance.home=${DOMAIN_HOME} -Dtangosol.coherence.log=jdk -Dtangosol.coherence.wka1=server1.example.com -Dtangosol.coherence.wka2=server2.example.com -Dtangosol.coherence.localhost=server1.example.com -Djavax.xml.soap.MessageF actory=oracle.j2ee.ws.saaj.soap.MessageFactoryImpl -Dweblogic.transaction.blocking.commit=true -Dweblogic.transaction.blocking.rollback=true -Djavax.net.ssl.trustStore=${WL_HOME}/server/lib/ DemoTrust.jks"

This would change the default multicast configuration to a proper and recommended unicast configuration.

Another possibility is to do what is in the high availability guide. The reason I don't like doing it that way is that the end result will be that the servers will have both configurations and that could lead to misunderstandings and confusion, not to mention I'm not convinced it even works.

Starting Weblogic Nodemanager from OEM GridControl 11g

After fighting for a long time with starting Weblogic nodemanager using Oracle Enterprise Manager GridControl 11g. I think I finally found the solution.

The problem was that when running the directive in GC that should start the nodemanger, it would do it, but never finish the task. That made it impossible to continue the GC flow. Apparently Nodemanager writes stuff to stderr, so the solution was actually quite simple. Redirect stderr to stdout. So doing something like this worked.

$ nohup ./startNodeManager.sh > /tmp/nm.log 2>$1 &

This finally made it work. Yay!

Scripting Oracle SOA Suite 11g installation

Installing the Oracle SOA Suite 11g using the silent installer in a script can be very frustrating because the Universal installer actually spawns a new process during execution. So running the following in a script will give you erroneous behavior.

$ ./runInstaller -silent -response $RESPONSE_FILE -invPtrLoc $ORACLE_HOME/oraInst.loc -jreLoc $JAVA_HOME &
The problem is that the script continues its execution path as soon as the installer spawns of its child process. Looking over the documentation didn't reveal much, but I did eventually find a solution, which of course is fairly obvious. Just append the -waitforcompletion flag to the execution line like this.

$ ./runInstaller -silent -response $RESPONSE_FILE -invPtrLoc $ORACLE_HOME/oraInst.loc -jreLoc $JAVA_HOME -waitforcompletion &
This will ensure that the internal child process completes before the initial process is considered done.

Removing Indicator entries in Ubuntu 11.10

On my computer at home I never user a dedicated email client (Thunderbird in 11.10) and therefore wanted to remove the entry in the Message Indicator. I also have an aversion against Gwibber so that had to go as well. Normally I just remove the applications from the computer, but in 11.10 you can't remove Gwibber without removing the entiry ubuntu desktop (WTF?). Anyway I finally found this post explaining how to do it.

Unfortunately it didn't work for me, so I removed the entries instead of blacklisting them like this:


$ rm /usr/share/indicators/messages/applications/thunderbird
$ rm /usr/share/indicators/messages/applications/gwibber.indicator
After logging out and back in, the entries was gone. Excellent!

Problem with Oracle Service Registry domain template

My normal way of creating scripted domains shown here. Doesn't work very well with OSR, for some reason the domain template created by the pack script is corrupt. It took me a lot of digging to figure out what the problem was. Apparently the template-info.xml file contained inside the domain template jar file is incorrect and therefore unparseable which is also indicated by the error shown.

com.oracle.cie.domain.script.jython.WLSTException: com.oracle.cie.domain.script.jython.WLSTException: com.oracle.cie.domain.script.ScriptException: unable to parse "template-info.xml" from template jar <TEMPLATE FILE NAME>

The workaround was to unzip the domain template jar file and change the line shown later in the template-info.xml file. Then zip the resulting files into a new jar file. So something like this.


$ mkdir template
$ unzip template.jar -d template
$ cd template
$ vi template-info.xml


Then change the following line:


<comp-ref name="Oracle Service Registry" symbol="Oracle_Service_Registry_11.1.1_registry111_ORACLE_HOME" version="11.1.1"></comp-ref>

To the following:

<comp-ref name="Oracle Service Registry" symbol="Oracle_Service_Registry_11.1.1_registry111_ORACLE_HOME" version="11.1.1.0"> </comp-ref>

Notice the version number change. Then zip up the template jar file again with:

$ zip -r newtemplate.jar *

WLST should be able to read the resulting newtemplate.jar template file. Don't ask me why this is a problem, but at least that fixed it for me.

How to change Datasources in WLST offline


This is the second post in a series of posts giving small examples of things you can do to create WebLogic domains using offline WLST. Use this post as a starting point.


Changing Datasources is a very common task when creating domains using a predefined template. Alternatively you can create new datasources. But this examples shows how to change a datasource used by Oracle SOA Suite. The following snippet of WLST code shows how to change the URL, Password and DB user of a datasource.


DB_URL = 'jdbc:oracle:thin:@'+DB_HOST+':'+DB_PORT+'/'+DB_SERVICE
#SOADataSource
cd('/JDBCSystemResources/SOADataSource/JdbcResource/SOADataSource/JDBCDriverParams/NO_NAME_0')
set('Url',DB_URL)
set('PasswordEncrypted',' mynewpassword')
cd('Properties/NO_NAME_0/Property/user')
set('Value', 'DEV_SOAINFRA') 



Remember to change the folder (the cd command) to the right datasource before setting the values.

How to change machine settings in WLST offline

This is the first post in a series of posts giving small examples of things you can do to create WebLogic domains using offline WLST. Use this post as a starting point

When doing a scripted installation and configuraiton of a weblogic domain you will want to change as much of the needed configuration as possible using offline wlst.

One thing I find myself doing everytime I create a new domain is changing the settings of the machine (nodemanager) configuration and that is changing its setting from SSL (default) to plain because I find that SSL communication to the nodemanagers is seldom necessary. So when doing a scripted installation I would want a similar configuration change done by the script. The following is a snippet of WLST to achieve exactly that.

cd('/AnyMachine/Machine1/NodeManager/Machine1') set('ListenAddress','server1hostname')
set('NMType','plain') 

The important thing to notice here is setting the NMType variable to plain.