new Puppet 3 Weblogic provisioning module

The last few weeks I was busy re-writing of my puppet WLS module so it fully supports the power of Puppet 3 (thanks for more than 4000 downloads on puppet forge and all the github downloads).

With Puppet 3 we now can use Hiera, Iterations and Lambdas expression. This does not sound like a big change but with Hiera and the new Puppet Language features, I can define big WebLogic Domains without losing maintainability ( Got one customer with 5 Clusters, 25 Managed + JMS servers, more than 100 queues & topics).

With Puppet version 2 and my WLS module you need to have a lot of files (site specific classes) with at least more than 2000 a 5000 lines, here is an example of this . In this ORAWLS module I use the default/minimal approach ( declare once in Hiera) so you won't need to declare the same parameters over and over again. Off course you can still define everything like you did in the WLS module.

When you want to know more about Hiera you can take a look at this documentation page. For the new Puppet 3 language features look at this slideshare presentation.

In this post I will show you the new features and how it works with Hiera. Here you can download or take a look at the new orawls github code, or download it from puppetlabs forge. I Also made a vagrant reference implementation ( which you can use to test it yourself ),  this reference will create a 3 node WebLogic 10.3.6.0.6 Domain with 2 Clusters together with some JMS modules and Queues.

The first big change is that the puppet install classes does not contain any site specific code, I have now only 2 classes which I can use everywhere

• One for the WebLogic Admin server which contain all the possible WebLogic options
• One for WebLogic nodes which only installs the software, copy domain and configure the nodemanager )

Let's begin with installing the WebLogic software.  For this I only need to add this to a class.

include orawls::weblogic

orawls::weblogic is a class and in Puppet 3 it will look in Hiera to find the parameter values ( it uses the following hiera lookup module_name::class_name:parameter_name )

Here is a snippet of my common.yaml file

# global WebLogic vars
wls_oracle_base_home_dir: &wls_oracle_base_home_dir "/opt/oracle"
wls_weblogic_user:        &wls_weblogic_user        "weblogic"
wls_weblogic_home_dir:    &wls_weblogic_home_dir    "/opt/oracle/middleware11g/wlserver_10.3"
wls_middleware_home_dir:  &wls_middleware_home_dir  "/opt/oracle/middleware11g"
wls_version:              &wls_version              1036

# global OS vars
wls_os_user:              &wls_os_user              "oracle"
wls_os_group:             &wls_os_group             "dba"
wls_download_dir:         &wls_download_dir         "/data/install"
wls_source:               &wls_source               "/vagrant"
wls_jdk_home_dir:         &wls_jdk_home_dir         "/usr/java/jdk1.7.0_45"
wls_log_dir:              &wls_log_dir              "/data/logs"

#WebLogic installation variables 
orawls::weblogic::version:              *wls_version
orawls::weblogic::filename:             "wls1036_generic.jar"
orawls::weblogic::middleware_home_dir:  *wls_middleware_home_dir
orawls::weblogic::log_output:           false

# hiera default anchors
orawls::weblogic::jdk_home_dir:         *wls_jdk_home_dir
orawls::weblogic::oracle_base_home_dir: *wls_oracle_base_home_dir
orawls::weblogic::os_user:              *wls_os_user
orawls::weblogic::os_group:             *wls_os_group
orawls::weblogic::download_dir:         *wls_download_dir
orawls::weblogic::source:               *wls_source

Next step is to add some WebLogic 10.3.6 patches (BSU)
Here is the code in the admin class which does a hiera lookup of bsu_instances and call create_resources for every BSU entry.

$default_params = {}
$bsu_instances = hiera('bsu_instances', [])
create_resources('orawls::bsu',$bsu_instances, $default_params)

and the matching yaml entry

# patches for WebLogic 10.3.6
bsu_instances:
  'BYJ1':
    patch_id:                "BYJ1"
    patch_file:              "p17071663_1036_Generic.zip"
    log_output:              true

  The same mechanism for all installing all the Fusion Middleware software

# FMW installation on top of WebLogic 10.3.6
fmw_installations:
  'osbPS6':
    fmw_product:             "osb"
    fmw_file1:               "ofm_osb_generic_11.1.1.7.0_disk1_1of1.zip"
    log_output:              true
  'soaPS6':
    fmw_product:             "soa"
    fmw_file1:               "ofm_soa_generic_11.1.1.7.0_disk1_1of2.zip"
    fmw_file2:               "ofm_soa_generic_11.1.1.7.0_disk1_2of2.zip"
    log_output:              true

The next step is create a WebLogic domain.

When you only have one domain then you can set the following hiera variables. These entries are also automatically picked up by the other WebLogic classes. With 2 domains or more you need to set the domain parameters everything you call a module function.

logoutput:                     &logoutput                     true

# when you have just one domain on a server
domain_name:                "Wls1036"
domain_adminserver:         "AdminServer"
domain_adminserver_address: "10.10.10.10"
domain_adminserver_port:    7001
domain_nodemanager_port:    5556
domain_wls_password:        "weblogic1"
domain_user_config_file:    "/home/oracle/oracle-Wls1036-WebLogicConfig.properties"
domain_user_key_file:       "/home/oracle/oracle-Wls1036-WebLogicKey.properties"

# create a standard domain
domain_instances:
  'wlsDomain':
     domain_template:          "standard"
     development_mode:         false
     log_output:               *logoutput

For more examples take a look at the readme page of het orawls module or look at the vagrant reference implementation.

The next part is used for bulk insert of WebLogic artifacts and do this with minimal configuration. this feature has some requirements

• puppet version > 3.2 ( make use of iterations and lambda expressions )
• set --parser future ( puppet agent config )

I also use hiera_array instead of hiera, this will search for this entry in all the hiera data files. This way I can separate application or cluster code in its own yaml file and created in one go.

example of how to call this wlstbulk define

$allHieraEntries = hiera_array('jms_module_jms_instances')

orawls::utils::wlstbulk{ 'jms_module_jms_instances':
  entries_array => $allHieraEntries,
}

And the matching hiera entry, this one supports global parameters so you won't need to add these params to every WLST entries.

This saves you a lot of extra configuration especially with many WLST entries and make your definitions much clearer

managed_servers_instances:
   - clusterOne:
      global_parameters:
         log_output:           *logoutput
         weblogic_type:        "server"
         script:               'createServer.py'
         params:
            - "listenAddress    = 8001"
            - "nodeMgrLogDir    = '/data/logs'"
      wlsServer1_node1:
         weblogic_object_name: "wlsServer1"
         params:
            - "javaArguments    = '-XX:PermSize=256m -Xms752m -Xmx752m'"
            - "wlsServerName    = 'wlsServer1'"
            - "machineName      = 'Node1'"
      wlsServer2_node2:
         weblogic_object_name: "wlsServer2"
         params:
            - "javaArguments    = '-XX:PermSize=256m -Xms752m -Xmx752m'"
            - "wlsServerName    = 'wlsServer2'"
            - "machineName      = 'Node2'"

So I hope this module will help you with all the WebLogic provisioning tasks.

Creating your own Virtualbox Development Images

For my Oracle Puppet provisioning development I can't do without these create image tools: Packer and Vagrant in combination with Oracle VirtualBox or VMware.  In this blogpost I will explain what these tools can do for you and how you can make your own images and use puppet as provisioning tool.

With Vagrant you can create your own virtual images and it can start puppet or chef to do all the server provisioning. Besides this, Vagrant can also manage all the network configuration or make a multi machine configuration. One of the great things of Vagrant is when you make a mistake you can start over by destroying the image ( vagrant destroy ) or start the provisioning again ( vagrant provision).

For Vagrant you need to download and import a small image box ( here is  an overview of the all public boxes which you can use right away )  like CentOS or Ubuntu. This image already contains the Virtualbox guest additions, which Vagrant will use to setup the shared folder etc.

Packer is a tool which can create this vagrant box for you. With this you can create a virtual image which also can be used for Amazon EC2, VirtualBox or VMware. In my case I need to have a particular image with the latest VBox guest additions and the latest Puppet RPM's. With packer I am now be able to create the same images as which my customer uses and test it on my own laptop.

How does this work?.  ( you can download a full working example at my Github page which will create a CentOS 6.4, 5.8 or Ubuntu images and this will also add Puppet )

First we need to define a json file and a Red Hat kickstart file.

here is an example of a CentOS 6.4 example.  This file contains the url of CentOS minimal or network iso, the Vagrant setup steps and post installation scripts, like the puppet install.

and the CentOS kickstart file


Next add the packer directory to your path variable , go to the packer github folder and use this command
packer build centos-6.4-x86_64.json

Wait some minutes and you will have a new box. Packer also shrinks this image to 600mb and is now ready to use it for Vagrant.


Next step is to download my vagrant github folder
When you go to this directory and use the vagrant up command, this will download the CentOS box, create a Virtualbox image and start the Puppet provisioning.

here is my example of a vagrant single node configuration, this will forward some ports and starts the Puppet site.pp class.

Here is an overview of my puppet folder

This is a shared folder ( virtual image and your machine ) and you can edit these Hiera and Puppet files in your fav editor, Add the needed puppet modules and start the provisioning again by using this command: vagrant provision or do it all over again by destroying it first: vagrant destroy -> vagrant up

Also vagrant supports multi machine configuration, just look at Vagrantfile_multinode.

Here is an working 3 node example, 3 VirtualBox Servers with a working WebLogic 10.3.6.0.6 Cluster which uses a private network.
https://github.com/biemond/biemond-orawls-vagrant 

Hope these will also help you.