Wednesday, December 3, 2008

Flat file to xml with Oracle Service Bus

With OSB you can easy transform a formatted flat file to a xml. To do this in OSB you have to create a MFL and XQuery definition. MFL is the output of the Format Builder tool where you can create descriptions of non XML files. XQuery can use the output of MFL to generate the xml.

This is an example of the flat file
10;0123456789;MR;JOHN;SMITH
20;0123456789;ACCT1
30;0123456789;SALARY;500000
30;0123456789;BONUS;1000
20;0123456789;ACCT2
30;0123456789;OTHER;100
10;1234566790;MR;DAVID;DOE
20;1234567890;ACCT1
30;1234567890;SALARY;10000

And this is the record definition of the flat file

one block is made of:
one record 10 (customer)
1 to N record 20 per record 10 (accounts)
1 to N record 30 per record 20 (transactions)

record 10
customer id
customer lastname
customer firstname

record 20
customer id
account id

record 30
customer id
label
value

Create a new MFL in the Oracle Workshop for Weblogic ( Eclipse plugin )

This will start Format builder tool where we can define the records groups and test the result.

First we add group 10 ( customer ) as child of the main element. Then we define the fields of recordtype 10. As child group of customer we add the Account group etc.

Here is a picture of the customer group where I enabled that the group is tagged option, with as value 10; and the group occurence is unlimited.
I add a child field which can occur once and has ; as delimiter

The last field of the record has as delimiter \n ( end of line )

Now we can test it ( menu tools and test ). Load the non xml data and transforms this to xml.

I made a new xml schema with I can use as target in Xquery. This is the xsd
Now we can create a new XQuery file where we can use the just created MFL definition as non xml source type and the xml schema as target type.


Let's map the source element to the targets element.
We are finished with XQuery and now we can create a proxy service. This proxy does the transformation and pass the xml output to an other service.

Create a new proxy service with messaging as service type. As Request message type we have to use the just created MFL.

Create a new message flow in this proxy service where we add 3 operations ( 2 assign and one insert )

In the first assign we will use the XQuey file and drag the personel in the body variable to the XQuery variable. Pass the output of this assign to the xml variable

In the second assign we will create a new body and pass this to the body variable

The last operation is insert, In this operation we copy the content of the xml variable to the body variable.

That's all now we have converted the flat file to a xml which we can passed on to other business services.

10 comments:

Shining Bulls said...

Hi I have been trying to do create this proxy but we dint create and business service or any thing and in the routing table it is asking for the service to be invoked which service do we need to specify????

Do we create some business service??
can you help me with this?

Edwin Biemond said...

Hi

you need to have an endpoint of the proxy service. You transform text to xml and then?. So you have to make a file adapter business service which put the output on a disc and select this bs in the routing table.

thanks Edwin

Nitin said...

Hi Edwin thanks for the reply but i am still unable to create this proxy service without problems i have created a proxy service with messaging service as the service type then specified the mfl file we created in this example as the request message type. response type as none.
1.) In the transport i was not sure what to chose is it http/jms or anything else??
I chose jms and the default end point as jms://localhost:7001/weblogic.jms.XAConnectionFactory/JmsRequest

left the jms tranport/ message content and security tabs to defaults
and then specified the message flow as explained by you

Can you explain the file adapter business service specific settings ?

Edwin Biemond said...

Hi,

can you take a look at this
http://biemond.blogspot.com/2008/10/oracle-service-bus-103.html

and here an example how to create a jms queue http://biemond.blogspot.com/2008/10/jms-in-jdeveloper-11g-and-weblogic-103.html

thanks

Anonymous said...

Hi biemond..
The thing is like I was trying out with this example with a csv file.
like
1,xyz
2,yyy
3,www

I was able to test the MFL and was able to transform the non-xml to xml. The problem is in the Query transformation part.

I couldn't get all the elements in the source side(when referring to non-xml data). Is there any thing that I need to do other that this??

Edwin Biemond said...

Hi, can you send me the testcase
and let me know what the result should be.

biemond at gmail dot com

thanks

Anonymous said...

Hi Edwin,

I made whole service and trying to test . i am literally confused what to pass as input payload is it flat file data or xml. I think it should be flat file data as we made proxy service based on mfl. Please guide me.

Edwin Biemond said...

Hi,

your flat file is the input. the proxy service knows this and translate this to xml and then pass it on as xml

thanks Edwin

Anonymous said...

What type of transport i need is it http or jms? i selected http and
When i post flat file i get following error
Failure while unmarshalling message: Failed to transform MFL content from binary to XML

Please help.

Anonymous said...

Hi

I get the same error as above "Failure while unmarshalling message: Failed to transform MFL content from binary to XML".
I am using a flat file (csv), transport is simple file.
In test mode of the sbconsole i can see only an empty body as first line in the invocation trace - the error handler...??