Tuesday, March 31, 2015

Converting SAP IDoc XML date and time to a database date value

In one of my last blog entries, I descriped how to convert a xsd:string date and time value from SAP IDoc into the more appropriate xsd:dateTime value. While for most cases, specifying the required date and time parameters of xsd:dateTime is sufficient, it might not for database writes. Databases often require the precision of the optional milliseconds and especially timezone parameters of xsd:dateTime when writing the a DATE field.

The following POC will show how to extend the XSL Transformation to add entirely specify all parameters of xsd:dateTime. The POC is realizing a database adapter service to write a new empoyee to the well-known HR schema. Therefore, a BPEL process is exposing a SOAP-based web service endpoint to provide input data in a SAP IDoc like style ... hence all data are defined as xsd:string, no matter their contents' meaning.

SCA composite of the POC
The input data require firstname, lastname and hire date and time. The web service returns the hire date formated as xsd:dateTime after successful insert to the database.
XML Schema Definition of input and output data
The BPEL process itselfs stores the received input to an input variable. The following XSL transformation we focus on maps the input data to the database write variable. After successful write to the database by invocation of the endpoint reference of the database adapter, the returned data are mapped to the output data of the process.

BPEL Process "AddEmployee"
Let's have a detailed look at the XSL transformation that is "building" our xsd:dateTime datatype. The graphical representation already shows the usage of the xp20:implicit-timezone() function from XPath 2.0 (green icon).

XSL transformation to "build" an xsd:dateTime value
While the date and time concatination is known from my previous poc, we will focus on the milliseconds and timezone parameters. The milliseconds are simply set to zero (.000) while we make use of the service engines' timezone setting to specify the timezone. In the end, we end up with a pattern of yyyy-mm-ddThh:mm:ss.s+zzzzzz instead of the less precise pattern yyyy-mm-ddThh:ss:mm from the previous POC. A detailed description of the pattern can be found in the W3C recommendation of XSD.

<top:hireDate>
  <xsl:value-of select='concat(
    substring(/client:request/client:hireDay,1.0,4.0),"-",
    substring(/client:request/client:hireDay,5.0,2.0),"-",
    substring(/client:request/client:hireDay,7.0,2.0),"T",
    substring(/client:request/client:hireTime,1.0,2.0),":",
    substring(/client:request/client:hireTime,3.0,2.0),":",
    substring(/client:request/client:hireTime,5.0,2.0),".000",
    xp20:implicit-timezone())'/>
</top:hireDate>

Testing the XSL transformation in the XSLT tester in JDeveloper, everything is working as expected.

Local testing in JDevelopers' XSLT tester

Testing the POC in Oracle Enterprise Manager shows everything working as expected.

Testing the POC in Oracle Enterprise Manager with sample data

An xsd:dateTime value with full precision is returned

References:

Node selection of Oracle's XSLT engine

Oracle Busisness Process Management Suite and Oracle SOA Suite enable XML Stylesheet Transformations (XSLT). The XSLT engine is part of the mediator component engine, which is indeed the former enterprise service bus (ESB) implementation of Oracle before Oracle Service Bus (OSB) was bought.
For implementation purpose, it is sometimes useful to know exactly the node selection behavior of the XSLT engine. In most cases, XSLT relies on XML Path Language (XPath) node selection mechanisms. But what happens, if the expression would match more then one node? Well, let's try!

The POC is quiet simple. A synchornous BPEL process exposing a SOAP web service interface accepts a list of animals as request.

SCA Composite of the POC

For the response, a list of the first, last and XPath node-selected animal will be returned.

XML Schema definition of request and response of the SOAP web service endpoint

The BPEL process itself is quiet simple: Received input data from the input variable are transformed to the output variable to be returned to the requester.

BPEL Process "SortMyAnimals"


The XSL Transformation uses XPath expression to map the first animal of the input list (/request/animals/animal[1]), the last animal of the input list (/request/animals/animal[last()]) and the animal that is selected by the XSLT engine we are actually interested in (/request/animals/animal) ... since we did not define its position in the list.

  <xsl:template match="/">
    <ns1:response>
      <ns1:firstanimal>
        <xsl:value-of select="/ns1:request/ns1:animals/ns1:animal[1]"/>
      </ns1:firstanimal>
      <ns1:lastanimal>
        <xsl:value-of select="/ns1:request/ns1:animals/ns1:animal[last()]"/>
      </ns1:lastanimal>
      <ns1:nodeselectedanimal>
        <xsl:value-of select="/ns1:request/ns1:animals/ns1:animal"/>
      </ns1:nodeselectedanimal>
    </ns1:response>
  </xsl:template>


For a sample input of three animals cat, dog and cow (in that order), the XSLT tester in JDeveloper returns the cat, that means the first element that is matching the XPath expression.

Local testing in JDeveloper using XSLT tester
After deployment of the POC to our environment, we will run the same example to proof what we found out.

Testing the POC in Oracle Enterprise Manager
 And yes, the result is the same:

Testing result in Oracle Enterprise Manager

Summery: Not specifying the exact node in an XPath expression in an XSL Stylesheet Transformation will return the first node matching the path criteria.

References:

Sunday, March 22, 2015

Converting SAP IDoc XML date and time to structured XML Schema xsd:dateTime data type

In a recent project, we worked with the Oracle Application Adapter for SAP R/3 in order to retrieve data from and publish data to the customers SAP system. Did you ever have a look at an XML-based intermediate document (IDoc) from SAP? Well, good news is that you get structured data. Each IDoc type comes with a schema definition as XML Schema Definition (XSD) which can be retrieved from the SAP system using Oracle Application Explorer. Bad news is that absolutly everything is stored as xsd:string. No matter the content of the element, if it's some numeric values, text, date or time values ... the values are stored as string. SAP doesn't make use of the more appropriate default datatypes or the XML Schema Definition.
However, other sources do. So if you want to use for example the header data of an XML IDoc DC40 fragment to call a web service, you need to transform the string based date value CREDAT and the string based time value CRETIM to the more appropriate xsd:dateTime.

<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"
            targetNamespace="urn:sap-com:document:sap:idoc"
            xmlns="urn:sap-com:document:sap:idoc"
            elementFormDefault="qualified">
   <xsd:element name="IDOCTYPEOFYOURCHOICE">
      <xsd:complexType>
         <xsd:sequence>
            <xsd:element name="IDOC" maxOccurs="unbounded" minOccurs="0">
               <xsd:complexType>
                  <xsd:sequence>
                     <xsd:element name="EDI_DC40">
                        <xsd:complexType>
                           <xsd:sequence>
                              ...
                              <xsd:element name="CREDAT" minOccurs="0">
                                 <xsd:simpleType>
                                    <xsd:restriction base="xsd:string">
                                       <xsd:maxLength value="8"/>
                                    </xsd:restriction>
                                 </xsd:simpleType>
                              </xsd:element>
                              <xsd:element name="CRETIM" minOccurs="0">
                                 <xsd:simpleType>
                                    <xsd:restriction base="xsd:string">
                                       <xsd:maxLength value="6"/>
                                    </xsd:restriction>
                                 </xsd:simpleType>
                              </xsd:element>
                              ...
                           </xsd:sequence>
                           ...
                        </xsd:complexType>
                     </xsd:element>
                     ...
                  </xsd:sequence>
                  ...
               </xsd:complexType>
            </xsd:element>
         </xsd:sequence>
         ...
      </xsd:complexType>
   </xsd:element>
</xsd:schema>


You can achieve this by assigning string data in the correct pattern to an xsd:dateTime element. This is possible since xsd:dateTime is accepting string inputs. However, the pattern of xsd:dateTime has to be respected. For our example, it is enough to target the pattern "yyyy-mm-ddThh:mm:ss" without the optional milliseconds and time zone parameters. Check out detailed information in XML Schema Definition Part 2 Datatypes.

The little POC for this is based on a synchronous BPEL process waiting for input data similar to XML IDoc data. An XSL Transformation is used to transform the string-based input data to xsd:dateTime data. Additionally, the XSL engine will add a generated xsd:dateTime value of the runtime environment. The following figure shows the BPEL process.

BPEL process GenerateTime
The service component architecture diagram is as simple as that, showing the exposed service endpoint and the BPEL process.

SCA composite overview of the POC
The input data to the request contains two elements date and time. Both of them are simple types based on xsd:string and restricted in length of 8 respectively 6 characters to represent a date of patter "yyyymmdd" and time of pattern "hhmmss".

XML Schema definition of the POC
The response data on the other hand contains two xsd:dateTime elements. One of them will be the target of the XSL transformation while the other one will hold engine generated data.

The following screenshot is taken from the XSLT editor in JDeveloper with SOA extension.

XSL Transformation is XSLT editor
We are using substrings to extract the year, month and day fragments from the input string date. The same way, we are extracting the hour, minute and second fragments from the input time value. Afterwards, we are concatinating all fragments including the required separators to the desired pattern of the xsd:dateTime based value ns1:customDateTime. Please find below the source code.

  <xsl:template match="/">
    <ns1:response>
      <ns1:customDateTime>
        <xsl:value-of select="concat(

          substring(/ns1:request/ns1:date,1.0,4.0),'-',
          substring(/ns1:request/ns1:date,5.0,2.0),'-',
          substring(/ns1:request/ns1:date,7.0,2.0),'T',
          substring(/ns1:request/ns1:time,1.0,2.0),':',
          substring(/ns1:request/ns1:time,3.0,2.0),':',
          substring(/ns1:request/ns1:time,5.0,2.0))"/>
      </ns1:customDateTime>
      <ns1:generatedDateTime>
        <xsl:value-of select="xp20:current-dateTime()"/>
      </ns1:generatedDateTime>
    </ns1:response>
  </xsl:template>



Now let's test the process (and XSL transformation) from Enterprise Manager, providing some sample data ...

Input data for testing
... and we are successfully receiving the transformed data as response:

Output data of the test
Notice that our transformed xsd:dateTime value only comes with the precision of the provided input data while the generated timestamp comes with the complete, detailed xsd:dateTime pattern including the optional parameters of milliseconds and time zone.

Here you can download the complete POC.