Tuesday, July 7, 2015

WSO2CON Asia


The three-day conference will include some innovative keynote speeches and interactive sessions on dealing with some of the most current enterprise challenges involving IoT, Big Data Analytics, Cloud, Security, Integration, and DevOps. It will also give me the opportunity to interact and exchange ideas with leading CxOs, Enterprise Architects, and Developers.

Visit at http://asia15.wso2con.com/

Tuesday, June 23, 2015

Connect camera to Raspberry Pi and stream video

In my previous blog posts I have described how to configure Raspberry Pi for first time and how to prepare SD card with raspbian image.

Work with Raspberry without Key board, Mouse or Monitor
Write raspbian disk image to SD and Backup your SD Card on Linux

Connecting Raspberry Pi camera module with a Raspberry Pi is pretty simple. All you need is connect camera ribbon cable to CSI (Camera Serial Interface) port as follows. Please note that you have to aware about the correct side of the ribbon cable.

The flex cable inserts into the connector situated between the Ethernet and HDMI ports, with the silver connectors facing the HDMI port. The flex cable connector should be opened by pulling the tabs on the top of the connector upwards then towards the Ethernet port. The flex cable should be inserted firmly into the connector, with care taken not to bend the flex at too acute an angle. The top part of the connector should then be pushed towards the HDMI connector and down, while the flex cable is held in place.

Here is small video guide created by Raspberry Pi team about, how to connecting the camera correctly.


After you connecting the camera correctly, it is needed to enable kernel modules and drivers for camera functionality. So use "sudo raspi-config" to start configurations.


Then Enable camera and restart your Raspberry Pi. Once device is booted use following command to start the camera. If you have a display connected with raspberry Pi, it will display the video feed from camera. Also RED LED in the camera module will turn on by indicating camera activity.

raspivid -t 0

Then press CTRL+C to terminate the camera feed.

Now we need an internet connection to install the streaming server. I preferred vlc for video streaming since it supports h264 encoding and RTSP protocol. To install vlc, you need to update your software repositories in Raspberry Pi. Use this command to update software repositories.

sudo apt-get update

Then install vlc;

sudo apt-get install vlc

Now you are ready to test video streaming. So in first you need to run command with following format. It start raspivid and pipe output to console. Then cvlc tool can use to encode and stream video feed.

raspivid -o - -t 0 -w <width> -h <height> -fps <frame rate> -b <bit rate> |cvlc -v stream:///dev/stdin --sout '#rtp{sdp=rtsp://:<port>/}' :demux=h264

You need to replace <width> with frame width and <height> with frame height which is compatible with 4:3 or 16:9 accept ratios. Replace <frame_rate> with the value that you need to grab frames per second. Higher frame rates contains smooth video but need higher bit rate and high speed network connection. Video bit rate should be assigned as bit per second value. Higher bit rates deliver much clear video and deliver higher video quality. But it also required high speed stable network connection. Finally <port> is used to write stream data to network and you will be able to access video stream via the specified port of your network interface.

Finally you should have to find IP address of your Raspberry Pi device. If you don't know your IP address using ifconfig command.

This is a sample command for stream video with 640x480@15fps 378kbps. This stream can be access through the port 8554.

raspivid -o - -t 0 -w 640 -h 480 -fps 15 -b 393216 |cvlc -v stream:///dev/stdin --sout '#rtp{sdp=rtsp://:8554/}' :demux=h264

So if you got IP address like 192.168.43.245, then you can access your video stream using your vlc media player in computer or with your mobile phone which are connected with the same network which your Raspberry connected.

Lets assume IP of your Raspberry Pi is 192.168.43.245, you can access video stream via rtsp://192.168.43.245:8554/

If you have MX Player in your android device you can access video stream via Network stream.



This is how I run it at my office. You can see my mobile is also showing the video feed.





Wednesday, June 10, 2015

Write raspbian disk image to SD and Backup your SD Card on Linux

In Linux, we have lot more cool stuffs to create raspberry SD card and backup your SD card in to PC or Laptop. You might need root permission to execute commands with dd tool.

Identify the SD card
  1. Remove if you have any SD card inserted and then run df -h to see what devices are currently mounted.

  2. Insert the SD card and run df -h again. The device that wasn't there last time is your SD card. The left column gives the device name of your SD card. It will be listed as something like /dev/mmcblk0p1 or /dev/sdd1.

    The last part ("p1" or "1" respectively) is the partition number, but you want to write to the whole SD card, not just one partition, so you need to remove that part from the name (getting for example /dev/mmcblk0 or /dev/sdd) as the device for the whole SD card. Note that as in screenshot the SD card can show up more than once in the output of df: in fact it will if you have previously written a Raspberry Pi image to this SD card, because the Raspberry Pi SD images have more than one partition.
  3. Now that you've noted what the device name is, you need to unmount it so that files can't be read or written to the SD card while you are copying over the SD image. So run the command below, replacing mmcblk0p5 with whatever your SD card's device name is (including the partition number) If your SD card shows up more than once in the output of df due to having multiple partitions on the SD card, you should unmount all of these partitions.
    umount /dev/mmcblk0p5
Write raspbian disk image to SD
    WARNING: 
    You might need some knowledge about Linux terminal commands and if something went wrong, you might even lost your data.
    Please note that the use of the dd tool can overwrite any partition of your machine. If you specify the wrong device in the instructions below you could delete your primary Linux partition. Please be careful.

    In the terminal write the image to the card with dd command, making sure you replace the input file if= argument with the path to your .img file, and the /dev/mmcblk0 in the output file of= argument with the right device name (this is very important: you will lose all data on the hard drive on your computer if you get the wrong device name). Make sure the device name is the name of the whole SD card as described above, not just a partition of it (for example, sdd, not sdds1 or sddp1, or mmcblk0 not mmcblk0p1)

    dd bs=1M if=~/Documents/2015-05-05-raspbian-wheezy.img of=/dev/mmcblk0


    Note that if you are not logged in as root you will need to prefix this with sudo. The dd command does not give any information of its progress and so may appear to have frozen. It could take more than five minutes to finish writing to the card. To see the progress of the copy operation you can run pkill -USR1 -n -x dd in another terminal (prefixed with sudo if you are not logged in as root). The progress will be displayed (perhaps not immediately, due to buffering) in the original window, not the window with the pkill command. Another option to monitor progress is to use Pipe Viewer (pv). Pipe dd input part through pv to the output part of dd:

    dd bs=4M if=2014-09-09-wheezy-raspbian.img | pv | dd of=/dev/mmcblk0

    Remove SD card from card reader, insert it in the Raspberry Pi, and have fun.


    Backup your SD Card

    You can simply backup your SD card to PC or Laptop using dd tool. Just only have to swap if and of in the image writing command:

    dd bs=1M if=/dev/mmcblk0 of=~/Documents/raspi-backup.img

    But if the SD card is with higher capacity and plenty of free space; (example of 32GB with only 4GB is used) the image file is a waste of space since it took entire SD card size from your hard drive (32GB image file will save in your hard drive according to the example).

    To avoid hard disk space wastage, you can use gzip to compress your SD card image while you creating it.

    dd if=/dev/mmcblk0 | gzip > ~/Documents/raspi-backup.img.gz

    Then your image will compressed it self and you can extract it later if required. How ever if you need to write your compressed image directly without extracting it from compressed archive, you can do it with following command.

    cat backup.img.gz | gunzip | dd of=/dev/mmcblk0


    Sunday, May 31, 2015

    Work with Raspberry without Key board, Mouse or Monitor

    Probably you might wondering how this could do. But it is possible if you have computer with DHCP enabled wired Ethernet. However this can only do in Raspberry devices which has RJ45 Ethernet port. So compute module and Model A+ away from this. But still you can prepare and configure SD card using Model B/B+ and then can use it with A+ without an issue.

    So what we need:
    1. Raspberry Pi with Ethernet port
    2. Latest Raspbian Wheezy image (http://downloads.raspberrypi.org/raspbian_latest)
    3. Win32DiskImager (http://sourceforge.net/projects/win32diskimager/files/latest/download)
    4. Ethernet cable
    5. DHCP enabled Ethernet network with internet connectivity (Probably it is your broadband router)
    6. Advanced IP Scanner (http://www.advanced-ip-scanner.com/)
    7. Bitvise SSH Client (http://dl.bitvise.com/BvSshClient-Inst.exe)
    Now extract the downloaded Wheezy image and insert your SD card to your computer. And then run Win32DiskImager.exe. It should find your SD Card drive or if not select it.  Select the file '#.img' image file you wish to use and then press write.


    Once it completes, we are ready to go, insert your SD card into the Raspberry Pi. And the connect your Raspberry to Ethernet network and give power to Raspberry. Then it starts booting and you can see green status indicator is blinking. Be patient for minute and Run Advanced IP Scanner. You could see IP address of your Raspberry device. If your device is not listed, try to enable all resources in Advanced IP Scanner, options and try again.


    Now get IP of Raspberry and connect with Raspberry using bitvise SSH client. By default Raspbian OS came with user pi and password raspberry. Once you prompt for Host Key, click accept and save.


    Once you connected with the Raspberry, you can configure your raspberry using "raspi-config" tool. This tool should be execute with super user privileges. So use "sudo raspi-config" to start configurations. With that tool you are able to expand your file system, enable Raspberry camera firmware and can do several other configurations. In here you must enable boot to desktop function. Otherwise you won't be able to connect to desktop remotely as explained in this post later.


    After you are finishing the configurations, you might ask to reboot. After rebooting, bitvise will connect you with Raspberry automatically. Then open a terminal session and you can update your system with "sudo apt-get update"


    Still we don't have any GUI from raspberry. So now we can install xrdp server in raspberry pi to access its GUI over a remote desktop connection. To install xrdp server, use "sudo apt-get install xrdp". Then you will asked to download and install xrdp server, and press "y" and enter to install.


    Now its time to connect to your raspberry desktop via remote desktop connection. Before you start go to "Remote Desktop" tab in bitvise and remove tick in "use ssh login credentials". Now provide user name as "pi" and password as "raspberry".


    To connect with remote desktop, just click on "New Remote Desktop" option in bitvise client. Then you asked to connect with Raspberry pi over the remote desktop connection and just click connect button. You can maximize screen to get full screen view.


    Anyway if you spend much long time in terminal you might only get a black screen, in that case, close remote desktop connection and restart raspberry using "sudo reboot". And try to connect again with remote desktop after device connected with bitvise.

    Saturday, May 23, 2015

    XML: Transformation with XSL

    XSL Transformations is a language for transforming XML documents into other XML documents, text documents or HTML documents.
    If you went through my previous post on Parse and Validate xml, you already aware with the xml that I used to create POJOs. Here the xml that I used and if you aware with its second line you can see I linked xsl with xml in there.

    <?xml version="1.0" encoding="UTF-8" standalone="no"?>
    <?xml-stylesheet type="text/xsl" href="Students.xsl"?>
    <studentsns:students xmlns:studentsns="http://www.charitha.org/students"
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xsi:schemaLocation="StudentsSchema.xsd">
     <studentsns:student id="1">
      <studentsns:name>Charitha</studentsns:name>
      <studentsns:year>2009</studentsns:year>
      <studentsns:nic>895623145v</studentsns:nic>
     </studentsns:student>
     <studentsns:student id="2">
      <studentsns:name>Achithra</studentsns:name>
      <studentsns:year>2009</studentsns:year>
      <studentsns:nic>881234566v</studentsns:nic>
     </studentsns:student>
     <studentsns:student id="3">
      <studentsns:name>Malan</studentsns:name>
      <studentsns:year>2011</studentsns:year>
      <studentsns:nic>921123903v</studentsns:nic>
     </studentsns:student>
    </studentsns:students>
    

    This is the style sheet that I used to style my xml. In that xml I used  xml-stylesheet to stylize the document.

    <?xml version="1.0" encoding="UTF-8"?>
    
    <xsl:stylesheet version="1.0"
     xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:studentsns="http://www.charitha.org/students"
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xsi:schemaLocation="StudentsSchema.xsd">
    
     <xsl:template match="/">
      <html>
       <body>
        <h2>Student Details</h2>
        <table border="1">
         <tr bgcolor="#9acd32">
          <th>Student ID</th>
          <th>Student Name</th>
          <th>Reg. Year</th>
          <th>NIC Number</th>
         </tr>
         <xsl:for-each select="studentsns:students/studentsns:student">
          <tr>
           <td>
            <xsl:value-of select="@id" />
           </td>
           <td>
            <xsl:value-of select="studentsns:name" />
           </td>
           <td>
            <xsl:value-of select="studentsns:year" />
           </td>
           <td>
            <xsl:value-of select="studentsns:nic" />
           </td>
          </tr>
         </xsl:for-each>
        </table>
       </body>
      </html>
     </xsl:template>
    
    </xsl:stylesheet>
    

    In this style sheet I used xsl:template tag to define styles for the document which is going to transform as HTML. So I wrote HTML contents as usual and specifically used xsl namespace to define special transformation instructions. Since we need to display student data from students list, we need to iterate through each student in students list. So in here we are using xsl:for-each tag and select "studentsns:students/studentsns:student". Then inside repetitive table rows we are selecting attributes by using "@" symbol before the attribute name and selecting tags by their name with namespace. xsl:value-of tag giving the value for selected tag or attribute.

    If you open that xml with your web browser after placing above xsl with in the same directory of your xml, you will see the stylized xml in the browser.


    Normally most of web browsers rendered xml from file source and display styled xml as HTML. Now we are going to transform xml in to HTML using javax.xml.transform.TransformerFactory. It is quite simple and we can generate HTML file from our xml as follows.

    // stream source sml
    StreamSource xmlSourse = new StreamSource(new File("students.xml"));
    // stream source xsl
    StreamSource xslSourse = new StreamSource(new File("students.xsl"));
    // stream result output html
    StreamResult htmlOutput = new StreamResult(new File("students.html"));
    
    try {
     TransformerFactory.newInstance().newTransformer(xslSourse).transform(xmlSourse,
     htmlOutput);
    } catch (TransformerException | TransformerFactoryConfigurationError e) {
     System.err.println("Transformation failed: " + e.getMessage());
    }
    

    Then students.html file generates in the same directory where your xml is placed and it is consists with populated data.


    <html xmlns:studentsns="http://www.charitha.org/students"
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <body>
     <h2>Student Details</h2>
     <table border="1">
      <tr bgcolor="#9acd32">
       <th>Student ID</th>
       <th>Student Name</th>
       <th>Reg. Year</th>
       <th>NIC Number</th>
      </tr>
      <tr>
       <td>1</td>
       <td>Charitha</td>
       <td>2009</td>
       <td>891623897v</td>
      </tr>
      <tr>
       <td>2</td>
       <td>Achithra</td>
       <td>2009</td>
       <td>881234566v</td>
      </tr>
      <tr>
       <td>3</td>
       <td>Malan</td>
       <td>2011</td>
       <td>921123903v</td>
      </tr>
     </table>
    </body>
    </html>
    

    So if we opened the html, it will display same as the styled xml which we opened earlier.

    Sunday, May 17, 2015

    XML Parsing & Validation

    In my previous post, I discussed how to create an xml from POJOs with namespaces. In here I'm going to describe how to parse and validate an xml. This is the xml which generated by previous example.

    <?xml version="1.0" encoding="UTF-8" standalone="no"?>
    <?xml-stylesheet type="text/xsl" href="students.xsl"?>
    <studentsns:students xmlns:studentsns="http://www.charitha.org/students"
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xsi:schemaLocation="StudentsSchema.xsd">
     <studentsns:student id="1">
      <studentsns:name>Charitha</studentsns:name>
      <studentsns:student>2009</studentsns:student>
      <studentsns:nic>1234567890v</studentsns:nic>
     </studentsns:student>
     <studentsns:student id="2">
      <studentsns:name>Achithra</studentsns:name>
      <studentsns:student>2009</studentsns:student>
      <studentsns:nic>9876543210v</studentsns:nic>
     </studentsns:student>
     <studentsns:student id="3">
      <studentsns:name>Malan</studentsns:name>
      <studentsns:student>2011</studentsns:student>
      <studentsns:nic>1597532632v</studentsns:nic>
     </studentsns:student>
    </studentsns:students>
    

    Also I'm using following xml schema to validate above xml.

    <?xml version="1.0" encoding="UTF-8"?>
    <schema xmlns="http://www.w3.org/2001/XMLSchema" targetNamespace="http://www.charitha.org/students"
     xmlns:studentsns="http://www.charitha.org/students" elementFormDefault="qualified">
     <element name="students">
      <complexType>
       <sequence>
        <element ref="studentsns:student" minOccurs="0" maxOccurs="unbounded" />
       </sequence>
      </complexType>
     </element>
    
     <element name="student">
      <complexType>
       <sequence>
        <element name="name" type="string" />
        <element name="year" type="string" />
        <element name="nic" type="string" />
       </sequence>
       <attribute name="id" type="integer" use="required"/>
      </complexType>
     </element>
    </schema>
    

    In here I'm going to explain 3 parser APIs for xml which known as DOM, SAX, and StAX. These parser APIs have their own advantages and also drawbacks. So first I'd like to list these three APIs and then compare and contrast differences between them.
    • DOM - pull the whole thing into memory and walk around inside it. Good for comparatively small chunks of XML that you want to do complex stuff with. XSLT uses DOM.
    • SAX - Walk the XML as it arrives watching for things as they fly past. Good for large amounts of data or comparatively simple processing.
    • StAX - Much like SAX but instead of responding to events found in the stream you iterate through the xml
    DOM

    First lets look at DOM parser. It uses DocumentBuilderFactory to build the XML document and defines a factory API that enables applications to obtain a parser that produces DOM object trees from XML documents. Since we are going to validate the xml with the schema, we need to aware with Document namespace. I'm using javax.xml.parsers.DocumentBuilder to obtain DOM Document instances from the empty XML document which I instantiated using DocumentBuilderFactory. An instance of DocumentBuilder class can be obtained from the documentBuilderFactory.newDocumentBuilder() method. Once an instance of this class is obtained, XML can be parsed from a variety of input sources. These input sources are InputStreams, Files, URLs, and SAX InputSources. But in here I'm using it only to create new XML document instead of parsing existing one. Then I'm creating a new org.w3c.dom.Document by parsing my xml file using the documentBuilder.

    DocumentBuilderFactory documentBuildFactory = DocumentBuilderFactory.newInstance();
    documentBuildFactory.setNamespaceAware(true);
    DocumentBuilder documentBuilder = documentBuildFactory.newDocumentBuilder();
    Document document = documentBuilder.parse("students.xml");
    

    Now we can validate parsed document against xml schema which mentioned in above. javax.xml.validation.SchemaFactory is a schema compiler. It reads external representations of schemas and prepares them for validation. So we need to specify default namespace for the schema when we call for new instance of schemaFactory. In validation process we create javax.xml.validation.Schema object from schema file by using schemaFactory instance. Schema object represents a set of constraints that can be checked/ enforced against an XML document and it is an immutable in-memory representation of grammar.

    SchemaFactory schemaFactory = SchemaFactory.newInstance(XMLConstants.W3C_XML_SCHEMA_NS_URI);
    Schema schema = schemaFactory.newSchema(new File("studentsSchema.xsd"));
    Validator validator = schema.newValidator();
    validator.validate(new DOMSource(document));
    

    Then we can extract elements from parsed document using root element of the document. Since our xml has Student list, I'm using ArrayList to hold each student record as POJO and NodeList to hold student elements from parsed xml.

    ArrayList < Student > students = new ArrayList < Student > ();
    
    // Extract nodes list from the xml file
    NodeList nodes = document.getDocumentElement().getChildNodes();
    
    // Iterates through all nodes
    for (int i = 0; i < nodes.getLength(); i++) {
     Node node = nodes.item(i);
     if (node.getNodeType() == Node.ELEMENT_NODE) {
      Student student = new Student();
      student.setStudentId(Integer.parseInt(node.getAttributes()
       .getNamedItem("id")
       .getNodeValue()));
    
      NodeList childNodes = node.getChildNodes();
    
      for (int j = 0; j < childNodes.getLength(); j++) {
       Node childNode = childNodes.item(j);
       if (childNode.getNodeType() == Node.ELEMENT_NODE) {
        // extract the content
        String elmBody = childNode.getLastChild().getTextContent();
        // check type of the content
        switch (childNode.getNodeName()) {
         case "studentsns:name":
          student.setStudentName(elmBody);
          break;
         case "studentsns:year":
          student.setRegYear(Integer.parseInt(elmBody));
          break;
         case "studentsns:nic":
          student.setNic(elmBody);
          break;
         default:
          break;
    
        }
       }
      }
      students.add(student);
     }
    }
    

    SAX

    Implementation of SAX validation is quite simple than implementing a DOM validation. We are using schema factory as same as the DOM validation. But when we are parsing our xml, we don't want to create DOM source from our xml. Instead of that we are passing javax.xml.transform.stream.StreamSource.StreamSource from xml File to validator.

    SchemaFactory schemaFactory = SchemaFactory.newInstance(XMLConstants.W3C_XML_SCHEMA_NS_URI);
    Schema schema = schemaFactory.newSchema(new File("studentsSchema.xsd"));
    Validator validator = schema.newValidator();
    validator.validate(new StreamSource(new File("students.xml")));
    

    Since SAX parsing is event driven, we need to use event handler to handle events and build POJOs from xml. So we are creating separate handler class by extending org.xml.sax.helpers.DefaultHandler class. When parser starts parsing with a xml stream, startDocument method called and once stream ended, endDocument method called. Once parser detects start or end of element, it called startElement and endElement methods respectively with following parameters.

    • uri The Namespace URI, or the empty string if the element has no Namespace URI or if Namespace processing is not being performed.
    • localName The local name (without prefix), or the empty string if Namespace processing is not being performed.
    • qName The qualified name (with prefix), or the empty string if qualified names are not available.

    In startElement method, there is an additional parameter is there to specify the attributes attached to the element. If there are no attributes, it shall be an empty Attributes object. Each element body content can be extracted through the characters method. Finally if there any exception occurred during parsing, error method is called.

    public class StudentsSAXHandler extends DefaultHandler {
    
     private ArrayList<Student> students;
     private Student student;
     private String elmBody;
    
     @Override
     public void startDocument() throws SAXException {
      students = new ArrayList<Student>();
     }
    
     @Override
     public void endDocument() throws SAXException {
      // printing student details
      for (Student s : students) {
       System.out.println(s.toString());
      }
     }
    
     @Override
     public void startElement(String uri, String localName, String qName,
                              Attributes attributes) throws SAXException {
      if ("student".equals(localName)) {
       student = new Student();
       student.setStudentId(Integer.parseInt(attributes.getValue("id")));
      }
     }
    
     @Override
     public void endElement(String uri, String localName, String qName) throws SAXException {
      switch (localName) {
       case "student":
        students.add(student);
        break;
       case "name":
        student.setStudentName(elmBody);
        break;
       case "year":
        student.setRegYear(Integer.parseInt(elmBody));
        break;
       case "nic":
        student.setNic(elmBody);
        break;
       default:
        break;
      }
     }
    
     @Override
     public void characters(char ch[], int start, int length) throws SAXException {
      elmBody = new String(ch, start, length);
     }
    
     @Override
     public void error(SAXParseException e) {
      System.err.println("Parsing error: " + e.getMessage());
     }
    
    }
    

    Now we have handler for the SAX parser, so then we are going to create parser with that handler. Since we already validate our document with SAX validator, we just only need to create instance from javax.xml.parsers.SAXParserFactory to parse our xml with our namespace. Then we are creating javax.xml.parsers.SAXParser. For ease of transition, this class continues to support the same name and interface as well as supporting new methods. An instance of this class can be obtained from the javax.xml.parsers.SAXParserFactory.newSAXParser() method. Once an instance of this class is obtained, xml can be parsed from a variety of input sources. These input sources are InputStreams, Files, URLs, and SAX InputSources. In here I'm using xml file as input source.

    // Get SAX Parser Factory
    SAXParserFactory factory = SAXParserFactory.newInstance();
    factory.setNamespaceAware(true);
    factory.setSchema(schema);
    SAXParser parser = factory.newSAXParser();
    parser.parse(new File("students.xml"), new StudentsSAXHandler());
    

    StAX

    In StAX validation, javax.xml.stream.XMLInputFactory is used to build xml from input stream or from file. Then using xml input factory we are creating javax.xml.stream.XMLStreamReader interface object and it is used to build javax.xml.transform.stax.StAXSource.StAXSource which can be validate using the validator.

    XMLInputFactory factory = XMLInputFactory.newInstance();
    SchemaFactory schemaFactory = SchemaFactory.newInstance(XMLConstants.W3C_XML_SCHEMA_NS_URI);
    XMLStreamReader xmlStreamReader = factory.createXMLStreamReader(new FileInputStream("students.xml"));
    Schema schema = schemaFactory.newSchema(new File("studentsSchema.xsd"));
    Validator validator = schema.newValidator();
    validator.validate(new StAXSource(xmlStreamReader));

    Then we only have to iterate through each element in node in reader object to build our POJOs by parsing xml.

    ArrayList < Student > students = null;
    Student student = null;
    String elmBody = null;
    
    while (xmlStreamReader.hasNext()) {
     switch (xmlStreamReader.next()) {
      case XMLStreamConstants.START_DOCUMENT:
       students = new ArrayList < Student > ();
       break;
      case XMLStreamConstants.START_ELEMENT:
       if ("students".equals(xmlStreamReader.getLocalName())) {
        students = new ArrayList < > ();
       } else if ("student".equals(xmlStreamReader.getLocalName())) {
        student = new Student();
        student.setStudentId(Integer.parseInt(xmlStreamReader.getAttributeValue(0)));
       }
       break;
      case XMLStreamConstants.CHARACTERS:
       elmBody = xmlStreamReader.getText().trim();
       break;
      case XMLStreamConstants.END_ELEMENT:
       switch (xmlStreamReader.getLocalName()) {
        case "student":
         students.add(student);
         break;
        case "name":
         student.setStudentName(elmBody);
         break;
        case "year":
         student.setRegYear(Integer.parseInt(elmBody));
         break;
        case "nic":
         student.setNic(elmBody);
         break;
        default:
         break;
       }
       break;
      default:
       break;
     }
    }
    

    Monday, May 11, 2015

    XML: Create XML from POJOs

    XML is using to describe data. So we can use xml in many areas to store and communicate data. In here I'm showing how to build an xml structure using java application to store data in POJOs.

    First I crated Java project using eclipse and create POJO class to model Student object. In my student object I have Student Id, Name, Registered Year and NIC number. With that class we can create several student objects and can transform them in to XML file.

    In XML creation I used javax.xml.parsers.DocumentBuilderFactory. It is also used to build the XML document and defines a factory API that enables applications to obtain a parser that produces DOM object trees from XML documents. I'm using javax.xml.parsers.DocumentBuilder to obtain DOM Document instances from the empty XML document which I instantiated using DocumentBuilderFactory. An instance of DocumentBuilder class can be obtained from the documentBuilderFactory.newDocumentBuilder() method. Once an instance of this class is obtained, XML can be parsed from a variety of input sources. These input sources are InputStreams, Files, URLs, and SAX InputSources. But in here I'm using it only to create new XML document instead of parsing existing one.

    Then I'm creating a new org.w3c.dom.Document object using documentBuilder and appending elements in to it from my POJOs.

    DocumentBuilderFactory documentBuilderFactory = DocumentBuilderFactory.newInstance();
    documentBuilderFactory.setNamespaceAware(true);
    DocumentBuilder documentBuilder = documentBuilderFactory.newDocumentBuilder();
    Document document = documentBuilder.newDocument();
    

    Now its time to create XML elements and append them to the created document object. First we have to create a root element and add if any XML processing instructions available before the root element. In this example I'm using xml-stylesheet to style the XML. In here I have used org.w3c.dom.Element interface to create XML elements in document. The Element interface represents an element in an HTML or XML document. Elements may have attributes associated with them; since the Element interface inherits from Node, the generic Node interface attribute attributes may be used to retrieve the set of all attributes for an element. There are methods on the Element interface to retrieve either an Attr object by name or an attribute value by name. In XML, where an attribute value may contain entity references, an Attr object should be retrieved to examine the possibly fairly complex sub-tree representing the attribute value. On the other hand, in HTML, where all attributes have simple string values, methods to directly access an attribute value can safely be used as a convenience.

    // Create the root element
    Element studentsElement = document.createElement("studentsns:students");
    studentsElement.setAttributeNS("http://www.w3.org/2000/xmlns/","xmlns:studentsns", "http://www.charitha.org/students");
    studentsElement.setAttribute("xmlns:xsi","http://www.w3.org/2001/XMLSchema-instance");
    studentsElement.setAttribute("xsi:schemaLocation", "StudentsSchema.xsd");
    document.appendChild(studentsElement);
    

    Then I'm appending my XML processing instructions to document before the root element.

    // Create processing instructions for style sheet
    Node style = document.createProcessingInstruction("xml-stylesheet", "type=\"text/xsl\" href=\"students.xsl\"");
    document.insertBefore(style, studentsElement);
    

    Now its time to iterate through the Student objects list and append student details to XML.

    for (Student s : students) {
       // create student element
       Element studentElement = document.createElement("studentsns:student");
       studentsElement.appendChild(studentElement);
       studentElement.setAttribute("id", String.valueOf(s.getStudentId()));
       // create name element
       Element nameElement = document.createElement("studentsns:name");
       nameElement.appendChild(document.createTextNode(s.getStudentName()));
       studentElement.appendChild(nameElement);
       // create year element
       Element yearElement = document.createElement("studentsns:student");
       yearElement.appendChild(document.createTextNode(String.valueOf(s.getRegYear())));
       studentElement.appendChild(yearElement);
       // create nic element
       Element nicElement = document.createElement("studentsns:nic");
       nicElement.appendChild(document.createTextNode(s.getNic()));
       studentElement.appendChild(nicElement);
    }
    

    Finally we have to transform DOM object of document to XML using javax.xml.transform.TransformerFactory. An instance of javax.xml.transform.Transformer class can be obtained with the transformerFactory.newTransformer method. This instance may then be used to process XML from a variety of sources and write the transformation output to a variety of sinks.

    We have to create javax.xml.transform.dom.DOMSource as a holder for a transformation Source tree in the form of a Document Object Model (DOM) tree. And also we need javax.xml.transform.stream.StreamResult to stream XML in to file. Then we can finally create XML file output by using DOMSource and StreamResult.

    // transform DOM object to xml file
    TransformerFactory transformerFactory = TransformerFactory.newInstance();
    Transformer transformer = transformerFactory.newTransformer();
    DOMSource domSource = new DOMSource(document);
    StreamResult streamResult = new StreamResult(new File("student.xml"));
    transformer.transform(domSource, streamResult);
    

    Generated XML file will be available in project folder.

    Click here to download sample eclipse project.