Sunday, May 31, 2015

Work with Raspberry without Key board, Mouse or Monitor

Probably you might wondering how this could do. But it is possible if you have computer with DHCP enabled wired Ethernet. However this can only do in Raspberry devices which has RJ45 Ethernet port. So compute module and Model A+ away from this. But still you can prepare and configure SD card using Model B/B+ and then can use it with A+ without an issue.

So what we need:
  1. Raspberry Pi with Ethernet port
  2. Latest Raspbian Wheezy image (http://downloads.raspberrypi.org/raspbian_latest)
  3. Win32DiskImager (http://sourceforge.net/projects/win32diskimager/files/latest/download)
  4. Ethernet cable
  5. DHCP enabled Ethernet network with internet connectivity (Probably it is your broadband router)
  6. Advanced IP Scanner (http://www.advanced-ip-scanner.com/)
  7. Bitvise SSH Client (http://dl.bitvise.com/BvSshClient-Inst.exe)
Now extract the downloaded Wheezy image and insert your SD card to your computer. And then run Win32DiskImager.exe. It should find your SD Card drive or if not select it.  Select the file '#.img' image file you wish to use and then press write.


Once it completes, we are ready to go, insert your SD card into the Raspberry Pi. And the connect your Raspberry to Ethernet network and give power to Raspberry. Then it starts booting and you can see green status indicator is blinking. Be patient for minute and Run Advanced IP Scanner. You could see IP address of your Raspberry device. If your device is not listed, try to enable all resources in Advanced IP Scanner, options and try again.


Now get IP of Raspberry and connect with Raspberry using bitvise SSH client. By default Raspbian OS came with user pi and password raspberry. Once you prompt for Host Key, click accept and save.


Once you connected with the Raspberry, you can configure your raspberry using "raspi-config" tool. This tool should be execute with super user privileges. So use "sudo raspi-config" to start configurations. With that tool you are able to expand your file system, enable Raspberry camera firmware and can do several other configurations. In here you must enable boot to desktop function. Otherwise you won't be able to connect to desktop remotely as explained in this post later.


After you are finishing the configurations, you might ask to reboot. After rebooting, bitvise will connect you with Raspberry automatically. Then open a terminal session and you can update your system with "sudo apt-get update"


Still we don't have any GUI from raspberry. So now we can install xrdp server in raspberry pi to access its GUI over a remote desktop connection. To install xrdp server, use "sudo apt-get install xrdp". Then you will asked to download and install xrdp server, and press "y" and enter to install.


Now its time to connect to your raspberry desktop via remote desktop connection. Before you start go to "Remote Desktop" tab in bitvise and remove tick in "use ssh login credentials". Now provide user name as "pi" and password as "raspberry".


To connect with remote desktop, just click on "New Remote Desktop" option in bitvise client. Then you asked to connect with Raspberry pi over the remote desktop connection and just click connect button. You can maximize screen to get full screen view.


Anyway if you spend much long time in terminal you might only get a black screen, in that case, close remote desktop connection and restart raspberry using "sudo reboot". And try to connect again with remote desktop after device connected with bitvise.

Saturday, May 23, 2015

XML: Transformation with XSL

XSL Transformations is a language for transforming XML documents into other XML documents, text documents or HTML documents.
If you went through my previous post on Parse and Validate xml, you already aware with the xml that I used to create POJOs. Here the xml that I used and if you aware with its second line you can see I linked xsl with xml in there.

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<?xml-stylesheet type="text/xsl" href="Students.xsl"?>
<studentsns:students xmlns:studentsns="http://www.charitha.org/students"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="StudentsSchema.xsd">
 <studentsns:student id="1">
  <studentsns:name>Charitha</studentsns:name>
  <studentsns:year>2009</studentsns:year>
  <studentsns:nic>895623145v</studentsns:nic>
 </studentsns:student>
 <studentsns:student id="2">
  <studentsns:name>Achithra</studentsns:name>
  <studentsns:year>2009</studentsns:year>
  <studentsns:nic>881234566v</studentsns:nic>
 </studentsns:student>
 <studentsns:student id="3">
  <studentsns:name>Malan</studentsns:name>
  <studentsns:year>2011</studentsns:year>
  <studentsns:nic>921123903v</studentsns:nic>
 </studentsns:student>
</studentsns:students>

This is the style sheet that I used to style my xml. In that xml I used  xml-stylesheet to stylize the document.

<?xml version="1.0" encoding="UTF-8"?>

<xsl:stylesheet version="1.0"
 xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:studentsns="http://www.charitha.org/students"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="StudentsSchema.xsd">

 <xsl:template match="/">
  <html>
   <body>
    <h2>Student Details</h2>
    <table border="1">
     <tr bgcolor="#9acd32">
      <th>Student ID</th>
      <th>Student Name</th>
      <th>Reg. Year</th>
      <th>NIC Number</th>
     </tr>
     <xsl:for-each select="studentsns:students/studentsns:student">
      <tr>
       <td>
        <xsl:value-of select="@id" />
       </td>
       <td>
        <xsl:value-of select="studentsns:name" />
       </td>
       <td>
        <xsl:value-of select="studentsns:year" />
       </td>
       <td>
        <xsl:value-of select="studentsns:nic" />
       </td>
      </tr>
     </xsl:for-each>
    </table>
   </body>
  </html>
 </xsl:template>

</xsl:stylesheet>

In this style sheet I used xsl:template tag to define styles for the document which is going to transform as HTML. So I wrote HTML contents as usual and specifically used xsl namespace to define special transformation instructions. Since we need to display student data from students list, we need to iterate through each student in students list. So in here we are using xsl:for-each tag and select "studentsns:students/studentsns:student". Then inside repetitive table rows we are selecting attributes by using "@" symbol before the attribute name and selecting tags by their name with namespace. xsl:value-of tag giving the value for selected tag or attribute.

If you open that xml with your web browser after placing above xsl with in the same directory of your xml, you will see the stylized xml in the browser.


Normally most of web browsers rendered xml from file source and display styled xml as HTML. Now we are going to transform xml in to HTML using javax.xml.transform.TransformerFactory. It is quite simple and we can generate HTML file from our xml as follows.

// stream source sml
StreamSource xmlSourse = new StreamSource(new File("students.xml"));
// stream source xsl
StreamSource xslSourse = new StreamSource(new File("students.xsl"));
// stream result output html
StreamResult htmlOutput = new StreamResult(new File("students.html"));

try {
 TransformerFactory.newInstance().newTransformer(xslSourse).transform(xmlSourse,
 htmlOutput);
} catch (TransformerException | TransformerFactoryConfigurationError e) {
 System.err.println("Transformation failed: " + e.getMessage());
}

Then students.html file generates in the same directory where your xml is placed and it is consists with populated data.


<html xmlns:studentsns="http://www.charitha.org/students"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<body>
 <h2>Student Details</h2>
 <table border="1">
  <tr bgcolor="#9acd32">
   <th>Student ID</th>
   <th>Student Name</th>
   <th>Reg. Year</th>
   <th>NIC Number</th>
  </tr>
  <tr>
   <td>1</td>
   <td>Charitha</td>
   <td>2009</td>
   <td>891623897v</td>
  </tr>
  <tr>
   <td>2</td>
   <td>Achithra</td>
   <td>2009</td>
   <td>881234566v</td>
  </tr>
  <tr>
   <td>3</td>
   <td>Malan</td>
   <td>2011</td>
   <td>921123903v</td>
  </tr>
 </table>
</body>
</html>

So if we opened the html, it will display same as the styled xml which we opened earlier.

Sunday, May 17, 2015

XML Parsing & Validation

In my previous post, I discussed how to create an xml from POJOs with namespaces. In here I'm going to describe how to parse and validate an xml. This is the xml which generated by previous example.

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<?xml-stylesheet type="text/xsl" href="students.xsl"?>
<studentsns:students xmlns:studentsns="http://www.charitha.org/students"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="StudentsSchema.xsd">
 <studentsns:student id="1">
  <studentsns:name>Charitha</studentsns:name>
  <studentsns:student>2009</studentsns:student>
  <studentsns:nic>1234567890v</studentsns:nic>
 </studentsns:student>
 <studentsns:student id="2">
  <studentsns:name>Achithra</studentsns:name>
  <studentsns:student>2009</studentsns:student>
  <studentsns:nic>9876543210v</studentsns:nic>
 </studentsns:student>
 <studentsns:student id="3">
  <studentsns:name>Malan</studentsns:name>
  <studentsns:student>2011</studentsns:student>
  <studentsns:nic>1597532632v</studentsns:nic>
 </studentsns:student>
</studentsns:students>

Also I'm using following xml schema to validate above xml.

<?xml version="1.0" encoding="UTF-8"?>
<schema xmlns="http://www.w3.org/2001/XMLSchema" targetNamespace="http://www.charitha.org/students"
 xmlns:studentsns="http://www.charitha.org/students" elementFormDefault="qualified">
 <element name="students">
  <complexType>
   <sequence>
    <element ref="studentsns:student" minOccurs="0" maxOccurs="unbounded" />
   </sequence>
  </complexType>
 </element>

 <element name="student">
  <complexType>
   <sequence>
    <element name="name" type="string" />
    <element name="year" type="string" />
    <element name="nic" type="string" />
   </sequence>
   <attribute name="id" type="integer" use="required"/>
  </complexType>
 </element>
</schema>

In here I'm going to explain 3 parser APIs for xml which known as DOM, SAX, and StAX. These parser APIs have their own advantages and also drawbacks. So first I'd like to list these three APIs and then compare and contrast differences between them.
  • DOM - pull the whole thing into memory and walk around inside it. Good for comparatively small chunks of XML that you want to do complex stuff with. XSLT uses DOM.
  • SAX - Walk the XML as it arrives watching for things as they fly past. Good for large amounts of data or comparatively simple processing.
  • StAX - Much like SAX but instead of responding to events found in the stream you iterate through the xml
DOM

First lets look at DOM parser. It uses DocumentBuilderFactory to build the XML document and defines a factory API that enables applications to obtain a parser that produces DOM object trees from XML documents. Since we are going to validate the xml with the schema, we need to aware with Document namespace. I'm using javax.xml.parsers.DocumentBuilder to obtain DOM Document instances from the empty XML document which I instantiated using DocumentBuilderFactory. An instance of DocumentBuilder class can be obtained from the documentBuilderFactory.newDocumentBuilder() method. Once an instance of this class is obtained, XML can be parsed from a variety of input sources. These input sources are InputStreams, Files, URLs, and SAX InputSources. But in here I'm using it only to create new XML document instead of parsing existing one. Then I'm creating a new org.w3c.dom.Document by parsing my xml file using the documentBuilder.

DocumentBuilderFactory documentBuildFactory = DocumentBuilderFactory.newInstance();
documentBuildFactory.setNamespaceAware(true);
DocumentBuilder documentBuilder = documentBuildFactory.newDocumentBuilder();
Document document = documentBuilder.parse("students.xml");

Now we can validate parsed document against xml schema which mentioned in above. javax.xml.validation.SchemaFactory is a schema compiler. It reads external representations of schemas and prepares them for validation. So we need to specify default namespace for the schema when we call for new instance of schemaFactory. In validation process we create javax.xml.validation.Schema object from schema file by using schemaFactory instance. Schema object represents a set of constraints that can be checked/ enforced against an XML document and it is an immutable in-memory representation of grammar.

SchemaFactory schemaFactory = SchemaFactory.newInstance(XMLConstants.W3C_XML_SCHEMA_NS_URI);
Schema schema = schemaFactory.newSchema(new File("studentsSchema.xsd"));
Validator validator = schema.newValidator();
validator.validate(new DOMSource(document));

Then we can extract elements from parsed document using root element of the document. Since our xml has Student list, I'm using ArrayList to hold each student record as POJO and NodeList to hold student elements from parsed xml.

ArrayList < Student > students = new ArrayList < Student > ();

// Extract nodes list from the xml file
NodeList nodes = document.getDocumentElement().getChildNodes();

// Iterates through all nodes
for (int i = 0; i < nodes.getLength(); i++) {
 Node node = nodes.item(i);
 if (node.getNodeType() == Node.ELEMENT_NODE) {
  Student student = new Student();
  student.setStudentId(Integer.parseInt(node.getAttributes()
   .getNamedItem("id")
   .getNodeValue()));

  NodeList childNodes = node.getChildNodes();

  for (int j = 0; j < childNodes.getLength(); j++) {
   Node childNode = childNodes.item(j);
   if (childNode.getNodeType() == Node.ELEMENT_NODE) {
    // extract the content
    String elmBody = childNode.getLastChild().getTextContent();
    // check type of the content
    switch (childNode.getNodeName()) {
     case "studentsns:name":
      student.setStudentName(elmBody);
      break;
     case "studentsns:year":
      student.setRegYear(Integer.parseInt(elmBody));
      break;
     case "studentsns:nic":
      student.setNic(elmBody);
      break;
     default:
      break;

    }
   }
  }
  students.add(student);
 }
}

SAX

Implementation of SAX validation is quite simple than implementing a DOM validation. We are using schema factory as same as the DOM validation. But when we are parsing our xml, we don't want to create DOM source from our xml. Instead of that we are passing javax.xml.transform.stream.StreamSource.StreamSource from xml File to validator.

SchemaFactory schemaFactory = SchemaFactory.newInstance(XMLConstants.W3C_XML_SCHEMA_NS_URI);
Schema schema = schemaFactory.newSchema(new File("studentsSchema.xsd"));
Validator validator = schema.newValidator();
validator.validate(new StreamSource(new File("students.xml")));

Since SAX parsing is event driven, we need to use event handler to handle events and build POJOs from xml. So we are creating separate handler class by extending org.xml.sax.helpers.DefaultHandler class. When parser starts parsing with a xml stream, startDocument method called and once stream ended, endDocument method called. Once parser detects start or end of element, it called startElement and endElement methods respectively with following parameters.

  • uri The Namespace URI, or the empty string if the element has no Namespace URI or if Namespace processing is not being performed.
  • localName The local name (without prefix), or the empty string if Namespace processing is not being performed.
  • qName The qualified name (with prefix), or the empty string if qualified names are not available.

In startElement method, there is an additional parameter is there to specify the attributes attached to the element. If there are no attributes, it shall be an empty Attributes object. Each element body content can be extracted through the characters method. Finally if there any exception occurred during parsing, error method is called.

public class StudentsSAXHandler extends DefaultHandler {

 private ArrayList<Student> students;
 private Student student;
 private String elmBody;

 @Override
 public void startDocument() throws SAXException {
  students = new ArrayList<Student>();
 }

 @Override
 public void endDocument() throws SAXException {
  // printing student details
  for (Student s : students) {
   System.out.println(s.toString());
  }
 }

 @Override
 public void startElement(String uri, String localName, String qName,
                          Attributes attributes) throws SAXException {
  if ("student".equals(localName)) {
   student = new Student();
   student.setStudentId(Integer.parseInt(attributes.getValue("id")));
  }
 }

 @Override
 public void endElement(String uri, String localName, String qName) throws SAXException {
  switch (localName) {
   case "student":
    students.add(student);
    break;
   case "name":
    student.setStudentName(elmBody);
    break;
   case "year":
    student.setRegYear(Integer.parseInt(elmBody));
    break;
   case "nic":
    student.setNic(elmBody);
    break;
   default:
    break;
  }
 }

 @Override
 public void characters(char ch[], int start, int length) throws SAXException {
  elmBody = new String(ch, start, length);
 }

 @Override
 public void error(SAXParseException e) {
  System.err.println("Parsing error: " + e.getMessage());
 }

}

Now we have handler for the SAX parser, so then we are going to create parser with that handler. Since we already validate our document with SAX validator, we just only need to create instance from javax.xml.parsers.SAXParserFactory to parse our xml with our namespace. Then we are creating javax.xml.parsers.SAXParser. For ease of transition, this class continues to support the same name and interface as well as supporting new methods. An instance of this class can be obtained from the javax.xml.parsers.SAXParserFactory.newSAXParser() method. Once an instance of this class is obtained, xml can be parsed from a variety of input sources. These input sources are InputStreams, Files, URLs, and SAX InputSources. In here I'm using xml file as input source.

// Get SAX Parser Factory
SAXParserFactory factory = SAXParserFactory.newInstance();
factory.setNamespaceAware(true);
factory.setSchema(schema);
SAXParser parser = factory.newSAXParser();
parser.parse(new File("students.xml"), new StudentsSAXHandler());

StAX

In StAX validation, javax.xml.stream.XMLInputFactory is used to build xml from input stream or from file. Then using xml input factory we are creating javax.xml.stream.XMLStreamReader interface object and it is used to build javax.xml.transform.stax.StAXSource.StAXSource which can be validate using the validator.

XMLInputFactory factory = XMLInputFactory.newInstance();
SchemaFactory schemaFactory = SchemaFactory.newInstance(XMLConstants.W3C_XML_SCHEMA_NS_URI);
XMLStreamReader xmlStreamReader = factory.createXMLStreamReader(new FileInputStream("students.xml"));
Schema schema = schemaFactory.newSchema(new File("studentsSchema.xsd"));
Validator validator = schema.newValidator();
validator.validate(new StAXSource(xmlStreamReader));

Then we only have to iterate through each element in node in reader object to build our POJOs by parsing xml.

ArrayList < Student > students = null;
Student student = null;
String elmBody = null;

while (xmlStreamReader.hasNext()) {
 switch (xmlStreamReader.next()) {
  case XMLStreamConstants.START_DOCUMENT:
   students = new ArrayList < Student > ();
   break;
  case XMLStreamConstants.START_ELEMENT:
   if ("students".equals(xmlStreamReader.getLocalName())) {
    students = new ArrayList < > ();
   } else if ("student".equals(xmlStreamReader.getLocalName())) {
    student = new Student();
    student.setStudentId(Integer.parseInt(xmlStreamReader.getAttributeValue(0)));
   }
   break;
  case XMLStreamConstants.CHARACTERS:
   elmBody = xmlStreamReader.getText().trim();
   break;
  case XMLStreamConstants.END_ELEMENT:
   switch (xmlStreamReader.getLocalName()) {
    case "student":
     students.add(student);
     break;
    case "name":
     student.setStudentName(elmBody);
     break;
    case "year":
     student.setRegYear(Integer.parseInt(elmBody));
     break;
    case "nic":
     student.setNic(elmBody);
     break;
    default:
     break;
   }
   break;
  default:
   break;
 }
}

Monday, May 11, 2015

XML: Create XML from POJOs

XML is using to describe data. So we can use xml in many areas to store and communicate data. In here I'm showing how to build an xml structure using java application to store data in POJOs.

First I crated Java project using eclipse and create POJO class to model Student object. In my student object I have Student Id, Name, Registered Year and NIC number. With that class we can create several student objects and can transform them in to XML file.

In XML creation I used javax.xml.parsers.DocumentBuilderFactory. It is also used to build the XML document and defines a factory API that enables applications to obtain a parser that produces DOM object trees from XML documents. I'm using javax.xml.parsers.DocumentBuilder to obtain DOM Document instances from the empty XML document which I instantiated using DocumentBuilderFactory. An instance of DocumentBuilder class can be obtained from the documentBuilderFactory.newDocumentBuilder() method. Once an instance of this class is obtained, XML can be parsed from a variety of input sources. These input sources are InputStreams, Files, URLs, and SAX InputSources. But in here I'm using it only to create new XML document instead of parsing existing one.

Then I'm creating a new org.w3c.dom.Document object using documentBuilder and appending elements in to it from my POJOs.

DocumentBuilderFactory documentBuilderFactory = DocumentBuilderFactory.newInstance();
documentBuilderFactory.setNamespaceAware(true);
DocumentBuilder documentBuilder = documentBuilderFactory.newDocumentBuilder();
Document document = documentBuilder.newDocument();

Now its time to create XML elements and append them to the created document object. First we have to create a root element and add if any XML processing instructions available before the root element. In this example I'm using xml-stylesheet to style the XML. In here I have used org.w3c.dom.Element interface to create XML elements in document. The Element interface represents an element in an HTML or XML document. Elements may have attributes associated with them; since the Element interface inherits from Node, the generic Node interface attribute attributes may be used to retrieve the set of all attributes for an element. There are methods on the Element interface to retrieve either an Attr object by name or an attribute value by name. In XML, where an attribute value may contain entity references, an Attr object should be retrieved to examine the possibly fairly complex sub-tree representing the attribute value. On the other hand, in HTML, where all attributes have simple string values, methods to directly access an attribute value can safely be used as a convenience.

// Create the root element
Element studentsElement = document.createElement("studentsns:students");
studentsElement.setAttributeNS("http://www.w3.org/2000/xmlns/","xmlns:studentsns", "http://www.charitha.org/students");
studentsElement.setAttribute("xmlns:xsi","http://www.w3.org/2001/XMLSchema-instance");
studentsElement.setAttribute("xsi:schemaLocation", "StudentsSchema.xsd");
document.appendChild(studentsElement);

Then I'm appending my XML processing instructions to document before the root element.

// Create processing instructions for style sheet
Node style = document.createProcessingInstruction("xml-stylesheet", "type=\"text/xsl\" href=\"students.xsl\"");
document.insertBefore(style, studentsElement);

Now its time to iterate through the Student objects list and append student details to XML.

for (Student s : students) {
   // create student element
   Element studentElement = document.createElement("studentsns:student");
   studentsElement.appendChild(studentElement);
   studentElement.setAttribute("id", String.valueOf(s.getStudentId()));
   // create name element
   Element nameElement = document.createElement("studentsns:name");
   nameElement.appendChild(document.createTextNode(s.getStudentName()));
   studentElement.appendChild(nameElement);
   // create year element
   Element yearElement = document.createElement("studentsns:student");
   yearElement.appendChild(document.createTextNode(String.valueOf(s.getRegYear())));
   studentElement.appendChild(yearElement);
   // create nic element
   Element nicElement = document.createElement("studentsns:nic");
   nicElement.appendChild(document.createTextNode(s.getNic()));
   studentElement.appendChild(nicElement);
}

Finally we have to transform DOM object of document to XML using javax.xml.transform.TransformerFactory. An instance of javax.xml.transform.Transformer class can be obtained with the transformerFactory.newTransformer method. This instance may then be used to process XML from a variety of sources and write the transformation output to a variety of sinks.

We have to create javax.xml.transform.dom.DOMSource as a holder for a transformation Source tree in the form of a Document Object Model (DOM) tree. And also we need javax.xml.transform.stream.StreamResult to stream XML in to file. Then we can finally create XML file output by using DOMSource and StreamResult.

// transform DOM object to xml file
TransformerFactory transformerFactory = TransformerFactory.newInstance();
Transformer transformer = transformerFactory.newTransformer();
DOMSource domSource = new DOMSource(document);
StreamResult streamResult = new StreamResult(new File("student.xml"));
transformer.transform(domSource, streamResult);

Generated XML file will be available in project folder.

Click here to download sample eclipse project.