Amazon EC2 Micro Instance Roundup

Since some time now, Amazon offers a free (for 1 year) Micro Instance in their cloud for new sign ups. Micro instances are widely discussed as easy and relatively cheap way to run a cloud-based application (e.g. website/blog). Below a collection of a number of links regarding Amazon’s Micro instance hosting.

Some Fundamental Components

Amazon Elastic Compute Cloud (Amazon EC2): Who came up with this product name? It’s a VPN hosting service, allowing you to run UNIX/Windows virtual machines on the Amazon servers.

Amazon Elastic Block Store (EBS): Amazon offers 10 GB of EBS storage for free as part of the micro-instance. This storage is external to your instance, which is nice, since the data is available when you shutdown/restart the instance or switch to a completely new image.

Amazon Simple Storage Service (Amazon S3): 5 GB for free are offered for the Amazon S3. This is another way to store data external to your instance. However, in difference to the EBS, which can be mounted simple as a node in a unix file system, S3 requires one to use Amazon’s S3 access API.

Performance and Pricing

“Amazon EC2 Micro Instance and Stolen CPU“: Cheap, but sometimes slow. In particular, if the CPU is heavily used for a few seconds, an automatic throttling might occur.

“Amazon EC2 “micro instances” vs. Google App Engine“: EC micro not necessarily cheaper than GAE, especially when considering additional costs for EBS storage.

“What is the real price of an Amazon EC2 machine?“: Some consensus that EC is rather expensive.

“Amazon EC2 Micro instances deeper dive“: Calculates that a micro instance gets ca. ¼ of the allocation of a small instance. Suggests that rackspace and other competitors have better offerings.

“WordPress at Amazon For $10 a Month“: Shows how to build up a website on the Amazon infrastructure costing only $10 a month.

“Deploying node.js on Amazon EC2“: Estimates to run a node.js server would cost about $15 a month using a micro instance.

Tutorials

“Amazon EC2 Cloud Installation.“: A nice tutorial walking through the various steps in setting up a micro instance.

“Quick start to using Amazon EC2 as a free VPS!“: Gives a good introduction to various aspects of setting up a micro instance.

“A LAMP guy’s n00b quick start to Amazon Web Services” Some discussion here on EBS storage vs. instance level storage.

Others

“Amazon Linux vs. Ubuntu for Amazon EC2“: Amazon Linux seems to be running okay. However, maybe it’s better to stick with a more ‘standard’ distribution.

Inside GWT RPC: Understand and Use the Serialization Format

GWT RPC is a technology with both great promises as well as many practical pitfalls. On the one hand, GWT RPC magically handles the transport of objects from a Java server to a JavaScript based client, without any need to create and maintain legacy data formats (e.g. in JSON). On the other hand, caused by the architecture of GWT RPC, it is easy to bloat up the size of JavaScript clients, since any for any object transported from server to client, the client needs a type-specific scaffold to instantiate the types in JavaScript. Calling eval(…), in contrast, is a much more lightweight solution. Moreover, serializing and deserializing large object graphs can be a performance hog, both for server and client.

Below a collection of a few interesting links, which can help in understanding GWT RPC better and using it beyond its initially devised use cases. In particular:

• To have a detailed understanding of how the GWT serialization format works on the inside (“Understand the GWT Serialization Format”),
• How to consume a GWT RPC service from a Java (JSE) client (“Consume GWT RPC using a Java Client”), and
• How to generate messages in the GWT serialization format using an environment different from the GWT provided servlet (“Generate Textual Representations of Java Objects using GWTs Serialization Format”)

Understand the GWT Serialization Format

Stackoverflow “GWT RPC data format”

This discussion provides details on the GWT serialization format, particularly by referring to this presentation. Looking at the format in detail, it indeed seems to resemble a remote method invocation protocol (RMI) rather than a sole remote procedure call protocol. This is not necessarily good, since a lot of meta-data has to be sent with every request (e.g. class on which method is to be invoked).

Consume GWT RPC using a Java Client

Stackoverflow “Invoke a GWT RPC service from Java directly”

It appears that attempting to consume GWT RPC services from Java clients has mostly been pursued for testing purposes. However, it seems that some of the discussed libraries might be out-dated and might necessarily work with the most recent version of GWT (since the GWT serialization format can change from version to version).

Generate Textual Representations of Java Objects using GWTs Serialization Format

Simpler and Speedier GWT with Server Side RPC Serialization – TechHui

This article describes how data can be serialized using GWTs server side serialization and written into the initially downloaded HTML. This can speed up startup time of applications, since it leads to a reduction of RPC calls. I am not quite sure but there might have been mention of this technique in one of the Goolge IO talks on GWT performance. There can be other use cases for attempting to do the GWT serialization ‘by hand’, for instance if one wants to avoid using a Servlet container (tomcat, jetty) and replace it by something nicer like netty. The JavaDoc of the following classes are a good starting point to explore further how to perform GWT serialization Google Web Toolkit JavaDoc Class RPC, Restlet JavaDoc Class Object Representation

 

 

 

 

 

Xtend Tutorial

The world is rife for a new dominant programming language! Actually, has been rife for a couple of years, which, however, does not stop Java and C++ from dominating the screens of many unfortunate developers. Apart from the inarguably impressive features offered by ‘new’ languages, such as Scala, Goovy, Ruby, Haskell, Clojure, Dart? and Go, I am highly suspicious of these languages: What happens to the billions of lines of code written in legacy programming languages? Will my Groovy app work with the next update of Groovy, or will it work in 10 years, when I have to update my app? I don’t want something big, not the total reinvention of computing but would already be happy with more concise syntax to create inline objects, type inference and simple closures.

Xtend defines itself as a language for Java developers. In specific, it allows to write code, which is similar to Java but improved in many ways. Compiling code written in the Xtend language generates Java source files. These files can easily be used with any ‘legacy’ Java application or library. I have assembled a little tutorial and introduction to get started with the Xtend language. Find below:

• How to install the Xtend Eclipse Plug-in
• How to materialize Xtend Example Project in your Workspace
• How to create Your Own Xtend Project
• A discussion of Xtend Advantages and Disadvantages
• Conclusions

Install the Xtend Eclipse Plug In

UPDATE: As Daniel pointed out below, Xtend is now available on the Eclipse Marketplace. Installing it from there rather following the instructions in this section, might be much easier!

• Open the install new software dialog in eclipse
• Connect to the XText release repository: http://download.eclipse.org/modeling/tmf/xtext/updates/composite/releases/
• Select all packages “TMF Xtext-2.1.0″ (or of a newer version if available)
• This includes the “Xtend2 SDK” mentioned in the Xtend tutorial. The other packages need to be selected since otherwise the Xtend editors will not work.

• Click Next again
• Accept EPL licence agreement
• Restart eclipse

Materialize Xtend Example Project in your Workspace

After the installation of Xtend is complete, the best way to get started is to install the Xtend example

• Create a new Example Project as shown below

• Or under the new project wizard select “Examples / Xtend Examples / Xtend Tutorial”

• Initially the project was not built correctly in my eclipse.

• However, after closing and reopening the project everything appeared to be (maybe the Infinitest plugin in my installation has caused the problem, so I have deactivated it).
• Below a screenshot of the examples provided by Xtend:

• You can right click the project and select “Run as JUnit test” to execute some of the examples

Create Your Own Xtend Project

After browsing and testing the provided examples, it’s time to create a new project with Xtend support.

Xtend seems to be nicely integrated with the eclipse PDE. Therefore, its easiest to create an Xtend project as eclipse Plug In Project. Using the PDE tooling such projects can be easily deployed as executable applications on numerous platforms. However, since Xtend generates plain Java files, we can always compile the generated files as independent Java application. Below the steps to create a simple Hello World application.

• Use Eclipse’s New Project Wizard to create a new ‘Plug-in Project’
• Adjust the source folder and output folder to sensible values (Maven anyone?) and create the project

• Open the MANIFEST.MF file in the folder META-INF
• Add the package org.eclipse.xtext.xtend2.lib to the [Imported Packages] on the [Dependencies] tab

• Also add the following packages:

com.google.inject

org.aopalliance.aop

org.eclipse.xtext.base.lib

• Create a new source folder for your project: src/main/xtext (Right click the project and select Build Path / New Source Folder)
• Now create a new package in the source folder src/main/xtext
• For instance, de.mxro.examples.xtend1
• Right click this package and select New / Xtend Class

• Select the name “HelloWorld”
• When prompted to add the Xtext nature to the project, click [Yes]

• Right click your project and select “Properties”
• Open the properties for Xtend / Compiler
• Change the output folder to src/main/java and deselect “Override existing files”
• Click [Apply] to apply the changes

• Go back to your file HelloWorld.xtext and define the following class:

class HelloWorld {

    def public void helloWorld() {

        val message = “Hello, World!”;

        System::out.println(”’«message»”’)

    }

}

Tip: writing the symbol “«” can be achieved by pressing CTRL + SHIFT + < (or ,)

• Now check the folder src/main/java. Here, a file HelloWorld.java should have been generated as follows

import org.eclipse.xtext.xtend2.lib.StringConcatenation;

@SuppressWarnings(“all”)

public class HelloWorld {
public void helloWorld() {
final String message = “Hello, World!”;

   StringConcatenation _builder = new StringConcatenation();

   _builder.append(message, “”);

   System.out.println(_builder);

}

}

Does not look 100% efficient but that’s the tradeoff for source code generation. I did not find out how to define static methods using Xtend, so we need a standard Java class to run this code:

• In the source folder src/main/java add a Java class “Application”, while checking create main(String[] args).
• Implement the following class

public
class Application {

    /**

     * @param args

     */

    public static void main(String[] args) {

        new HelloWorld().helloWorld();

    }

}

• Right click the file Application.java and select “Run as Java Application”

Volia! You should be friendly greeted with the output “Hello, World!”

Xtend Advantages and Disadvantages

After following this little exercise and browsing the Xtend documentation, these are a number of advantages and disadvantages I see in this languages:

Advantages

• Allows writing more concise code for the Java platform.

It is no secret that Java syntax is unnecessarily verbose. This sets the bar relatively low for better languages for the Java platform. Xtend has a number of very nice features to allow writing more concise Java code, my favourite example being type inference.

• Is fully compatible with most Java technologies.

Although the JVM has received considerable attention with alternative Java byte code generating languages such as Scala, not everything Java is Java byte code. Most notably, the Android platform draws on the Java language but is not compatible with generated Java byte code. The Google Web Tookit (GWT) is another technology working with the Java language but not with the JVM. Xtend enables to write code, which interoperates both the platforms build on top of the JVM (since its Java source files can be compiled into byte code) as well as platforms build on top of the Java language syntax.

• Performance

Since the Xtend language is close to the Java language and generates relatively optimal Java source files, its performance can be expected to be far superior to other improved Java dialects, especially Groovy. Moreover, since it does not generate byte code but Java source files, more compiler optimization can be applied to the applications.

Disadvantages

• Tight coupling with eclipse tooling

Xtend and Xtext are doubtlessly eclipse technologies through and through. This is a definite advantage for starting to use these technologies since they offer nice integration with the eclipse IDE. As a downside, however, code written in the Xtend language might be difficult to use with other tools apart from eclipse. On the other hand, the Java code generated by Xtend is vanilla Java code (apart from the lightweight Xtend library used in the generated code). This code, of course, can be used by most other tools. Therefore, I have chosen above to place the xtend source files not in the main Java source code folder of the project (src/main/java) but into a separate folder (src/main/xtext). This enabled to let Java code be generated into the src/main/java folder. From there, it can be picked up and used by other tools such as Maven.

• In places, insufficient documentation

The documentation available for Xtend is far from extensive. In comparison, the documentation available for Groovy is extensive including books and various web sites.

Conclusion

I believe that Java and C++ are the very best programming languages around. Not in terms of developer productivity, measured in implemented features/hours of course, but in terms of creating sustainable code. Most Java or C++ applications written a decade ago will still run today. Java programs even without the need to be recompiled. This makes one rather confident that a decade from now, still people will be able to compile and run written in Java and C++ today. Adopting terms from the finance industry, they are simply too big to fail.

Give that, I am very suspicious of any new programming language. That is it more elegant and allows me to be more productive are simply insufficient arguments. Code is too difficult to write that I would want to run into the risk of having to dump it ever. Xtend offers a neat compromise. It allows to write code in a more modern, more productive language but yet defining the application in plain old Java code. Therefore, I see Xtend primarily as a useful tool to generate Java tool. Even if Xtend becomes deprecated at some point in time, one always has the Java code to fall back to.

Mistakes to Avoid

As mentioned above, installing the plugin following the documentation provided on the Xtend tutorial page has not been a smooth process. Below a number of mistakes in installing the plugin, which are better avoided.

Do not install only the Xtend2 SDK

Installing only the Xtend 2 SDK package as shown below …

.. and creating a new class …

… will result in a problem being reported by Xtend.

Do not Install the Xpend/Xtend All-in-One Package

Selecting “Xpand/Xtend All-in-One” …

… with all these packages …

… will not allow you to create the Xtend example project.

Do Not Create an Xtext Project to Start an Xtend Project

Xconfused? An Xtext project does not automatically allows to get started with Xtext. It is better to create an eclipse Plug-in project as shown above.

References

Xtend Official Overview

Xtend 2 – the Successor to Xpand post by creator Sven Efftinge

Key-Value Stores for Java

One common theme in discussions of persistence is to be critical of the traditional SQL databases, which have been successfully used in business applications for decades. As alternative to these SQL databases, often so called NoSQL solutions are advocated. NoSQL, essentially, dramatically simplifies the kind of data, which can be managed by the database. Often, a simple key-value store is at the heart of the database. Such stores allow the user to store data in a similar way as it has long been known for the Map data structure. However, whereas the traditionally Maps have been used in-memory, e.g. to hold data temporarily, which fits on one system’s memory, NoSQL key value stores are designed to persist and scale up to millions or billions of key-value records, distributed among potentially thousands of servers. As can be expected, in the Java space, there is not one all-encompassing NoSQL solution but various (mostly free and open source) offerings compete with varying features.

In the following I discuss two categories of implementations:

1. Object Prevalence engines: which assure objects stored in memory are never lost and constantly backed up onto a hard disk.
2. Persisted Databases engines: which allow to work with key-value databases, which are larger than the systems memory by persisting parts of them onto a hard disk (or another persistence backend).

Object Prevalence

Object Prevalence systems keep all records in-memory. However, they are designed to keep a synchronized persisted copy of all objects on the hard disk. In case of a system failure or reboot they therewith can easily restore the state before the interruption. Object prevalence systems are often used to speed up applications, for which all required data can be stored in memory (Villela, 1^st of August 2002).

One prominent example of an object prevalence system is the open source solution Prevayler (http://prevayler.org). A significant limitation for such object prevalence systems is, of course, that the amount of data the application can work with is limited to the memory available on one server. To allow systems to scale, an object prevalence system can be distributed among various systems. A very well-performing open source solution for this purpose is hazelcast (http://www.hazelcast.com/).

Engine	Project home	Description
Prevayler	http://prevayler.org	Lightweight solution to build prevalent systems.
Space4j	http://www.space4j.org/	Keeps an incremental record of all operations saved on disk in order to recreate a state in memory (but also has the option to create snapshots).
MegaMap	http://megamap.sourceforge.net/	Allows to work with Maps larger than the memory and persist these onto disk. However, not developed for a while and not fault tolerant.
hazelcast	http://www.hazelcast.com/	Allows to distribute objects held in memory among a cluster of physical systems (within a network or connected through WAN).

Persisted Databases

While object prevalence systems are arguably one of the best performing solutions to store the data of an application, there are many use cases, in which data is only rarely accessed and therewith not required to be available in memory. For these purposes key-value stores, which keep only a fraction of their data in the memory are best suited. Below a number of examples implementing this pattern. To get started quickly, I think jdbm2 is a good option, for large scale solutions, you might have to consider Berkely DB – but this might end up being a pricy pathway.

Engine	Project home	Description
jdbm2	http://code.google.com/p/jdbm2/	Persists a HashMap or TreeMap using Java Serialization.
Banana DB	http://people.apache.org/~kalle/bananadb/	Persists a Map in a file. Potentially inefficient read operations.
BabuDB	http://code.google.com/p/babudb/	Persists key-value pairs of byte[] values.
Berkeley DB	http://www.oracle.com/technetwork/ database/berkeleydb	Key-value store solution from Oracle. Rather restrictive open-source licence.
JOAFIP	http://joafip.sourceforge.net/	Rather than providing a simple key-value store, JOAFIP attempts to dynamically persist an object tree to the disk (to manage object trees too large to fit into memory).

Resources

JDBM

“HIVE-1754 – Remove JDBM component from Map Join” Apache Hadoop has removed the JDBM dependency from their codebase due to poor performance. However, they seem to have replaced the persistence backed map with a map solely in memory. So they did not really find a better alternative for JDBM but rather removed the necessity for file-based persistence altogether. The old implementation of JDBM used by them can be found in the source repositories: e.g. BaseRecordManager in hive-0.5.0, or how it was used in HashMapWrappper also in hive-0.5.0.

“JDBM2 released” Blog post with some additional info concerning JDBM2

SQL Alternatives

For an SQL database with a very low footprint and high performance see: http://www.h2database.com/

Other Resources

“An introduction to object prevalence” (Villela, 1^st of August 2002)

“Anti-RDBMS: A list of distributed key-value stores” (Jones, 19^th of January 2009)

“NoSQL Services Available” (Menon, 29^th of March 2011)

“The Jalapeño Persistence Library for Java” (intersystems.com)

“POJO Persistence” (on this blog)

Prevayler (http://prevayler.org)

hazelcast (http://www.hazelcast.com/)

A Quick Reflection on Exceptions in Java

Exceptions are a necessary evil of most software development endeavours. As much as we would like to design applications, which are equivalent to mathematical functions ‘give me one input and always I will trustfully return you the same output’, the real world IOExceptions, OutOfMemoryExceptions, DivideByZeroExceptions and their various evil cousins force us to consider a versatile and often unpredictable state often out of our control.

The Java programming language, for instance, fundamentally provides four ways to inform a caller of a method of an ‘exceptional’ state:

• Return Values of methods allow passing information of an invalid state to the caller of the method. Often, a specific “null” value is used to denote “Sorry, I can’t do what you ask me to do, ’cause of some unknown condition”.
• Checked Exceptions can be added to the signature of methods to indicate that these methods could not be executed as planned. In difference to using return values, Checked Exceptions allow to express richer descriptions of the failure conditions. For instance, an IOException can be cast if an unexpected error occurred while reading/writing data from a source other than the memory.
• Unchecked Exceptions can be thrown anywhere and must not be declared as part of message signatures. Unchecked Exceptions where originally envisioned to capture unexpectable exceptions, for instance an OutOfMemoryException when the JVM runs out of memory.
• Callbacks are usually not as prominently used in Java as they are in JavaScript or node.js etc. However, they can be a powerful tool in Java. In principle, rather than return the result of a method as a return value, an object is passed to the method and the method calls specific methods of this object depending on the result of the method. The GWT RPC mechanism is a nice example for this, where a call to the remote server results either in a method onSuccess(…) or onFailure(…) to be called.

Some Advantages and Disadvantages

Choice always comes with its challenges, so we remain with the question, which of these mechanisms to use to report unexpected states during the execution of a method to the consumer of the method. Below, a brief discussion of potential advantages and disadvantages of the types discussed so far:

Return Values

Reporting unexpected states during the execution of a method in a return value is usually not a very good idea. The reason for this is that interpreting the result of the method becomes significantly more complex for the caller of this method. For instance, if the method divide(x,y) would report the divide by 0 exception through the return value, it would not be possible to use a number-based type such as Integer as the return type. Instead, a type like IntegerOrException would have to returned, which complicates using the method greatly.

Checked Exceptions

Once propagated as innovative feature of the Java programming language, checked exceptions are now often portrayed as one of its most serious design flaws. Like reporting unexpected conditions in return values, defining checked exceptions significantly complicates consuming methods. This is particularly paramount since Java has adopted a quite verbose syntax for dealing with exceptions, which quickly undermines the elegance of any code fragment. Basically, any call to a method with checked exceptions quickly explodes from 1 LOC to 5 LOC (try … catch …).

Unchecked Exceptions

Unchecked Exceptions are the unfriendly brothers of checked exceptions. They cause an application to immediately stop working and displaying a lengthy error report. Although it seems unintuitive at first, to write applications which are constantly on the verge of total collapse (e.g. an unchecked exception is thrown), relying on unchecked exceptions can lead to surprisingly reliable applications. Moreover, it is also possible to explicitly handle unchecked exceptions (using the good ol’ try … catch …) to avoid a production system from crashing on unexpected exceptions. However, the number of such explicit ‘checkpoints’ for unchecked exceptions is usually far lower than for checked exceptions.

For me, unchecked exceptions actually come in two flavours: those caused by ‘throw new RuntimeException(…)’ statements and those cause by violated assertions (assert text !=null). The latter are specific in that these exceptions will most likely be thrown only in development environments. For production, all assertions can be ‘switched off’, which can increase execution speed and a reduction in ‘application breaking’ exceptions being thrown.

Callbacks

Callbacks, in my opinion, are an underused ‘feature’ of the Java programming language. The node.js folks are not too wrong claiming that any operation depending on external resources (remote server, file system, …) should be done in an asynchronous ways. This is enabled by callbacks, since it is undefined, when the return methods will be called. Callbacks are usually a good choice if the caller of a method needs to respond differently to different unexpected states. Was there an error reading from the filesystem or sending a call to the server, or both? In this case, callbacks allow for an elegant way to make the caller of a service aware of these states, which must be handled, since each state can be represented by one method on the callback object.

Conclusion: A User-Centric Perspective

It is difficult to say which is the ‘right’ way of handling unexpected states since each one has their own advantages and disadvantages. I think one could sometimes even make a case for checked exceptions. In general, I find it very helpful to aid my decisions by thinking of the type of the unexpected state as well as the nature of the user of the provided method.

Do you think the unexpected state is ‘impossible’?

Yes. Sometimes the program can be expected to work in a certain way and an unexpected condition is simply unthinkable. An example for such a condition would be one key being existent in a Map twice. In such a case, assertions are the way to go, assume that after the application is tested and goes in production such highly dysfunctional conditions should have been eradicated.

No. If the exception condition can be expected, assertions are not a good option. For instance, an error might be encountered while reading a file from the disk. Such exceptional circumstances are likely to happen in production environments. Therefore, I would use callbacks to inform the caller of the method of potential exceptions occurring during execution of the method.

Is the unexpected state caused by an improper use of your interface?

Yes. Every object, even those not explicitly extending an interface have the programmatic interface of the sum of methods it implements. Conceptually, there are certain rules how these methods need to be used above providing the right data types for the parameters. For instance, it is not sensible to insert an element into a set, which is already defined in this set. In this case, I prefer to use unchecked exceptions, which provide the user of your interface with a clear message of what is going wrong (e.g. ‘Element X cannot be inserted since it is already defined in the set’). Note that the preconditions in Google Guava are a nice tool for this purpose.

No. If the unexpected state is not cause by an improper use of your interface, the impossibility question listed above can be applied to determine wither to use assertions or callbacks.

Resources

Blog post “Programming antipatterns“

 

JavaDoc Editor for eclipse

Formatting JavaDoc using plain HTML can be a troublesome and time-intensive experience. Today I installed the JDocEditor plugin for eclipse, which allows editing JavaDoc in eclipse using a small rich text editor.

Here, a quick evaluation, a few screenshots and a little getting started guide.

Evaluation

Good:

• Free
• Helps to reduce the hassle of dealing with line breaks and paragraphs in JavaDoc
• Allows to compose lists and do simple formatting like using bold and italics

Not so good:

• The editor has no native support for JavaDoc annotations such as @link, …
• The editors handling of line breaks, paragraphs and basically any more sophisticated formatting can be unpredictable at times

Conclusion:

• Good tool to enhance productivity in editing JavaDoc documentation with simple formatting.

Screenshots

The Editor:

Generated JavaDoc (rendered):

Generated JavaDoc (html):

Getting Started

• Install using the update site http://www.certiv.net/updates
• After the plug in has been installed, add the view of the plugin to your workspace as shown below

• The view can be found under the category JavaDoc Editors/ JDocEditor

Resources

StackOverflow Discussion “JavaDoc editor for Eclipse to create formatted text”

Blog post on JDocEditor from 2005

Models Driven Development and Domain Specific Languages

The question of the right programming language is one that has always spurned much controversy. Likewise, the idea that one day we could develop software by simply ‘drawing’ expressive models has as many advocators as opponents.

The idea of domain specific languages could be one which helps us to advance both of these controversies. Both programming languages and traditional models (think UML) are ultimately both just abstractions, models to use another word. More specifically, most visual modelling techniques as well as programming languages follow well-defined (more or less) grammars. In a domain specific language we utilize the power of these grammars to solve problems in one domain.

A very good introduction as well as guide to domain specific languages (DSL) has just been released by Markus Völter:

MD*/DSL Best Practices V 2.0

Below a number of citations from this document on a number of matters:

General Purpose Programming Language vs. DSL

“[T]he ability to extend existing languages (such as it is possible with MPS, Spoofax, and to some extent with Xtext2), makes it possible to build domain specific languages as extensions of general-purpose languages. So instead of generating a skeleton from the DSL and then embedding 3GL code into it, one could instead develop a language extension, that inherits for example expressions and/or statements from the general-purpose base language. This makes a lot of sense: imagine the development of a language for asynchronous, reactive programming. In this case it is very useful to be able to inherit expressions from a general-purpose base language.”

Graphical vs. Textual Notation

“Things that are described graphically are easier to comprehend than textual descriptions, right? Not really. What is most important regarding comprehensibility is the alignment of the concepts that need to be conveyed with the abstractions in the language. A well-designed textual notation can go a long way. Of course, for certain kinds of information, a graphical notation is better: relationships between entities, the timing/sequence of events or some kind of signal/data flow.”

Tooling Matters

“Defining languages and notations is not enough per se – you have to provide good tool support for them, too. […]To increase usability, DSL editors need to be able to cope with wrong or incomplete models as they are entered by the users. Ideally, it should even be possible to persist them. Of course, as long as models are wrong or incomplete they cannot be processed any further. In the context of textual languages, this might mean that you design a somewhat “looser”, more tolerant grammar, and enforce correctness via constraints.”

Critique

I personally see especially great value in building DSLs around general purpose languages, such as Java, JavaScript, Groovy, Scala, etc. Some frameworks have already gone in long way in embedding DSLs in Java Syntax. See here for instance the Mirror Project. This project aspires to make it easier to interact with the Java Reflection API. The resulting calls come close to expressions in natural language.

new Mirror().on(target).set().field(fieldName).withValue(value);

Another example is the mocking framework Mockito. This framework utilizes a DSL to specify the behaviour of ‘mocked’ objects. For instance:

when(mockedList.get(anyInt())).thenReturn(“element”);

Another premier example is the hamcrest library, which is also mainly used to support unit tests.

assertThat(theBiscuit, is(equalTo(myBiscuit)));

Further ‘DSLs’ implemented in Java based on the hamcrest library are listed in the project, which use the hamcrest library.

Another example of a DSL directly implemented in Java is the JooQ library, which integrates SQL with the Java syntax. Below an example from the jOOQ project website:

create.selectFrom(BOOK)

.where(PUBLISHED_IN.equal(2011))

.orderBy(TITLE)

Many APIs are also implemented in a DSL-like fashion. For instance, the API for the NextReports framework (Dinca-panaitescu, 2011):

FileOutputStream stream = new FileOutputStream(“test.html”);

FluentReportRunner.report(report).connectTo(connection).withQueryTimeout(60).withParameterValues(createParameterValues()).formatAs(ReportRunner.HTML_FORMAT).run(stream);

Of course, there is still some way to go to improve those DSLs. Especially end users are likely to struggle with some intricacies of general programming languages. That said, I still believe that it would be easier to build powerful tools around the limitations of general purpose languages than to develop these ‘from scratch’ for other languages.

References

MD*/DSL Best Practices V 2.0 (Völter, 2011)

“Model Driven Development and Domain Specific Language Best Practices”, Jean-Jacques Dubray on Mar 28 2011 on infoq.com

Presentation “Real Software Engineering” by Glenn Vanderburg presented at Lone Star Ruby Conference 2010. (Advances that the code is the model)

“Is modelling about to overtake coding? I’m a happy SAP business consultant “, Thierry Crifasi on SAP Community Network posted on Dec. 13, 2010 (Advances that models should take over coding)

 

JDepend Plugin for Eclipse

The other day I came across another interesting static source code analysis plugin for eclipse: JDepend. JDepend calculates a number of quality metrics for Java packages (therewith it has a slightly different unit of analysis to other tools, which focus on methods, classes or projects).

I found that Andrey Loskutov has created an excellent eclipse plugin based on JDepend called JDepend4Eclipse.

After installation of the plugin, context menus for packages (not projects) will feature another item ‘Run JDepend analysis’.

Executing this analysis will result in a nicely formatted report being generated. The results can easily be interpreted with the JDepend documentation.

I tend to build very small modules, in which I place a kind of factory for concrete implementations along with some essential interfaces in the main package of the module. It seems that JDepend is not fond of this practice, since the factory depends on the concrete implementations (in the .internal* packages) and these in turn depend on the essential interfaces within the main package. However, I am not yet convinced that this practices is very bad, as long as circular reference between modules are avoided.

Eclipse and SVN to manage Apache Web Server

In order to configure a simple Apache Web Server, I found myself many a time connecting to a UNIX system using putty, fiddling around with various configuration files using pico or vim. I don’t want to count the times I deleted some configuration I later missed dearly or forgetting to close some xml element. Two of the main problems for me were:

(1) As much as some might be convinced of the productivity of text-based editors like pico and vim, for me it just doesn’t fly. Of course one shouldn’t use the mouse for everything but windows, the mouse cursor, tabs, and various visualizations such as trees seem just too useful to me to be completely abandoned.

(2) I have become too used to the convenience of a versioning system when developing source code. Although I know there are various ways to backup a UNIX-based system, I would like all the convenience modern source code management systems offer in organizing the versions of text-based files.

I decided to try using two established technologies to address these problems:

• Eclipse is a well-known integrated development environment primarily for the Java programming language, allowing for many conveniences in editing text-based files.
• Subversion (or SVN) is a centralized versioning system for just about anything. Although Subversion can be very cumbersome at times (talk about deleting folders …), it is supported by many platforms and can become almost usable when one knows of its limitations.

The following is a description of the necessary steps to set up a development environment for Apache web server configuration files (and basically any other kind of configuration files one might encounter on a remote server) build on eclipse and subversion.

Step 1: Install Subversion on your web server

Installing Subversion on the various flavours of UNIX is very straightforward and Google quickly yields useful instructions. Here, for instance the instructions for Ubuntu Server.

Please note that you need to install subversion on your web server primarily for the SVN command line client, so omit any configuration steps regarding setting up a subversion server.

You can check the success of this step by typing in your remote console ‘svn –version’.

Step 2: Install Subversion Server

It is sensible to install a subversion server on a different machine than your web server. This will also allow for an easy way to backup various configuration files from your web server.

Again, there are numerous instructions for setting up a subversion server. See for instance, this post for setting one up on Ubuntu. One of the easiest ways is using VisualSVN to install a subversion server as a windows service.

Step 3: Create a Repository on your Subversion Server

You can either use one for your already set up repositories (if you use subversion for other project) or create a new repository dedicated for web server configurations.

In VisualSVN, you can right click ‘Repositories’ and select ‘Create New Repository’ which opens the dialog given below.

Step 4: Create a project in your repository

Since the folders in which the configuration files lie on the web server are already filled with files and folder, we need first to create an empty project on the subversion server.

In VisualSVN, we can right click the newly created repository and create a new Project Structure as shown below.

Step 5: Add the folders on your web server to version control

After we have created a ‘skeleton’ project on the SVN server, we can start to upload the configuration files from the web server.

For this we go to the folder, from which we want to upload the files. For instance, /etc/apache2.

cd /etc/apache2

In this folder, we initialize the SVN checkout operation:

sudo svn checkout https://[your SVN server]/svn/webRepro/webMyApacheConf/trunk/ .

Next, we can add the folders we want under version control. For instance, the folder conf:

sudo svn add conf

Finally, we commit and upload the files to the SVN server:

sudo svn commit

Step 6: Import the files into eclipse

In order to import the files uploaded to the subversion server, we need to install the Subversive eclipse plugin. After doing so, the following option should be available under File / Import:

In the following dialog, we first specify our SVN repository (../webRepro/) and the project (../webRepro/webMyApacheConf). Note that you should select the ‘trunk’ folder rather than the project root.

Chose to check out the project as a project configured using the New Project Wizard.

As type for the project, select General / Project.

Step 7: Edit configuration files in eclipse

Now, the configuration files from your web server should be available as eclipse project.

Just click and edit these files in the eclipse editors. If you have made changed, right click the project and select Team / Commit, which will commit the changed files to the subversion server.

Step 8: Import changed files to web server

Finally, if you have changed files in eclipse, you need to reimport these files onto your web server. For this connect to your web server and navigate to the directory you have placed under version control. For instance:

cd /etc/apache2

To download the changed files, type

sudo svn update

As a result, the changed files should be downloaded to your web server.

Then, don’t forget to restart your server to see the effect of the changes.

 

 

 

 

Convert IMG to ISO on Windows 7

In order to mount an image file on VirtualBox, I needed to convert an IMG image file into an ISO format (VirtualBox does not support mounting IMG files). There seem to be a myriad of applications available for this purpose; many of them commercial or at least shareware.

With OSFMount I found a free and handy tool for this purpose. Download and install the software, then mount the IMG file you need to convert.

Right click the mounted drive and select “Save to image file…”.

Choose “RAW CD Image (*.iso)” and Voilà! You have an ISO file, which can be mounted by VirtualBox.