Clojure is fun and Clojure macros are even more fun to use and really useful for developing domain specific languages. You can do amazing things using macros once you get used to them. This post describes how we can use Clojure proxy and proxy-super in combination with macros to implement DSLs which needs interactions between both Java and Clojure.

Recently I was developing DSL for Apache Storm as an experiment and had to bridge Storm’s Java API and Clojure. There is a Clojure DSL available for Storm in Storm project itself. But I wanted to implement DSL similar to popular stream processing language StreamIt. I mainly followed existing Storm Clojure DSL implementation and built my DSL based techniques used in that.

Existing Storm Clojure DSL is also done by bridging Storm’s Java API and Clojure and it uses combination of Java interfaces and Clojure reify to allow users to define...

Sri Lankan Cricket Team reached world cup finals four times before this and finally became 2014 World T20 Champions!! Congratulation Mahela, Sanga and Sri Lankan Cricket Team.

Above photo was taken from espncricinfo.com.

Me and my wife celebrated our 4th wedding anniversary with our daughter on 4th March 2014. Here are some pictures from that day.

According to Wikipedia

The singularity, is a hypothetical moment in time when artificial intelligence will have progressed to the point of a greater-than-human intelligence, radically changing civilization, and perhaps human nature.

Recent acquisitions by Google, IBMs recent developments related to Watson, and other day to day news such as deep neural networks to implement human like behaviors shows us that we are hardly trying to moving towards singularity.

But in my opinion we should try to integrate (or interface) human brain into computers and find a way to utilize best of both worlds as shown in TV series Intelligence rather than trying to make human like computers. And this way we may not need to fear about machines trying to dominate humans, situations like in Matrix or Terminator. Even though these humans are connected to virtual world they will always be human as long as we...

Qualcomm has made developing BLE based solutions practical and easy with their Gimbal Proximity Beacons. I don’t have experience with other BLE solutions like Estimote except with BLE Shield for Arduino. But Gimbals beacons, SDK and their documentation allowed me to integrate Gimbal Series 10 beacons into iOS app within 1 hour.

Gimbal provides Cloud based API for managing and querying beacons. Developers can manage their beacons through Gimbal portal and iOS/Android applications can use Gimbal SDK for discovering beacons or for tasks such as geofencing. One disadvantage is they hide the original UUID for the beacon and we can only discover beacons through their API which gets the UUIDs for your beacons on the fly. So internet connection will be required most of the times.

I specifically said most of the times because, I was able to detect my beacon using Gimbal app even when I am...

This post describes necessary steps required to setup 2-node Hadoop YARN cluster using Hadoop 2.0.6-alpha release. This post is based on these¹ posts² and can be considered as a combination of both posts with extra steps necessary to setup Hadoop 2.0.6-alpha. Steps I discussed here should also work for Hadoop 2.1.0-beta.

User accounts, /etc/hosts file modifications and password less SSH

You can follow the steps described in this post¹ under section “User creation and other configurations steps” to setup necessary user accounts and password less SSH. One thing to make sure that password less ssh for localhost, in addition to slaves. Other post² doesn’t mention about password less SSH for localhost, but it is important. Otherwise startup scripts will ask you to type the password during data node and node manager startup.

Configuring Hadoop

You can follow the steps 4, 5, 6, 7, and 8...

Clojure web application development is still in it’s early stages and there aren’t lot of web applications written in Clojure. Recently I had to write add file uploading support to web application I’m developing in Clojure(using ring and compojure). I found following two articles by googling, but those lacks some of the basic internal details.

• File upload in Clojure & Compojure
• File uploads with Clojure, Ring and Compojure

Rest of this post will discuss some of things I have discovered while implementing file upload support.

Below is the code for a simple compojure web application which implements file upload support.

(ns file-upload.core
  (:use [compojure.core]
        [ring.middleware.params]
        [ring.middleware.multipart-params]
        [ring.adapter.jetty]
        [hiccup.core]))

(defn home-page []
  (html [:form {:action "/file" :method "post" :enctype

I did some coding in Scheme and Ruby last couple of months and felt that writing Java code is not fun anymore. But Java as a platform is really good for
most of the tasks and I started to learn some Clojure because there is support for reusing Java libraries in Clojure and it’s a really good mordern [Lisp](http://en.wikipedia.org/wiki/Lisp_(programming_language) derivative.

I’m new to Clojure and I found the series of blog posts by Eric Rochester is really helpful to understand basic stuff for a beginner like me. Please keep in mind that there are some incompatibilities in the code of above blog posts with the new Clojure versions.

As a start I started to re-wrtie a partially done Java Web Service in
Clojure as a REST service. So I googled for resources on Clojure web
appliction development and found this tutorial. That is really good tutorial for beginners and it covers most of the...

Scrapy is a web crawling and scraping framework
written in python. The framework is really simple to understand and easy
to get started with. If you know little bit of Python, you should be
able to build your own web scraper within few minutes. In this post I’m
going to describe how you can use Scrapy to build recursive blog
crawler. Building a recursive scraper using Scrapy is pretty simple, yet
it’s getting started guide doesn’t help people who are unfamiliar with
the framework to write a recursive scraper.

I’m going to use blog.scrapy.org as the target
blog and techniques I am discussing will work on other sites as well
with simple changes to information extraction queries. If you are new to
Scrapy it’s required to read the Scrapy tutorial first.
I also assume that you have Scrapy installed in your machine.

Let’s create a Scrapy project first using following command:

scrapy