Saturday, 5 October 2013

Functional Programming with Groovy

I have recently started the Coursera Functional Programming with Scala course (taught by Martin Odersky - the creator of Scala) - which is actually serving as an introduction to both FP and Scala at the same time having done neither before. The course itself is great, however, trying to watch the videos and take in both the new Scala syntax and the FP concepts at the same time can take a bit of effort.

I wanted to work through some of the core FP concepts in a more familiar context, so am going to apply some of the lessons/principles/exercises in Groovy.


Functions:

If you have done any Groovy programming then you will have come across Groovy functions/closures. As Groovy is dynamically typed (compared to Scala's static typing), you can play it fairly fast and loose.

For example, if we take the square root function that is demonstrated in the Scala course, it is defined as follows:



As you can see, Scala expects the values to be typed (aside, you don't actually always need to provide a return type in Scala). But in the Groovy function it is:



A groovy function can be defined and assigned to any variable, thereby allowing it to be passed around as a first class object. If we look at the complete solution for calculating the square root of a number (using Newton's method - To compute the square root of "x" we start with an estimate of the square root, "y" and continue to improve the the guess by taking the mean of x and x/y )


So that's in Scala, if we try Groovy we will see we can achieve pretty much the same thing easily:


Recursion (and tail-recursion):

As FP avoids having mutable state, the most common approach to solve problems is to break the problem down in to simple functions and call them recursively - This avoids having to maintain state whilst iterating through loops, and each function call is given its input and produces an output.

If we again consider an example from the Scala course, with the example of a simple function that calculates the factorial for a given number.

This simple function recursively calculates the factorial, continuing to call itself until all numbers to zero have been considered. As you can see, there is no mutable state - every call to the factorial function simply takes the input and returns an output (the value of n is never changed or re-assigned, n is simply used to calculate output values)

There is a problem here, and that is as soon as you attempt to calculate the factorial of a significantly large enough number you will encounter a StackOverflow exception - this is because in the JVM every time a function is called, a frame is added to the stack, so working recursively its pretty easy to hit upon the limit of the stack and encounter this problem.  The common way to solve this is by using Tail-Call recursion. This trick is simply to have the last code that is evaluated in the function to be the recursive call - normally in FP languages the compiler/interpreter will recognise this pattern and under the hood, it will really just run the code as a loop (e.g. if we know the very last piece of code in the block of code is calling itself, its really not that different to just having the block of code/function inside a loop construct)

In the previous factorial example, it might look like the last code to be executed is the recursive call factorial(n-1) - however, the value of that call is actually returned to the function and THEN multiplied by n - so actually the last piece of code to be evaluated in the function call is actually n * return value of factorial(n-1).
Let's have a look at re-writing the function so it is tail-recursive.


Now, using an accumulator, the last code to be evaluated in the function is our recursive function call. In most FP languages, including Scala, this is enough - however, the JVM doesn't automatically support tail-call recursion, so you actually need to use a rather clunkier approach in Groovy:

The use of the trampoline() method means that the function will now be called using tail-call recursion, so there should never be a StackOverflow exception. It's not as nice as in Scala or other languages, but the support is there so we can continue.


Currying:

This is like function composition - the idea being you take a generic function, and then you curry it with some value to make a more specific application of the function. For example, if we look at a function that given values x and y, it returns z which is the value x percent of y (e.g. given x=10, y=100, it returns the 10 percent of 100,  z=10)

The above simple function is a generic mechanism to get a percentage value of another, but if we consider that we wanted a common application of this function was to always calculate 10% of a given value - rather than write a slightly modified version of the function we can simply curry the function as follows:

Now, if the function tenPercent(x) is called, it uses the original percentage() function, but curries the value 10 as the first argument. (If you need to curry other argument positions you can also use the rcurry() function to curry the right most argument, or ncurry() which also takes an argument position - check the Groovy docs on currying for more info)


Immutability:

Immutability is partially supported in Java normally with use of the final keyword (meaning variables can't be changed after being initially set on object instantiation). Groovy also provides a quick and easy @Immutable annotation that can be added to a class to easily make it immutable.  But really, there is more to avoiding immutable state than just having classes as immutable - As we have functions as first class objects, we can easily assign variables and mutate them within a function - so this is more of a mindset or philosophy that you have to get used to. For example:

The first example is probably more like the Groovy/Java code we are used to writing, but that is mutating the state of the list - where as the second approach leaves the original list unchanged.


Map Reduce:

As a final note, there are some functions in FP that are pretty common techniques - the most famous of which these days (in part thanks to Google) is Map-Reduce, but the trio of functions are actually Map, Reduce(also known as Fold) & Filter - you can read more about the functions here (or just google them!), but these functions actually correlate pretty nicely to core Groovy functions that you probably use a lot of (assuming you are groovy programmers).

Map

map is the easiest to understand of the three. It takes in two inputs - a function, and a list. It then applies this function to every element in the list. You can basically do the same thing with a list comprehension however. 
Sound familiar? This is basically the .collect{} function in Groovy

Reduce/Fold

This one is a bit more complicated to descibe, but is the same as the .inject{} function in groovy

Filter

And another simple one - filtering out a list for desired elements, this is Groovy's .findAll{} function



As I said at the start, I am new to FP and coming from an OO background, but hopefully the above isn't too far from the truth!  As I get further through the Coursera course I will try to post again, maybe with some of the assignments attempted in Groovy to see how it really stands up.


Some useful references:

Thursday, 23 May 2013

6 Tips: Getting your Hackathon project into Production

Recently there has been a steady stream of stories about projects that started as a Hackathon idea going into production at some of Silicon Valleys top companies. It's well known that Facebook's 'Like' button started off as a Hackathon project, and Ebay just announced a Hackathon project has led to them using Node.js for the first time in production. So what will give your project a chance of making it out to production for your users to benefit from? The team @ NerdAbility.com are giving you six tips to get things moving when your boss gives you some time to hack away!

1. Share Your Project


So you have made something really cool, maybe on your own or with some other devs. Now is the time to share your Hackathon project and get other people within the company excited! If it's a tool for the dev team why not give a lunch-time talk or send out a note with a quick-start guide. Once people see that you have made something cool and interesting it is surprising how quick it can get added to priority list for more work / release.

2. Don't worry (too much) about the technology


Hackathons are meant to be a chance to try out something new or maybe do something differently, so don't let people tell you your work has to be production ready right away. Often you will do your best work when you are excited and lets face it all of developers love shiny new things! But, once you have the concept nailed and something implemented be prepared to be flexible. Getting a new technology live can often be time consuming, so sometimes you will need to give and take to see your project get the light of day.

3. Be prepared to say 'OK that didn't work'


So you have tried to make something and, well, it didn't quite work out. This is still a valuable project! You will have learnt new things and hopefully take away a few lessons. The most important thing to do is make sure you and your team do another Hackathon! Still share your work with the rest of the team and who knows it may inspire someone else. Next time you can take a fresh idea and see what happens!

4. Upload your project to GitHub


So you have finished your kick ass project and your company decides on a change of direction, your project just isn't needed. Shock horror, this does happen. Why not try to get permission to put your project onto GitHub? This is a great way to share your work and who knows it may be just what another developer / company is looking for. Then instead of your code being in production in one place, it could be used all over the world. Dream big.

5. Work with the business guys


Great things can happen in pairs! Use your technical genius to think up a solution for someone in the business with a problem or idea. If you address a hidden problem or tap into an undiscovered opportunity you will look like hero. The great thing about this is your work will hopefully have business value so you instantly know it will have a better chance getting released. Also working with someone with a different perspective could open your mind to come up with an out of the box solution that will blow the socks off the rest of the dev team!

6. Launch it yourself


If you have built a cool application in your own time, then just launch it yourself! There are plenty of cloud providers offering free basic hosting such as Appfog, AWS and CloudBees amongst many others. Just be careful of any legal complexities that may occur from having users such as data protection and anti-SPAM regulations.

Finally.....


We loved this presentation by Philip Su, Facebook Site Lead London, from back in February this year on Facebook's engineering process and how they use Hackathons as part of their development culture.

Check it out:

Sunday, 12 May 2013

Essential Resources to Become a Life Long Learner (in tech)

Is one of your New Year Resolutions to re-skill? Thinking about re-training for a new career (or even just a new hobby) in tech? Then you're in luck! 

Today, more than ever, the barrier to entry for starting to learn a new technology or programming language is all but nonexistent, all you really need is a computer (or even a mobile device) and a web connection and you are pretty much good to go - just choose your preferred technology, an IDE and get started.  Almost everything is open source or at least free to use for a single developer just out to learn and there is a wealth of blogs, articles and Q'n'A sites ready to help you with tutorials, walk-through's and helpful advice - many of these backed with ready and running code bases on GitHub free for you to play with and generally work out what is going on.

However, with all these resources it can sometimes be a bit daunting with so much content. Once you have chosen a language how do you know where to start? Here are some of our favourite sites and resources that we have discovered and found useful in learning new skills:

  • iTunes U - a lesser known category on iTunes is their academic section, iTunes U(niversity) featuring loads of podcasts and lectures from a range of academic organisations, and some of this stuff is serious! Several large universities have uploaded full lecture series there, and by and large they are free to download (yes, you have to install iTunes, which sucks, we know).  Want to take the full term of Stanford university's iOS course? Its up there. Want to learn AI for chess playing from Cambridge uni? Yep, got that too. And for free.
  • MIT OpenWare - MIT have been one of the strongest advocates of open sourced education. A lot of there lecture series are online (can also be found on iTunes, but can be avoided).  Is it just us who thinks its amazing that anyone around the world with a web connection can get educated by the most prestigious academic organisations around?
  • Khan Academy - there is a lot of hype around this one, well funded with some pretty big names supporting it (jQuery creator John Resig is a Dean there), a not-for-profit aiming at providing free education for everyone. The academy provides lots of video based courses as well as interactive challenges and detailed stats on how you are doing.
  • Udacity - this is another recent, well-funded startup trying to tackle free higher education for all. Founded my three robotocists it is slowly building a very respectable catalogue of uni level courses ranging from CS101 to AI for robotics. As with the Khan academy, the lectures are purely for the web so the videos are clear and designed for remote learning (different from the filmed university lectures which are targeting classroom based learning).  We have recently created and Open Sourced the Spring-Social implementation of the Khan Academy API - so if you are working with the JVM and want to have a play with the Khan Academy API then check it out on GitHub
  • CodeAcademy - we have mentioned before we are fans of code academy, code academy is an in-browser development environment that walks you through programming exercises to help you learn with your hands - currently supporting JavaScript, HTML, Ruby and Python
  • Free eBooks - there are loads of great free eBooks available online, so many there is no point listing them, instead I will just point you here. Which leads nicely on to the next point..
  • StackOverflow - what really needs to be said about SO? It is the definitive q'n'a site for tech. If you are just starting learning head over and sign up, the help from the incredibly active community over there will be invaluable (although be sure to read the posting guides, they can be a little unforgiving at times!).
  • Coursera - Another massively popular online learning resource, this one recently generated a lot of interest with its recent Scala course taught by the original creator of the language!  We are currently working on some secret integration with Coursera at NerdAbility, and you will soon be able to integrate your Coursera account and show off which courses you have completed!

Hopefully the above resources help on your path to re-skilling.  In reality, getting your hands dirty with code and trying to solve problems and fix errors is the best way to learn, so don't forget to get stuck in - and maybe when you are more confident try answering questions on StackOverflow!

Of course, with these new found skills you will want to show them off, so we'd recommend heading to NerdAbility and registering (if you haven't as already) and update your skills, add your StackOverflow profile and even add a custom section talking about what you are learning (employers always love to know that candidates are proactive and motivated when it comes to learning new things and keeping up with technology). 

Leave your comments with any other tools and resources you have found useful in your journey of becoming a life long learner.

Tuesday, 19 March 2013

Tech: Building an RSS Reader Android App

This tutorial will walk through building an RSS reader on the Android platform (focusing on 3.0 + as it will be using Fragments). All the code is available as a complete, working Android app that you can fork/download and fire up straight away on a compatible Android device/emulator. So feel free to go grab that from GitHub before continuing.

It is not an unusual requirement for mobile apps these days to be able to consume an RSS feed from another site (or many) to aggregate the content -  Or maybe you just want to build your own Android app now that Google has announced it will be retiring Reader.

Those of you who have worked with RSS in other JVM languages will know that there are plenty of libraries available that can handle parsing RSS - however, because the android platform doesn't actually contain all the core java classes, almost all of the RSS libraries have not been supported.

Fear not though, as Java's SAX parser is available, so with a bit of code we can get a custom RSS parser up and running in no time!

This walk through will cover off the basics of getting an RSS reader app up and running quickly and will also cover some details of Android's fragment system for tablet optimization as well as some things to watch out for (such as changes in the platform that mean network operations cannot be run in the main thread, which requires some tweaking if you have worked on earlier versions).

All the code for the app is also available on our GitHub so feel free to fork that and try loading it up on your Android device/emulator.

Parsing an RSS Feed:


So to get started we will look at parsing the feed - if you have any experience parsing XML using SAX in Java then you will know how easy this is. All we need to do is to tell the parser which XML nodes we care about, and what to do with them.

If  you have never implemented a SAX parser before, there are three primary methods that we will override: 
  • startElement() - This is called by the parser every time a new XML node is found
  • endElement() - This is called by the parser every time an XML node is closed (e.g. </.. )
  • chars() - this is called when characters are found between nodes



Because we only really care about capturing data from the leaf nodes, our startElement() method is left empty. The chars() element has to be watched, as there is no guarantee when it will be called (e.g. in a node like hello world  this method might be called several times between the start and end) so every time we will just append the contents to a StringBuffer - that way we can be sure that we will have captured all the data in the node.  By the time the endElement() method is called, we know that we have the contents of the node itself, and we just have to store the data.  At this point, we just quickly knocked up a POJO with the attributes that we wanted to capture - the Strings that we match on are the node names from the ATOM RSS feed (that Blogger uses) - if you are using another feed, just have a quick look at the feed and update the node names appropriately.

Using our Feed in an Android App

So, that was easy right? Once that parser has run through (and you could use that code standalone in any java app really) then you will have a list of Java objects that have the core details about the latest blog posts on the feed (title, author, datecreated, content, etc) - So now lets look at using it in an Android app.

We will assume a basic understanding of the Android SDK and the concept of Fragments, so won't go back to basics with that stuff.

What we will do, is create a basic ListFragment and an RSSService class that we will use to populate the list. In our ListFragment we will simply tell our RSS service to populate the list:



Simple right?

Let's take a look at what our helpful RSS service is doing for us.



The first thing to note is that this class is extending Android's AsyncTask- The reason for this is that since Android 3.0, you are no longer able to perform network operations in the main application thread, so being as our class is going to have to fetch some RSS feeds we are going to have to spin off a new thread.

As you can see, the constructor just sets some context info that we will use later on, and then builds a progress dialogue - this is then displayed in the onPreExecute() method - This lets us show a "loading" spinning disk whilst we fetch the data.

Android's AsyncTask's primary method that handles the actual work that you want to do asynchronously is called "doInBackground()" - In our case, this is simple - we just invoke our SAX RSS parser and fetch our feed data:



Finally, we will override the "onPostExecute()" method of the async class to use our newly fetched list to populate our ListFragment.  You note how when we overrode the doInBackground() method earlier we set the return to List of Articles (where Article is my simple POJO containing my RSS blog post info) - well this must correspond to the argument of the "onPostExecute()" method, which looks like this:



Actually, all we really needed to do in this method would be pass our new List or articles to the ListFragment and notify it of the change as below:



However, in our application we have added a bit more sugar on the app - and we have actually backed the app with a simple DB that records the unique IDs of the posts and tracks whether or not they have been read to provide a nicer highlighting of listed blog posts.

So that's it - there's plenty more you can add to your RSS reader app, such as storing posts for reading later, and supporting multiple feeds/feed types - but feel free to fork the code on GitHub, or just download on to your android device to enjoy all our NerdAbility updates!

Application running in Andriod emulator
RSS application running in an Andriod  tablet emulator




Thursday, 31 January 2013

Tech: A Rough Guide to Atmosphere with Spring, Tomcat & Jersey: Avoiding the Pitfalls

Welcome to the second  installment of the NerdAbility tech guide series. This post looks at the Atmosphere framework! Enjoy....

Edit: I Previously forgot to include the  project sources and people have rightly asked in the comments about it. I couldn't find the example I wrote this blog post from, but I found a similar example on my github. You can find it here

More and more it's becoming a common requirement to support some sort of bi-directional (asynchronous) channel in modern web applications. Atmosphere is a great library which does much of the heavy lifting if you're running on a Java, servlet based stack. It offers support for WebSockets, Server Side Events (SSE), Long-Polling, HTTP Streaming (Forever frame) and JSONP. There are, however, some pitfalls and, if you've not got a lot of experience in the area, it can be frustrating trying to work out whats going wrong.

Having run through the process once I thought it'd be nice to cover my implementation in some detail. Maybe you'll see something which can help you with your own configuration or implementation problems.

Overview of the stack

This is what I had in place before integrating atmosphere.
  • Application server: Tomcat 7.0.32 (Must be greater than 7.0.30 for atmosphere to work correctly).
  • Container: Spring 3.2 RC1 ( though anything post 3 is fine )
  • Build Enginer/ Dependency Manager: Maven3
  • Front-End: Javascript / JQuery

    Include the Dependencies

    You'll want to add the following entries to your POM :

    Properties:

    Dependencies:

    Just a quick note about the cors-filter dependancy, you'll only need it if you want Cross-Origin-Resource-Sharing support. I thought about covering it in this article, but maybe i'll write it up as a follow up

    The web.xml

    If you're not using servlet 3 already, time to upgrade. Change your web-app config to look like this:

    The next step is to include the standard spring dispatcher etc to your configuration. You will probably you'll already have this anyway.


    The Atmosphere Controller 

    Using The Atmosphere Servlet

    I fully accept that everyone has their own pet way of setting up servlets, this is just one possible mutation. You want to add the following to your web.xml:

    Pitfall 1: The broadcasterLifeCyclePolicy tells atmosphere to destroy any connections that been closed by the client, enabling it will prevent OOM (Out of Memory) issues in your application.
    Pitfall 2: The recoverFromDestroyedBroadcaster prevents an exception being thrown when you try to reuse a broadcaster that you have recently destroyed ( useful if you want your client to subscribe using its own personal channel ).
    So now you're able to use spring MVC alongside your atmosphere code without them stepping on each others' toes. Lets move on.

    The Controller Class

    Mine was simple enough. It basically passes off the broadcasters that it creates to another service which whatever listeners you have configured ( Mine was RabbitMQ ) will call when they have an appropriate message.

    
    @Configurable: Using @Configurable is a great way to use aop classweaving to inject spring dependencies into non-managed spring classes. There is extensive documentation on how to set this up in spring 

    Pitfall 3: Make sure you set the suspend period, or you're going to be dealing with a java.net.SocketException: Too many open files in pretty much no time.

    The Service Class

    Also very simple. Using a map to store the in use channel ids -> broadcasters, the service is simply responsible for looping through the registered broadcasters and broadcasting a generic message.

    The Async Class

    Probably you'll have some asynchronous process which is waiting for messages, there are many possible implementations that this may take so i'll just show one of the popular ones RabbitMQ.
    You'll need to make sure you add the following dependency to your pom:

    RabbitMQ has a pretty clean java integration which is nice and quick to set up. The first component is a context configuration file where you bind your queues, connection factory, admin, exchanges and listeners.


    Notice the reference to broadcastQueueConsumer we used there ? Lets write that now.

    I've omitted the declaration of the Simple Message Converter in the context file but please dont forget about it. Once you're done you've got a pretty clean implementation of the serverside and should be ready to move onto the frontend.

    JQuery frontend implementation

    Really its quite simple to build a small subscription handler. I've added an example of mine below :
    Pitfall 4: Make sure you set enableXDR to true if you want to support cross domain requests.

    Pitfall 5: To send custom request params it seems like you need to pass via url (maybe jfrancard has fixed now but not sure)

    Thanks for reading, and feel free to ask any questions via a comment below or our twitter @nerd_ability

    Wednesday, 16 January 2013

    Tech: How to Fix 'SSLPeerUnverifiedException: peer not authenticated' Exception in Groovy / Java

    This is the first in a series of tech posts on the NerdAbility blog. We are aiming to blog any useful tips / gotchas we come across when developing with various technologies. We will still be posting tech recruitment insights and NerdAbility news, so stay tuned!

    How to Fix 'SSLPeerUnverifiedException: peer not authenticated' Exception in Java / Groovy


    When developing with web services in Java you may come across the need to connect to a HTTPS URL, for example when creating a REST client. In some cases there will be an issue with the type of certificate the web server is using, resulting in a SSLPeerUnverifiedException.

    To solve this you could previously export the servers SSL certificate via firefox / chrome and load this directly into the cacerts keystore (jvm's default trusted keystore). In recent versions of firefox / chrome this feature seems to have disappeared.  In this post we will show you how to grab the certificate using command line tools and then load it into the cacerts keystore. Finally we give an example of connecting to a HTTPS URL with Groovy using RESTClient.

    Please note this guide is for Linux / Mac users. Windows users may be able to follow along using cygwin, but we have not tested this. If you are using an alternative trusted keystore in your application, use this instead of cacerts in the examples.

    Prerequisites: Before loading any key into your cacerts keystore, please verify you are happy with the certificate and its authenticity, and issuer. You can do this by using a tool like this one.

    Disclaimer: Follow this guide at your own risk, we can not be held liable / accountable for any damage or issues caused to you or your systems.

    Step 1: Download and Store the Certificate


    To download and store the certificate run the following command, changing $ADDRESS for the sites address. For example https://www.facebook.com would become facebook.com:

    echo -n | openssl s_client -connect $ADDRESS:443 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > /tmp/$ADDRESS.cert

    To check the certificate was grabbed, you can run:

    cat /tmp/$ADDRESS.cert

    This will output the certificate and you should see something like:

    -----BEGIN CERTIFICATE-----
    *DATA*
    -----END CERTIFICATE-----


    Step 2: Load this into the default keystore for the JVM CACERTS


    First of all you need to locate the cacerts keystore for the JRE you are using. To find out the version of java run the following command:

    java -version

    This should give you something similar to:

    java version "1.6.0_11" Java(TM) SE Runtime Environment (build 1.6.0_11-b03) Java HotSpot(TM) 64-Bit Server VM (build 11.0-b16, mixed mode)

    Next take the Java version number from the previous output, in this case 1.6.0_11 and use locate to find the cacerts keystore for this Java install:

    locate cacerts | grep "1.6.0_11"

    The output should give you something similar to:

    /usr/lib/jvm/jre1.6.0_11/lib/security/cacerts

    Now you have enough information to import the key into the keystore. Run the following command, replacing the $ADDRESS with the address variable you used earlier, the $ALIAS with a name for the certificate i.e. facebook. Replace the $PATH variable with the path to the cacert (the output from the locate command we just ran). Also we have added the -storepass argument, passing the default password for the cacerts keystore. You will want to change this, if you have not already, and should be prompted to do so.

    sudo keytool -importcert -alias "$ALIAS" -file /tmp/$ADDRESS.cert -keystore $PATH/cacerts -storepass changeit

    Once you run this you will be shown the certificate and prompted to confirm you want to import the certificate:

    Trust this certificate? [no]:  yes
    Certificate was added to keystore

    Now you should have the certificate ready for use in your application, providing it is configured to use the default keystore and runs on the JVM we configured the certificate for!

    Step 3: Test It!


    Here is some example Groovy code using RESTClient:



    Thursday, 10 January 2013

    TechBritain Launch Interactive UK Tech Startup Community Map

    Tech Britain has just launched an interactive map of the UK tech startup community! This great service allows you to explore the tech startup scene around you, including hangouts, investors, companies and communities. 

    Tech Britain Interactive Map
    TechBritain.com UK Interactive Startup Map
    If you are working in a startup why not explore who is around you, or where you can grab a coffee and meet up with other startup folk? Maybe you are looking to move to a startup or just get involved, you will find all this and more!

    Tech Britain is also taking submissions for anything that they have missed, so if your company isn't featured you can sign up and submit a listing here.

    Remember if you are wanting to get involved with the UK tech startup scene also come check out NerdAbility.com and create a free profile to show how awesome you are!