SE-Linux: check and deactivate

I quite often run into security problems when testing stuff on my machine.
This is to remind me how to deactivate SE-Linux, so I can see if it is causing the problem...

sudo getenforce
> Enforcing

If "sudo getenforce" returns "Enforcing", you have SE-Linux, so turn it off.

sudo setenforce 0

then

sudo getenforce
> Permissive

... now test the thing...

making Makefile Self documenting

I have use 'make' on the last few projects, just to control all the CMD line build tools we now need to use.

I found a neat trick a little while ago, that automatically shows you what CMDs are available.

#
# Why use Makefile?
# because you get help lists & auto-complete on complex commands
#

help:           ## Show this help.
 @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//'

# linebreak
: ## ======================================================================

# make all output silent - ie: no CMDs shown
#.SILENT:

gitStatus: ## show GIT status
  @git status -b --column -s

Simple copy this into the top of your Makefile, and add comments as shown using ## after the cmd name.
Now when you call 'make' without any arguments, you will see a list of available commands with their descriptions.

A brilliant tool for projects with loads of cmd line tools to remember.

How to get AWS EC2 instance metadata from Java

I have recently found that I can access EC2 machine instance metadata using curl.
Which is brilliantly useful.

However, I wanted to get the same data inside my Java applications.
So I build a utility to make it available...

package com.mendeley.weblet.oauth.utility;

import com.sun.jersey.api.client.Client;
import com.sun.jersey.api.client.ClientResponse;
import com.sun.jersey.api.client.WebResource;
import com.sun.jersey.api.client.config.ClientConfig;
import com.sun.jersey.client.apache4.config.DefaultApacheHttpClient4Config;

import java.net.URI;


/**
 * Utility class to access EC2 instance Meta-data
 *
 * @See (http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html)
 */
public class EC2Metadata {

    /**
     * Some of the Metadata types
     */
    public enum Type{
        ami_id("ami-id"),
        ami_launch_index("ami-launch-index"),
        ami_manifest_path("ami-manifest-path"),
        block_device_mapping("block-device-mapping/"),
        hostname("hostname"),
        instance_action("instance-action"),
        instance_id("instance-id"),
        instance_type("instance-type"),
        kernel_id("kernel-id"),
        local_hostname("local-hostname"),
        local_ipv4("local-ipv4"),
        mac("mac"),
        network("network/"),
        placement("placement/"),
        public_hostname("public-hostname"),
        public_ipv4("public-ipv4"),
        public_keys("public-keys/"),
        reservation_id("reservation-id"),
        security_groups("security-groups"),
        services("services/");

        private String name;

        private Type(String name){
            this.name = name;
        }
    }


    /**
     * Get metadata using Type enum
     *
     * @param type
     * @param timeout
     * @param defaultValue
     * @return
     */
    public static String retrieveMetadata(Type type, int timeout,String defaultValue) {
        return retrieveMetadata(type.toString(),timeout,defaultValue);
    }

    /**
     * Get metadata by String value
     * Allows further metadata to be retrieved.
     * See AWS documentation for more info.
     *
     * @param type
     * @param timeout
     * @param defaultValue
     * @return
     */
    public static String retrieveMetadata(String type, int timeout,String defaultValue) {
        try{
            URI uri = URI.create("http://169.254.169.254/latest/meta-data/" + type);
            System.out.println(uri.toString());

            ClientConfig config = new DefaultApacheHttpClient4Config();
            config.getProperties().put(ClientConfig.PROPERTY_CONNECT_TIMEOUT,timeout);

            Client client = Client.create(config);
            WebResource webResource = client.resource(uri);

            ClientResponse response = webResource.get(ClientResponse.class);

            String results = response.getEntity(String.class);
            return results;

        }catch(Throwable t){
            return defaultValue;
        }
    }


    public static void main(String[] args) {
        String myEC2Id = retrieveMetadata(Type.instance_id,1000, "null");
        System.out.println("The Instance Id is " + myEC2Id + " .");
    }

}

I will try to get it into Github at some point, until then, please use it as you see fit.
Obviously I take no responsibility for the end of the world as you know it, should it occur.

Using ENYOjs to build a Chrome App, a few pointers

As per usual, I am trying something new... discovered a few problems, then solutions, and need to remind myself about the solutions for the next time.

I am building a Chrome App using ENYO, partly because I actually need an app right now, and partly because I think ENYO will make a perfect quick build solution for the next time.

Anyway...

Problem 1: document.write error
Solution: use renderInto.

var app = new UMLEditor({name: "app"});

app.renderInto(document.body);        

Problem 2: localstorage warning
Solution: refactor to use chrome.local.storage
   Open: source/data/sources/localStorage.js
   Change: e.localstorage to chrome.local.storage


Simple problems, simple solutions, but annoying to look up again next time.

Groovy Eclipse compiler versions and ShortTypeHandling

Just a short note to remind myself.

If you come across a classNotFoundException: org.codehaus.groovy.runtime.typehandling.ShortTypeHandling

Then the chances are you have 2 versions of Groovy being used.
1 pre Groovy 2.3.5, and 1 after it.

Reminder: use this to investigate (remember to look at parents too)
mvn dependency:tree -Dverbose

I believe this is a Java 8 compatibility change, but would need to investigate to be sure.

Anyway, go here to find out more:
http://glaforge.appspot.com/article/groovy-2-3-5-out-with-upward-compatibility 

UML Diagram Editor is ready

I have been working on a small chrome App, a tool for building UML diagrams from text. It leverages PlantUML, which I have found to be a great tool, but seriously lacks a decent desktop app to use it. So I build one, or at least the start of one.

You can find it here:
UML Diagram Editor

I have been teaching my colleagues to create Sequence Diagrams to plan agile stories. They can capture, communicate and confirm all the tasks required for a story with all the participants and interested parties, before even creating the tickets to achieve it.

As far as clear communication goes, I now see sequence diagrams as equal importance to using BDD style acceptance criteria. As both allow everyone up and down the technical/business chains to understand exactly what it planned, and what is required.

To use sequence diagrams for you project...

Keep it simple, only one feature per story.
It is more likely to finish on time, and easier to describe fully.
Any scope creep goes into a new story, to be prioritized into the backlog.

When planning the story, simply create a sequence diagram describing the flow that will occur when the feature is finished. Include UIs, acceptance criteria, validation, message formats, button clicks, and anything else you believe is needed to describe the solution you are going to build.

Now just look at your completed diagram, the tasks should fall out of it very readily, separated by component, and position in the flow. Dependencies will also be very obvious, so those tasks can be done in order.


I have found this to be a superb solution to the issue of communication within a team, as well as externally, as it provides an easy point of focus and discussion. When it is easy to edit or modify the UML diagram, it becomes a live document, and the only one needed if the tasks are kept small.

Why you want to build your own platform. PAAS on top of IAAS.

The problem:

PAAS and auto-scaling are brilliant, I love them, but I hate the hidden price...Vendor lock-in.
IAAS and cloud VMs are great, I love them, but I hate the hidden price...Vendor lock-in.

Neither solution works in the long run.

Either we run into a missing facility in the PAAS, or our costs scale out of control as IAAS gains traction with the developers, testers and everyone starts spinning up VMs all over the place.

We end up creating a Heath Robinson-esque machine, solving each little problem by adding one more part to our monster.  The complexity grows and grows and grows. Soon enough no more features can be added as keeping the machine running takes all our time.

Eventually we are going to realise that we need more/better/bigger and will have to change everything or go and find it somewhere else and we still have to change everything.

So vendor lock-in is the biggest elephant in the room, and we face it where ever we look at the moment. Yes, you can have all their shiny features, but you must create everything their way, and woe betide you if you think you can change provider easily.

Everything we do becomes tightly coupled to the providers infrastructure or systems:
development, deployment, testing, roll-back, DevOps, Sysops, metrics, and the list goes on...

There are ways around these issues, but they always feel like edge case coding, and I know another edge case is waiting just around the corner, in fact I have 3 in the backlog.

Are you ready? Here are the words, just in case:
"We welcome our Heath Robinson overlords..."

The dream:

For a long time, I have had a dream ... of a system that allows everyone to get the best of all worlds, without any group suffering to support other, and everyone is capable of working to improve their conditions.

• my applications should be able to live anywhere
• without requiring a complete refit, refactor and rebuild just to move home
• I can code it, test it, build it, deploy it and manage it easily, uncoupled from the infrastructure
• No more herculean efforts just to get tests to run locally or in the build
• Use the best of breed as standards, but avoiding the Heath Robinson effect
• A simple configuration system that has defaults for everything, yet all can be overridden
• Simple to change, and old parts replaced or new injected without downtime

How to do it?
This thought has bothered me for a long time.

The solution:

Eventually, I fell back on time honoured solution.
To de-couple 2 things, create an abstraction layer between them.

An Infrastructure Abstraction Platform: IAP.

Create an  abstraction layer, a platform for my code, that runs on top of any IAAS, that can manage itself, and all the applications inside it, as well as interface with the containing IAAS, but avoid coupling my applications to the IAAS. Everything external to my platform and applications should be attached through discovery and configuration, so an IAAS provider change should only require a redeploy and a config change.

If you have any idea how much time/effort this would take, you can understand my reluctance to even think about starting it. That was until I discovered Vertx.

Next:

[Coming soon: What Vertx lets me achieve]
[Coming soon: Why I use ENYO to build UIs]
[Coming soon: Why I can now build my own platform on IAAS]