Cymen Vig

Wordwrap Kata

Since performing the wordwrap kata for those in the office as a practice run it has been much easier to keep on doing the kata every day. In Robert Martin’s example of the kata the code ends up with a branch at the bottom of the method like this:

1
2
3
4
5
6
if ...
  ...
elsif ...
  ...
else
  ...

My first change was figuring out how to simplify this down to an if-else and now I’ve progressed on my kata to the point I can eliminate the if-else with some refactoring at the end. Now I need to practice this approach with the screen setup used when performing the kata (we use a projector in the office which demands low resolution and large font sizes). I’m also trying to configure vim so I an run the tests within vim and have them displayed in a small part of the screen so the red/green status stays on screen.

HTTP Server Project

I’ve spent some time looking at using Netty in the second version of my HTTP server. It would be fun to have a server that supports SPDY. Twitter contributed a SPDY implementation to Netty which I think is low enough in the stack that it would work below my HTTP server. It does look a little tricky though and time wise I’d like to go back to focusing on the task so I’m considering skipping it for this rewrite and trying it on a future one.

Internal Project

The rails-based project with the interactor approach is going slowly. Brian explained how another internal tool uses that approach however some aspects of it lag behind the currently favored or at least discussed approach. It is an interesting challenge. Note I edited this after realizing the work we’re going to do is in the upcoming week not for this week.

Apparently, some people think that having big orange blobs that show up on your windows when you happen to have three fingers on your touchpad is a “good idea” (one of them and another). It turns out these are called “Unity MT” handles and even if you set all three to disabled Gnome goes back to enabled with default settings. Perhaps the real problem is that I’m using Gnome. The only way I found to get these to go away is to set each action to an unlikely key sequence using CompizConfig Settings Manager. If you actually disable Unity MT completely you loose the ability to resize windows which isn’t very useful. There is likely another way to fix this but I have zero interest in spending more time figuring that out.

Monday

I tried explaining the Liskov substitution principle to my fellow apprentices. It was a bit tricky to explain particularly as Robert Martin’s example in PPP and the definition are tied to static type systems. However it is still an important principle in a dynamic language and one can see that in the expectation of behavior that PPP reiterates throughout the chapter on the topic.

Tuesday

I helped Doug with some office improvements by configuring two new Cisco wireless access points. I met with some other apprentices and Paul to kick off an internal project that should be interesting.

Problem: Can’t find the maven repository for XYZ but have jar file

I wanted to use netty 3.3.1.Final in a Clojure project however according to the old netty site they left the jboss.org foundation which appears to have complicated the inclusion of the most recent versions of netty in the “central maven repository”. There is probably a maven repository out there for netty which has 3.3.1.Final but I can’t find it.

Solution: Use a local maven repository

If you can download the jar file then you can use the solution of creating a local maven repository and adding the jar file to it. I did this with these steps for netty 3.3.1.Final:

1. lein plugin install lein-localrepo "0.3"

2. Download netty 3.3.1.Final from netty.io/downloads/ with

1
<code>wget http://netty.io/downloads/netty-3.3.1.Final-dist.tar.bz2</code>

1. Uncompress the tar file:

1
<code>tar jxf netty-3.3.1.Final-dist.tar.bz2</code>

1. Add the jar file to the local maven repository:

1
<code>lein localrepo install netty-3.3.1.Final/jar/netty-3.3.1.Final.jar io.netty/netty 3.3.1.Final</code>

Now the project.clj can look like this for netty 3.3.1.Final:

1
2
3
4
(defproject clojure-netty "1.0.0-SNAPSHOT"
  :description "FIXME: write description"
  :dependencies [[org.clojure/clojure "1.4.0-beta1"]
                 [io.netty/netty "3.3.1.Final"]])

Note that I chose to put netty into the local repo as io-netty/netty. I should have used local.repo/netty however lein-localrepo doesn’t have remove implemented yet so I left it as is. Using local.repo/netty would reduce confusion when sharing your project on github.

Update: Manually removing items from local maven repository

Having netty as io-netty/netty in my local repository bothered me. I found that the maven repository existed as ~/.m2/repository and removing netty was accomplished with:

1
<code>rm -rf ~/.m2/repository/io</code>

After verifying the io directory held only the netty installation. I was then able to reinstall netty locally with:

1
<code>lein localrepo install netty-3.3.1.Final/jar/netty-3.3.1.Final.jar local.repo/netty 3.3.1.Final</code>

Now my project.clj should be clearer:

1
2
3
4
(defproject clojure-netty "1.0.0-SNAPSHOT"
  :description "FIXME: write description"
  :dependencies [[org.clojure/clojure "1.4.0-beta1"]
                 [local.repo/netty "3.3.1.Final"]])

SOLID is an acronym from Robert C. Martin to aid in remembering these basic principles:

1. Single responsibility principle
2. Open/closed principle
3. Liskov substitution principle
4. Interface segregation principle
5. Dependency inversion principle

Liskov substitution principle

The principle was defined by Barbara Liskov and can be paraphrased as: Subtypes must be substitutable for their base types. Martin discusses this principle in Agile Software Development: Principles, Patterns and Practices with some observations:

• A typical violation of the Liskov substitution principle is seen in code that branches by type inspection (see below). This is a violation of the open/closed principle as an addition of another type related to payment (ie PayPal) would require changes to this code.

  1
  2
  3
  4
  5
  6
    if (object.Type == "MoneyOrder") {
      ...
    }
    else if (object.Type == "PersonalCheck") {
      ...
    }
  

• A more complex example of a violation can be seen when attempting to shoe horn an additional object on top of a base class that does not have the same behavior. Martin’s example is a Square class that inherits from a Rectangle class. The issue with this is that if Rectangle has methods to set the height and width of the rectangle then Square must also have these methods. But in Square, when either height or width is set the other length must also be set (in other words, if I create a Square with height 5 and width 5 and then set the height to 10 the width must also be set to 10 otherwise it would not be a square). The problem is setting the height of width of a Square has a side effect and because of this it violates the Liskov substitution principle. Martin’s example of this violation is a method that calculates the area of a Rectangle. It would be reasonable to expect setting width = 5 and height = 4 would return area = 20 however for Square this is not the case – it will return 16 if the height is set last or 25 if the width is set last! It is of course possible to override the base method that calculates area however the test will still fail. A Square cannot be substituted for a Rectangle.
• It is reasonable to expect the behavior of classes to conform to the base type. That just because the Square class is a valid Rectangle does not mean it is a valid design because the clients of the class cannot treat them the same!
• ISA is about behavior: you might consider a Square is a Rectangle during design but a test that expects to set the width to 5 and the heigh to 4 and get an area of 20 for any Rectangle would fail! So by behavior, a Square is not a Rectangle. In other words, the consumers of your classes expect reasonable behavior (and when this behavior is violated they themselves will have to violate the Liskov substitution principle to work around the issue or try to fix the design).
• Testing is a great way to clarify and document the behavior of a class.

Martin includes more complex examples of the violations along with good and bad solutions to violates. He also identifies some quick methods of finding potential violations:

• A derived class has an override of a method with an empty body (aka a degenerate function).
• A derived class throws an exception on a method that base classes do not.

Why does the Liskov subsitution principle matter today with dynamic code?

Because at it’s heart as Martin reiterates the principle is really concerned with behavior. It doesn’t really matter if the language is Ruby or C#: consumers of classes will have reasonable expectations of behavior. If these expectations are violated the quick fix is a violation of the open/closed principle and that has a negative impact on the project as it is now harder to maintain, less resuable and less robust.

Why is a “quick fix” a violation of the open/closed principle?

I’ll write about the open/closed principle another day but a quick summary of it is that classes (or other entities) should be open for extension but closed for modification. The reason a quick fix of the Liskov substitution principle is almost certainly a violate of the open/closed principle is that the easiest solution is to inspect the type of the object instance and handle the type with different behavior differently. In other words, to branch on object type. This violates the open/closed principle because any time a new type is introduced the class that holds this branching logic will need to be updated.

Resources

• Barbara Liskov’s keynote address to OOPSLA 1987 conference (in which the principle was first discussed, unfortunately behind ACM paywall)
• Robert C. Martin - The Liskov Substitution Principle (PDF, second column for The C++ Report)
• Robert C. Martin - Agile Software Development: Principles, Patterns and Practices

Installing Go on Ubuntu

To install Go on my Ubuntu system I used the directions at golang.org/doc/install.html using ~/dev/go as my build location. The initial attempted to run “./all.bash” failed. Go requires bison which can be installed with “sudo apt-get install bison”. The second run of “./all.bash” succeeded and after a couple minutes ~/dev/go/bin contained the 6g set of executables for amd64 (as opposed to 5g for arm or 8g for 386 or something else on other operating systems).

The final step was to add the folowing to my ~/.bash_profile:

1
2
3
4
export PATH=$PATH:~/dev/go/bin
export GOROOT=$HOME/dev/go
export GOARCH=amd64
export GOOS=linux

And then run “source ~/.bash_profile”. Note that you can also get the 8g compiler on the amd64 platform by running “GOARCH=386 ./all.bash” instead of “./all.bash” along with installing gcc-multilib (“sudo apt-get install gcc-multilib”).

Vim Syntax Highlighting

The go repository includes a syntax highlighter file for Go in “go/misc/vim/” which you can copy to your vim syntax directory with “cp $GOROOT/misc/vim/syntax/go.vim in ~/.vim/syntax/”. Then enable the syntax highlighter for go files by editing ~/.vim/ftdetect/go.vim and adding:

1
au BufRead,BufNewFile *.go set filetype=go

Getting the Go Koans

One set is at go_koans: github.com/VictorDenisov/go_koans. A quick git clone and a “make” and you’re all set.

• I’m running Ubuntu 11.10 amd64 but likely very similar install for other versions.

Update: More Go Koans

One of the craftsmen at 8th Light has written go-koans: https://github.com/sdegutis/go-koans!

For my HTTP server project I want the option of configuring it on the command line for at least the port, document root and potentially IP to listen on (or all). I came across Building a Clojure app with a command-line interface? on StackOverflow which informed me that clojure.tools.cli was the right path to take over the with-commmand-line.

Here is an example of using clojure/tools-cli which revealed some Clojure syntax I’d overlooked (more on that later):

1
2
3
4
5
6
7
8
9
10
11
(ns server.core
  (:gen-class)
  (:require [clojure.tools.cli :as c]))

(defn -main [& args]
  (let [[options args banner]
        (c/cli args
          ["-port" "Port to listen on" :default 5000]
          ["-root" "Root directory of web server" :default "public"])]
    (println "port:" (:port options))
    (println "root:" (:root options))))

• options is a hash-map of the parsed options

• args is the other arguments on the command line

• banneris a string one can print for command line options – for this program, it would contain:

  Usage:

  Switches Default Desc ——– ——- —- -port 5000 Port to listen on -root public Root directory of web server

Note that tools.cli throws an exception when an unrecognized command line option is provided like so:

1
2
Exception in thread "main" java.lang.RuntimeException:
  java.lang.Exception: '-z' is not a valid argument

I guessed it would default to showing the banner it generates but it does not do so.

Basic Clojure Syntax I wasn’t aware one could take a returned vector and assign its contents to multiple variables with let as done in this line from above:

1
(let [[options args banner] ...

In a prior post, I explained how I was passing back a response to an HTTP request as a hash-map of hash-maps. This can start to get confusing and I think the approach used above can make this simpler – instead of a hash-map of hash-maps consider returning a vector of hash-maps.

Today I performed the word wrap kata mentioned yesterday in Ruby as a practice run after lunch. It went fairly well. I was of course nervous and managed to fat fingered or overlook a few things in the usage of my editor (vim) and a missing comma in a method definition (Clojure on the brain). The feedback was to run each test directly after writing it to show the red (fail) before writing code to make it green (pass). Another suggestion was to have more narration or use music. Now that we are doing a review process for katas before they are performed at 8th Light University on Fridays I can probably get some good feedback on beneficial narration. I think good narration is likely the hardest part of performing a kata.

I think Doug had me perform the kata sooner than later even though I didn’t feel quite prepared enough in order to show me the value of doing katas. I struggle to see the value of katas but I suspect it is fine to ignore that way of looking at it as the value is in the doing not in the thinking (before the fact). It seems somewhat like chanting common in Buddhism. It’s foreign if not slightly jarring to those not used to it and the value is very hard to see as a non-participant. Both suggest daily practice too. I’ll keep going down the path of the kata.

The rest of the day I spent on some edges of Clojure that will be useful for in my HTTP server:

• How to get an uberjar to be runnable with java -jar

• How to use clojure.tools.cli to parse command line options

The bare minimum to get a leningin uberjar to run properly on the command line is:

1. The namespace that contains -main method must have :gen-class applied like so:

1
2
(hello-world.core
  (:gen-class))

1. The project.clj must have a :main setting that points to the namespace containing -main:

   … :main hello-world.core …

Now both lein run and java -jar hello-world-1.0.0-SNAPSHOT-standalone.jar produce the output Hello world!. This project is on github as “hello world”.

HTTP server

I discussed my HTTP server with Doug and he mentioned one option I had thought about earlier: why not start over? The main issue is not writing it with a test first approach which means the benefits of emergent design and series of simple tests serving as documentation aren’t present. The github repository for the second attempt is at clip-clop.

Code Kata

I’m working on the wordwrap code kata which is an exercise in writing a method to wrap text to a specific column length. So for example, “word word word” with column length 6 would wrap to:

word word word

And “word word” with column length 3 would wrap to:

wor
d
wor
d

While that wrapping style might seem odd that is the aim of the code kata.

I find doing katas to be a real struggle. The idea is to do things in a specific rehearsed sequence that shows a progression from nothing to a tested method. Doug encouraged me to think of katas as a form of acting and I’m attempting to do so (clarification: Doug actually said that about a specific part of the word wrap kata).

Uncle Bob has more thoughts on the word wrap kata that explore why sometimes it can result in a dead end that requires backtracking.

The refactoring done over the last couple days to the HTTP server has begun to pay off. I am now able to test my HTTP response methods without having to use HTTP. The tests are still somewhat complicated as a hash of request headers is passed in along with an input stream containing any payload. I suspect if I had stuck with testing all along the structure of my code would be different (Uncle Bob has something to say about this in The Transformation Priority Premise). But at this point it is what it is – I can’t go back and recreate the likely more simplistic and more testable code along with the minor tests that both verify correctness and serve as documentation. I can try my best but it just isn’t possible to do and I think that is an important discovery.

The interesting thing about this experience is the different feel in development. Without testing code does seem to get written faster but refactorings induce more anxiety. The other track of testing while developing provides a steady stream of assurances in the form of passing tests while going “up the mountain” so to speak. Refactorings don’t have the same issue because the tests stand ready to either verify your work or reveal your poor testing ability.

In terms of this project, it felt like going up two mountains: the first was getting the server working and the second was refactoring it without tests in order to be more suitable for testing and writing those tests. The end affect on myself as the programmer is that testing is not particularly enjoyable with this approach. It is hard to fight the ingrained “it works why are you doing this” and continue work that has no immediate benefit. This is the opposite of the experience of writing a test and then writing the code to pass that test.

To sum up the issue, the key points are:

• code written using test first is different in structure than code written with tests after

• code first enforces a simple progression of tests that result in a solution which is hard or impossible to do with tests after because the structure of the code isn’t conducive to simple tests that build up

• refactoring code in order to make it testable is certainly possible but it isn’t enjoyable and this is important because what isn’t enjoyable becomes a chore

• this things are present even in small projects where spec-based testing was put aside

On a side note, speclj can do exactly what I want in terms of printing the two values it is comparing if one uses the should= operator. As Myles commented on a prior post, the startup time issue can be resolved in most cases by using “lein spec -a” which keeps speclj running on a hot JVM watching for changes in your spec files and rerunning on those observed changes.

I spent the day mulling over how to refactor my server to achieve two goals:

1. Be more testable without having to mock input and output streams

2. Allow logging of requests similar to Apache which is a cross-cutting concern: the log needs to know client details (IP), request details (method type, path with query) and response details (response code, response content length)

I knew that if I had some sort of data structure for each method type (where a method is GET, POST, etc) it would be easier to do this. My code took the easy route and just wrote straight to the output stream. I was able to mock an output stream using a ByteArrayOutputStream which has a handy toString() method but this seem like a bit much. I’d have to parse my output and what if there was a bug in that parser? Plus it’s more code to test and more code to write.

After more time than seemed reasonable I suddenly concluded I could indeed return a hash-map quite easily in clojure like so:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
(defn my-method [request]
  (hash-map
    :Status-Line {
      :HTTP-Version ...
      :Status-Code 200
      :Status-Message "OK"
    }
    :Headers {
      :Date (date-in-gmt)
      :Server "blah blah/1.0"
     }
    :Body {
       ...
    }))

Why did it take so long to see that? I’m not sure… But I’m happy with my abstraction as I can now test my methods by passing in a hash map (the request) and getting back a hash map (the response).

The other issue I ran into was how to delay creating the body of the response. There isn’t always a body for an HTTP response however when there is I’d rather push it directly out of my program than hold it in a buffer. That sounded familiar and I realized a closure would be appropriate. This made me slightly nervous as file input output in a closure seems risky however my IO functions seemed high enough and were only relying on filename so it seemed safe. For an HTTP GET, my hash map response looks like:

1
2
3
4
5
6
7
8
9
(defn my-method [request]
  (hash-map
    :Status-Line {
      ...
    }
    :Headers {
      ...     }
    :Body (fn [output] (copy (input-stream filename) output)))))
    ; copy input-stream from clojure.contrib.io

The closure is closed in the main loop of the server (well, in the thread responsible for the handling the request):

1
2
3
4
    ...
    (if (contains? response :Body)
      ((:Body response) out))
    ...

I haven’t run ApacheBench against the server after this change but I will after some more cleanup work to get all the response methods using this approach. Hopefully, the server isn’t leaking file handles or consuming memory in some odd way due to this approach.