Saturday, November 12, 2011

Go has turned 2 years!

Indeed on Nov 10 2009 the Go-team at Google made their work public.
Go has come a long way in these 2 years, thanks to the brilliant team and hundreds of contributors from all over the world. Early 2012 Go 1 will be launched as a hallmark to the world announcing that it is ready for mainstream adoption, being stable and supported for years to come. The past year has seen a great number of refinements and additions so that Go is now faster than ever, building upon a complete standard library and surrounded by hundreds of peripheral 3rd party projects
(see http://godashboard.appspot.com/package).
Go has also made its way in the cloud in Google App Engine, where it will soon be equal in possibilities to the Java and Python environments.
Already now the list of business projects using Go is impressive:
http://go-lang.cat-v.org/organizations-using-go
In only 2 years it can present an equal or greater amount of references than Ruby:
(http://www.ruby-lang.org/en/documentation/success-stories/).
Every day on github alone around 10 projects using Go are created or updated (https://github.com/languages/Go/created)
The release of Go1 will also see the publication of an ever growing number of published books on Go and its applications.
See also the GopherTimes (www.gophertimes.com) for  announcements of the steps and milestones in the evolution and adoption of Go.

Sunday, November 6, 2011

An error-handling scheme with closures

Every time when a function returns we should test whether it resulted in an error: this can lead to repetitive and tedious code. Combining the defer/panic/recover mechanism with closures can result in a far more elegant scheme that we will now discuss. However it is only applicable when all functions have the same signature, which is rather restrictive. A good example of its use is in web applications, where all handler functions are of the following type:
                                         func handler1(w http.ResponseWriter, r *http.Request) { … }

Suppose all functions have the signature:     func f(a type1, b type2)
The number of parameters and their types is irrelevant.
We give this type a name:  fType1 = func f(a type1, b type2)
Our scheme uses 2 helper functions:
i)                     check: a function which tests whether an error occurred, and panics if so:

func check(err os.Error) { if err != nil { panic(err) } }


ii)                   errorhandler: this is a wrapper function. It takes a function fn of our type fType1 and returns such a function by calling fn. However it contains the defer/recover mechanism, outlined in § 13.3

func errorHandler(fn fType1) fType1 {
    return func(a type1, b type2) {
        defer func() {
            if e, ok := recover().(os.Error); ok {
log.Printf("run time panic: %v", err)
            }
        }()
        fn(a, b)
    }
}

When an error occurs it is recovered and printed on the log; apart from simply printing the application could also produce a customized output for the user by using the template package (§ 15.6).

The check() function is used in every called function, like this:

func f1(a type1, b type2) {
    …
    f, _, err := // call function/method
    check(err)
    t, err := // call function/method
    check(err)
    _, err2 := // call function/method
    check(err2)
    …
}

The main() or other caller-function should then call the necessary functions wrapped in errorhandler, like this:

func main() {
    errorHandler(f1)
    errorHandler(f2)
    …
}

Using this mechanism all errors are recovered and the error-checking code after a function call is reduced to check(err).In this scheme different error-handlers have to be used for different function types; they could be hidden inside an error-handling package. Alternatively a more general approach could be using a slice of empty interface as parameter and return type.

Friday, October 28, 2011

Using memoization for performance

      When doing heavy computations one thing that can be done for increasing performance is not to repeat any calculation that has already been done and that can/must be reused. Instead cache the calculated value in memory, which is called memoization. A great example of this is the calculation of the Fibonacci series:
to calculate the n-th Fibonacci number, you need the 2 preceding ones, which normally have already been calculated. If you do not stock the preceding results, every higher Fibonacci number results in an ever greater avalanche of recalculations, which is precisely what the following program does:

1st version:
package main
import (
    "fmt"
    "time"
)

func main() {
    result := 0
    start := time.Nanoseconds()
    for i:=0; i <= 25; i++ {
        result = fibonacci(i)
        fmt.Printf("fibonacci(%d) is: %d\n", i, result)
    }
    end := time.Nanoseconds()
    fmt.Printf("the calculation took this amount of time: %f\n", float32(end - start)/1000000000)
}

func fibonacci(n int) (res int) {
    if n <= 1 {
        res = 1
    } else {
        res = fibonacci(n-1) + fibonacci(n-2)
    }
    return
}

Simple stock the n-th Fibonacci number in an array at index n, and before calculating a fibonnaci-number, first look in the array if it has not yet been calculated.
This principle is applied in the 2nd version; the performance gain is astounding, time both programs for the calculation up to the 40th Fibonnaci number:
                          normal: the calculation took this amount of time: 4.730270 s
                       with memoization: the calculation took this amount of time: 0.001000 s
In this algorithm memoization is obvious, but it can often be applied in other computations as well, perhaps using maps instead of arrays or slices.

2nd  version:
package main
import (
        "fmt"
        "time"
)

const LIM = 41
var fibs [LIM]uint64

func main() {
        var result uint64 = 0
        start := time.Nanoseconds()
        for i:=0; i < LIM; i++ {
                result = fibonacci(i)
                fmt.Printf("fibonacci(%d) is: %d\n", i, result)
        }
        end := time.Nanoseconds()
        fmt.Printf("the calculation took this amount of time: %f\n", float32(end - start)/1000000000)
}

func fibonacci(n int) (res uint64) {
        // memoization: check if fibonacci(n) is already known in array:
        if fibs[n] != 0 {
                res = fibs[n]
                return
        }
        if n <= 1 {
                res = 1
        } else {
                res = fibonacci(n-1) + fibonacci(n-2)
        }
        fibs[n] = res
        return
}

Saturday, September 24, 2011

Comparing the performance of Go on Windows and Linux.

I measured the performance of a simple Go channel benchmarking program on Windows versus Linux on the same system with as processor an Intel Core 2 Duo (T9300 2.5GHz) and 4 GB of memory.
The Windows OS is a 64 bit Windows 7 Professional; the Linux OS is a 64 bit Linux Ubuntu 11.04 (Natty).
The Linux Go release was: 6g release .r60 (9481).
The Windows Go releases are:  8g release .r60 (9684) and 6g version weekly.2011-07-07 (9153+).
I used the following program:

package main

import (
        "fmt"
        "testing"
        "runtime"
)

func main() {
        runtime.GOMAXPROCS(1) // runtime.GOMAXPROCS(2)
        fmt.Println(" sync", testing.Benchmark(BenchmarkChannelSync).String())
        fmt.Println("buffered", testing.Benchmark(BenchmarkChannelBuffered).String())
}

func BenchmarkChannelSync(b *testing.B) {
        ch := make(chan int)
        go func() {
                for i := 0; i < b.N; i++ {
                        ch <- i
                }
                close(ch)
        }()
        for _ = range ch {
        }
}

func BenchmarkChannelBuffered(b *testing.B) {
        ch := make(chan int, 128)
        go func() {
                for i := 0; i < b.N; i++ {
                        ch <- i
                }
                close(ch)
        }()
        for _ = range ch {
        }
}

These properties were compared:
1)       A synchronous channel versus a buffered channel
2)       The value of GOMAXPROCS (1 versus 2, with value 2 both cores should be used)
3)       Windows performance versus Linux
4)       On Windows: a 32 bit Go-compiled program versus a 64 bit program.

The number of measurements for each result was between 4 and 5 x 106   ;  each result gives how many nanoseconds one operation took.

Here are the results in ns/op:       


GOMAXPROCS
Synchronous
buffered
Windows (8g)
1
428
180

2
3577
3762
Windows (6g)  
1
426
179

2
3603
4000
Linux(6g)
1
16642
210

2
17625
212



We can see the following:
1)       A buffered channel performs better than a nonbuffered channel as is to be expected:
2.4x better  on Windows (8g and 6g)
                               81x  better on Linux

2)        Influence of GOMAXPROCS 1 versus 2:
On Windows (8g and 6g) this did not behave as expected: the nonbuffered channel performed 8.4x worse for GOMAXPROCS=2 versus 1, and the buffered channel even 21.5x worse. Moreover buffering with GOMAXPROCS=2 even gives a slightly worser performance than the synchronous channel, in contrast to 1)!
On Linux also the results for GOMAXPROCS=2 are almost the same than value 1 (in fact very slightly worse).  There is no improvement with increasing GOMAXPROCS.

3)        Linux versus Windows
Windows performs slightly better for buffered channels and much better (39 x) for synchronous channels; I don’t know a reason for the latter.

4)       Windows 8g versus Windows 6g:           they have the same performance.


                We can conclude for this kind of problem (filling and reading a channel):
-          Buffered channels perform better than synchronous channels (much better on Linux)
-          Increasing GOMAXPROCS is not useful here; the task being divided over the 2 cores creates an immense overhead.
-          Windows performs on par with Linux, and even much better for synchronous channels.
-          Windows 8g performs the same as Windows 6g.
                

Sunday, August 14, 2011

Why Go ?

After all, we have C and C++ when we need performance, Java and .NET for complex business oriented projects, Python and Ruby for nimble and flexible applications. Why should there be any need for another language?
Because C, our main system language on which nearly all other languages are built, is more than 40 years old and is getting less and less adapted to our distributed, networked and multicore computing world. 
Go is an attempt to make a new foundation for the coming computing era, by proposing a C for the 21st century. Such an immense undertaking could only be successful when initiated by a team of distinguished software engineers and backed up by a big company. 
And that is the case: the designers of Go are renowned software pioneers, creators and co-creators of Unix, C, Unicode and the Java hotspot compiler, namely Ken Thompson, Rob Pike and Robert Griesemer. The backing company is Google Inc., which has, considering its immense software and hardware infrastructure, a big business case for a  language which could improve upon C++, Python and Java: and indeed Go is already in use within Google itself since May 2010.
Go is not a completely new or experimental language, instead it combines good things from different existing languages and software engineering principles, thereby improving on many of the shortcomings of the above mentioned languages.
Let’s make this clear with a few arguments:

1)      Go is completely free and open source, so you don’t owe Google anything (except your gratitude if you are so inclined); it isn’t bound to Google and could grow also outside of it (just like C was not bound to Bell Labs). It runs on Linux, OS X and Windows.

2)      An important design point is simplicity: Go has a familiar syntax for developers accustomed to C++, Java, and .NET, but it is very much cleaned up, made concise, less keywords and constructs but more powerful. The code layout obeys a few simple rules and is imposed by a formatting tool (gofmt), so no fruitless discussions about code layout styles anymore, and the code is very readable, which is of the utmost importance in large projects.

3)      It is a statically typed language, so most of the infamous runtime errors of dynamic languages are caught at compile time. It compiles to native code, with execution speeds at this moment superseding Python by a factor of 5 to 100, nearing C++ speeds within a factor of 2, and still improving. And it compiles extremely fast, contrary to the well known heavy compilations of C++ projects. So again well suited for large demanding projects, and accelerating the development life cycle.

4)      Go doesn’t allow you memory tricks in programming, saving the programmer from this league of errors. Programs execute in a Go runtime, which is not as heavy as the Java or .NET virtual machines, but which amongst other things takes care of garbage collection, so frees you from memory management and leaks. And it gives you reflection capabilities !

5)      At the same time Go does some type inferencing like dynamic languages, thereby reducing the clutter of declaration statements and giving it a Python-like feel.

6)      Go is object oriented, but not in the classic style of C++ / Java / .NET. Instead of a heavy weight class-hierarchy system, it uses lightweight but powerful types and exploits the interface principle to its extreme. This makes for reduced and very general library code.

7)      Functions are first class citizens in Go, not as much as in the typical functional languages like Haskell, F# or Scala, but higher order functions and closures are used widely in Go-code, also increasing its abstracting powers.

8)      Go is built from the ground up for concurrent and parallel execution on multicore and networked machines. Built in are goroutines and channels, the tools which realize a programming paradigm based on communication, not on locking.

9)      It is suited for cloud programming, being enabled in Google App Engine. At the same time it runs on ARM processors, making possible future smartphone applications.

10)   Go has, certainly also because of its open source character, a strong, lively and active community of contributors to the language, building packages or projects in it or experimenting with it. Real life examples of usage of Go in other organizations can be found at http://go-lang.cat-v.org/organizations-using-go

Give it a try. On this blog will appear more means to guide you with that.