Shrink your Go binaries with this one weird trick

Ok, I lied, there's no weird trick. However, you can easily reduce a Go binary size by more than 6 times with some flags and common tools.

Note: I don't actually believe a 30MB static binary is a problem in this day and age, and I would not trade (build time | complexity | performance | debug-ability) for it, but people care about it apparently, so here we are. For the record, the dev team cares about it, too.

It all started with an observation Jessie Frazelle made: go binaries compress to almost 1/5th of their size.

$ ls -l go go.xz
-rwxr-xr-x  1 filippo  staff  12493536 Apr 16 16:58 go
-rwxr-xr-x  1 filippo  staff   2647596 Apr 16 16:58 go.xz

Not a surprise if you look at the entropy graph (generated with binwalk). A lot of sections there should compress extremely well.

default binary entropy

When I mentioned this is the office John Graham-Cumming suggested that we should then just make binaries self-decompress at runtime. At first I though he was joking, then I realized that no, this almost made sense!

So this weekend was going to be about hard-core engineering, building a binary that decompresses a payload and then JMPs to it. But soon enough I found out that such a thing exists already, it's called UPX and is quite nice.

So this post will be about graphs instead!

But first, let me strip the binary

Before we go all in on compression, there's something we can do to make binaries smaller: strip them.

IMPORTANT: Go binaries are their own thing and they don't play well with the standard strip utility, so don't use that.

Instead, we can use the -s and -w linker flags like this:

$ GOOS=linux go build cmd/go
$ ls -l go
-rwxr-xr-x  1 filippo  staff  12493536 Apr 16 16:58 go
$ GOOS=linux go build -ldflags="-s -w" cmd/go
$ ls -l go
-rwxr-xr-x  1 filippo  staff  8941952 Apr 16 17:08 go

We already lost 28% of our weight! Here's the new entropy graph; you can see that the last pretty-low entropy section is gone.

sw binary entropy

Interestingly, what gets stripped is only the DWARF tables needed for debuggers, not the annotations needed for stack traces, so our panics are still readable!

$ cat ../hello.go
package main

func TheHitchhikersGuideToTheGalaxy() {  
    panic("DO NOT PANIC")

func main() {  
$ go build -ldflags="-s -w" hello.go
$ ./hello

goroutine 1 [running]:  
panic(0x569c0, 0xc82000a140)  
    /usr/local/Cellar/go/1.6.1/libexec/src/runtime/panic.go:464 +0x3e6
    /Users/filippo/code/gopack/hello.go:4 +0x65
    /Users/filippo/code/gopack/hello.go:8 +0x14

Enter UPX

Next step, let's run upx. It works out of the box on linux binaries built by 1.6, but for earlier versions you'll need goupx.

$ GOOS=linux go build cmd/go
$ ls -l go
-rwxr-xr-x  1 filippo  staff  12493536 Apr 16 16:58 go
$ upx --brute go
                       Ultimate Packer for eXecutables
                          Copyright (C) 1996 - 2013
UPX 3.91        Markus Oberhumer, Laszlo Molnar & John Reiser   Sep 30th 2013

        File size         Ratio      Format      Name
   --------------------   ------   -----------   -----------
  12493536 ->   2554140   20.44%  linux/ElfAMD   go

Packed 1 file.  
$ ls -l go
-rwxr-xr-x  1 filippo  staff  2554140 Apr 16 16:58 go.upx

We went from 12MB to 2.5MB! Combined with -s and -w, we can reduce the binary size to just 15% of the default build. Almost 7 times smaller!

Here's a graph of the sizes obtained by each technique, applied to the Go compiler, to Gogs and to hello.go.

Obviously decompression is not free, but the overhead should only be at process start. In my benchmarks the UPX version of go was taking 160ms more to start, and hello 15ms.

Here's all the data and here is the simple build script I used.

It's getting better

One final note: a lot of work went into the 1.7 cycle to shrink the binaries, including new conditional tree pruning and generic SSA savings.

Here's the size of the same binaries built with Go tip, where 100% is the size of the default 1.6.1 build.

If you like reading interesting facts about Go in a totally-not-Buzzfeed style, you might want to follow me on Twitter.