12 Indispensable Go Packages and Libraries

Go is an amazing language with a lot of momentum, and it's focused on simplicity. This approach is evident in its standard library, which provides all the essentials, but not much more.

Fortunately, Go has a vibrant community that creates and shares a lot of third-party libraries. In this tutorial, I'll introduce you to 12 of Go's best packages and libraries. Some of them have relatively narrow scope and can be added to any projects, while others are huge projects that you can incorporate into massive, large-scale distributed systems.

Awesome Go

Before diving into the libraries themselves, let me introduce you to Awesome Go, a very active and curated list of Go libraries and other resources. You should visit every now and then and check what's new.

1. Golang-Set

Go has arrays, slices and maps, but it doesn't have a set data structure. You can mimic a set with a map of bools, but it's nice to have an actual data type with the right operations and semantics. This is where golang-set comes in. Here is a basic example of creating a new set, adding items and testing for membership:

package main

import (
    "fmt"
	"github.com/deckarep/golang-set"
)


func main() {
	basicColors := mapset.NewSet()
	basicColors.Add("Red")
	basicColors.Add("Blue")
	basicColors.Add("Green")

	if basicColors.Contains("Green") {
		fmt.Println("Yay! 'Green' is a basic color")
	} else {
		fmt.Println("What a disappointment! 'Green' is not a basic color")
	}


	if basicColors.Contains("Yellow") {
		fmt.Println("Yay! 'Yellow' is a basic color")
	} else {
		fmt.Println("What a disappointment! 'Yellow' is not a basic color")
	}
}

Output:

Yay! 'Green' is a basic color
What a disappointment! 'Yellow' is not a basic color

Note that the package name is "mapset". In addition to the basics, you perform all set operations like union, intersection, and difference. You can also iterate over the set values:

package main

import (
    "fmt"
	"github.com/deckarep/golang-set"
)


func main() {
	basicColors := mapset.NewSet()
	basicColors.Add("Red")
	basicColors.Add("Blue")
	basicColors.Add("Green")

	otherColors := mapset.NewSetFromSlice([]interface{}{"Orange", "Yellow", "Indigo", "Violet"})
	rainbowColors := basicColors.Union(otherColors)

	for color := range rainbowColors.Iterator().C {
		fmt.Println(color)
	}
}

2. Color

Let's continue with the color theme. When writing command-line programs, it is useful to use colors to highlight important messages or distinguish between errors, successes, and warnings.

The color package gives an easy way to add some color to your programs (see what I did there?). It uses ANSII escape codes and supports Windows too! Here is a quick example:

package main

import (
    "github.com/fatih/color"
)

func main() {
	color.Red("Roses are red")
	color.Blue("Violets are blue")
}

The color package supports mixing colors with background colors, styles like bold or italic, and sprinkling color with non-color output.

package main

import (
    "github.com/fatih/color"
	"fmt"
)

func main() {
	minion := color.New(color.FgBlack).Add(color.BgYellow).Add(color.Bold)
	minion.Println("Minion says: banana!!!!!!")

	m := minion.PrintlnFunc()
	m("I want another banana!!!!!")

	slantedRed := color.New(color.FgRed, color.BgWhite, color.Italic).SprintFunc()
	fmt.Println("I've made a huge", slantedRed("mistake"))
}

The color package has other useful features. Go ahead and explore more.

3. Now

Now is a very simple package that provides a convenience wrapper for the standard time package and makes it easy to work with various date and time constructs around the current time.

For example, you can get the beginning of the current minute or the end of the Sunday closest to the current time. Here is how to use "now":

package main

import (
    "github.com/jinzhu/now"
	"fmt"
)

func main() {

	fmt.Println("All the beginnings...")
	fmt.Println(now.BeginningOfMinute())
	fmt.Println(now.BeginningOfHour())
	fmt.Println(now.BeginningOfDay())
	fmt.Println(now.BeginningOfWeek())
	fmt.Println(now.BeginningOfMonth())
	fmt.Println(now.BeginningOfQuarter())
	fmt.Println(now.BeginningOfYear())

}

Output:

All the beginnings...
2017-06-04 16:59:00 -0700 PDT
2017-06-04 16:00:00 -0700 PDT
2017-06-04 00:00:00 -0700 PDT
2017-06-04 00:00:00 -0700 PDT
2017-06-01 00:00:00 -0700 PDT
2017-04-01 00:00:00 -0700 PDT
2016-12-31 23:00:00 -0800 PST

You can also parse times and even add your own formats (which will require updating the known formats). The Now type embeds time.Time, so you can use all of the time.Time methods directly on Now objects.

4. Gen

The gen tool generates code for you—in particular, type-aware code that tries to alleviate the gap of not having templates or generics in Go.

You annotate your types with a special comment, and gen generates source files that you include in your project. No runtime magic. Let's see an example. Here is an annotated type.

// +gen slice:"Where,Count,GroupBy[int]"
type Person struct {
    Name string
	Age int
}

Running gen (make sure it's in your path) generates person_slice.go:

// Generated by: gen
// TypeWriter: slice
// Directive: +gen on Person

package main

// PersonSlice is a slice of type Person. Use it where you would use []Person.
type PersonSlice []Person

// Where returns a new PersonSlice whose elements return true for func. See: https://clipperhouse.github.io/gen/#Where
func (rcv PersonSlice) Where(fn func(Person) bool) (result PersonSlice) {
    for _, v := range rcv {
		if fn(v) {
			result = append(result, v)
		}
	}
	return result
}

// Count gives the number elements of PersonSlice that return true for the passed func. See: http://clipperhouse.github.io/gen/#Count
func (rcv PersonSlice) Count(fn func(Person) bool) (result int) {
	for _, v := range rcv {
		if fn(v) {
			result++
		}
	}
	return
}

// GroupByInt groups elements into a map keyed by int. See: http://clipperhouse.github.io/gen/#GroupBy
func (rcv PersonSlice) GroupByInt(fn func(Person) int) map[int]PersonSlice {
	result := make(map[int]PersonSlice)
	for _, v := range rcv {
		key := fn(v)
		result[key] = append(result[key], v)
	}
	return result
}

The code provides LINQ-like methods to operate on the PersonSlice type. It's simple to understand and nicely documented.

Here is how you use it. In the main function, a PersonSlice is defined. The age() function selects the age field from its Person argument. The generated GroupByInt() function takes the age() function and returns the people from the slice grouped by their age (34 is just Jim, but 23 has both Jane and Kyle).

package main

import (
    "fmt"
)

// +gen slice:"Where,Count,GroupBy[int]"
type Person struct {
	Name string
	Age int
}

func age(p Person) int {
	return p.Age
}

func main() {
	people := PersonSlice {
		{"Jim", 34},
		{"Jane", 23},
		{"Kyle", 23},
	}

	groupedByAge := people.GroupByInt(age)

	fmt.Println(groupedByAge)
}


Output:

map[34:[{Jim 34}] 23:[{Jane 23} {Kyle 23}]]

5. Gorm

Go is known for its spartan nature. Database programming is no different. Most popular DB libraries for Go are pretty low-level. Gorm brings the world of object-relational mapping to Go with the following features:

Associations (Has One, Has Many, Belongs To, Many To Many, Polymorphism)
Callbacks (Before/After Create/Save/Update/Delete/Find)
Preloading (eager loading)
Transactions
Composite Primary Key
SQL Builder
Auto Migrations
Logger
Extendable, write Plugins based on GORM callbacks

But it doesn't cover everything. If you come from Python, don't expect SQLAlchemy magic. For more fancy stuff, you'll have to go a lower level. Here is an example of how to use Gorm with sqlite. Note the embedded gorm.Model in the Product struct.

package main

import (
    "github.com/jinzhu/gorm"
    _ "github.com/jinzhu/gorm/dialects/sqlite"
)

type Product struct {
  gorm.Model
  Code string
  Price uint
}

func main() {
  db, err := gorm.Open("sqlite3", "test.db")
  if err != nil {
    panic("failed to connect database")
  }
  defer db.Close()

  // Migrate the schema
  db.AutoMigrate(&Product{})

  // Create
  db.Create(&Product{Code: "L1212", Price: 1000})

  // Read
  var product Product
  db.First(&product, 1) // find product with id 1
  db.First(&product, "code = ?", "L1212")

  // Update - update product's price to 2000
  db.Model(&product).Update("Price", 2000)

  // Delete - delete product
  db.Delete(&product)

6. Goose

One of the most important tasks when working with relational databases is managing the schema. Modifying the DB schema is considered a "scary" change in some organizations. The goose package lets you perform schema changes and even data migrations if needed. You can goose up and goose down to go back and forth. Mind your data, though, and make sure it doesn't get lost or corrupted.

Goose works by versioning your schema and using migration files corresponding to each schema. The migration files can be SQL commands or Go commands. Here is an example of a SQL migration file that adds a new table:

-- +goose Up
CREATE TABLE person (
    id int NOT NULL,
    name text,
    age int,
    PRIMARY KEY(id)
);

-- +goose Down
DROP TABLE person;

The -- +goose up and -- +goose down comments tell goose what to do to upgrade or downgrade the schema.

7. Glide

Glide is a package manager for Go. Under a single GOPATH, you may have many programs that have conflicting dependencies. The solution is to have each program manage its own vendor directory of package dependencies. Glide helps with this task.

Here are the features of glide:

Support versioning packages including Semantic Versioning 2.0.0 support.
Support aliasing packages (e.g. for working with github forks).
Remove the need for munging import statements.
Work with all of the go tools.
Support all the VCS tools that Go supports (git, bzr, hg, svn).
Support custom local and global plugins.
Repository caching and data caching for improved performance.
Flatten dependencies, resolving version differences and avoiding the inclusion of a package multiple times.
Manage and install dependencies on-demand or vendored in your version control system.

The dependencies are stored in glide.yaml, and glide provides several commands to manage dependencies:

     create, init       Initialize a new project, creating a 
                        glide.yaml file
     config-wizard, cw  Wizard that makes optional suggestions 
                        to improve config in a glide.yaml file.
     get                Install one or more packages into 
                        `vendor/` and add dependency to 
                        glide.yaml.
     remove, rm         Remove a package from the glide.yaml 
                        file, and regenerate the lock file.
     import             Import files from other dependency 
                        management systems.
     name               Print the name of this project.
     novendor, nv       List all non-vendor paths in a 
                        directory.
     rebuild            Rebuild ('go build') the dependencies
     install, i         Install a project's dependencies
     update, up         Update a project's dependencies
     tree               (Deprecated) Tree prints the 
                        dependencies of this project as a tree.
     list               List prints all dependencies that the 
                        present code references.
     info               Info prints information about this 
                        project
     cache-clear, cc    Clears the Glide cache.
     about              Learn about Glide
     mirror             Manage mirrors
     help, h            Shows a list of commands or help for 
                        one command

8. Ginkgo

Ginkgo is a BDD (Behavior Driven Development) testing framework. It lets you write your tests in a syntax that resembles English and allow less technical people to review tests (and their output) and verify that they match the business requirements.

Some developers like this style of test specification too. It integrates with Go's built-in testing package and is often combined with Gomega. Here is an example of a Ginkgo + Gomega test:

1	actual, err := foo()
2	Ω(err).Should(BeNil())
3	Ω(actual).ShouldNot(BeNil())
4	Ω(actual.result).Should(Equal(100))

9. Etcd

Etcd is a reliable distributed Key-Value store. The server is implemented in Go, and the Go client interacts with it though gRPC.

It focuses on the following:

Simple: well-defined, user-facing API (gRPC).
Secure: automatic TLS with optional client cert authentication.
Fast: benchmarked 10,000 writes/sec.
Reliable: properly distributed using Raft.

Here is an example of connecting to the server, putting a value and getting it, including timeouts and cleanup.

func test_get() {
    cli, err := clientv3.New(clientv3.Config{
		Endpoints:   endpoints,
		DialTimeout: dialTimeout,
	})
	if err != nil {
		log.Fatal(err)
	}
	defer cli.Close()

	_, err = cli.Put(context.TODO(), "foo", "bar")
	if err != nil {
		log.Fatal(err)
	}

	ctx, cancel := context.WithTimeout(context.Background(), 
                                       requestTimeout)
	resp, err := cli.Get(ctx, "foo")
	cancel()
	if err != nil {
		log.Fatal(err)
	}
	for _, ev := range resp.Kvs {
		fmt.Printf("%s : %s\n", ev.Key, ev.Value)
	}
	// Output: foo : bar
}

10. NSQ

NSQ is a great distributed queue. I've used it successfully as a primary building block for large-scale distributed systems. Here are some of its features:

Support distributed topologies with no SPOF.
Horizontally scalable (no brokers, seamlessly add more nodes to the cluster).
Low-latency push based message delivery (performance).
Combination load-balanced and multicast style message routing.
Excel at both streaming (high-throughput) and job oriented (low-throughput) workloads.
Primarily in-memory (beyond a high-water mark messages are transparently kept on disk).
Runtime discovery service for consumers to find producers (nsqlookupd).
Transport layer security (TLS).
Data format agnostic.
Few dependencies (easy to deploy) and a sane, bounded, default configuration.
Simple TCP protocol supporting client libraries in any language.
HTTP interface for stats, admin actions, and producers (no client library needed to publish).
Integrates with statsd for real-time instrumentation.
Robust cluster administration interface (nsqadmin).

Here is how to publish a message to NSQ (error handling is elided):

package main

import (
  "github.com/bitly/go-nsq"
)

func main() {
  config := nsq.NewConfig()
  p, _ := nsq.NewProducer("127.0.0.1:4150", config)

  p.Publish("topic", []byte("message"))
  p.Stop()
}

And here is how to consume:

package main

import (
  "sync"
  "fmt"
  "github.com/bitly/go-nsq"
)

func main() {
  wg := &sync.WaitGroup{}
  wg.Add(1)

  config := nsq.NewConfig()
  q, _ := nsq.NewConsumer("topic", "channel", config)
  handler := nsq.HandlerFunc(func(message *nsq.Message) error {
      fmt.Printf("Got a message: %v", message)
      wg.Done()
      return nil
  })
  q.AddHandler(handler)
  q.ConnectToNSQD("127.0.0.1:4150")
  wg.Wait()
}

11. Docker

Docker is a household name now (if your family members are mostly DevOps people). You may not be aware that Docker is implemented in Go. You don't typically use Docker in your code, but it is a significant project and deserves to be recognized as a hugely successful and popular Go project.

12. Kubernetes

Kubernetes is an open-source container orchestration platform for cloud-native applications. It is another monster distributed system implemented in Go. I recently wrote a book called Mastering Kubernetes where I go in detail over the most advanced aspects of Kubernetes. From the Go developer's point of view, Kubernetes is very flexible, and you can extend and customize it via plugins.

Conclusion

Go is a great language. Its design philosophy is to be a simple and approachable language. Its standard library is not as comprehensive as some other languages like Python.

The Go community stepped up, and there are many high-quality libraries you can use in your programs. In this article, I introduced 12 libraries. I encourage you to look for other libraries before jumping in and implementing everything from scratch.