In this blog post, I will explain how to expand environment variables in a configuration file.

If you are looking for the solution, you can jump directly to the solution section.

Reason

Sometimes, your Go application needs to share some configuration with other applications, especially secrets like API keys. We do like DRY principle. In modern times and at a medium to large scale, this problem is solved via rather complex (more complex than this tutorial) tools like this. However, when you have a small system or you can accept secret/configuration management technical debt, there are always easy hacks like this one.

Background

Let’s suppose we have this configuration file:

$ cat config.yml

environment: test
key: abc123def456

Let’s see how basic YAML parsing works:

package main

import (
    "fmt"
    "io/ioutil"

    "github.com/go-yaml/yaml"
)

type Config struct {
    Environment string `yaml:"environment"`
    Key         string `yaml:"key"`
}

func main() {
    confContent, err := ioutil.ReadFile("conf.yml")
    if err != nil {
        panic(err)
    }

    conf := &Config{}
    if err := yaml.Unmarshal(confContent, conf); err != nil {
        panic(err)
    }
    fmt.Printf("config: %v\n", conf)
}

After this point we can pass conf.Key to wherever it’s needed. Easy peasy.

Problem

We have another application running on the same server with our Go app. This application needs the same API key configured. If we replicate the configuration, we break DRY principle. We don’t want a complicated centralized secret manager.

Solution

Since both applications are on the same server, we can use environment variables. Let’s change the configuration as follows:

$ cat config.yml

environment: test
key: ${API_KEY}

Of course, we need to add export API_KEY=abc123def456 to environment variable configuration file (for instance ~/.bashrc or ~/.bash_profile) or we can specify it while running the application.

Then, we need to replace environment variables with their values while reading the configuration:

package main

import (
    "fmt"
    "io/ioutil"

    "github.com/go-yaml/yaml"
)

type Config struct {
    Environment string `yaml:"environment"`
    Key         string `yaml:"key"`
}

func main() {
    confContent, err := ioutil.ReadFile("conf.yml")
    if err != nil {
        panic(err)
    }
    // expand environment variables
    confContent = []byte(os.ExpandEnv(string(confContent)))
    conf := &Config{}
    if err := yaml.Unmarshal(confContent, conf); err != nil {
        panic(err)
    }
    fmt.Printf("config: %v\n", conf)
}

os.ExpandEnv expands environment variables in given string. Voila! conf.Key is now assigned with value of API_KEY environment variable.

Conclusion

This is a kind of hack, not a nice solution. However, it comes handy for small systems.