Publish Go binary to Amazon S3 and Homebrew

August 30th, 2016 • 500 Words

After the setup of a go command line tool with Cobra it’s now time to release it to the public and publish it to Homebrew for easy installation on MacOS. Together with the steps to use AWS S3, CloudFront and SSL Certificate Manager for easy web hosting it’s a pretty neat setup for your Homebrew application formula and go binary.

Prepare an Amazon S3 bucket

The created heft CLI application has a command called version which responds with v0.0.3 as the built version. The Makefile uses goxc to build a binary stored in build/ and now we want to create an Amazon S3 bucket to store the binary for public access:

$ > aws s3api create-bucket \
--bucket downloads.heft.io \
--region eu-west-1 \
--create-bucket-configuration LocationConstraint=eu-west-1

Now that we have created the S3 bucket to store all files, we need to enable the built-in Amazon S3 feature for hosting static websites and configure the default index document:

$ > aws s3 website s3://downloads.heft.io/ \
--region eu-west-1 \
--index-document index.html

The first command already responded with the public URL of the S3 bucket, in this case downloads.heft.io.s3.amazonaws.com. But no worries, we will be able to access the files in this bucket with a custom domain using Amazon CloudFront in the end.

As we do not plan to store any confidential data in the Amazon S3 bucket, we can just enable general public read access to all objects stored in it:

$ > aws s3api put-bucket-policy \
--bucket downloads.heft.io \
--region eu-west-1 \
--policy '{
      "Version": "2012-10-17",
      "Statement": [
          {
              "Sid": "Allow Public Access to All Objects",
              "Effect": "Allow",
              "Principal": "*",
              "Action": "s3:GetObject",
              "Resource": "arn:aws:s3:::downloads.heft.io/*"
          }
      ]
  }'

All needed steps to have a custom domain, use SSL and CloudFront for caching can be found in the post about hosting on AWS S3, CloudFront with SSL Certificate Manager.

Configure goxc

Now head over to the goxc configuration from the previous tutorial of our go application and remove copy-resources from the list of excluded tasks:

{
  "AppName": "heft",
  "ConfigVersion": "0.9",
  "ResourcesExclude": "LICENSE",
  "BuildSettings": {
    "LdFlags": "-s",
    "LdFlagsXVars": {
      "Version": "main.VERSION"
    }
  },
  "Tasks": ["interpolate-source", "default"],
  "TasksExclude": ["rmbin", "deb", "deb-dev", "downloads-page", "go-vet", "go-test"]
}

Building the project with make build will create a binary file and a compressed archive in the build/ directory:

$ > make build
$ > tree build

build
└── 0.0.3
    ├── darwin_amd64
    │   └── heft
    └── heft_0.0.3_darwin_amd64.zip

Extend the Makefile

The binary file is needed for the make install command and the compressed archive will be used for Homebrew. As we do not handle any sensitive information we can upload the complete build/ folder to the S3 bucket. For easy usage extend the Makefile with a publish command:

publish: build
  aws s3 sync $(PATH_BUILD)/$(VERSION) s3://$(S3_BUCKET_NAME)/$(VERSION)

The S3 bucket name is new a variable which should be added to the top of our Makefile as well:

S3_BUCKET_NAME=downloads.heft.io

You can now call make publish and all compiled files of the current version are uploaded to Amazon S3.

Create Homebrew Formular

Homebrew is a widely used package manager for MacOS. It relies intensively on GitHub repositories for package management. Within the repository Homebrew uses Ruby classes called Formula. The Formula contains information about the application name, the URL where to download the archived binary and an SHA256 checksum for validation:

class Heft < Formula
  desc "Heft.io command line interface"
  homepage "https://heft.io"
  url "https://s3-eu-west-1.amazonaws.com/downloads.heft.io/0.0.3/heft_0.0.3_darwin_amd64.zip"
  version "0.0.3"
  sha256 "103b8b15fecdccfaaf0db70b773650798e930511378f456292c0592734b98a82"

  def install
    bin.install "heft"
  end
end

On MacOS you can easily generate the needed SHA256 checksum with the shasum command.

$ > shasum -a256 build/0.0.3/heft_0.0.3_darwin_amd64.zip
103b8b15fecdccfaaf0db70b773650798e930511378f456292c0592734b98a82  build/0.0.3/heft_0.0.3_darwin_amd64.zip

To store your Formula just create a new and empty GitHub repository and create a Formula directory with a .rb file named after the application name, in this case it’s called heft.rb:

$ > tree .

.
├── Formula
│   └── heft.rb
└── README.md

After pushing the ruby file to your GitHub repository it’s time to register the custom Formula in Homebrew, just run the following command to tap your namespace and repository:

$ > brew tap heft/cli git@github.com:heft/homebrew-cli.git

Now you can install the binary using Homebrew:

$ > brew install heft

The complete Homebrew Formula of the heft Command Line Application is available on GitHub for free.


  • React SPA with server-side rendering on AWS Lambda

    February 5th, 2020 • 1152 Words • EN

    React is great for creating websites and writing applications using JavaScript. But, whenever the use of React is rejected, one of the most common reasons is the lack of a simple implementation for server-side rendering (SSR). When you search on Google for this topic, you’ll find various approaches…

  • Deploy React SPA with CodePipeline and CodeBuild using AWS CDK

    January 23rd, 2020 • 748 Words • EN

    There are plenty of tools and services for continuous delivery available. Most of them are either directly built into the source code management tools you already use, or perfectly integrate with them. You might be familiar with CircleCI, Travis CI, GitLab CI, or GitHub Actions. Of course, AWS has…

  • AWS re:Invent 2019 Recap & Videos

    December 15th, 2019 • 360 Words • EN

    I was able to attend the AWS re:Invent 2019 conference. A week full of learning about current and new technologies, services, and general approaches is definitely overwhelming. There is no much content available, during the conference, and as videos and slide decks afterwards. I tried to list my…

  • CloudFormation Best-Practices

    May 1st, 2019 • 457 Words • EN

    You can find plenty of frameworks and tools to provision your AWS resources. Some of them do a great job for a specific purpose, others are more generic. Nevertheless, I do prefer to use native CloudFormation templates as much as possible. Prefix all the things Split up your CFN Stacks Nested…

  • Makefile Best-Practices

    April 30th, 2019 • 167 Words • EN

    The more projects you work on, the more streamlined your tooling gets. Hopefully. Various services using different languages have different tooling requirements, of course. A sweet Makefile can be the entry to a unified tooling interface. Hide Commands Wildcard Targets Foreach and Lists As long as I…

  • Last Update: 2020-02-17T10:07:11.183Z
    Stresemannstraße 132, 22769 Hamburg
    +49 151 54 64 90 55
    inbox@sbstjn.com