Why do I have to be inside the GOPATH to run go commands? A hack fix

Forest Johnson -

UPDATE 2018-07-24: It looks like the go project has come up with a new dependency management specification called "Modules" as a part of Go 1.11! https://github.com/golang/go/wiki/Modules

The Go modules specification may make everything I am doing here obsolete as it promises to allow you to store your code wherever you want.

Don't like how most features of the go development tools (go cli) require you to store your source code inside the $GOPATH in a very particular directory structure? Neither do most people when they start using go, and neither did I. It seems like most people I work with get over it eventually, but I never did. It still feels a bit insulting to me, and I've been thinking about it for a long time.

Eventually, I started doing a hack where I would store my go code wherever I want, then create a soft link in the right place in the GOPATH to point to my repository elsewhere. It worked, with two caveats.

  1. I would still have to cd into the gopath to run dep or go get
  2. Setting up the soft links in the first place is a pain

So what did I do? The same thing we do every night, brain. Try to automate it!

Introducing goo for go, to help you "stick your source code together" the way YOU choose!

goo is a bash script which addresses those two pain points in my soft link process directly. It automatically creates soft links, and it automatically changes directory into the GOPATH before running go commands. After writing it and installing it with a couple pointer scripts inside /usr/local/bin, I can run goo get or depp ensure from anywhere, as long as I'm inside a git repository that has exactly one git remote defined, or a git remote named origin defined. The script will use the git remote information to determine the correct folder for this project inside the GOPATH. Then it will create a soft link inside the GOPATH which points to the root of the current git repository. Finally, it will change directory to the position inside that soft link in the GOPATH which corresponds to your current working directory, and finally, run whatever command you intended: go for goo or dep for depp.

This is what the directory structure looks like on my machine:

/
├── GOPATH
│   └── src
│       └── github.com
│           └── MyOrg
│               └── my-go-project -> /home/forest/Desktop/Projects/my-go-project
└── home
    └── forest
        └── Desktop
            └── Projects
                ├── my-go-project
                └── my-javascript-project

What problems will this introduce? What weird edge-cases would be impossible to fix while using this tool?

  • Absolute paths inside the arguments of your goo or depp command will not be translated.
  • Relative paths that step outside the current git repository and are inside the arguments of your goo or depp command will almost certainly break stuff.
  • It will put a soft link inside your GOPATH. As far as I have seen, this is fine. But it could cause a problem somehow in theory.

The good news is, we can think of this tool as an alternate User Interface on top of the existing go and dep tools. It doesn't, or shouldn't, change anything in terms of how those tools work or what they are doing. It just automates the process of running go and dep inside your GOPATH.

Updated 2018-07-16: added support for SSH as well as HTTPS git remote URLs as well as error message on unsupported remote url formats. Fixed issue where host/user folder doesn't exist yet inside gopath.

UPDATE 2020-01-30: Bug fixes for cgit URLs and support default GOPATH of $HOME/go

UPDATE 2020-03-14: Bug fix to support repository names that have a period in them.

/usr/local/bin/goo

#!/bin/bash

/home/forest/Desktop/programs/goo.sh "$@"

/usr/local/bin/depp

#!/bin/bash

GOO_COMMAND="dep" /home/forest/Desktop/programs/goo.sh "$@"

/home/forest/Desktop/programs/goo.sh

#!/bin/bash

source ~/.profile

GOO_COMMAND=${GOO_COMMAND:-go}  
if [ "$GOPATH" = "" ]; then  
  GOPATH="$HOME/go"
fi

if [[ $PWD/ = $GOPATH* ]]; then  
  "$GOO_COMMAND" "$@"
else  
  GIT_REMOTES=$(git remote 2> /dev/null)
  if [ $? -eq 0 ]; then
    ORIGIN_REMOTE="origin"
    HAS_ORIGIN=$(echo "$GIT_REMOTES" | egrep "^origin$" | wc -l)
    REMOTES_COUNT=$(echo "$GIT_REMOTES" | wc -l)
    if [ $REMOTES_COUNT -eq 1 ]; then
      HAS_ORIGIN=1
      ORIGIN_REMOTE="$GIT_REMOTES"
    fi 
    if [ $REMOTES_COUNT -gt 1 ] && [ $HAS_ORIGIN -ne 1 ]; then 
        (>&2 echo "error: unsupported: no git remote named 'origin' was found. Implement this in 'goo' or create the origin remote.")
        exit 1
    fi
    if [ $HAS_ORIGIN -eq 1 ]; then

      GIT_CLI_ORIGIN_FETCH_URL_LINE=$(git remote get-url "$ORIGIN_REMOTE" 2>/dev/null)

      #Case:  SSH  
      #Fetch URL: git@github.com:MyOrg/my-repo.git
      GO_PACKAGE=$(echo "$GIT_CLI_ORIGIN_FETCH_URL_LINE" | sed -e 's/\([^@]*@\)\?\([^:]*\):\([^.]*\)\.git/\2\/\3/')
      GIT_HOST_AND_ORG_OR_USER=$(echo "$GIT_CLI_ORIGIN_FETCH_URL_LINE" | sed -e 's/\([^@]*@\)\?\([^:]*\):\([^\/]*\)\/.*/\2\/\3/')


      #Case:  HTTPS
      #Fetch URL: https://git.domain.com/MyOrg/my-repo.git
      #           git.domain.com/MyOrg/my-repo.git
      #           https://git.domain.com/cgit/MyOrg/my-repo
      if [ "$GIT_CLI_ORIGIN_FETCH_URL_LINE" = "$GO_PACKAGE" ] || [ "$GIT_CLI_ORIGIN_FETCH_URL_LINE" = "$GIT_HOST_AND_ORG_OR_USER" ]; then
        GO_PACKAGE="$(echo "$GIT_CLI_ORIGIN_FETCH_URL_LINE" | perl -pe 's|^(https?://)?([^/\s]+)/([^.\s]+)(\.git)?|\2/\3|')"
        GIT_HOST_AND_ORG_OR_USER="$(echo "$GO_PACKAGE" | perl -pe 's|(.*/[^/\s]+)/([^\s]+)|\1|')"

        #echo "GO_PACKAGE: $GO_PACKAGE"
        #echo "GIT_HOST_AND_ORG_OR_USER: $GIT_HOST_AND_ORG_OR_USER"
      fi

      if [ "$GIT_CLI_ORIGIN_FETCH_URL_LINE" = "$GO_PACKAGE" ] || [ "$GIT_CLI_ORIGIN_FETCH_URL_LINE" = "$GIT_HOST_AND_ORG_OR_USER" ]; then
        (>&2 echo "error: unsupported: Unable to parse GIT_CLI_ORIGIN_FETCH_URL_LINE: $GIT_CLI_ORIGIN_FETCH_URL_LINE. Implement this as a different case in 'goo'")
        exit 1
      fi 

      #echo "GIT_CLI_ORIGIN_FETCH_URL_LINE: $GIT_CLI_ORIGIN_FETCH_URL_LINE"
      #echo "GO_PACKAGE: $GO_PACKAGE"
      #echo "GIT_HOST_AND_ORG_OR_USER: $GIT_HOST_AND_ORG_OR_USER"

      CURRENT_PATH_INSIDE_REPO=$(git rev-parse --show-prefix | sed -e 's/\/$//')
      CURRENT_PATH_INSIDE_REPO_LENGTH=$(echo "$CURRENT_PATH_INSIDE_REPO" | wc -c)
      REPO_PATH="$PWD"

      if [ $CURRENT_PATH_INSIDE_REPO_LENGTH -gt 1 ]; then
        REPO_PATH=$(echo "$REPO_PATH" | sed -e "s|\/$CURRENT_PATH_INSIDE_REPO\$||")
      fi

      if [ -d "$GOPATH/src/$GO_PACKAGE" ] && [ ! -L "$GOPATH/src/$GO_PACKAGE" ]; then
        (>&2 echo "error: unsupported: The directory \"$GOPATH/src/$GO_PACKAGE\" already exists. Normally I would create a soft link here, but I can't. Implement this in 'goo' or delete it out of the GOPATH and try again.")
        exit 1
      else 
        if [ ! -L "$GOPATH/src/$GO_PACKAGE" ]; then
          mkdir -p "$GOPATH/src/$GIT_HOST_AND_ORG_OR_USER"
          ln -s "$REPO_PATH" "$GOPATH/src/$GO_PACKAGE"
        fi
      fi

      if [ -L "$GOPATH/src/$GO_PACKAGE" ]; then
        pushd "$GOPATH/src/$GO_PACKAGE/$CURRENT_PATH_INSIDE_REPO" > /dev/null
        "$GOO_COMMAND" "$@"
        GO_EXIT_CODE=$?
        popd > /dev/null
        exit $GO_EXIT_CODE
      fi
    fi 
  else
    (>&2 echo "error: unsupported: either git is not installed or you are not currently inside a git repository. Implement this in 'goo' or make this folder a git repository.")
    exit 1
  fi
fi