SequentialRead - Greenhouse Update 4

🤔 Say that you have a computer and it has internet access

That computer can browse the web, but "the web" (or, web users around the globe) typically can't browse files or web applications running on that computer. Publishing a server on the internet takes work, costs money, and requires maintenance and upkeep. It can be especially difficult if you want to use your computer, not rent someone else's. If you live in a dormitory or if you get internet access via wifi from your neighbor or via tethering to your phone, you may not be able to perform the necessary modifications to your router's configuration even if you knew how.

Greenhouse provides an easy way to make one or more computers into servers that anyone in the world can connect to securely & reliably, regardless of where or how the servers are connected to the internet.

Greenhouse is being designed from the ground up as a trustless service, that is, you don't have to trust me or whoever's running the service to keep your data secure — it's designed so it can't access your data in the first place.

For more information about why I am building this, see greenhouse.server.garden and The "Pragmatic Path" 4-Year Update: Introducing Greenhouse!

You may also check out the source code at git.sequentialread.com/forest/greenhouse

Previous post in this series: Greenhouse Development Update 3 - August

Next post in this series: 🥳 Greenhouse Enters Alpha Test Phase!! 🎉

September is ending. Wake up!!!11!!!!!!!!

No, seriously, as I write this, it's about 22 minutes until midnight, that time when the calendar rolls over to October 1st.

I spose I've delayed writing a Greenhouse update post for September until now because I was secretly hoping I could get the alpha version released this month. Well, it's definitely not happening at this point. However, at least I can say it's not for a lack of trying.

This month I tried to make a short list of tasks that I'd have to complete before releasing an alpha version of greenhouse. To me, alpha means that it's relatively unstable, it probably doesn't have very many users, and it's free; payments have not been implemented yet.

I believe I started out with a list of about 8 items. By the time I had completed half of them, the list had grown to 17 items 😛

Here is my todo list today:

- [.] alpha test
  - [X] view logs from CLI
  - [X] view logs from desktop app
  - [X] delete tunnels from desktop app
  - [X] greenhouse webapp external domain support (greenhouse-alpha.server.garden)
  - [X] greenhouse webapp scheduled tasks: external domain validation, rebalance
  - [X] greenhouse webapp auto-start cloud-side and daemon
  - [X] greenhouse webapp general cleanup
  - [X] greenhouse internal how-to pages (using-your-own-domain-name-with-greenhouse)
  - [ ] greenhouse webapp alpha test invite/auth system
  - [X] desktop app listening port chooser
  - [X] desktop app file chooser
  - [ ] greenhouse daemon / cli / desktop app development environment quickstart docs
  - [X] Desktop app build / packaging Linux
  - [.] Desktop app build / packaging Windows
  - [.] Desktop app build / packaging MacOs
  - [X] curl | sh installation method for daemon/CLI on Linux
  - [ ] Desktop app installs background service on first run on MacOS
  - [X] Some sort of installation method for daemon/cli on windows?
  - [ ] Telemetry/monitoring: greenhouse
  - [ ] Telemetry/monitoring: greenhouse-desktop
  - [ ] Telemetry/monitoring: greenhouse-cli
  - [ ] Telemetry/monitoring: greenhouse-daemon
  - [ ] Telemetry/monitoring: threshold

By now it looks like it's grown to 24 items, but on the bright side, only 8 of them aren't started yet.

I wasn't sure just how quick-and-dirty I wanted the alpha version to be. For example, does "alpha test version" mean it only works on Linux? Does it mean the Windows and Mac implementations are there, but super hacky?

I spent a lot of my time this month grappling with these decisions. Often times I would investigate what work would actually be required & attempt to decide based on the technical reality I was facing, rather than a theoretical estimate.

I think I've been erring on the side of "do it right to begin with" to some extent, at least where the amount of work associated with that decision is minimal.

Here's a brief list of decisions I've made so far:

The linux installer for the alpha version will just be a shell script, aka the world-famous curl ... | sudo sh
The linux shell script installer will only support systemd-based linux distributions, however there will be reasonable DIY instructions for folks who don't use systemd.
There are two different installers for linux:
- The shell script which installs the background service and command line tool
- A source repo or debian package for the desktop application.
Windows and Mac are fundamentally different from linux in terms of use case: On these operating systems, the desktop application is king, it is not optional. Everything else is based on the desktop application installation.
For the desktop application, I will leverage the fman build system (fbs) Python/QT framework and its installer support. I decided on this framework early on, because I wanted an alternative to electron, something which could produce a more lightweight application experience while at the same time minimizing the pain associated with cross-platform development.
- On linux, it generates .deb and .rpm packages, potentially others as well. I haven't looked into "PPA"-style ways of hosting these yet, for now they will be direct downloads.
- On Windows, it creates an executable Nullsoft Scriptable Installation System NSIS installer. This is actually quite nice, as far as I can tell this installer gives a great user experience when implemented correctly, but it also doesn't limit you in terms of what you can do.
- On MacOS, it creates a .dmg disk image containing a .app package. The good news is, mac users love this and they know exactly what to do with it. The bad news is it's not an installer, so we don't get an opportunity to run a script while it's being installed. I will probably end up building the daemon and CLI installer into the desktop app on MacOS; it won't install until the first time you launch the app.
You will be able to use your own domain name with the alpha version, although this feature is going to be sort of hacky for now. Most of the advanced DNS-related features will have to come later.

Here is a screenshot of the work-in-progress alpha version of the web application:

greenhouse admin panel with "choose your greenhouseusers.com subdomain" feature, a bandwidth usage graph, and cross platform client software installation options

I think I got a lot done this month, for one thing, I finished the CLI (command line interface). Here's a breif overview of what it looks like:

$ greenhouse

Usage: greenhouse COMMAND [...]

Commands: status, register, tunnel, ls, rm, logs

To read the instructions for a specific command, you can run for example:

greenhouse help register
greenhouse tunnel -h
greenhouse ls --help

Advanced users may wish to configure this cli with environment variables. 
For instructions on how to do that, run 'greenhouse help config'

$ greenhouse status

Greenhouse Daemon:
    Server Name: thingpad
    Configured Tunnels: 1

    Caddy Server Status:
        Enabled: true
        Running: true
        PID: 411094
        Up for 3m29s
        Health Check: not implemented yet

    Threshold Tunnel Client Status:
        Enabled: true
        Running: true
        PID: 411088
        Up for 3m29s
        Health Check: not implemented yet
 
Greenhouse Account:
    Email Address: forest.n.johnson@gmail.com
    Allocated Port Range: 10000-10019

    Authorized Domains: 
        - greenhouse-alpha.server.garden
        - forest.greenhouseusers.com

    Client States:
        greenhouse_internal_node:
             Current: ClientConnected
            Previous: ClientClosed
        thingpad:
             Current: ClientConnected
            Previous: ClientClosed

$ greenhouse tunnel

Error: expected at least 3 arguments for tunnel command, but only saw 0:

Usage: greenhouse tunnel LISTEN_URL to LOCAL_URL
   OR: greenhouse tunnel --json '{...}'

  Open a new tunnel. 
  LISTEN_URL supports the protocol schemes https://, http://, tls://, and tcp://.
  LOCAL_URL supports the protocol schemes http://, tcp://, and file://
	For details on the format that --json accepts, run 'greenhouse help json'

  e.g:
  greenhouse tunnel https://www.cli-user.greenhouseusers.com to http://localhost:80
  greenhouse tunnel https://files.cli-user.greenhouseusers.com to file:///home/cli-user/Public
  greenhouse tunnel tcp://:10014 to tcp://localhost:22
  greenhouse tunnel --json '{"domain": "cli-user.greenhouseusers.com", "public_port": 443, ...}'
		
$ greenhouse tunnel https://nginx.forest.greenhouseusers.com to http://localhost:80

Now applying new tunnel configuration...

  - waiting for underlying services to start
  - creating threshold tunnels
  - testing threshold tunnels
  - configuring caddy
  - waiting for caddy to obtain https certificates from Let's Encrypt
  - final testing

Your tunnel was configured successfully!

I also made serious headway on getting the software built and deployed for all platforms. At this point, windows is most of the way there. ¡Bienvenido a IIS!

screenshot of a windows desktop displaying IIS (internet information services, the default web server on windows) being tunneled through greenhouse

I managed to get the greenhouse-daemon (called greenhouse-background-service on windows 😇) installing as a windows service via the NSIS installer, and it even creates a special service user for that as well. I mainly wanted to do that to isolate greenhouse for its own protection; make it a bit harder for malware running on the windows machine to swipe the encryption keys that are used for TLS, for example. I'm still fairly green on file permissions / security on Windows, so if any of yall are Windows experts, I'd love to hear your thoughts in the comments!

There are still quite a few bugs and issues on Windows, but just the fact that I was able to get everything going via a single .exe installer is exciting!

I also got the desktop application running on MacOS, in the picture below I am using the Sosumi snap to virtualize MacOS on my linux workstation.

Here's the roadmap as it stands today. The main changes since last time; the cross platform installer feature was moved from the beta phase to the alpha phase, the CLI was finished, and a lot of other features went from "MVP implemented" to "ship it".