Dropserver Progress - June 2024

This is the progress report for Dropserver for June 2024. Here is last month’s report.

I started on the Tailscale integration, which began by getting the tsnet package, which resulted in a shock.

Go get tailscale.com/tsnet … 😲

After entering the command the machine churned for several minutes while unfurling a seemingly endless listing of dependent packages required by tsnet. Yikes.

tsnet loads lots of dependencies from the tailscale.com package which is expected. But these in turn load some surprising stuff. In particular, Dropserver now depends on the AWS SDK because tailscale.com/ipn/store has an option of storing stuff on S3.

Having unused dependencies like that isn’t great, but I could live with it. And the reality is I’m likely going to end up depending on AWS SDK to allow pushing appspace backups to S3-compatible storage some day.

But I got a shock when I compiled ds-host with tsnet: the built size went from 28 megs to 47! Yikes. Hello bloat. 😞

tsnet creates an entire networking stack in user-space, so it’s not surprising that the build got bigger, but this is a lot. For this and other reasons, I am going to try to make it possible to build a Tailscale-free version of ds-host.

Integrating Tailscale (Easier Said…)

It’s easy to say I’ll integrate Tailscale into Dropserver. But there are a number of decisions to make on how this is actually going to work.

I’m realizing that there are likely two types of people who may want to use Dropserver via Tailscale: those who have a Tailnet in active use and those who will use Tailscale for the sole purpose of simplifying their self-hosting adventure with Dropserver.

I have to consider that active Tailscale users will not allow Dropserver to do anything dubious in their Tailnet and they will likely control things themselves in the ACLs. Meanwhile, users without Tailscale may not know much about ACLs and may want to minimize their interactions with the Tailscale admin panel.

In either case I want to preserve the ideals of Dropserver while making Dropserver nodes “good citizens” of a Tailnet. And as always security is critical. Meeting all these goals won’t be easy.

I’ll write a full post on the integration when I am further along, but here are some quick gotchas that I’m up against:

• Tailnets do not support subdomains for a node. Woops! Subdomains are the common way of creating an address for an appspace. There is a workaround: each appspace will be its own Tailscale node.
• ACLs control who can access each node and are edited by hand in a JSON file on the Tailscale website. One goal of Dropserver is to not have to hand-edit config files when hosting a new app or adding a user!
• It’s possible to edit the ACLs via an API, but there are no granular permissions. An API client that can edit the ACLs can make an absolute mess of your tailnet. I don’t imagine anybody that actually uses Tailscale for things other than Dropserver giving DS the keys to their tailnet. No way.

Despite these (and other issues) I think I see a path forward, but I’m only partway through a functioning implementation so I am not confident I’ve got it right. Stay tuned.

Appspace Meta DB Versioning and Migrations

As mentioned in last month’s progress report integrating Tailscale means changing the Appspace Meta DB schema, where appspace users are stored. This means that for the first time I will have different schemas for the Appspace Meta DB in the wild. A backup of an appspace that is sitting in my Downloads folder will likely be at versions 0 (sans Tailscale). All currently used appspaces will have their Appspace Meta DB at version 1.

I need to work through that situation while preserving the goal that Dropserver should “just work” and users should not get weird error messages or become stuck unable to use their data.

To handle this Dropserver will determine the schema of any Appspace Meta DB it comes across and act accordingly. It will migrate up to the current version when appropriate. Some work alredy done:

• Change migration code for Appspace Meta DB. Commit 8fa902.
• Add table to allow multiple auths in Meta DB. Commit 66271e.

Next I need to work migrating all Meta DBs when the ds-host instance changes, and performing migrations when necessary when importing appspace data from backups.

What’s Next?

I’ll keep working on Tailscale integration!

However between Summer vacation and non-Dropserver work my progress will continue to be slowed. In fact I’m going to call it now: there will be no Progress Report for July as too little will get done. I’ll be back at it in August. See you then!