Dropserver Progress - September 2023

October 3 2023

This is the monthly progress report for Dropserver for September 2023. The previous report is here.

The Big Picture

I worked on finishing app changelogs and started working on installing apps from a URL. This is all part of “Dropserver App Packaging and Distribution” project. I also spent some time thinking about how to present Dropserver.org, how to make it more easily usable, and whether working on app distribution is the best way to spend my time now.

Thinking and Questioning

When working on a large project over a long period of time there is always a risk that going on vacation will cause a rupture in the sense of progress that may trigger a slide into apathy. Luckily I don’t feel apathy, but the break has forced me to think about where Dropserver is at and how I get it to where I would like it to be. I went down these mental quests:

Low Friction Dropserver Offering

It’s no secret that I’m the only person who runs a Dropserver instance, and my wife is the only non-admin user in existence. That’s fine, for now. DS still has a lot of holes, and installing and running an instance is too demanding right now. But this can’t go on forever. I need to think about how I can make it easier for people (even technical users) to try and use Dropserver.

Ultimately the easiest way for someone to get on Dropserver is by signing up through a commercial hosting service. So that’s what I need to work towards. (I have known this for a long time of course, but every once in a while I have to sit and ponder every other option before returning to my starting point.)

But even with that, I would still be suffering from a lack of users because of a lack of apps.

Re-Orient Dropserver.org Website

A developer posted a cool project built on Glitch. Since this is the kind of project I’d like to see built for Dropserver, I wondered if a developer like Casey would consider developing an app for Dropserver if they even heard about it? Looking at the Dropserver.org home page with fresh eyes, it became immediately clear it wouldn’t do the trick. It’s focused towards the end-user, who scientifically speaking can not exist in this world right now. So what’s the point of pitching them?

Then, while looking around for mentions of Dropserver on various sites, I found this discussion on Reddit that shows that there are many home application server offerings. A comment by ocdtrekkie spells out exactly the problem: how do you differentiate yourself? The Dropserver.org doesn’t tell a developer why they are better off developing for Dropserver. Yet developer experience is one of the differentiating factors that I want to build into Dropserver. I need to rethink the Dropserver.org site with that in mind.

What to Work On

The problem with spending so much time thinking is that nothing gets built, and everything comes into question. For a period I thought I might punt on installing apps from distribution URLs. I decided to carry on due to the realization that the order of things I release doesn’t matter: it all needs to get built sooner or later. The worst thing I can do given my limited resources is to stop mid-project to focus on something else because I imagine somehow it will serve DS better. This is wasteful, and I should just build the thing that I can build most effectively right now.

After spending so much time on app installation and distribution, I should just knock it out and move on.

Some Code Progress, Finally

Finish Work on Changelogs

I talked about changelogs in the last update. On September 13 I committed the changes.

Work on Apps from Websites

The goal for Dropserver version 0.12 is that a user should be able to install an app by pasting a URL where that app can be found. Not only can the app be installed using this method, but it should be possible to fetch updates and install the latest version, and do so automatically if the user so desires.

A further constraint is I want developers to be able to publish their apps as static website. Static website hosts are widely available with a generous free tier, and developers routinely host their blogs and other sites on them. The idea is that if ds-dev can generate a set of files then the developer just has to upload them to their static host of choice, and the app is published.

All this presents me with a number of questions:

What is this URL? What does it look like?
What data resides at that URL?
What other files need to be generated?
How will internationalization and localization fit into all this?

The current design calls for the URL to point to a JSON file that contains an array of versions, each with URLs to fetch the complete manifest (JSON), the package file (tar.gz), the icon file, and the changelog file (plain text).

URLs

I would like for URLs to be clean, so a user should be able to paste olivierforget.net/leftovers/ to get the leftovers app. But that URL should serve a HTML page describing the app. I could use HTTP Headers to tell the server I want the data file, but remember I want this to be trivial to host. I don’t want the dev to fiddle with settings in their static host, which is usually required when you want to serve different content based on headers.

So I am thinking of using either a .well-known path like this:

olivierforget.net/leftovers/.well-known/dropserver/app-listing.json

My fear is that the interpretation of trailing slash in the original URL will cause some problems.

The other option would be to put a <meta> or <link> tag in the HTML page served that points to the data file, however it’s not clear either of these are appropriate for the task.

For now user will have to paste a complete URL to the JSON file.

SSRF Protection

The first thing I ran into when starting to actually implement fetching that URL is the potential for SSRF — Server-Side Request Forgery. Fetching apps remotely can’t be an avenue for probing your local network.

While mitigations for SSRF are well understood, they present a problem for home-lab / self-hosters: the main idea of the mitigation is to prevent connections to servers that have a non-public IP. But if you are hosting at home or on a private network, well, that’s all your IPs.

After asking on Mastodon it seems the solution is to block all connections to local addresses except those white-listed by the instance admin via a config. This is probably the direction I’ll take.

Here is a good post on preventing SSRF attacks in Go, and a library to go with it.

What’s Next?

In October I’ll continue working on fetching apps from URLs and on the generation of the files necessary to host an app.

I am still being pulled away from Dropserver by unrelated projects, so I don’t expect to be able to make leaps of progress, but at least I shouldn’t spend too much time thinking!