Dropserver Progress - July 2023

This is the second progress report for Dropserver. This one will be modest unlike the previous epic.

The Big Picture

I continued to work on app packaging and managed to tag a release. To celebrate, I immediately went to work fixing some of the things that didn’t make it into the release.

Finishing Up App Packaging

App Name and Description

I had to create validations for app name, and decide how to deal with violations. The main factor is app name length.

App name length

I had to decide how to limit application names and short descriptions that are entered by the app developer in the manifest.

Every app platform I have come across limits the length of app names and other strings. Names should be limited in length because it makes it easier for users to scan when apps are displayed alongside other apps. But how does a hard limit square with the realities of responsive design? A name might be cumbersome to display on a phone screen but have lots of room to spare on a large display.

Let’s be honest: maximum app name length is completely arbitrary. For this reason I settled on an arbitrary value, but only give a warning and not a fatal error if the name is too long. I’ll deal with exceedingly long app names using CSS.

Don’t be naïve: count graphemes

The naïve approach to counting the number of characters in a string is to count the length of a string. This is quite wrong unless you’re dealing strictly with ASCII characters. The answers in this StackOverflow question cover a lot of ground.

Thankfully I found a good Go library that counts graphemes.

Commit e495dad.

Remove Bad URLs From Manifest

In earlier work bad URLs (like Javascript:) were marked with a warning, but that meant they were still displayed in the admin frontend, which is exactly what you don’t want. Ideally the bad urls would be visible but not rendered as a clickable link.

I don’t have the code paths to do this easily and reliably, so I do the easy thing and overwrite bad urls with an empty string.

Commit 8738eea.

Specify Package Base Filename

When you create a package file using ds-dev, you now specify a base filename. So:

ds-dev -create-package=/path/to/dist/ -app=/path/to/app -pagkage-name=leftovers

…creates leftovers-0.1.2.tar.gz.

I thought about putting the package name in the manifest, but it doesn’t make sense. It’s not a property of the app. It has no meaning. The package could be called anything and package file names can change from version to version, the system doesn’t care. For that reason it’s an argument to the packaging tool.

Commit 0222007.

Set Arbitrary Reasonable Manifest Size

While limiting app name length (as well as for other strings in the manifest) is mostly a UI issue, there remains the possibility that someone will abuse the manifest with absurdly long strings, hoping to cause the system to have to deal with all that data will cause a DDOS of some sort.

To prevent this I decided that manifests should only be so big. You can do anything you want, but beyond a certain size, the manifest will not be processed at all.

This works because there is a lot of room between the size of a reasonably large manifest and the amount of data needed to cause problems in ds-host.

The max size is 10kB. Commit bbbaf0c.

Validate Icon Image

I added validations for the app icon. Using a go library I check the icon size in pixels, I check that it’s square, and I warn if the icon size is too large (in kB).

Along the way I learned that you can compress PNGs a lot with tools like tinypng.com/ (which are probably wrappers aroudn open source tools).

I don’t allow gifs for app icons, but I learned that PNGs can be animated. I don’t want app icons to move in the admin panel, so I check that the PNG is not an animation.

Commit 8daf0f8.

Sometime after this something happened in the social media world: The Twitter app was rebranded to “X” and the icon changed. At least one app platform warned its users about the change:

See this post on Mastodon. This made me realize that I would probably eventually validate that the icon has not changed significantly from version to version. This will require doing a similarity analysis between icons, luckily this go library might work just fine.

Domain Change Tool

I wanted to test all the changes I made for App Packaging against my personal “production” instance of Dropserver, but I did not want to upset this instance at all, since I rely on it. Downloading all the data from that instance and sticking it on my dev VM is easy enough, but the problem is that there a lot of places in the DB and appspace data that stores domain names.

I decided to start work on a “domain change” tool in a branch (I won’t release it for a while) that would go through the DB and appspaces and change the domain name. That would allow me to take the data of any instance of ds-host and change the domain names so that it can run under a different domain.

This is useful for testing purposes. But it’s also handy if a user loses control of their domain and they want to reboot their instance under a different domain, for example.

Work is in the change-domain-tool branch.

After digging through to find all the spots where a domain is written, I had a tool that did the job. It worked well to test all the changes against my “prod” data. A nice boost in confidence for this release and a useful tool already showing value.

Documentation

I wrote a reference page for the app manifest.

I also wrote a high level overview of how to build a Dropserver app. It is meant to give potential devs a quick view of things that are different with Dropserver app development, along with some gotchas.

Finally I also added a page for packaging a Dropserver app. It’s a stub really for now.

I found myself struggling a bit writing these docs. On one hand I want things explained. On the other, each word of docs I write needs to be maintained. Outdated docs are worse than no docs. So I have to be concise and direct, so I can stand a chance to keep it all fresh.

Release v0.11.0 🥳🎉

Finally on July 19 I released App Packaging. Mastodon post.

Post-Release Fixes

First thing to do after releasing is to fix and tweak the things that were missed in the release.

Tweak Migrations Validations

In the released version of Dropserver, an app must provide migration functions in pairs (a down-migration for each up-migration). This is to encourage devs to give users a way to back out of an app upgrade that they are not happy with.

Unfortunately the mere presence of a down-migration function does not guarantee that it will do its job satisfactorily. I worried that a dev faced with a fatal error because of a missing migration would simply add a stub for the down migration. This is the worst possible outcome, because the system will assume the migration took place, leading to data corruption.

I decided a warning would be plenty good enough. Commits 3f19c30 and 502448e.

Display Migrations in a Grid

In both ds-dev and ds-host I wanted to show migrations that ship with an app as clearly as possible. I settled on a grid layout which I think is satisfactory.

Commits 71bd4b1 and 0dc3c20. Yes, there is an alignment problem, I know.

ds-dev UI and Code

I changed all Vue component code to use the <script setup> style. It’s a huge improvement in developer experience but refactoring each component is some very boring labor. This is where I wish I were a bit more adept at using AI to code because it would do quick work of this task. Oh well, it’s done.

What’s Next?

The big thing I’d like to work on next is app distribution and the ability to install and update an app automatically using just the URL at which the app is published. This is the “Distribution” part of “App Packaging and Distribution”.

But right now I’m headed for a vacation during a good chunk of the month of August so I am not going to start on that until September.

See you then!