Olivier Forget

Dropserver Progress - February 2026

Olivier Forget — Sat, 07 Mar 2026 11:35:00 -0800

This is the progress report for Dropserver for February 2026. The previous report is here.

This month I kept grinding away on removing DropIDs from the hot path of authentication in appspaces. See the November-December update for an explanation of what it’s all about.

Showing User Conflicts in the UI

A consequence of removing dependence on DropIDs and also adding authentication via Tailscale means that each appspace user can be identified through different identifiers (their tailnet ID, their DropID, and in the future probably their email address.)

Having multiple ways for each user to authenticate is great but you might get in a situation where one appspace user might be associated with two separate ds-host instance users. This is a confusing situation to be in and is best not allowed. Similarly it is possible to associate one instance user with two appspace users. Again, confusing. But also, it means the system doesn’t know which appspace user to authenticate when an instance user wants to access an appspace. It’s best to disallow this situation.

Note that if I do my work correctly ds-host will prevent you from creating these conflicts in most cases. For example it won’t let you select an instance user for an appspace user if that instance user is already associated with a different appspace user.

But conflicts are still unavoidable. I am serious about making appspaces “movable”, meaning that you can export the data and import it on a different instance and, as much as it’s possible, things should work. In other words an appspace, along with its list of users and associated auth identifiers, can be imported into a ds-host instance, and there is no guarantee that this set of auth ids doesn’t conflict with auth ids and instance users of the new host.

As such, I have to assume conflicts can happen and make it possible for appspace owners to deal with them.

The first step is to display the conflicts as clearly as possible. See below for where I’m at with this right now. The first screenshot shows no conflicts, just one warning about a user that is not an instance user at all. (It’s technically possible to have non-instance appspace users, but it’s not implemented yet.)

Now if I change the DropID of the first user to “dropid.local.dropserver.org/altoli” and then set the DropID for the “Alt Oli” instance user to the same, then everything falls apart, and orange appears everywhere.

(nb: I still need to replace “User 1” with the actual instance user name and profile pic, and tweak some alignments, but you get the idea.)

If an appspace owner encounters this, clicking on the “Edit” link allows them to remove or change the problematic auth identifiers. Hopefully that’s clear enough.

Instance User Display Name and Images

I had punted on display name and images for ds-host instance users until now. With instance users becoming more important and DropIDs becoming irrelevant I needed to give instance users these display names and profile pics, and make them available to all other instance users so they can see for example who owns an appspace that they are using, etc…

Moving Along

Overall decent progress in February, especially considering that I was on vacation from the 20th onwards. I can see the light at the end of the tunnel with this work. See you next month.

Dropserver Progress - January 2026

Olivier Forget — Sun, 15 Feb 2026 16:35:00 -0800

This is the progress report for Dropserver for January 2026. The previous report is here.

I started the year by refreshing my personal note taking app, then I got back to work making ds-host more flexible with domain names, in particular making them unnecessary.

Updates to Illuminoted Dropserver App

Illuminoted is an app I wrote after getting fully fed up with the state of note-taking apps. It’s a very personal app, full of quirks and inadequate user experiences which is why I haven’t yet formally published it online as an installable Dropserver app, though I probably should.

As with all my personal apps I only touch the code when it’s sorely needed. With the Leftovers app I usually go half a year before I get in there and change things up. This is a key thing about home cooked apps: unlike commercial apps that are always after a bigger number for their KPIs, the changes are few and far between. Or rather you only change them when you strongly feel that something needs to change. Maybe there is a badly needed feature that you just don’t want to work around anymore, or a papercut bug that has snagged you so many times you’ve run out of virtual band-aids. I find that a few months is the right amount of distillation to surface the truly necessary changes.

Oddly, with Illuminoted, though it’s the app I use more than any (I’m on it all day every day, 19,360 notes deep, documenting all work stuff, non-work coding stuff, but also mundane things like my car tire shopping and research notes) it’s the one that has gone the longest without any significant upgrade.

October 2023! That’s the last time I did anything. Naturally after such a long time I had a long list of things improve. I didn’t get to all of them but I did knock out quite a few wins and I am happier for it. You can get an idea from the commit log.

I now have better threads management, better navigate-to-note support, better UX to create a new note, and proper full text search thanks to sqlite’s FTS5.

Back To Work Removing DropIDs

In the previous progress report I talked about my goal of removing the hard dependency on domain names in ds-host and how that meant I had to remove the hard requirement for users to set up a “DropID”.

This creates a cascade of changes around users and appspaces. I’ll try to summarize it all below:

Eliminate “remote” appspaces

These are appspaces hosted on other ds-host instances that can be accessed via federation (DropIDs were central to this). It turns out that a bug in my federation code prevented any such federation, so the “remote appspaces” could only be on the instance the user is on. A convenient mistake…

Make it possible for users to access an appspace on the same instance

Originally “remote” was used for both other instances and appspaces owned by others on the same instance. With federation gone, the latter capability had to be restored.

Double down on multiple identifiers for appspace users

When I integrated Tailscale into ds-host I made sure to use that network’s native IDs to identify users in an appspace. A good move, but then I had multiple ways of identifying a user: DropID and Tailnet ID. Once you allow that, the potential for conflicts is inevitable. An appspace user could match with multiple ds-host users and a ds-host user could match multiple appspace users. To avoid problems I wrote a system to detect such conflicts and report them to the frontend.

A reminder that all of this stems form the principle that appspaces should be “moveable”: you can download an appspace from a ds-host instance and move to another without losing all connections to the users. This is possible because appspace users are identified with established globally unique identifiers, not the instance’s user ID.

It’s good to have this work done because I will probably add more types of identifiers in the future, like the venerable email address.

Cleanup on aisle 6

Finally I was able to remove remote appspaces from the code as well as the faulty “federation” code. A few months ago I said I was about to do a “cleanup on aisle 6”. This was that cleanup, and it felt great to mop up over 2000 lines of code away.

What’s Next

I still have a number of loose ends to finish up before I can ship these changes, sadly. As always the pace of development of Dropserver isn’t what I’d like it to be. So it goes.

I’m late posting this progress update. Even later than usual in fact. January was stressful for reasons unrelated to Dropserver. See you for the February update, hopefully not as late this time!

Dropserver Progress - November December 2025

Olivier Forget — Sat, 10 Jan 2026 10:23:00 -0800

This is the progress report for Dropserver for November and December 2025. The previous report is here.

In November I began work on removing the dependence on domain names in ds-host.

Domain Names in Dropserver

When I started writing Dropserver I thought of it as a very standard server program. The main difference was that it served apps that could be easily installed through an interface.

Lately I have started peeling away the assumptions that go into making a server. In particular I started seeing that the server might actually be serving on different networks, or have requests funneled to it via a diversity of technologies.

The first sign that I had made some incorrect assumptions came when I wrote the integration with Tailscale.

In a typical server program the configuration file expects that there is a domain name that points to the server. Because of course there is, right? What’s a web server without a web address?

So ds-host was built with the idea that there was a domain name, and that the appspaces would live at subdomains on that domain, and that’s how you could refer to an appspace, because its address is its name. Nice and easy.

Then came Tailscale, and each appsapce is its own Tailscale node, and all of a sudden appspaces have two names: the subdomain based one, and the Tailscale one, which is somename.tailxxxxx.ts.net for example.

The Tailscale integration works really well. It’s a fantastic way of accessing a personal web-app from your phone or laptop from anywhere in the world. In fact it’s so good that if you only use Dropserver for personal stuff, a domain name pointing to your Dropserver instance is kind of unnecessary.

Not just unnecessary! It can be painful to set up and it costs money too.

Using a domain name to access ds-host typically involves making your instance accessible via a public IP, which makes it a target of bots and crackers. You need to buy and renew that domain name, point it to the right IP etc… All these things are surmountable, especially for regular web developers. But my goal is to bring self-hosting to non-tech people, and ditching the requirement for a domain name is a big win.

Going beyond Tailscale I am thinking about creating an integration with a tunneling service. Those also sometimes offer use of their own domains, so once again, no need to bring your own.

In the end the goal is to let people run Dropserver on some small computer in their home, and use one of these tunneling for software-defined network services to make that instance accessible as appropriate for their use case. But this all means that “the domain is defined in the config file” is completely inappropriate.

Removing the Requirement for Domain Names

The problem right now is the entire Dropserver code assumes a domain is set in the config file. When you think you’re building something in a bog standard way from day one, as I was doing with Dropserver, the assumptions you made have had all that time to take root and spread their tentacles all over the code.

I’ll need to figure out how I “name” appspaces when their subdomain+domain is no longer necessarily a thing. But that will be for a different time.

As I explored the code, I found that a big dependent of domain name is the “DropID” concept that I hallucinated long ago and baked deep into the system.

A DropID is a handle (username) and a domain. It’s like an email address, but there is no mailbox. It’s just a way of uniquely identifying a user (the ds-host instance ensures usernames are unique, and on a public domain name the domain part is unique too.)

Why did I create this thing?

Appspaces, Users, and DropIDs

I dislike having to create a username and password for every new service I do anything with. I dislike using OAuth because it gives too much control to the identity provider (usually giant megacorps) over my access to that service.

Should Dropserver become popular one day, with each person running their own instance, then joining a friend’s appspace would mean becoming a user on their instance (username + password). That’s friction and I don’t like it.

So I decided that Dropserver should be federated in some way. Two instances would be able to talk to each other and a user of one instance could obtain a login token to a different appspace from their own instance.

(Totally not reinventing OAuth over here 🤦‍♂️)

This works if instances are on the public internet with a registered domain and TLS. That was my assumption back then so I didn’t question it.

(I am not going to go into the details of how this worked because I can tell this post is going to be long enough.)

Instead of referring to users by the instance’s user_id, appspaces referred to users by their dropIDs, which are ids that are globally unique, and that allows people to log in from a different instance.

There are few things wrong here. First, it was half-baked. This kind of thing needs a lot of work to get right. I didn’t spend the time, and it showed in the completely lame implementation.

Second, really not a great idea to reinvent the wheel.

Third, it’s only useful when people have their own instance. It leaves potential users who have not gone down the road of setting up the ds-host (meaning everybody on the planet except me and maybe one or two other nerds) completely out of the loop.

Walls Versus Ramps

This could be its own blog post, but I’ve been thinking a lot about walls versus ramps. In short: what I should focus on is an easy way for non-Dropserver people to gain access to a DS appspace. Let them get into the system in the easiest way possible for the first time. Give them a ramp.

I’ve been talking lately about making Dropserver work on computers users already have at home like a Mac or Windows machine (a ramp) instead of asking them to learn Linux (wall).

By implementing DropIDs I was assuming many people would successfully climb walls. Instead I should focus on building ramps that help people get into Dropserver to begin with.

It’s a New Year

In 2025 I managed to ship Tailscale integration, and got ds-host builds for the Mac and for arm64. Then I got to work on simplifying configuration and installation of Dropserver. Work is slow, but steady.

In 2026 I’ll continue to make Dropserver easier to get into (ramp) by reducing the requirements for a basic minimal install that is useful and safe for serving personal web apps.

Dropserver Progress - October 2025

Olivier Forget — Sat, 15 Nov 2025 15:19:00 -0800

This is the progress report for Dropserver for October 2025. The previous report is here.

This month I continued thinking about how to make ds-host easy to install, but I also got to work on some real improvements. In particular I found out that my MacOS build of ds-dev was broken…

Fixing Releases to Support arm64

My Github Action that builds the Dropserver releases used the macos-latest image to build ds-dev. That image used to be Intel-powered. At some point Github decided to make it an arm64 machine.

The result: my builds of ds-dev that stopped working on Intel.

Since I had been wanting to release an arm64 version of ds-host and ds-dev for Linux to support the Raspberry Pi and other SBCs I decided to rewrite the Github Actions to fully support multi-arch and multi-OS releases.

I also separated the building of release artifacts from the actual release, allowing me to build for platforms even if I don’t release them. This lets me test on unreleased platforms without having to build on that platform.

See all these commits.

0.14.2 Release for Linux/Arm64

Dropserver 0.14.2 includes binaries for Linux-arm64 for ds-host and ds-dev.

I tested ds-host out on my old Raspberry Pi 4B (4 Gigs of RAM) and it worked fine. Sandbox start times are a bit slower (300ms instead of 150ish), but that’s still highly usable. So now Dropserver works on a small cheap SBC! Nice.

0.14.4 Release ds-host for MacOS on Intel and Arm64

I have long had a ds-dev build for Mac, and while working on the Tailscale integration I tested ds-host locally on my Mac to save time. So I have some confidence that Dropserver works fine on the Mac.

I only had to make it so the “linuxy” bits don’t show thorough on the Mac and it was ready to go. Commit 4a6803a disables cgroups and bwrap on darwin.

Dropserver 0.14.4 🎉

The rest was mostly about docs, and me learning how to run a service on a Mac. Launchd is quite different from systemd. It likes to quietly do nothing that you’re asking it to do instead of printing an error.

For now the docs show how to run ds-host as your main user. This is of course not ideal, and it leads me to the next thing:

Sandboxing on MacOS

I talked last month about how I am moving away from two-layer (Deno + bubblewrap) sandboxing of app instances. I’m trusting Deno more as the primary sandbox and I think a sandbox around ds-host and its Deno sub-processes makes more sense as the secondary sandbox.

This was a good opportunity to experiment with the Mac’s sandboxing abilities.

Unfortunately I didn’t get far.

sandbox-exec should be very effective as it’s pretty low-level, but it’s not easy to get it working. If you block everything by default there is a lot of work needed to figure out what needs to be allowed for the program to work. I didn’t have the time to go all the way with this so I’m punting.

If I can’t get that to work I can at least start with an allow by default, and then restrict reading and writing to the user’s files outside of the ds-host data dir. That would protect the user’s stuff at least.

Launchd also has some capabilities, like RootDirectory, which is essentially chroot, but I haven’t played with that, and it’s likely not the way to do things anymore.

Finally it seems the way things are done now is through the App Sandbox. ds-host itself would call the Mac’s sandboxing features to specify how it wants to be contained. Although this answer indicates it’s a very coarse kind of sandboxing, so less valuable (but easier to get right no doubt!)

If you are knowledgeable about sandboxing things on the Mac and you want to help out, please reach out!

Work on ds-dev for Windows

Two years ago I tried to get ds-dev to work on my Windows machine. I was traveling recently with my Windows Laptop so I decided to take another stab at it.

First I had to bring that branch up to speed. It was over 150 commits behind. Fun.

Then I set things up so I can test the code from a Linux WSL2 instance of Debian in addition to normal Windows OS. That way as I mess around to get things to work on Windows I can verify that the tests still pass on Linux.

I made some progress on the smaller more obvious fixes. Almost everything is a question of how to handle paths, really.

But the real crux of the problem is centered on the Deno sandbox. The problem is that there are paths everywhere: some get passed to Deno via cli args, some are written to the import map, some are written to JS code as imports, some fetch files from Go’s virtual filesystem, some from Deno, etc… In each case I’m not completely clear what format (slash type, “C:\\”??) is understood or expected.

So there is a lot more work to do there, but one day I hope ds-dev, and then ds-host will work on Windows natively.

Here’s to Another Mid-Month Update

And that is all for October. I am quite happy with the progress I made. I have releases for hardware that is more likely to be available and familiar to people, making Dropserver more accessible. Installation process is still bad, but I’m on it.

I said a month ago I’d try to write this sooner after the end of the month. Looks like I failed. Oh well.

I’m excited to share the “cleanup on aisle 6” type of work I’m doing in November. See you in early December! Or mid-December.

Dropserver Progress - September 2025

Olivier Forget — Tue, 14 Oct 2025 19:27:00 -0700

This is the progress report for Dropserver for September 2025. The previous report is here.

Bubblewrap woes

Dropserver uses Bubblewrap as a second layer sandbox. bwrap uses Linux namespaces to create its sandbox, and as such things can sometimes get prickly.

For some reason, on Debian 12 I can not get ds-host to start a sandbox with bwrap enabled when it is running from systemd. It works fine when I start it up manually, but as a systemd service, no go. I tried all kinds of things and I just can’t get it to work. Something is different in this situation and I don’t know what and I just can’t get it to work.

The folks over at Sandstorm have also had some issues with Linux distros (Ok I mean Ubuntu) making it harder for some namespace features to work.

There are real security issues to be fair. This thread gives some idea of what’s on the maintainers' minds.

De-emphasizing Bubblewrap

For my project, I think it’s best to continue to support Bubblewrap as long as it works. However I will not recommend it so much anymore.

It’s harder and harder to get it to work
It doesn’t offer significant additional security. Deno is fairly mature and stable and should be enough.
My goal is to support platforms other than Linux, where this kind of fine-grained sandboxing doesn’t exist anyways.

I’d rather focus on making sure the way I use Deno is safe rather than trying to wrestle with bwrap, which is only available on one platform.

I’ll rearrange the docs to put the bwrap stuff in a separate page for those who feel they really want this.

Tailscale in the Real World

I was thrilled to finally ship Tailscale integration and I still think it’s transformative for people who self-host.

However my experience trying to get some people hooked up with the app on their phones has been less than great. Installation is easy and quick, but for some people, it’s a legit battery drain.

Nobody likes their phone losing battery faster than normal. It’s just something about the current moment in smartphone tech. Batteries last long on a good phone, but battery vampires are never welcome. So I have some people who have the Tailscale app but leave it turned off until they need it, which… OK. 😔

I know Tailscale has done a lot of work on battery issues for their apps, but you can still see a number of complaints in recent app reviews. I suppose there is only so much they can do yet I still hope it gets better somehow.

Occasionally laggy

Once in a while, I’ll open the Leftovers app and it will sit there with the refresh spinner running. If I refresh the app it sits there trying to load.

At other times, in fact most of the time, it’s blazingly fast.

But when it stalls we’re talking 7 to 10 seconds of lag here. A painful wait.

I’m pretty sure it’s a Tailscale issue. I’ve only seen this behavior on the apps I access from Tailscale, and the logs don’t indicate a slow request processing time (I do need better request logging in ds-host though).

I have a suspicion that it’s a DNS issue, but I suppose it could be anything in the process of connecting a tunnel. When I tried to observe the requests by connecting the phone to the Chrome debugger, I was unable to reproduce the issue. Weird.

Also, it’s not always the case. In September I experienced the lag frequently. Now in October it seems much better? Though it still happens. Maybe TS was having some difficulty, though their status panel never showed a problem.

Leftovers App Work

I improved the Leftovers app.

I added a ❄️Freezer❄️ section. You can now specify that items are in the freezer and set their expiry time in months instead of days. Commit 21dfb72.
The data automatically reloads every time the app visibility changes to “visible”. This helps ensure you’re always looking at the freshest (ha) food listing. Commit 605d18b.
I added a little spinner that shows the data loading. A tap causes the data to load again. I fiddled with CSS animations until I could get the spinner to slow down convincingly instead of abruptly stopping. Sometimes I like playing with CSS. (See commit above.)
Fought a battle with package.json and postcss.config.js and vite.config.js and yarn and npm to upgrade to Tailwind 4. I never enjoy playing with these sickos. Commit 41b2557.

Version 0.7.0 of Leftovers is now out. The previous significant update was 18 months earlier. I enjoy the pace of development of home-cooked apps.

Brainstorming Better Install Experience for ds-host

I spent the rest of the month thinking about how I can improve the installation experience for ds-host. As it stands it’s really pretty poor. I keep running into this myself when I want to quickly install it to test something out and I just can’t do it quickly, even though it’s my project and I’ve done it many times. 😭

Dropserver is my attempt at making self-hosting more approachable to non-tech users. But right now the installation experience completely defeats the whole effort of the project. So it’s what I’m tackling next, and I intend to make it good.

It’s going to require a rework of the config file and a number of assumptions that are baked into the code (I talked about domain names, “DropIDs”, and other things last month), but it needs to be done.

I think this will have to be a separate blog post though.

Onwards

See you next month. I’ll try not to wait til mid-month to write the progress update. I already have exciting things to write about for October!

Dropserver Progress - August 2025

Olivier Forget — Sun, 14 Sep 2025 16:45:00 -0700

This is the progress report for Dropserver for August 2025. The previous report is here.

This month I worked on getting an instance of ds-host running in my home. Now that it’s trivially easy to make an appspace a Tailscale node, running my instance away from the public internet is the way to go.

But there are caveats! And new footguns! And countless little decisions I made while developing ds-host that are no longer helpful! Finding all those out, and working on solutions occupied plenty of time.

Domain Names Should Not Be Required

The ds-host config file asks for the domain name of the instance. Oof. What if you don’t have one? Why do you have to have one?

I used to think all instances would be on the public internet, with a domain or a zone pointing to their public IP. I no longer think this way.

It’s perfectly reasonable, and in fact it’s probably a good approach to run ds-host on a machine at home, and access it exclusively through Tailscale. This takes care of all the networking challenges in one shot while being far more secure than exposing your instance to the public internet. It’s a giant win for self-hosters.

So domain names should not be a requirement. While I was pulling on that thread, I wondered about all the ways someone might want to connect to their instance. It’s quite a matrix! See issue 142.

Dropping DropIDs

What are DropIDs? I never got around to writing that up in the docs. So here is how they came about:

One of the principles of Dropserver is that you can move your service from one instance to another without undue hurdles to climb (illustrated by the move-your-sh**.png icon on the home page). Since a user on the appspace may not be a user in the new instance, appspace users have to be identified by some sort of globally unique name.

Imagine for a moment that I was even more naive than I am and had used the instance’s user ID as the appspace user ID? Is user #2 in one instance the same as user #2 in a different instance? Of course not.

So I cooked up DropIDs. I was fully into the “every instance will have a public domain” phase of Dropserver, so using the instance’s domain name made sense! Your DropID is your instance’s domain with a username appended as a path, so somedomain.com/oli could be my DropId.

With that as the user identifier in appspaces, an owner could move an appspace to a new instance and there would be no confusion as to who’s who in the list of appspace users.

But wait, did I mention that DropIDs were also part of:

Remote logins

DropIDs enabled remote logins to appspaces from other instances. If an appspace is on ownerdomain.com and a user is on an instance at userdomain.com, then that user could “join” that appspace from within their instance, and log in from there by clicking on a link from within their instance (no additional password needed.)

I was really keen on this “federation” feature in ds-host. I didn’t want people to have to create accounts at countless Dropserver instances to be able to participate in other people’s appspaces.

I implemented all this, but I did not give this capability the dev time it deserved. And something like this takes a lot of work to get right.

Now that I fully expect many instances will be hidden from the internet and have no public domain name the whole idea falls apart.

Use email instead

I still need a unique identifier because moving appspaces around is still key.

I realized that the obvious one to use is email addresses. Everybody’s got one, and they are globally unique. Everybody knows what they look like and can recognize their friend’s addresses (sometimes).

The only hiccup is that ds-host currently does not send mail so I’ll need to be clear that the email is for identification, not for sending.

No more remote logins?

I still think it’s important to make it as easy as possible to let other appspace users log in to an appspace. But the picture is more complicated now. Tailscale is great, but nobody I know has the Tailscale app, especially among my non-technical friends.

There is more thinking to do, but anything based on public domain names I now consider DOA. So the remote login aspect will go away in a future update along with DropIDs.

I do hope to bring it back someday, but it will be quite different.

ds-host At Home: New Challenges

Installing ds-host is a pain. I know this and plan on improving the situation as much as possible, but more concerns came up as I went through my own install at home.

Backing up

Having my own instance at home brings up some new questions: what happens if this cheap hardware I bought gives up the ghost? I have lots of data on some of my appspaces and I do not wish to lose it. (My chat-style note/journaling app has 17,679 hand typed entries as of this moment.)

I had to get Proxmox to generate backups of the VM once per night and place them on my Synology NAS. That’s good, but ds-host itself should have built-in backup capabilities. It should be able to backup itself and all its appspaces periodically. You should also be able to backup appspaces to a remote location automatically. More things to think about.

Monitoring

On my older instance I installed Grafana and Prometheus in a docker compose setup to monitor the ds-host instance and its VM. It worked fine, but it’s an unreasonable lift for non-technical people. Also, it generates tons of graphs that won’t make sense to most people.

This led me to wonder what could be done to give just enough visibility and what are the use cases for visibility into runtime metrics?

Running Dropserver should just work. Things that just work don’t require dashboards full of graphs of low level metrics. It’s important not to carry over habits from running public SaaS apps into the world of self-hosting private apps from home. These are different things, like a Formula 1 car is different from a Honda Civic. One requires constant care and attention by engineers and the other just runs.

But sometimes even the Honda needs some love, so perhaps there should be something like a “check engine light” on ds-host, and a page that gives current metrics that can be shared on a forum to help with diagnosis and resolution.

More thought needed there.

Onwards

That was long! The time spent pondering things this month will help me put my efforts in the right place in the following months.

See you then!

Dropserver Progress - June and July 2025

Olivier Forget — Wed, 06 Aug 2025 12:22:00 -0700

This is the progress report for Dropserver for June and July 2025. The previous report is here.

In June I finally finished up the integration of Tailscale into Dropserver, then went on vacation. What remained of July was not terribly productive, but it was a good opportunity to think about where I should be going with this project.

Tailscale Integration

There were 21 commits in June. Lots of little things of course but some were the result of finally figuring out an effective UI for creating and managing a Tailscale node.

Tailscale connection UI

I have found that the most reliable way to create a node in fewer steps is to use an auth key generated from the Tailscale admin panel.

By comparison, when using a login link, tags have to be specified (without typos) and are not applied reliably. Furthermore you usually end up interacting with the Tailscale admin UI anyways to remove key expiry for example (which should happen automatically, but once again, not reliable.)

So I simplified the UI by removing the tags input line and I encourage users to create an auth key. Though if they don’t that’s fine too. Dropserver will show the login link and will detect misconfigurations (missing tags, etc…) and guide the user to a fix.

I also tweaked the Control URL. No more “leave blank to use Tailscale”. Now Tailscale.com is explicitly shown as the default control URL along with an affordance to “use alt control URL”.

By eliminating tags from the inputs the dialog becomes 25% simpler (3 lines instead of 4). Of those three lines, the “Hostname” line has a good default, the “Control URL” will be left untouched by most users, so all that’s left is the “Auth Key”. Not bad? See image below.

Here is what it looks like if you created a node but you are missing tags (and also do not have HTTPS turned on):

Commits fa8ebe8, 3f66f20 and 136d0d9.

Bug / poor behavior in tsnet

While working on validating Control URLs I found that tsnet does not error or timeout if you tell it to use a Control URL that is non-existent. I filed a bug with Tailscale.

If a user puts a typo in their control URL they will get no feedback whatsoever because of this. The UI will just sit there saying it’s waiting on a connection. Not Cool.

Release (Finally)

I tagged and pushed v0.14.0 on June 28. 🎉

And promptly went on vacation. 😴

What Next?

Back from vacation I found myself not too terribly motivated to dive back into Dropserver. I usually prefer to leave with an unfinished project waiting, it makes it easier to get back into it. Also Tailscale integration took far too long, so I’m not surprised to feel the need to take some distance.

After a few days hints of motivation appeared. With Tailscale done, I feel free to work on anything that could improve Dropserver. The question is what. Here are some possible directions:

Make DS easier to install on more hosts

This is the direction that I thought I should do while working on the Tailscale stuff. The Tailscale integration into Dropserver means you can have a usefully connected instance without doing anything like buying a domain name, opening ports, and making your network reachable by the outside world.

Since Dropserver already focuses on making app installations trivially easy, the only remaining hurdle is the installation of Dropserver itself! I had originally thought that I’d offer a DS hosted service to get around this problem, but at this point I don’t think that’s the right approach. DS is about self hosting. (And I don’t want to try to run such a service!)

So making ds-host as easy as possible to install, and making it installable on the computers and OS that a user has at home is key. This implies work on a number of areas:

Rethink the config file to eliminate as many things that can be moved to the admin UI.
Add a way to change the ds-host config without hand-editing a config file (maybe using a CLI or GUI program)
Use operating system defaults for data and working directories instead of asking the admin to specify them.
Find ways of packaging DS so that it can be more easily installed on Linux
Make a build and maybe a package for Raspberry Pi
Make a build for MacOS. ds-host works fine on MacOS, I ran it on that platform throughout the development of the Tailscale integration.
Continue to try to get it to run on Windows. That starts with getting a version of ds-dev to run on Windows, then work on ds-host.

Honestly that’s a lot and I won’t tackle all this at once. In the meantime there are other things I should look at.

Improve user management

Tailscale integration forced some changes in how I think about users (both appspace users and users of the instance). In particular each user can have multiple ways of authenticating (username + password, tailnet ID, etc…). Also, the concept of “DropID” which ties a user’s identity to a domain name no longer makes sense since ds-host instances can be hidden from the public Internet thanks to Tailscale.

It would be worthwhile to remove the concept of DropID from the system (though it is used in quite a few places so that has a domino effect unfortunately), and develop the “Contacts” functionality to make adding users to appspaces easier.

Better backups

One project I’m definitely working on now is to move my own personal instance of ds-host from a VM on the public cloud to my Proxmox instance running right here in my office. This is what Tailscale enables and I’m excited to disappear my instance from the public net.

But this begs the question: what happens if my VM or its host blows up? All of a sudden I am responsible for my own backups 😓. I have 17,299 notes in my personal note taking/journaling app that runs on Dropserver so this is quite important to me!

Of course I set up automated VM backups but it would be nice to have more granular backups of just the Dropserver data. It’s clear that ds-host should come with the ability to generate and save backups of the instance data, or of individual appspace data, to an external source automatically.

Work on more apps

When I work heavily on a capability like Tailscale integration, I don’t have any time left to improve the few existing apps or add new ones. I wouldn’t mind spending some time on this now.

Time Marches On

I don’t expect to get huge amounts done in August with the kids out of school but whatever I do, I’ll try to make it a small project to avoid stretching months between releases.

Dropserver Progress - May 2025

Olivier Forget — Tue, 10 Jun 2025 17:45:00 -0700

This is the progress report for Dropserver for May 2025. The previous report is here.

Quick one this month because I was away on vacation for a good chunk of May.

Tailscale Integration

Tailscale integration is nearly done now. I’m knocking out items on my todo list. The main thing I did this month was to experiment again with the open source alternatives to Tailscale Headscale and Ionscale.

I would really like for Dropserver to work with open alternatives to Tailscale, which is why I spent a decent amount of time trying both of them again.

Headscale

A major blocker with using Headscale with Dropserver is that it does not support serving over HTTPS. This means you do not get a Secure Context, which can affect some apps. In my Leftovers app, the lack of secure context blocks use of the camera, and you can’t take pictures of the food you put in the fridge. Bummer.

I had filed an issue some time ago which resulted in a long discussion but little action (I personally do not have it in me to implement this for the Headscale project, sorry). Now, there is a new issue by the maintainer laying out the roadmap for this feature to be implemented. So maybe Headscale will some day get HTTPS support for serve.

A separate issue I ran into was that tailnet peers are apparently not sent to tagged devices. This is a problem because Dropserver Tailnet nodes are tagged since they are non-user devices, and the list of peers is used to associate tailnet users to Dropserver and appspace users.

Ionscale

I had to fiddle quite a bit to get Ionscale going. There are a number of issues in the docs that slowed me down.

However I did get it working (to a point). Tags were not being sent to the node which caused ds-host to warn that a tag had to be applied to the node for it to work. Luckily this was resolved promptly.

I even got serve over HTTPS to work (though not before filing another issue). Ionscale is able to generate a certificate for your node if you provide it with a DNS nameserver and an API key.

Ionscale requires an OIDC provider to support the concept of a “user”. Wihtout that you don’t get peers at the node, and you have the same problme as with Headscale. I didn’t have it in me to sort out an OIDC situation after all the time spent on this, so I decided to move on. If someone wants to use Ionscale with Dropserver and it doesn’t work despite setting up OIDC, I hope they’ll file an issue.

Docs

I started writing docs for connecting to Dropserver and Appspaces using Tailscale or its alternative open source clones.

Good docs will be important here so I’m starting now.

Naming things

I grew uncomfortable having references to “Tailscale” in the UI when there is a chance that Headscale or Ionscale is used. The term tsnet which is the tailscale library has no meaning for users. I decided to change the UI to use the term “tailnet” as much as possible. I found that this term is used by both Headscale and Ionscale, and it is apparently not trademarked.

I’ll still use Tailscale when it’s directly relevant.

Finally I use it in the title of the UI box: “Tailscale Node”. If a user connects to a non-Tailscale tailnet, the title changes “Tailnet Node”. Tailscale has a lot of name recognition, and I think it will be easier for users to get what that UI offers by using the full brand name.

The Ever Dwindling TODO List

I’m at the phase of the project where the TODO list is no longer growing. In fact at this point it’s pretty short and it’s getting shorter.

I’m not going to say what I hope to have done by the end of June. Nope. Not gonna say it.

I’ll see you right here next month, whatever happens (or doesn’t happen).

Dropserver Progress - April 2025

Olivier Forget — Wed, 07 May 2025 12:05:00 -0700

This is the progress report for Dropserver for April 2025. The previous report is here.

Tailscale Integration for the Admin Side of Dropserver

Recapping where I was: I got Tailscale integration working pretty well for appspaces (an appspace is a running app in Dropserver, see here.) Meaning that you can access the apps that you host on Dropserver via tailnet node in addition to via the regular Internet. Now I want to give access to the admin side of ds-host via Tailscale. This is where Dropserver users go to install apps, create appspaces, and generally manage their Dropserver stuff.

After completing the work of making my tsnet integration code more generic (it was heavily geared towards serving appspaces) it was time to work on connecting a node for user and admin side of ds-host.

Setting up users with tailnet credentials

As with appspaces I decided that access via a tailnet node should be one of multiple ways of accessing Dropserver. A user can have tailnet credentials, and can also have a username and password should they choose to log in via a more traditional network.

I had to make having an email and password optional. If a user connects via a tailnet, the authentication is taken care of, so forcing them to choose an email and pw is just an obstruction. This required some DB changes because of course the email and password columns were NOT NULL and I had a unique index on email. There was also a domino effect of changes in the UI.

Commits f493857 and 420dcdd.

Associating tailnet peers with ds-host users

After pondering on the possible situations that a ds-host admin may face:

They have existing users on their ds-host instance, and just connected to a tailnet. They have to associate peer users from that tailnet with existing ds-host users.
They are connected to a tailnet and want to give access to someone new. This implies creating a new ds-host user with credentials from that tailnet.
They made a mistake in any of the above and have to break associations and start over

I settled on these UI choices:

When looking at tailnet settings, for each peer user of the tailnet, the admin can use a dropdown to select the ds-host user that should be associated, the last option in the dropdown is “create new user”.
When looking at a user, the admin can select which tailnet peer user to associate.

In both cases, the UI respects the requirement that there is only a one-to-one relation between ds-host and tailnet users. One tailnet user can not be associated with two ds-host users, and vice versa.

Commits d629d08 and ca99594.

Connecting tsnet to user and admin routes

As mentioned above a tailnet is but one way to access your Dropserver instance. ds-host has multiple servers: one for responding to requests from the Internet, and one that is on a tailnet and responds to requests from that world.

Both serve routes like the user home page, and the admin pages just the same, but the authentication and authorization steps are different. Also, the “Internet” routes include the login, logout, and user registration pages, which are not reachable while accessing ds-host from a tailnet (because they make no sense there).

So there are two routers, one called FromPublic and the other FromTSNet.

The FromPublic router can serve login/logout routes, and takes a request’s cookie to determine the authenticated user, adds the ID to the request’s context and passes control to the “user routes” router.
The FromTSNet router looks up the user id based on the tailnet peer’s ID, puts that in the context and passes the control onwards.

This arrangement helps me keep things decoupled and clear.

Once I had it all wired up, it worked on the first shot*. I was able to access ds-host running on my iMac from my phone through a tunnel managed by Tailscale. It felt great. I feel like I’m almost there.

Commits 3355234 and 757df05.

Getting close

With that functional, I turned to my todo list which is still quite long but many items are fairly quick. I started by knocking out validations, which was a bit of work because Tailscale’s naming limitations are not intuitive:

Machine names: alpha numeric and dashes are allowed; must start and end with alpha or numeric.
ACL tags: alpha numeric and dashes allowed, must start with alpha (not numeric!), can end in dash 🤷‍♂️

Commit ab6183b.

Bye For Now

I’ll be unable to work for part of May, so the next progress report will be shorter.

*: 11 months of work were necessary to get to that magical “it worked on the first shot”.

Dropserver Progress - March 2025

Olivier Forget — Wed, 09 Apr 2025 12:49:00 -0700

This is the progress report for Dropserver for March 2025. The previous report is here.

Dropserver.org Docs Pages and Modern CSS

I spent some time at the beginning of the month making the Docs pages of Dropserver.org easier to use. In particular I wanted to have the navigation in the sidebar instead of a separate page, this makes casually browsing the docs far more user-friendly.

But this was not just an effort to improve the docs pages. I want to improve my HTML and CSS skills so I took the time to really dig into new techniques for this little project.

Some years ago I got sucked into the whole Tailwind thing, but I have regrets. The ds-host UI is made with Tailwind, and while it was quick to get going and have it look decent, now it feels like tweaking the look of the UI is impossible. Also I dislike having so many class names in the HTML.

So part of this is exercising my atrophied HTML and CSS muscles, so that I can rebuild my Tailwind code some day.

I could have just used Starlight since Dropserver.org is built with Astro but instead I reverse engineered (aka copied) their layout and learned a few things along the way.

It took a bit of time, but I was able to get familiar with new (to me) CSS features like :has() and :not() and clamp() and others.

That’s just the tip of the iceberg. Features like @scope and @layers will really help keep CSS stylesheets manageable, which is the challenge that drew me to Tailwind in the first place.

See the result here: Dropserver.org/docs/. Note the collapsible menu works without JS!

Tailscale For the Dropserver Admin Panel

Work continues on making the admin and user routes available via Tailscale. Yes that’s right all the work I’ve done so far was focused on making tsnet nodes for individual appspaces, so now I have to do similar (but different!) work to make the user’s ds-host control panel accessible from yet another tsnet node.

In order to not repeat myself the first task is to take as much of the code I wrote for serving appspaces via tsnet and make it generic enough so that it can be used to serve other things.

This is progressing well. It’s the kind of work that forces one to think more clearly about the core role of each part of the code. It’s a lot of work, but the result is clearer code.

Commits aa891dd, ea6093a, and aafdce9.

Thinking about publishing a generic library for integrating tsnet?

Having done so much work to figure out tsnet and build out a higher level API around it I wonder if I should publish this as a library.

I don’t have a lot of confidence in some areas of my integration with tsnet, so I would like other eyes to look at it. It’s much easier for Tailscale experts to point out mistakes and offer suggestions on better ways of doing things in a standalone library than in my gigantic Dropserver codebase.

Plus, some projects might adopt it and point out bugs (highly likely) and even contributes fixes (LOL I wish 😓).

Unsure if I am going to do this, but I am putting it out there.

Keepin’Keepin’On

Short one this time. I’d rather spend time making progress than writing about the lack of progress in a “Progress Report”.

See you next month.

Dropserver Progress - February 2025

Olivier Forget — Wed, 12 Mar 2025 11:33:00 -0800

This is the progress report from Dropserver for February 2025. The previous report is here.

This month was extra-short thanks to a much-needed ski vacation, but I did manage to make some solid progress on integrating Tailscale into ds-host.

Tailscale Users as Appspace Users

After grinding on Vue templates, DB calls and everything in between, it’s now possible to assign multiple auth credentials to a single user. This means an appspace user can access the appspace via Tailscale, or via the public network or local network interchangeably.

The UI needs some polish to say the least but the essence is there:

Create a new user or assign the Tailscale identity to an existing appspace user:

See who the peers are and the connected appspace user:

All this works while the Tailscale peers can change at any instant. The data is kept up-to-date on the frontend UI through a system of events inside of ds-host, and Server Sent Events to the browser (see September 2024 progress report). While I don’t love all-JS frontends, Vue reactivity is great for reflecting the latest state as soon as it shows up.

Commit: “feat(ds-host): implement multiple auths per appspace user, including tsnetid” 10f4770

Identifying Tailscale and Headscale Users

Tailscale users have a pretty solid looking ID that I understand is stable and unique. Using that to identify peers is natural, but there are a couple of gotchas.

Headscale user IDs are plain integers starting with 1. So user ids are 1, 2, 3, 4… There can be many Headscale instances in the world therefore there are many Headscale users with ID 1, ID 2, etc… If an admin changes / moves / rebuilds their instance, relying solely on the user ID like I do with Tailscale isn’t enough.

To mitigate this I am doing two things:

ids in the appspace DB are stored as <id>@<headscale-control-domain>. If you change to a different Headscale instance at a different domain, this will ensure that user-1 in one instance is not the same as user-1 in a different instance.
I added an extra_name column in the DB, which stores the email address of the Headscale user. I haven’t implemented this part yet but it should help deal with situations where a Headscale user is not who the system thinks they are.

Commit: “add extra name to appspace user auth” b77fd83.

Tags, Key Expiry and Auth Keys

Tailscale expects non-user nodes to be “tagged” (see Tailscale docs.) As such appspace nodes should be tagged. This triggers the following do-list:

UI to input tag when creating node
read actual tags for the node
disable use of appspace node if there are no tags

With the above done, I set up the UI to warn the user when the node has no tags, and point them to a remedy.

Auth Keys

If you try to create an appspace Tailscale node the ds-host UI will display a link to click, which takes you to Tailscale (or your Headscale instance) where you authenticate to finish creating the node. It’s convenient and requires no prior steps to complete.

An alternative is to create an auth key in Tailscale (or Headscale). Conveniently you can select tags to set on nodes created with that auth key.

I added support for creating a node using an auth key. In many cases this will be a superior workflow for users, so long as they have the forethought of creating an auth key. It’s also possible to create multiple nodes with the same auth key.

Node key expiry

Tailscale nodes expire, but you can disable the expiry date of the node if it’s meant to run unattended like a server. In fact, if you create a node with a tag, key expiry is disabled automatically.

If in spite of the work above the appspace node is created without a tag and an expiry date is in effect, the UI points out the issue along with steps to resolve. (It’s easy to do, but if you don’t know you don’t know.)

With the above, the user should be able to create a node and get it to usable state even if things didn’t go according to plan the first time around. ds-host detects all “gotcha” situations that I can think of and points the user in the right direction to resolve. This is key to get Tailscale working for users who may not be familiar with it.

Commits: “add tsnet tags input and key expiry date to status” 604d0e3 and “create tsnet node using AuthKey” 7f122d4.

Getting CLoser

It feels like Tailscale integration is finally getting closer to being done. I’m punting on a few things that are related but not critical: handling avatars, and better user management.

I am now working on making the ds-host admin panel accessible from Tailscale as well. Naturally it’s not nearly as simple as I would like it to be.

Dropserver Progress - January 2025

Olivier Forget — Wed, 12 Feb 2025 10:37:00 -0800

This is the progress report for Dropserver for January 2025. While the previous report was a wrap up of 2024, this one is a look ahead for 2025.

The Battle Between the Drive and The Grind

A New Year’s celebration adds a new notch to a years-long project. A new year is a time to take stock of how far a project has come along, and what it still needs.

Safe to say that for Dropserver I’m not beaming with confidence. There are a few things making it feel like an endless grind. Some features take forever, like Tailscale integration which is kicking my butt.

Also, after realizing that the problem space is bigger than I had anticipated and my ideas on how to solve it shifts, I have to contemplate doing more work in a direction I hadn’t anticipated.

On the other hand, there are things that drive me forwards: there is light at the end of the Tailscale tunnel, and its integration into Dropserver is key to reaching the project’s goals.

This is no celebration but the current political environment in The US of A (f*** yeah?) is making self-hosting from home something many Americans may want to look into. As dark as that is, it pushes me forwards.

Finally I am motivated because I have realized what I need to do to make Dropserver stand out from the crowd of self-hosting platforms: make Dropserver very to install.

The way forward

I think the most game-changing characteristic I can bring to the table with Dropserver is to make it very easy for anybody to install. This means:

use a cheat code for networking (the cheat code is “Tailscale”)
make ds-host installable on popular operating systems (MacOS, Windows)
use GUI installers and wizards
ditch the config file

There are many self-hosting platforms out there, but most only work on Linux. Any platform that requires Linux has to first convince the potential user that they want to mess with Linux. For many people, that’s a foreign country they’ve never heard about and have no intention of visiting. A real demotivator.

I already build and run ds-host on my Mac. Unlike platforms based on Linux namespaces (aka containers), there is nothing about Dropserver that requires it to run on Linux. So it will be no problem to make a build for MacOS. Windows support will be harder simply because Windows is so different from Linux. But I should get to it because Windows market share is completely ridiculous if you believe StatCounter.

The next challenge will be to get rid of that config file somehow. Ideally the service needs to start via a GUI and live in the tray. No CLI or config file editing should ever be needed.

Further challenges will be to leverage the MacOS and win32 sandboxes to good effect to provide additional protection around Deno (I currently use bwrap on Linux). In addition to all that, Dropserver needs to update itself gracefully.

Note that the no-CLI and no-config approach applies to Linux too. Despite what I said above, buying a Raspberry Pi with the graphical OS pre-installed is within the reach of many. I’ll be exploring a release for arm64 and techniques for making it easy to install on a Pi.

These and a few other adjustments in my approach to Dropserver are where I see things going in 2025 …if I can just ship Tailscale integration already! 😭

Tailscale Integration Work

The current focus of the Tailscale integration work is to merge the users who appear as peers of the appspace node into the appspace’s users.

Tailscale is just one way to access an appspace on Dropserver. As such, a user might access it via Tailscale one day, and via the local network another day. Although this adds complexity, I am convinced that it is required to support multiple simultaneous ways of accessing an appspace, and not be “either-or”. This aligns with how I use 3rd party services when home hosting.

As a result each appspace user can have different “identities” depending on the network used to access the appspace. I have to consider this in all aspects of authentication and user management.

Then, it turns out tailnet users are… different to work with. Like I’ve mentioned before, everything about working with tsnet is an exercise in reversing the flow of data for everything. In this case, users don’t come from my database, they come from the list of peer devices for the appspace node. This is only available when the node is connected and can change at any second. So I have to listen for changes to the peers, deduce the list of users from the devices, and then decide whether that tailnet user is a user of the appspace or not.

Or rather, I have to decide how to decide if the user is an appspace user. It depends on how the tailnet is managed. If someone is carefully crafting their ACLs in the Tailscale UI, then any peer of the node should be authorized. But if they use set-once-and-forget ACLs that treat all appspaces the same they will use Dropserver’s UI to authorize users for individual appspaces. (Some admins may not want to mess with Tailscale ACLs on a regular basis because they are, quite frankly, user-unfriendly.)

Originally I thought that “shared” users would automatically become appspace users because sharing is a deliberate action by the admin, and there is no need to second-guess it. But I had reverse that. Why? Because there is a decent chance that the person is already a user of that appspace, and auto-adding would create two users for the same person. This is obviously bad and confusing.

The goal in all this is to present the appspace owner with a clear view of appspace users along with their identifiers on Tailscale and elsewhere. And then give them a way to manage the Tailscale peers by turning them into new appspace users or adding their identifiers to existing users.

Eventually, to make user management less manual, I will develop the “Contacts” section of Dropserver. Once a user is in the list of contacts then auto-adding becomes possible in some cases. But that’s for later. The first release of ds-host with Tailscale will have a clunky UX, sorry.

The grind

add appspace users from tsnet (unfinished) bc8a63c
change tsnet backendURL property name to controlURL 4b1ea07
remove last_seen from appspace user and user auths 56532b0
refactor appspace users reactivity c767b84

Onwards

See you next month.

Dropserver Progress - Recap of 2024

Olivier Forget — Wed, 08 Jan 2025 10:39:25 -0800

Instead of going over the month of December (not much happened), here is a quick recap for the year in countdown form:

Four Releases

I released four versions of Dropserver:

0.13.0: Install/update apps from URL. See January.
0.13.1 brought a few bug fixes and improvements. See March progress report.
0.13.2 Adds a proxy for outbound requests, and many small tweaks. See May update.
0.13.3 See September progress report.

Three Main Areas of Work

I worked primarily on three new things in 2024:

Outbound requests from appspaces. Failed to ship (see this post)
Deno 2 support. Shipped but caused some backwards compatibility challenges.
Tailscale integration (not yet finished). See progress reports for April and onwards.

Two Dropserver Apps

I worked on the two publicly available non-demo apps for Dropserver:

Leftovers: small tweaks to polish the UX. This app is pretty decent now, a few quibbles aside.
ShoppingList: I made it a PWA and improved the UX for adding items to your list quickly. This one needs more work before I am happy with it.

One Goal

I am building Dropserver in an attempt to figure out how self-hosting personal web services can be brought within reach for non-technical users, or people who don’t want to become hobbyist sysadmins.

I am not where I want to be, and it seems the decks are stacked against this ever becoming a thing. But when I look at Plex, HomeAssistant, and Cloudron, I see that some formulas can lead to success.

Depending on Third Party Services When Self-Hosting

Olivier Forget — Wed, 01 Jan 2025 12:27:00 -0800

In our drive to take control of our web presence by self-hosting our services from home, it seems contradictory to depend on 3rd party services to make it all work. Companies like Mailgun, Tailscale, Cloudflare, Netlify, NextDNS, Linode, AWS, ngrok, and many more offer compelling solutions to the endless challenges of self-hosting. But does using them mean we are fooling ourselves about our independence from big tech?

Here is how I think of things as I decide what to use in my own home, and what I integrate into Dropserver.

Questions I ask myself

Naturally I ask myself if the service trustworthy and reliable, but a “yes” answer isn’t good enough. Services change, get bought, pivot and generally enshitt– No, it’s the first day of 2025, I won’t start the year by overusing 2024’s biggest word.

In any case it’s best to assume the worst and ask questions like these:

What are the consequences if it goes away?

If I depend on a service in my home-hosting setup, what are the consequences of the service going down, or becoming unusable? Is my instance dead? how much work to switch to something different?

Is there a drop-in replacement for the service? Can I actually use the replacement now or is the API different and we need to go beg maintainers of projects to support different services?

Are there less convenient but workable ways of getting around the loss of the service that I can actually implement in a reasonable amount of time?

How much of my data can the service harvest?

I’m sure the third party service says they’ll “never sell your data or use it to train LLMs” or whatever. But it’s just words. We all know how these things go. So what matters is how much of your data is actually visible to them? And are you OK with them breaking their promises with that data?

An email sending service like Mailgun sees the content and the recipient address. Depending on the kind of emails you send, that may be OK. Perhaps you use it to send notifications of mundane events. On the other hand maybe you use it to send out a very personal news to friends, in which case you have to decide if it’s OK for them to have that.

Another factor to consider is whether this data can be protected at all? Netlify may see every page on my website, but that website is public anyways, so it’s likely getting harvested whether I like it or not.

Can the service hold my data hostage?

How much data is held exclusively by that service? If I keep a bunch of files on an S3 storage service then that service’s disappearance might have profound consequences for me. You might say “oh come on that will never happen”. Perhaps, but consider that you might lose access to your account if, say, they accuse you of being a spammer for whatever reason. Your data is as good as gone in that case.

Given that one of the major reasons for self-hosting is data sovereignty, this is a big no-no. If I were to use an external storage service, I’d make sure I have copies of files in a completely independent service or a local drive.

An other service that might hold critical data is newsletter sending services. They usually manage the subscriptions and hold the data for all your subscribers. Make sure that stuff can be downloaded and imported elsewhere, and set up an automated recurring job that downloads that data locally.

Some services will have some data that is exclusively under their control. Ask yourself if you can live without it and act accordingly.

Escape hatches for 3rd party services

Given the above, what might we look for in 3rd party services for our home-hosting setup?

Use generic services

If I have a virtual machine on DigitalOcean and I stop trusting that company, I can create the same VM on Linode. Virtual machines are not a proprietary thing. To make sure I can do this readily, I should make sure data from the machine is backed up outside of DO and that I can recreate the VM quickly using infrastructure as code.

Even easier: many email sending services let you use SMTP to send through them, so with a little config change, you’ve escaped. This is good.

Use services that have open source self-hosted versions

While you probably signed up for a service because it was easy and convenient, if faced with a loss self-hosting their solution is less bad than being completely stuck. It’s an escape hatch. You don’t want to use it, and with some luck you won’t have to any time soon, but you definitely don’t want to be without it.

Use services that let you download data unattended

If the service holds data that you don’t want to lose, only use it if you can reliably and automatically download your critical data in usable form.

Closing thoughts

Personally I find that using third party services to help me with self-hosting is a good move. Their value is well worth the cost of a small inconvenience if I have to switch.

I think of commercial services as more tools in my toolbox. These tools do not displace my existing trustworthy tools, I can take advantage of them, but I have options if they become troublesome. I just make sure I don’t give my whole toolbox to a third party!

Different strokes for different folks

When evaluating the questions above absolutists will end up declining to use every third party service they can, and that’s fine.

My ambition with Dropserver is to (some day) bring home-hosting of personal web services to a broader audience. I’m pretty sure at this point it’s impossible without the help of third party commercial services to simplify the process. The key will be to thoughtfully integrate some services that help home-hosting without defeating the whole purpose.

Dropserver Progress - November 2024

Olivier Forget — Tue, 10 Dec 2024 10:39:25 -0800

This is the progress report for Dropserver for November 2024. Here is last month’s report. This month was all about Tailscale integration. Trying to get this stuff right isn’t easy! Dig in…

Config Conundrums

When I started Tailscale integration I reflexively put some configuration parameters in the ds-host config file.

Silly idea. First, I should not be adding things to this config file. Nobody wants to write a config file before running a new service. I should work to eliminate it not add to it.

More à-propos of Tailscale, it turns out there are no configuration parameters that exist instance-wide. (Well, almost. There is one. More on that some other time.)

If you speedrun through my notes from the last couple of months, it’s a continuous process of moving config values for tsnet nodes away from the instance level, to have them keyed by tailnet, and finally to have them attached to an appspace ID, if not to disappear entirely.

Implementing tsnet is an exercise in completely reverting the flow of configuration from your config file, to have it fed to you by the state of a (running!) Tailscale node.

Take HTTPS for example. For the “normal” server, whether to use HTTPS is a value in the config file that is read at ds-host startup, and is used to decide whether to start a TLS server.

I obviously thought there would be an equivalent config value for Tailscale nodes. Then I realized different users might use different tailnets for their nodes, so it became a per-tailnet setting. Then I realized that it shouldn’t even be a setting because you can’t even have a functioning HTTPS server on a tsnet node unless the backend is set up to enabled this.

By the end of November there are very few configuration values, and they are all per-appspace. The ds-host config file will have nothing related to Tailscale.

Trying Out Ionscale

As I integrate Tailscale into Dropserver, I want to make sure the open-source alternative backends also work. Ionscale is a self-hosted alternative backplane for Tailscale. It appears to have a good number of features, but unfortunately some docs are missing.

The installation docs are actually quite good. Concise, they got me to having an instance of Ionscale on a VM pretty quickly. Unfortunately, the usage docs are non-existent. It’s through trial and error that I was able to figure out how to make it work (I think).

Double unfortunately, as soon as I figured out the command needed, I got a panic. Not great! I filed an issue but after several days of not sharing from the dev, I deleted the VM.

Womp womp.

I’ll certainly revisit Ionscale at some point in the future and hopefully have better luck, but for now I need to move on to more familiar territory.

More Headscale

I had played with Headscale already. It’s more widely used than Ionscale even though it has fewer features (see below). Also it’s the closest thing to an officially supported open source backend for Tailscale. I think the dev even works at Tailscale.

One appspace on Tailscale and one on Headscale?

I wanted to know whether it was possible to have one node connected to the official Tailscale backplane and one connected to my own instance of Headscale. And have these nodes both connected simultaneously from one running Go program.

Knowing this would inform whether the ControlURL for tsnet is an instance-wide setting or not. So I made some experiments…

Turns out tsnet can connect to different backplanes simultaneously. That means different appspaces can be connected to different Tailscale backends (which will be great for Tailscale users who want to experiment with alternative backends) and an instance-wide config for this is unnecessarily limiting.

Also poof! 🪄 one less config a the instance level.

Headscale HTTPS serve issue

One bummer about Headscale is that it does not yet support serving over HTTPS. This means that an appspace connected to a Headscale instance is only accessible over plain HTTP. Of course, the connection is encrypted since it’s running through a WireGuard Tunnel. But if you open the link in the browser, it tells you it’s not secure.

This is problematic because many advanced web features require a Secure Context to be available. Without that, no PWA, no notifications, no camera access, etc… The Leftovers app would lose a lot of usefulness without HTTPS.

I opened an issue to add this feature, unfortunately right now I don’t have the time to implement it.

MagicDNS, TLS, and User Experience

So far in my Tailscale experiments I assumed that MagicDNS was on and HTTPS enabled. But that’s not a given. Even if the likely outcome is that someone using Tailscale to serve Dropserver appspaces will have these features on, they may not be on at first.

If a user has just signed up to Tailscale, these features are off. They may not be reading my instructions carefully (a safe assumption to make). Dropserver should connect to the tailnet with what’s available, and make clear to the user that they can get HTTPS and a URL by going in and enabling MagicDNS and TLS certs.

What’s more, since tsnet is good about forwarding network configuration changes to the client promptly (this is inherent to how Tailscale works) Dropserver should notice these changes and act accordingly.

The goal is that when a user makes changes to their Tailscale (or Headscale or Ionscale) configuration, the nodes managed by Dropserver instantly “do the right thing”. If HTTPS becomes available, then it starts ListenTLS() and if it becomes unavailable it stops that listener.

This means the user just needs to flip a couple of switches in the Tailscale admin UI and the ds-host admin panel updates to show them the full HTTPS URL for their appspace. No interaction required. No “…and then go restart your node” in the docs.

Commit 5d2114.

What’s Next

This month I will soon finish the config side of things (for appspaces at least) and I can move on to how to handle users, which should be fun.

Dropserver Progress - October 2024

Olivier Forget — Wed, 06 Nov 2024 13:06:25 -0800

This is the progress report for Dropserver for October 2024. Here is last month’s report.

Dropserver 0.13.3 released

As I mentioned last month I released ds-host and ds-dev version 0.13.3 with support for Deno 2.

I updated to 0.13.3 on my personal instance of ds-host, which happened without a hitch. I am still using Deno 1.x because I haven’t updated all my apps to work with Deno 2.

Update Leftovers and Shopping List for Deno 2

The changes to Leftovers and ShoppingList for Deno 2 were trivial in nature (though I still wish there were none). I did have to submit a one-line PR for deno-sqlite that removes a TypeScript error. The patch was merged and a new release tagged in a couple of days so I could use it right away. Nice!

Commit ba5446a for Leftovers. Commit 51b753c for ShoppingList.

I packaged and published each which reminded me that an automated packaging and distribution system using GitHub Actions (or similar) would be a nice addition.

Getting the new versions installed on my instance was trivial thanks to the work I did earlier on app distribution and installation.

ds-dev Screencast

I upgraded my screen-casting setup this Summer, so it was time to record my first proper video. It walks you through some of the experience of using ds-dev to develop an app for Dropserver.

Have a look at it in the app development tutorial page here.

I realized after publishing it and listening to it using headphones that the keyboard makes a booming sound when I type on it. I’ll try to fix that for the next one.

Vacation 🏖️

I took one week (plus weekends) off this month for a family reunion. One week is short enough that when I get back to work it doesn’t feel like I have left too long, so I was able to get right back to it.

Back to Tailscale Integration

Last month I thought the next big task would be to capture user info from the tailnet backend and merge that with appspace users. This is definitely looming, but there are a number of other details that need to be sorted out first.

I started on some basics like capturing various aspects of node status and reflecting it live on the ds-host frontend. This helps me understand what is happening with the node as it goes through various states. I’ll package the UI up better later, I consider this debugging output for now.

I’m finding Tailscale integration to be tricky to get right because data about each node (such as its URL, ability to use HTTPS etc..) comes from the Tailscale backend, and can change at any time.

Example: when creating a node, you can set a host name, like “leftovers” and the resulting URL will be https://leftovers.tailxyz123.ts.net. But! it might actually be leftovers-1.tail... Or it might start as “leftovers”, but it can be changed at any time from the Tailscale UI. So ds-host can express preference but it’s the backend that controls and can change things at any instant.

This is one reason I spent time last month working on a more solid events system using Server-Sent Events and I am taking the time to build out the events that carry Tailscale data all the way to the frontend. Things can change at any second, and I don’t want users to get into confusing situations because the data they see in the Dropserver UI (like the URL of the node) is no longer correct.

I am getting a handle on all of this. In fact I recorded a quick screencast to show the dynamic URL in action:

(Note the lack of booming keyboard sounds thanks to this trick!)

I’ll be doing more of this kind of work: absorbing data from the tailnet and merging it with ds-host’s view.

Keep On Keepin' On

Difficult day today in the US. But we keep going. One day at a time. See you next month.

Deno 2 and Backwards Compatibility in Dropserver

Olivier Forget — Thu, 10 Oct 2024 11:45:00 -0700

Deno 2 is here. My project Dropserver, an application platform for your personal web services, uses Deno as its app sandbox.

Deno 2’s arrival forced me to break backwards compatibility in Dropserver, which is something I really don’t like doing. Here’s an explanation of why this happened, how it chafes with my vision for Dropserver, and how I plan to avoid this in the future.

Deno code in Dropserver

Dropserver is written in Go, but it runs apps written in JavaScript (or Typescript) inside Deno. Dropserver injects code in the Deno sandbox to help run the app. This code ships with Dropserver to guarantee compatibility between the Go side and the Deno side.

To make it easier to write apps I publish a dropserver_app library that serves as a bridge between app code and the Dropserver code inside Deno. App devs import this library and call its exported members to interact with the Dropserver system, such as creating routes or fetching the authenticated user, etc…

To sum up Dropserver runs JavaScript in Deno:

code that ships with the Dropserver executable
code that I publish as a library in deno.land/x/
code written by app devs

If the only code was the one that shipped with Dropserver, I would have no problems: I would patch that and ship it and tell people to upgrade. But that’s not the nature of an application platform, is it? The idea is to run other people’s code, so when things change, how do you allow old apps to keep working?

What changed in Deno 2

Several types of changes took place:

Typescript version changed
Global space changes
Deno namespace changes

See Deno’s migration guide for full details.

TypeScript changes

Deno comes with TypeScript built-in, which means your version of TS is tied to your version of Deno. And TS makes breaking changes. In particular, in TS 5.6 errors in catch blocks are type “unknown” and you have to assert them before using them. So I went over my sandbox code and asserted error types. I changed this:

//...
catch(e) {
    // do somethign with error
}

…to this:

//...
catch(e) {
    if( e instanceof Error) {
        // do somethign with error
    }
}

I even had to submit a tiny patch to the deno-sqlite library because of this TypeScript change. Yawn. It’s boring menial labor, but if you don’t do it, TypeScript complains.

Deno changes

Deno removed window from the global space. So all references to window cause an error. This is trivial to fix: change window.foo to globalThis.foo.

They also moved many APIs from the Deno.* namespace to the standard library. In all cases the change to make is trivial, but if you don’t do it, you get a fatal error.

All these changes add up to a situation where I can’t expect Dropserver apps developed under Deno 1 to work in Deno 2 unchanged. And that’s a bummer.

Dropserver Vision

My vision with Dropserver is that an app you start using should continue to run for as long as possible. I would like a 10-year shelf life for apps, even if the developer abandons it. This is hard to pull off for many reasons, but it’s not impossible. I have been writing JavaScript for web browsers for nearly 20 years, and a lot of the code I wrote back then still works, unmodified, today.

While Deno follows web standards which promise stability, they also have their own APIs which means that there are going to be changes. Given Dropserver’s deep dependence on Deno for the app runtime, that’s rough, but I know how I got here.

Software changes

Note that I don’t blame Deno here. They went 4 years on 1.x. That’s pretty calm, especially for a VC funded team. If anything, I find that they are very careful with breaking changes. The changes needed are trivial to implement. By comparison, PHP breaks things every year, and PHP 8 was awful to migrate to (ask me how I know!) Over in Python world, it sounds like Python 2 to 3 was miserable.

The reality is software changes, regardless of where its funding comes from and what guides its decision-making. So the job is to deal with the changes.

Vision violation: Deno 2 is a breaking change for Dropserver

I had initially hoped I could just cruise into Deno 2 with no breaking changes (meaning that no changes would be needed to old apps to run on Deno 2.) After all I make a big deal of all the layers I have in place to allow these kinds of transitions.

Unfortunately the changes to TypeScript mean that even if I were able to patch over all the Deno changes, any TS checking would probably raise errors in existing app code. So there is no point in trying to make this non-breaking.

So does that mean my vision is unrealistic? No. But I need to make some changes to be better prepared for future changes.

How to avoid breaking changes in a breaking changes world

Here are some things I will do to make Dropserver more resilient to changes in Deno: (note that I use Deno 1 and 2 to illustrate the ideas, it’s obviously too late to do these things for v2!)

No TypeScript checking

Right now deno runs with --check=all when installing an app. The idea is to ensure that the types in use are compatible between the sandbox code (shipped with the ds executable) and the app code (written externally.) If TS raises errors, there is a good chance something went wrong somewhere, and that things will likely error out unexpectedly.

On the flip side, TypeScript is prickly, never really stable and error-prone. It’s possible to get TS errors on perfectly OK code (see the catch blocks above).

So TS checking will be turned off. If an app misbehaves or errors out, maybe the user will be able to enable it on a one-off basis to gather errors to send to the app developer (or to me 😬).

Monkey patch old Deno features

Since Dropserver’s sandbox code runs first, before the app code, I have some freedom to monkey patch the environment that the app code will see.

I could have re-added the window global to the sandbox, and patched the Deno namespace to implement the removed features. But this can cause problems: these were removed for a reason, and forcing them to stay could cause conflicts in the other direction.

So the correct thing to do is to monkey-patch, but only if the app is designed for Deno 1. The problem is I don’t have any way to know if an app expects to run in Deno 1 or Deno 2.

Add Deno version to app manifest

I’m unsure the exact way to approach this, but the app’s manifest should probably give an indication of what runtime it definitely works with. Maybe the app packaging code could inject "deno-version":"1.46.3" to the dropapp.json to represent the version of deno in use at packaging time. If that’s what’s on the system, then that’s what the app code works with, right? Then ds-host can read that and determine what features it needs to monkey-patch.

Or perhaps I need to have automated testing of the app as part of ds-dev, and have a field called tested-with in the manifest.

Let Dropserver control the Deno version

Ultimately, I think Dropserver will download its own copy of Deno and use that to run the sandbox. There are several advantages:

Simpler installation of ds-host, no need to also install Deno
Safer. DS could check for new versions and always run the latest release

It could also mean having multiple versions installed. In the current case, we’d have 1.46.3, and 2.0.0. And if the app says it was built or compatible with 1.x, then use that.

Then, when 1.x is no longer supported, Dropserver would fall back to monkey-patching Deno 2 so that it works like Deno 1 for the apps that need it.

Back to today’s reality

I shipped updates to Leftovers and ShoppingList. The user-facing changelog says this:

This release updates the code to support future software releases,
in particular it supports "Deno" version 2.

🤢 🙈

Dropserver Progress - September 2024

Olivier Forget — Mon, 07 Oct 2024 15:15:25 -0700

This is the progress report for Dropserver for September 2024. Here is last month’s report.

Going Off On a Tangent from Tailscale Integration

Tailscale integration is progressing. I am able to read information about the network, such as connection status, peers, etc… This data is dynamic and changes when the tailnet admin shares a node with someone, or a user connects a new device, etc… Some of this data has implications for the ds-host appspace owner. In particular, if an appspace node is shared with someone new, that person should be considered a user of the appspace, and that should be reflected on the frontend.

In 2024 you should not need to reload your interface if something changes externally. The UI the user is looking it should update dynamically to reflect the current reality. As such I am going to have to pipe some of that dynamic data from the tailnet to the frontend. This isn’t a big new thing for ds-host, I already have a rudimentary event system on the backend, and I use my self-developed protocol called “Twine” that can carry these events to the frontend. But…

Twine is overkill for frontend events

Twine is meant to offer advanced two-way communication between two running programs over a network. With Twine you can create subscriptions to a specific event and then unsubscribe. Twine is powerful, and I think I have some good ideas in there, but I’m realizing that frontends are best served with simple event streams. You don’t need powerful two-way communication for a typical frontend. Better to just get a firehose event stream.

Furthermore, Twine is under-developed: there are painful design mistakes, it’s cumbersome to use and debugging issues is unpleasant. I think it could be quite good but I would need to put a lot more effort into the protocol and the implementations for it to be a nice thing to use. Right now it’s just not, and I have too much on my plate to give it the attention it needs.

Send events to the ds-host fronted using Server-Sent Events

I decided that before sending events from the tailnet to the frontend, I should make sure I’m happy with how I do this. I don’t want to write more Twine code if I know I want to eliminate Twine. I don’t want to have two systems either: one Twine-based and one SSE-based. So I bit the bullet and decided to remove Twine from ds-host, at least for frontend events. (The communications between the sandbox and ds-host take place over Twine. In this case the two-way nature of it is useful. For now.)

To simplify things as much as possible, I decided that there would be a single SSE endpoint that would send events for all things that the logged in user might potentially be interested in. That doesn’t mean the user will see all events. The frontend may ignore some, or simply make a change to reactive data that will update in the UI if it is currently rendered. In other cases, like if the application they just started installing has completed, they might get a toast or something. The point is the frontend doesn’t subscribe to specific events. It’s a firehose. But the number of events should be small enough that it won’t matter.

Making all these changes forced me to go through all my backend event code. I refactored a number of areas. Events on the backend are more consistent now. They use the same generic base structs which have decent test coverage. I also found that some older parts of the code used events differently, so I made things more uniform throughout.

Commits 84b5dc8, df43dfa, 1a5eb34.

I’m amazed at how little JavaScript it takes to handle events on the frontend. I cut out a lot of code by dropping Twine. At some point I want to rebuild the frontend of ds-host to be plain server-rendered HTML pages, with some JS to make the dynamic parts live, and this lean events system is a better place to start.

One downside of SSEs is they take up one connection under http/1.1. But under http/2 that’s not an issue. ds-host should usually be served over h2, and even if not, it’s not a very request-hungry app, so it shouldn’t matter.

Deno 2 Drops

Deno 2 finally dropped! At least in release candidate form. So it was time to start testing and adapting my code to make sure everything would continue to work.

I made a number of changes to accommodate Deno 2. Most of the changes are minor, but existing apps need to be upgraded individually to work with Deno 2.

At the very least your app code should import version 0.2.2 of the dropserver_app library.

My vision for Dropserver is that apps should not need much maintenance, so asking devs to make changes to support Deno 2 bugs me. But it’s not easily avoidable right now. I’ll write more about Deno 2 and how Dropserver can better handle this kind of situation in a separate post.

Dropserver 0.13.3 Drops

On Oct 1 I tagged 0.13.3 of Dropserver which works with Deno 2 and includes the SSE changes mentioned above plus other commits that had accumulated lately.

There is no downside to upgrading to Dropserver 0.13.3, but I’d wait to upgrade your Deno install until you are sure all your apps support Deno 2.

Having said that it’s trivial to downgrade your Deno back to 1.x if things don’t work. (Trust me, I up/down-graded my Deno install a bunch of times over the last couple of weeks!)

What’s Next?

With the SSE work done, I’ll go back to the Tailscale integration work. The key thing is to make sense of the users from the peers list and merge them with appspace users. It’s going to be a bit messy

Tune in next month for the gruesome details.

Dropserver Progress - July and August 2024

Olivier Forget — Sun, 15 Sep 2024 12:05:00 -0700

This is the progress report for Dropserver for July and August 2024. Here is last month’s report.

Work on Tailscale integration is ongoing, but before heading off on vacation I took some time to improve a Dropserver app that I use daily.

Work on the ShoppingList App for Dropserver

I finally put a bit of polish into the Shopping List app for Dropserver and released it:

https://shoppinglist.olivierforget.net/

My wife and I had been using an earlier version of this for well over a year now and it was finally time to make it better.

When working on home-cooked apps, I like to use them without making changes for long periods of time. After so many months, it’s quite clear what needs to get worked on. When you work on your little app once per year, you make the best of that time by working on the things that really make a difference.

The result is a leaner app that performs the tasks it needs to and nothing else. No time to build up bloat.

Among the issues I tackled this time:

made it a PWA (essential for an app that is used daily)
improved search for items
make it much easier to add an item in an ad-hoc way
add selectable filter to the items list (all items, check stock items, items to buy)
UI tweaks and more

Already my wife is much happier with this version (win!). I am also pretty happy with where it’s at, but I can already see the outlines of the next things I need to add. However I won’t touch it for several months, and if I still feel the same then, I’ll go ahead and add those.

Tailscale Integration in ds-host

As mentioned before, integrating Tailscale means changing lots of things in Dropserver. There are lots of assumptions baked into the Dropserver code that get blown apart when working with a tailnet. As such, a lot of the work of integrating tailscale is backing out of those assumptions. This is a good thing overall. The Dropserver code will be more flexible and able to adapt to new features down the line thanks to the work happening now.

One specific thing that has to change is the schema of the appspace meta DB. It needs to support one appspace user signing in to the appspace using completely different identifiers (one is a tailscale ID, the other might be their Dropserver ID or email address). Since it’s the first time I change this schema I have to sort out how when and where these databases get migrated. I started work on this in June and finished in August (see commits b320d6e and 8f5ab79).

With that work done, I turned my attention to tsnet, the embeddable tailscale library, and the vast array of methods and types that might deliver the information I’m looking for from the tailnet:

is the appspace connected to the tailnet?
is it “logged in”? Or does the user need to visit a tailscale URL to authenticate the appspace (which happens if no Auth Key is used.)
who are the peers of this appspace node, and who among them can be considered users? (Meaning: who can I show in the appspace’s list of users and who gets authorized when a request arrives?)

Some of this was not completely straightforward to sort out, but I’m happy to say I made good progress, and I’m starting to look at putting all this together in an implementation.

Deno 2.0 Heating Up

A lot of activity on some issues I subscribe to on Deno’s Github project. It appears they’re getting ready to ship Deno 2.0.

My most desperately needed feature is allow-net-fetch or something like it. The Deno crew looked at it and deemed it was not a breaking change, therefore it will not make it in 2.0. However they seem to say they get the use case for this? So maybe later? For now more waiting…

I don’t know if they’ll make changes to --allow-net for 2.0. It would be wild if they didn’t. If ever there was a thing to change in the CLI it’s that set of permissions. See my post about allowing outgoing net requests. Also see this issue about how the net permissions don’t distinguish between listen and dial.

When the first release candidates for Deno 2.0 come out I will likely pause my work on Tailscale integration to make sure Dropserver works with Deno 2! So expect that to happen.

It’s no bad thing. I have a few little fixes and improvements I can ship along with 2.0 support. And tailscale integration is going to run long, so a release will be in order anyways.

Moving Forwards Slowly But Surely

Since I got back from my trip in early August, I can’t say I’ve been super productive. I’m actually at about 60% productivity (based on the number of Pomodoros I complete per week) and a lot of that is spent working on non-Dropserver stuff.

So things are moving slowly, but they’re moving. Hopefully I can rev things up a bit now that the kids are back in school.

Til the next one..

Dropserver Progress - June 2024

Olivier Forget — Fri, 05 Jul 2024 16:38:00 -0700

This is the progress report for Dropserver for June 2024. Here is last month’s report.

I started on the Tailscale integration, which began by getting the tsnet package, which resulted in a shock.

Go get tailscale.com/tsnet … 😲

After entering the command the machine churned for several minutes while unfurling a seemingly endless listing of dependent packages required by tsnet. Yikes.

tsnet loads lots of dependencies from the tailscale.com package which is expected. But these in turn load some surprising stuff. In particular, Dropserver now depends on the AWS SDK because tailscale.com/ipn/store has an option of storing stuff on S3.

Having unused dependencies like that isn’t great, but I could live with it. And the reality is I’m likely going to end up depending on AWS SDK to allow pushing appspace backups to S3-compatible storage some day.

But I got a shock when I compiled ds-host with tsnet: the built size went from 28 megs to 47! Yikes. Hello bloat. 😞

tsnet creates an entire networking stack in user-space, so it’s not surprising that the build got bigger, but this is a lot. For this and other reasons, I am going to try to make it possible to build a Tailscale-free version of ds-host.

Integrating Tailscale (Easier Said…)

It’s easy to say I’ll integrate Tailscale into Dropserver. But there are a number of decisions to make on how this is actually going to work.

I’m realizing that there are likely two types of people who may want to use Dropserver via Tailscale: those who have a Tailnet in active use and those who will use Tailscale for the sole purpose of simplifying their self-hosting adventure with Dropserver.

I have to consider that active Tailscale users will not allow Dropserver to do anything dubious in their Tailnet and they will likely control things themselves in the ACLs. Meanwhile, users without Tailscale may not know much about ACLs and may want to minimize their interactions with the Tailscale admin panel.

In either case I want to preserve the ideals of Dropserver while making Dropserver nodes “good citizens” of a Tailnet. And as always security is critical. Meeting all these goals won’t be easy.

I’ll write a full post on the integration when I am further along, but here are some quick gotchas that I’m up against:

Tailnets do not support subdomains for a node. Woops! Subdomains are the common way of creating an address for an appspace. There is a workaround: each appspace will be its own Tailscale node.
ACLs control who can access each node and are edited by hand in a JSON file on the Tailscale website. One goal of Dropserver is to not have to hand-edit config files when hosting a new app or adding a user!
It’s possible to edit the ACLs via an API, but there are no granular permissions. An API client that can edit the ACLs can make an absolute mess of your tailnet. I don’t imagine anybody that actually uses Tailscale for things other than Dropserver giving DS the keys to their tailnet. No way.

Despite these (and other issues) I think I see a path forward, but I’m only partway through a functioning implementation so I am not confident I’ve got it right. Stay tuned.

Appspace Meta DB Versioning and Migrations

As mentioned in last month’s progress report integrating Tailscale means changing the Appspace Meta DB schema, where appspace users are stored. This means that for the first time I will have different schemas for the Appspace Meta DB in the wild. A backup of an appspace that is sitting in my Downloads folder will likely be at versions 0 (sans Tailscale). All currently used appspaces will have their Appspace Meta DB at version 1.

I need to work through that situation while preserving the goal that Dropserver should “just work” and users should not get weird error messages or become stuck unable to use their data.

To handle this Dropserver will determine the schema of any Appspace Meta DB it comes across and act accordingly. It will migrate up to the current version when appropriate. Some work alredy done:

Change migration code for Appspace Meta DB. Commit 8fa902.
Add table to allow multiple auths in Meta DB. Commit 66271e.

Next I need to work migrating all Meta DBs when the ds-host instance changes, and performing migrations when necessary when importing appspace data from backups.

What’s Next?

I’ll keep working on Tailscale integration!

However between Summer vacation and non-Dropserver work my progress will continue to be slowed. In fact I’m going to call it now: there will be no Progress Report for July as too little will get done. I’ll be back at it in August. See you then!

Dropserver Progress - May 2024

Olivier Forget — Mon, 10 Jun 2024 18:09:00 -0700

This is the progress report for Dropserver for May 2024. Here is last month’s report.

May was a “Spring cleaning” month. Before launching into my next big project (Tailscale integration) I want to have a clean code base, or at least clean out the obvious relics of ancient ideas that never fully blossomed.

Here we go:

Remove Unused Appspace DB Code

Back in ancient times Dropserver apps could store data in a sqlite DB that was managed by Dropserver. Reads and writes went through a rather lame API from the sandbox side via a decidedly unfinished protocol. I disabled this a long time ago and recommend app devs use a WASM build of SQLite directly in their app.

Yet there was quite a bit of code left over from this “Appspace DB” that remained to this day. No more!

Cleaning this up was important because the Appspace DB code is easily confused with the “Appspace Meta DB” which holds data about the appspace and is going to get a lot of attention when I work on Tailscale. integration (see below).

Rethink Backwards Compabilitlity

Tailscale integration will bring new data types to the Appspace Meta DB, where users and their access types and ids are stored. This means I’ll have two versions of appspace meta DBs in the wild: the original version, and the new one with the ability to store tailscale data.

This launched me down a path of rethinking how to do backwards compatibility in Dropserver. Its important to get it right, and I’m pretty sure I got it wrong, so a rethink is in order.

See the post Building Backwards Compatibility into Dropserver for the latest thinking.

I don’t mention the Appspace MEta DB in the post, but the implication is that there is only one version of the Meta DB understood by a given version of Dropserver.

As such, the host will migrate the Meta DB of any appspace it encounters to its current version.

This means that if you upgrade ds-host and it supports new features and a new MetaDB schema, then all the appspaces get migrated to this schema. If you import a backup of an appspace and it needs to be migrated, that will happen too.

Should you want to use an appspace that was migrated on an older ds-host, then down-migration tools should be available too, possibly within ds-host itself.

With that in mind I was able to eliminate a lot of v0 code, which makes the codebase feel cleaner. I still need to write the Appspace Meta DB migration logic though.

Deno Config and Import maps

In addition to cleaning up the code, I have to keep an eye on the evolution of Deno to make sure I don’t fall too far behind.

With the advent of JSR I am feeling more pressure to support custom import maps for Dropserver apps. JSR leans on import maps to make adding packages more convenient, and I should support that. I spent some time exploring all the ways Deno uses import maps, and looked at the spec carefully and ran a number of experiments.

Dropserver uses import maps to prevent static access to files outside the allowed directories (see this issue comment). Allowing custom import maps means being careful not to give the app any ability to read a file outside of its direcotry.

I’m happy with what I learned and I almost started implementing this but something happened that made me put it aside for now.

File Descriptor and Goroutine Leaks

My personal Dropserver instance stopped working all of a sudden. Turns out I had too many open files? This was frustrating because that is not the first time it happens and I thought I fixed it last time. Evidently not.

Dropserver is supposed to run for a long time without needing any attention. When it blows up like that it’s really disruptive. These kinds of headaches are why people don’t want to run their own services. It’s a pain to have to deal with them when they don’t run.

My hunt for leaking file descriptors and goroutines was more thorough this time: see commits 6e0f48, 1dd6bc and 9f442f. I might write a blog post on what I learned along the way.

Release Dropserver 0.13.2

By now I’ve accumulated a number of minor improvements and code changes, so it would be good to push that out to the world.

Prior to shipping, a routine upgrade of node modules turned into a hour-long battle with my Tailwind and PostCSS and package.json files. I swore off Tailwind a while ago, but I have yet to rewrite the Dropserver frontend using regular CSS, so I’m subject to it ruining my joie de vivre for now. (The error was “Vite CJS Node API deprecated”; see this commit for the fix.)

Version 0.13.2 is finally out, it includes a proxy for outbound requests from the sandbox that I described last month, the fix to goroutine and fd leaks, and a few other things.

Compile ds-host for MacOS

For the Tailscale integration work I thought it might be easier if ds-host was running directly on my Mac. ds-dev already runs fine on the OS, so why wouldn’t ds-host?

One little commit later and ds-host runs on the Mac too! With the config set to just serve over localhost, I can access it via dropid.localhost and appspaces are accessed at their own subdomain like leftovers.localhost (Except on Safari, which doesn’t support subdomains on localhost 😠).

This is encouraging. I’ll need to make some tweaks, and try to get ds-host to run under the MacOS sandbox since Bubblewrap doesn’t work in this environment (On Liunx Bubblewrap is a second layer of sandboxing in addition to Deno) but I can foresee a future where I offer a build of ds-host for the Mac.

I am so excited about this I checked the price of used Mac Minis: under $100 if you’re willing to live with 10 year old spinning disks! Might want to spend a bit more, but self-hosting Dropserver on a Mac Mini would be a great option for people who know Macs and are intimidated by Linux.

Onwards

I just started work on the Tailscale integration. A tailnet is a rather different environment than the global internet, and as such a number of assumptions I naively baked into the Dropserver code are going to have to be rethought. So this is going to take some time.

In the meantime, I just looked at my Grafana dashboard on my personal instance and the goroutines and file descriptors lines are staying under control. Pfew!

(Although I know of some situations where I still have problems, like upgrading an app causes leaks. That’s for another day!)

See you next time.

Remembering My Time With Dick Rutan

Olivier Forget — Mon, 03 Jun 2024 14:47:00 -0700

Dick Rutan died earlier last month. This caused me to revisit a time in my life when I worked for him, flew with him, and shared in the misadventures of a failed project with him.

I worked for Dick from the Summer of 1998 to Spring 1999 while he was on his second attempt at being the first to go around the world in a balloon non-stop and non-refueled. He was already famous for his circumnavigation of Earth in the Voyager in 1986. That aircraft hangs in the National Air and Space Museum in Washington DC where I grew up. I spent a lot of time in that museum in my youth and already imagined myself as an aerospace engineer, so any Rutan project was tantalizing.

In 1998 I was studying engineering in Montreal. When I heard of Dick’s ballooning ambitions I traveled to Mojave to try to land a job on the project by just showing up.

It worked.

Here are a few vignettes from that era.

Dick the scamp

Dick was due to give a talk at the EAA Museum during Airventure 1998 in Oshkosh, WI. I was tagging along with him. We arrived a bit late at the venue and we entered through the main doors at the back of the auditorium. The crowd was already thick and facing the stage anticipating the arrival of the aviation legend. Since they were all facing away, nobody noticed our arrival. Dick moved forwards through the mass of people. He paused behind someone and muttered “Do you think he’s even going to show up?” then moved ahead. He did this to a few people, saying things along the lines of “How long is he going to make us wait?” and “Who does this guy think he is?” all the while moving ahead as if nothing had been said.

His victims were quite amused by their brush with the pilot after they figured out who was saying these things.

Dick says things

At that same Oshkosh event he had been talking mostly about his experience flying the Voyager. That is what he was known for. I remember asking Dick if the “Balloon Project” was going to be as great as the Voyager project.

He said: “It’s going to be even better.”

The “Balloon Project” was a failure. His first attempt ended with him bailing out before his balloon crashed and burned somewhere in the Midwest.

The second attempt was perpetually held in limbo state by lack of funds. It seemed nobody wanted to give us any money (I wonder why?). My emails to my parents are rundowns of all the ways in which we failed to unlock some much needed funds from rich people.

Dick had choice words for all these people and organizations who had all the money but none of the spirit and tenacity and courage that he had in spades.

But this limbo state also made my experience special. Had we gotten funding, we would have hired a bunch of people and would have gone into a pressure-cooker work environment. I had very little experience about anything at the time, so I would have been just one worker bee among others. It wouldn’t have been the same.

The way things worked out the project was at best half-dead until it was fully dead. I was the only person working apart from a tiny team of long-time Rutan acolytes. I talked directly with Dick and crew chief Bruce Evans on a regular basis. Dick called on me for various things, including helping him fly his hot air balloon.

The result is I got a meaningfully close look at the man, and I am grateful for it.

Dick Rutan, me and Bruce Evans in front the habitable high-altitude ballooning capsule I was helping to build.

Adventure is the essence of life

He wrote this on a poster of the voyager he gifted me: “Adventure is the Essence of Life.”

Even after hearing many of the stories first hand, it is still striking to read his obituary in the media. His collection of adventures lived are something to behold.

He could have stopped after Voyager and just talked about that on the speaking circuit for the rest of his life. But that’s not Dick.

One day he was sitting in the office in the balloon hangar bemoaning the state of affairs. He must have just been rejected for funding yet again and must have really felt that his chance was slipping away.

He said somberly: “I would hate it if the greatest thing I’ll ever do is behind me”.

He was over 60 at the time and already an aviation legend. But this really bugged him. He was made to go further and higher. What do you do after you get there?

“Life is a shit sandwich”

I remember a late evening meeting with a representative of a very big company headed by a very rich guy. We were on our last gasps but had high hopes. The funds would amount to a rounding error in their books. After the main meeting, Dick spoke alone with the rep.

I was poking around in the shop, heart in my stomach as I waited for the news.

He emerged gloomy. Burned again.

He found me and said: “Well Oliver, life is a shit sandwich and every other day you have to take a big bite.”

So much for optimism. Earlier in the project he had been answering any doubt about our chances with the mantra “it’s darkest before dawn.” We were now in the “shit sandwich” phase.

About a month later the Breitling Orbiter completed its circumnavigation of the planet and Dick’s ballooning ambitions were forever put to rest.

The Certified Flight Instructor

After the project died for good Dick offered to give me flying lessons. He would get me to my first solo in a power plane (I was already a solo glider pilot). After he died I spotted a number of people on social media reminiscing about their time learning to fly from Dick.

He loved to share the art of flying.

This piece of plastic is historic (or not)

One day towards waning days of the project he said he wanted to give me something. It was a piece of transparent plastic. “It’s from when we made the windows for the Voyager. You could go to the National Air and Space Museum, hold this up to the window of the Voyager, and it will fit perfectly.” Supposedly as they cut the pieces of plastic that made up the window, he kept a piece of discarded plastic as a keepsake.

I have no idea if this piece of plastic is actually from the Voyager. Maybe he kept it for fifteen years and finally decided to part with it, or maybe Dick just felt like seeing my eyes light up for a piece of trash. I wouldn’t put it past him, the scamp that he was.

That plastic may not have been from Voyager but it was definitely from Dick: full of optimism and historical aspirations but it requires one to be firmly grounded.

Goodbye Dick

Well Dick, you’ve taken the last bite of your shit sandwich.

Thanks for letting me tag along during a tiny sliver of your incredible life. It may not have been the most glorious part of your life, but for me it was a highlight.

Building Backwards Compatibility into Dropserver

Olivier Forget — Mon, 20 May 2024 17:45:00 -0700

An unfortunate pitfall of modern computing is to be forced to make the choice between upgrading an operating system and continuing to use a beloved old app. Forcing this on users brings out some choice words for the developers of the OS, but a system that can evolve to its full potential while running old code is hard to build.

I would like for Dropserver, an OS of sorts, to continue to run old apps for as long as possible even as it evolves. Here is how I’m thinking through it.

Layers

There are several layers of APIs between the application code and the core Dropserver internals. These layers provide opportunities to make backwards compatible adapters, or to hide the ugliness of aging legacy APIs. Here is the list of layers as things stand:

dropserver_app library. Imported by the app to interact with the Dropserver system, as shown in the tutorial.
dropserver_lib_support compatibility layer is a dependency of both dropserver_app and denosandboxcode. It’s a bunch of TypeScript types that ensures the layer above and the layer below can talk to each other when they are run together.
denosandboxcode ships with the Dropserver host, and is therefore part of its repo. It runs in the sandbox alongside the app code. It communicates with “Sandbox Services”.
“Sandbox Services” in Dropserver host. It listens for queries from the sandbox side and passes these on to the appropriate data model or whatever.
Data models and controllers in Dropserver host. These talk to databases or other forms of storage. It’s the end of the line.

Let’s look at how each layer help backwards-compatibility:

dropserver_app

The dropserver_app library can evolve without changing the underlying API. In other words, I can make improvements to the syntactic sugar without making any changes to the underlying APIs. As such there are no changes to be made that may cause older versions to no longer work.

However making changes to this library can put app developers in an annoying spot as we’ll see below.

dropserver_lib_support

dropserver_lib_support is the compatibility layer that must remain back-compatible with itself. If I want to change anything there, I have to add a new type, or a new optional field in an existing type. I can’t delete anything, ever, if I want old apps to run. It’s going to grow over time and will include bad function names like “getUsersV3”. However, because lib_support is not used directly by the app dev, they are not subjected to this. The new versions of dropserver_app will use the new APIs with nice names.

Again since dropserver_lib_support is hidden, it can be low level and it can be “ugly”. That’s fine, it’s hidden away. Types should be added slowly and carefully with an eye to maintaining them long term. The syntactic sugar and nice API can be left for dropserver_app.

denosandboxcode

In denosandboxcode I have the opportunity to handle old APIs by transforming them into new APIs. Remember that denosandboxcode ships with Dropserver. As a result it handles the latest API (at the time of shipping). But it also has to handle all the old APIs. The best possible outcome is that old API calls can be transformed into new API calls right there in the sandbox. That way the Dropserver host does not need to ship with a “Sandbox Service” that can handle multiple versions of an API.

Sandbox Services

It’s possible an old API can’t be coerced into a new API inside the sandbox for some reason, in which case the transformation may be able to take place at the service level. The difference between this and the sandbox is that the “Sandbox Service” runs on the host and has access to all the data about the appspace, users, etc… while that information is strictly limited inside the sandbox.

Core Models and Controllers

Finally, if all else fails, there is the possibility of versioning the very core of Dropserver’s handling of app and appspace data, like the users DB, etc… This manifests itself with parts of the host code named v0AppspaceRoutes and v0AppspaceUsers (note the v0.)

This was actually going to be the primary way of doing things but my thinking has evolved.

The Thinking Evolves

I originally set up a good chunk of Dropserver code with these v0 names. I did this thinking it was the right approach, but I just removed these. Here is why:

Versioning is viral: when you start having versioned data structures, some other structs that depend on them may need to be versioned as well. If you display any of this data to the user, the frontend code that handles this gets versioned too, not to mention the APIs used to fetch the data. Next thing you know you’ve got more versioned code than you ever wanted to see.
More code to maintain is bad: maintaining old code paths is hard. Keep these to a minimum. You do this by eliminating the difference between an old API and the current one as early as possible. That means inside denosandboxcode if at all possible. That way the core of Dropserver stays free of versioned madness.
The goal should be no changes anyways. API versions imply breaking changes. Nobody wants that. So why lay the groundwork for it?
Makes small additions hard. If I want to add a small feature, do I have to increment the version? That makes a big deal of every small change. I might even hesitate to make the change, holding it back until I have all the pieces that warrant a major API change. This doesn’t line up with how I’d like to work.
API versions imply big releases. A project like Android has API versions, and new versions coincide with new Android releases, which are announced at big flashy events every year. That’s not how I envision Dropserver. It’s a stable project. It must evolve of course, but not in that big flashy release manner that seems to be favored by companies that build planned-obsolescence into their products. By contrast, look at how LXD do it: they have a 1.0 API, and a myriad of API extensions. I like that better.

For all the reasons above I’ve eliminated the v0 code wherever I could.

What About Forward Compatibility?

One conundrum I still have to deal with is the situation where a new version of Dropserver includes a new API for app developers, and therefore is accompanied by a new version of dropserver_app. If an app developer uses the newest dropserver_app library to write a new app, does that app require the latest version of DS to run? Even if it doesn’t use the new API?

I’ve talked about backwards compatibility where an old app can continue to run on newer Dropserver versions. But what about a brand new app running on an older instance?

While I would like Dropserver instances to be updated regularly, realistically this won’t happen. Ideally a new app made with recent versions of the libraries I publish should be usable on older versions of Dropserver.

The solution here I think is to not have a monolithic dropserver_app.

C’Mon Man Just One More Layer

I haven’t fully worked this out but here’s where I think things are going to go:

dropserver_app will become a shell of sorts, and the app dev will import other libraries, like dropserver_users and dropserver_routes to interact with the DS system. The objects created by these libraries are passed into createApp().

So if an app only needs to use basic (older) route features, they can select the dropserver_routes@^1.0.0 and keep their app usable on old versions of Dropserver. If they need the newest features, they can use ^2.0.0.

createApp() can examine the objects passed in and determine the APIs that are required to run this app. This list of APIs used by the app is very handy for devX and UX reasons:

ds-dev can show the app developer the minimum version of Dropserver needed to run their app (you can assume Dropserver will have a map of API -> minimum Dropserver version)
When an app is published, that info can be embedded in the manifest and shown in the app distribution site.
When installing, the user can reliably be told the version of Dropserver they need.

Another advantage with this approach is that the app developer can use the latest version of the dropserver_routes without being forced to use the latest version of the users API (since they are separate libraries.) This means they can adopt a new feature without being cornered into adapting unrelated parts of their code.

Let’s Get Real

Creating a solid foundation for backwards compatibility for a platform that is not even off the ground yet is one of the more challenging aspects of Dropserver. I think it’s important though as it’s the kind of think that is hard to fix down the line.

This challenge is not academic. I have a bunch of Dropserver apps in various states of development, and I currently depend on four different apps on a daily basis. I’d hate to release a new version of Dropserver that would force me to update all of these before I can upgrade my personal instance.

I’m sure I’ll have to make more adjustments to my approach, but I think at a fundamental level the presence of multiple layers gives a lot of opportunities to fix problems surgically instead of needing to do big rewrites.

Dropserver Progress - April 2024

Olivier Forget — Sat, 11 May 2024 11:34:00 -0700

This is the progress report for Dropserver for April 2024.

Last month I said I would change how I do these. I want them shorter and more to the point. If any aspect of the work deserves a deeper technical dive it will be in a separate blog post.

Here we go:

Outbound Fetches Get Stuffed

Last month I tried to implement outgoing net requests in a safe way for the user and instance. I was unable to put something together that was flexible and powerful and safe. I ran into difficulties in a lot of places, but mostly the Deno net permissions are just not cut for this.

For all the details, see my post “Allowing Outbound Net Requests from a Dropserver App”.

My enthusiasm and optimism for Dropserver took a hit as I endured this setback. I committed the work I was doing on the outbound request proxy, and I may ship it later. But for now, my motivation to work on outgoing net requests is done.

I needed to move to greener pastures, and threw myself in Tailscale’s open arms.

Tailscale

I want to make Dropserver adoptable by as many people as possible. To this end I realized recently that Dropserver should be easy to install (think GUI Wizard on a popular operating system), and it should be easy to get it connected such that it can be reached and be safe. That last bit, it turns out, is a real pain.

I have a blog post in the works on the challenges of serving a small personal app on the web like what Dropserver will do. In fact, I got so frustrated with this problem that I wrote a prequel so I could vent some frustration before I finish the main post: “I Want to Surf The Non-World Non-Wide Web”.

Between domain names, DNS, TLS certs, and exposing anything to the public Internet there is no shortage of sharp edges to deal with. That is unless you summon the services of Tailscale.

Tailscale lets you create a private network that you can access from any of your devices. You can also invite other Tailscale users to one or more of your services. Powerful ACLs give you fine grained control over all access. They do all the tedious work for you: domains, TLS, etc… it’s all taken care of.

A prominent use case for Dropserver is hosting apps that are only used by yourself and a few other known people (like a life partner) or even just yourself. For example I use it to host my personal note-taking app, which is used by me only, but I like to access it from any device I have handy. But right now it’s exposed to the public internet, with the only thing stopping other people from reading my notes an auth system that sits at layer 7 😟.

So the question is: what if I integrated Tailscale into Dropserver? The service is really nice, and they offer a library and encouragement. Could that cut down on the set-up time and increase the usefulness of a Dropserver install while also improving its security?

After two weeks of playing around the answer appears to be a resounding “yes”. I’ll write a separate post specifically about this.

My optimism is way up thinking about how easy it would be for someone to get a functioning yet completely private and safe way to host their private apps thanks to Tailscale 🤩.

Headscale: Tailscale’s Open Source Cousin

If there is one downside to Tailscale, it’s that it’s a proprietary service run by a for-profit company which we should assume will get sold and enshittified at some point in the future. While we are lucky that they open-source most of their code, their control-plane is proprietary.

Headscale is the open source self-hostable alternative, which is supported and encouraged by Tailscale.

Ideally all Tailscale functionality that Dropserver uses would also work with a Headscale server, but that’s not going to be the case right away.

Unfortunately there are some features missing in Headscale, specifically Serve with TLS enabled and Funnel. Since HS is written in Go I’m looking at contributing by adding the missing pieces of TLS cert generation to Headscale. See this issue, and drop a reaction to let others know you’d like to see this happen as well, or comment if you want to help me get it built.

That’s a Wrap for April

I am not able to work as many hours on Dropserver these days because of “real” work constraints. But the good news is Tailscale has given me a new boost of enthusiasm. I think it’s a brilliant service and its a shoe-in for Dropserver and I’m excited to get to work on the integration.

Allowing Outbound Net Requests from a Dropserver App

Olivier Forget — Wed, 08 May 2024 16:56:00 -0700

With Deno 2.0 delayed again I recently tried to implement outbound net requests for Dropserver apps using Deno v1’s permission model. I was excited to offer this new capability for Dropserver apps but unfortunately things did not go as I had hoped.

Problem Description

In the current version of Dropserver an app is unable to make a dynamic request to another host. It’s blocked by the Deno sandbox: there is no --allow-net permission.

(NB: Static requests are allowed by Deno. A static request is one where the URL is hard-coded into the source code files, like a typical import * from 'some-url.com'; statement. When I talk about allowing outbound net requests I mean the dynamic type.)

Here are some use cases for outbound net requests:

An app needs access to a finite set of hosts, determined by the developer. Example: a dice roll app hits random.org for an integer.
An app that needs dynamic access to hosts that are directly decided by the user. For example a RSS reader. There is a 1-1 relation between the user subscribing to a feed and the app needing access to a new host.
An app that needs open ended access to the internet. For example a Fediverse server or a web crawler. One can not expect the user to approve every domain.

Regardless of which hosts are allowed and how, here are some things I want to guard against for the safety of the admin and users:

HTTP(S) only. No telneting around to see what services can be discovered.
Dial only. No listening! A dropserver app can not create listeners, that’s ds-host’s job. (Amazingly the Deno permissions don’t make that distinction.)
SSRF protection: no requests to the local network unless it’s explicitly allowed by the ds-host admin.

Also I’d like to log all requests, with at least the name of the host contacted (see below).

The most interesting apps need open-ended access to the net (3rd bullet above), so my goal was to solve that case.

Deno Permissions

Dropserver apps run in Deno because of its sandboxing properties. See this bit of history to know how we ended here. It’s an effective sandbox for limiting file read/write and execution access. But on the networking side, it’s far from great. Let’s look at how it works:

Permissions for the network come in the form --allow-net and --deny-net. So far so good. However, here are the gotchas:

--allow net permits outbound requests (dial) and listen! So if you naively rant all net access because you have an app that may want to reach any host (situation three above), that app can now create listeners (servers) on any port (within operating system restrictions).
The --allow-net and --deny-net are compared with the URL that is requested, not the resolved IP.

These two issues open a lot of potential security problems if I open net access for Dropserver apps.

Good to know: Deno uses reqwest

Deno uses reqwest internally (for now at least) to make requests on behalf of the running code. This is good to know because we can assume that Deno won’t implement something that is not part of reqwest. Currently there is an open issue for some sort of SSRF protection – it’s been open for 3 years. So I can’t expect that to happen.

Reqwest now allows for a custom DNS resolver which would help with SSRF protection. Interestingly this feature was requested by Ry, the creator of Deno, so I suspect they use it internally, but so far it’s not exposed in the CLI as far as I can see.

The HTTP_PROXY

One feature of reqwest that Deno exposes is the ability to declare a proxy for HTTP requests via environment variables like HTTP_PROXY. This is a good way to route requests through a proxy that Dropserver controls to prevent any unwanted request. Proxying requests would allow me to “see” and filter to my heart’s content. I could even reuse Dropserver’s own SSRF mitigation code. It’s the ultimate solution.

Unfortunately, while the proxy works as expected, it in no way prevents an app from making requests that bypass the proxy. Basically the proxy is only used for “fetch” requests. You can craft a request from low-level APIs, and I think you can even override the proxy using the fetch API.

The proxy is merely a “helpful suggestion”, so not usable as a security boundary. (That’s not a knock against the proxy implementation: a proxy, by itself, is not meant to be a security boundary.)

A Hail Mary: –allow-net-fetch

If only Deno could block all net access except those that go through the proxy, then I’d be set. I posted a request for --allow-net-fetch permission in the Deno issues hoping they find it an interesting approach. Since Deno 2 is still in the works, I am hoping there is still time to slip that in, if they haven’t already done something similar.

Non-Deno Solutions

I could run Deno inside a Linux namespace that gives me full control over the network. There are a few problems with that:

this only works in Linux. This is unfortunate because I am leaning towards making Dropserver installable on Mac and Windows natively to reach the broadest possible audience.
I currently use Bubblewrap to further isolate Deno sandboxes when ds-hostruns in a Linux environment but its net namespace support is on/off. Setting up the virtual networking elements to get a functioning network is not easy. And Bubblewrap doesn’t even run in Docker, or at least not easily.

So limiting an app’s net access to an acceptable degree without Deno’s help is complicated and will take a lot of my dev time, and it will only work on Linux at a time when I am eager to not limit Dropserver to Linux! So that’s a nope.

Report Hosts to User

While I am talking a lot about permissions, visibility into what hosts are being contacted and how often is an important feature for a user who is concerned about what an app is doing.

There are good reasons for an app to request open-ended access to the web as we’ve seen above. But how can a user be made to feel comfortable about an app’s behavior after they’ve granted that permission? The key is to give them visibility into what the app is actually doing with that permission.

If the user can see reports that point out the most contacted domains, the payload sizes, and other metrics that might reveal whether an app is behaving correctly or not.

I’ve always felt that it was weird when a system asks me to grant some permission but then offers zero information about the usage of that permission. If it’s so important to my safety that I have to grant a permission, why are you not showing me what is done with that permission?

With this in mind I worked on a logging proxy for outbound requests from the appspace sandbox.

Proxying outbound requests

I implemented a proxy using goproxy (see commit e78d447).

I learned along the way that MITM’ing the connection is very computationally expensive and slows down requests significantly when there are many at a time. With no MITM the performance is OK.

With no MITM the proxy only sees the host and not the full path, or the payload, but I can at least report that host X was contacted.

Maybe I’ll add an option to turn on MITM for an appspace. The user might do that if they get suspicious of an app’s activity.

Conclusion

After getting the proxy to workable condition I achieved 0.5 of the four requirements I laid out at the top. I only managed to log hosts and only if the app uses the Fetch API without defeating the proxy.

The other requirements (HTTP-only, No listening, and SSRF protection) I am not able to pursue without better tools from Deno, or I have to make Dropserver Linux-only and dive into Linux namespaces and veths galore.

What if Deno 2 doesn’t solve my issues? This would be bad, but I’m a crafty guy and I’ll figure something out.

I Want To Surf the Not-World Not-Wide Web

Olivier Forget — Wed, 17 Apr 2024 12:50:00 -0700

The World Wide Web is awesome. It’s “World-Wide”, and it’s a “Web”, meaning everybody and anybody can connect with everybody and anybody! Information is shared and everybody can read it! The WWW rocks.

Well, it rocks for things that are meant to publicly available.

Once you want to share information with just one or a few people things get a little weird.

If you want to host your own private little island on the web for just you and a few known friends or family you find that the web is hostile to your intentions.

“But that’s not the point of the web! It’s for sharing information publicly!” True, but there are other use-cases that could benefit from the amazing tech of the web, and they are being shunned right now, and it’s unfortunate.

I would like to leverage the amazing qualities of web technologies for a different use-case: private interactions. Essentially I want to use the web browser, but to browse the not-World not-Wide Web.

Web browsers are great

Web browsers are available on any operating system. Ubiquitous.
There are alternative browsers by different vendors. Choice.
They’re built on standards and don’t break backwards compatibility. Longevity.
Most web browsers have good security and isolation properties. Safety.
I can create a well formatted readable document with just a bit of HTML and CSS. Simplicity.
I can create a highly interactive application using JavaScript, if I choose to. No limits.

The browser platform is incredible. But the networking protocols they work with are utterly hostile to private interactions.

I have built web-apps that are private-only. One is Leftovers. I also have a shopping list web-app my wife and I use. My personal note-taking app is a web-app, I am its sole user. I have ideas for others. In fact I’m building an entire platform for supporting these use cases.

But the thing I keep coming up against is the “World” “Wide” “Web”.

The problem

Here is how the web works, basically:

you enter or click on an address, like http://olivierforget.net/blog/
that gets turned into a magic number, the IP, that looks like 50.18.142.31
your request is sent to the computer at that IP.
that computer sends you what it deems is at that address

Looks fine, right? Let’s look at those steps again in light of a private interaction:

you enter or click on an address. The domain is a public record that may even contain your personal info like your physical home address.
get that public IP address by contacting some 3rd party server that may or may not keep a record of the fact you looked up this address
the request is sent to the other computer over the public IP network.
It’s only after you’ve done all this that the other computer gets a chance to ask you for your username and password to get into a private interaction.

So much of this private interaction leaves a public trace.

It’s like having to set up a booth on Times Square to get a hug from your grandma.

It just doesn’t make any sense.

It also means that to serve something privately, you have to expose the computer to the public internet, which means you’re on the receiving end of bots and other unsavory characters. Remember how the great thing about the WWW is that it allows anybody and everybody to connect? Yeah, that applies to all the state actors and script kiddies too.

Setting up a computer with a public IP is like walking into a pirate bar with an “I Hate Parrots” t-shirt.

Don’t get me wrong, again. I love the World Wide Web… for Public information. But for doing private, family and friends stuff it’s just the wrong architecture.

What do I want?

I’d like to be able to leverage some of the technologies of the web, like the fantastic run-anywhere abilities of HTML+CSS+JS, but via a private network.

I don’t want to have to buy and renew a globally unique domain name just to share with a dozen people
I don’t want that domain name part of the public record along with my home address (and I don’t want to pay extra to hide personal details)
I should not need an external service to navigate to this domain, like a DNS server run by some company
I don’t want to have to go through a third party to obtain a TLS cert that goes into the public record just to get a secure context
I don’t want the computers where I host stuff for my friends and family to have reachable public IPs

What does it look like?

I don’t know exactly. This is a gripe / wishful thinking post, not a solution post. Sorry.

But I suspect it’s a bit like a Tailscale tailnet, but geared more towards personal interactions.

Perhaps web browsers would embrace Wireguard natively and treat such connections as secure. Something like that.

We’d need an alternative to the whole Public DNS thing. Maybe GNU Name system? But it needs to be user-friendly and browsers would have to support it.

So yeah, I’m just dreaming here.

Dropserver Progress - March 2024

Olivier Forget — Fri, 05 Apr 2024 12:30:00 -0700

This is the monthly progress report for Dropserver for March 2024. The previous report is here.

This happens to be the twelfth month for which I am publishing an update (the first progress update covered April-June 2023). A full year! Yay me.

It’s also time for me to change how I write these posts. I think I’ll make them shorter and more to the point and I will dedicate complete blog posts when a topic deserves deeper examination. For example, The State of Sandboxing in Dropserver is a post I keep going back to and that stands out in my list of posts, while the series of “progress updates” are just so much water under the bridge.

The Big Picture

I’m plugging away despite having less time to spend on Dropserver due to other projects. This reduction in time results in a slight decrease in time spent on the core of Dropserver, and a significant curtailing of peripheral work like experimenting with Lume and other side-quests.

I started work on the next big feature of Dropserver, but before that I did some maintenance work and pushed a minor version.

Prepare for Deno 2.0

The big 2.0 for Deno keeps getting pushed back. At this point I don’t know when it’s coming but it sounds like the previous timeframe of “April” will not happen.

Dropserver uses Deno’s sandboxing features to prevent app code from doing bad things, so a big change like whatever might be coming in Deno 2 keeps me on my toes.

Thankfully Deno published a guide for migrating to Deno 2 that describes the APIs that are definitely going away in D2. I made changes to the sandbox code to use the latest server APIs and std lib (commits 283b48 and b603e5.)

I am still struggling with a mistake I made long ago: I included the now deprecated Deno.RequestEvent TypeScript type as part of a type in dropserver_lib_support, which is kind of a bummer since that library is not supposed to change to preserve backwards compatibility. I may write a separate post about how that works and lessons learned there.

Inotify watchers fix

Back in December I experienced a (thankfully rare) anomaly on my personal ds-host instance. It turns out it was this bug that was causing ds-host to create and not clean up inotify watchers. I fixed it in commit 0580e5.

(I just looked at my personal instance which has been running this fix for a few weeks, and can confirm the inotify watchers are not growing unbounded. Pfew.)

Release 0.13.1

With these improvements committed, why not release a new version of Dropserver? I tagged 0.13.1 on March 13.

Next Feature: Outgoing Net Requests

Dropserver apps are currently not allowed to make outbound network requests. This is a huge limitation that I am keen to eliminate. I could of course add a --allow-net to the Deno sandbox and call it a day. But that would go against all the principles of security that I’ve established for this project.

I am spending time trying to figure out how to allow network access such that it is convenient for users and developers, and such that it does not pose a security risk to the user.

There is lots to consider, including how Deno does network sandboxing (not great currently, maybe better in the elusive 2.0?), the kinds of things I want to protect against (SSRF among others), etc… I have run across gotchas everywhere and it’s making my head hurt.

If I start writing about this here I won’t be able to publish this update any time soon. So it will go in a separate post some day.

Proxying outgoing requests

Here is something tangible I spent time on: I worked on adding an HTTP proxy in Dropserver that sits between the Deno sandbox and the outside world. The idea is that for HTTP fetches at least, I can see what the app is requesting, I can log it, I can even deny it if it doesn’t pass my SSRF filters.

I learned along the way the MITMing HTTPS connections is very computationally expensive. If an app needs to download modules from deno.land/x/ its startup time increases five-fold! Without MITM I can still log the destination host and run the connection through my SSRF filter, so that’s good enough.

This proxy is not a good security solution because it’s only effective if the app uses the “fetch” call, but it’s a starting point of something. While I suspect a proxy like this will play a role in a fully implemented secure outbound request solution, it’s not what it is now.

Still, I may include it in a minor point release along with the ability to see fetch history, purely as an informational thing.

Work On Leftovers App

I made some small improvements to the Leftovers app. I use this app daily at home so its little UX paper cuts got to me all the time. Finally it’s starting to feel quite pleasant to use.

A true home cooked app for home cooked meals.

This was also an opportunity to run through the release + publish flow for a Dropserver app. It worked, but it was a bit fiddly. I may need to create a Github Action that takes care of everything for the app developer. Push a git tag and it cuts a release and publishes an updated version of the website to GH pages. Boom done.

It was also an opportunity to experience the app upgrade process from the user’s point of view. Here again, more work needed 😔.

What’s Next

While I want to make progress on outgoing net requests, there are a few other things I should improve.

In particular with JSR out, and the realization that it uses import maps, I should probably support them in Dropserver apps. There are other reasons too: I’d like to support vendoring dependencies so that apps can be published with all code included, but that depends on import maps too. (And guess what? Vendoring affects outbound requests too: see the bit above about hitting deno.land/x/ through a MITM proxy!)

There are many other things I could work on, and I’m not sure what I will do… so see you next month!

Dropserver Progress - February 2024

Olivier Forget — Mon, 11 Mar 2024 09:56:00 -0700

This is the monthly progress report for Dropserver for February 2024. The previous report is here.

The Big Picture

This will be a short report because I spent half of this month on vacation, which limited my work. After Releasing 0.13 last month I am allowing myself some time to explore tangential projects while also chipping away at some Dropserver issues.

Getting Lume to Work

I have been trying to get Lume to work in a Dropserver app. I’d like to combine Lume, a static site generator (SSG), with a GUI frontend to make a Dropserver app that allows you to edit content in an online editor and publish it. It is meant to be a demo of the kind of app that can be built for DS. It is also a test of running a non-trivial project that was not intended to be run in Dropserver.

One issue I’ve run against is that Lume is clearly meant to be used as a CLI app, and not as a library within other code, especially if given limited permissions. The other issue is that Lume leverages existing libraries from the node ecosystem via Deno’s NPM support. While the NPM libraries work, they tend to expect full disk access permissions and fail hard when they don’t have it.

One such library is Browserslist, which is used by autoprefixer which is used in most projects that leverage PostCSS. The issue is that browserslist was not designed to run in a sandboxed environment like Deno. I decided to try to fix this.

Browserslist PR

PostCSS and autoprefixer are so commonly used in site generation tooling that I thought it would be worthwhile to try to help it run in Deno when permissions are limited. I submitted this issue and said I’d submit a patch.

It’s nice working on an existing and widely used project for a change. I spend a lot of time working on my own stuff, it feels good to get out of my shell.

I added an ENV variable that limits how far up the path Browserslist goes to look for config files. Credit to the maintainer, my PR was promptly merged (after I fixed linter errors – woops!) and a new version of bl was released, which is a nice feeling.

Since the Lume and autoprefixer devs are staying on top of dependencies, the current version of Lume requires the latest version of autoprefixer which requires the latest version of browserslist which includes my patch. Neat!

ENV vars in Dropserver sandbox?

I changed the browserslist library to limit its file reads via an environment variable, but the Dropserver sandbox has no mechanism to let developers specify environment variables for the sandbox. I’m contemplating adding this capability. I’ll consider it seriously if I find over and over again that a library wants to be controlled via environment variables.

I suspect this type of library is usually associated with CLI apps, and site generators. I do want to support site generators in Dropserver if possible. I think all the tools we build for the web that are CLI-based are a boon for technical people but leave regular users helpless.

Then again this may be a bridge too far. I haven’t thought much about it, but I imagine that letting app devs specify Environment variables is a big security issue.

(Note that there are other ways of preventing Browserslist from iterating up the path: put all the config files it looks for in the appspace data directory and it won’t go looking elsewhere for them. This is what I have been doing in my Dropserver Lume experiments.)

Make Appspace Data Directory the CWD

Since it is a CLI app, Lume puts a lot of stock in the CWD “Current Working Directory”. When the CWD is non-existent or unreadable, Lume errors out.

This one I could fix in Dropserver. I changed the sandbox code so that the appspace files dir is the CWD of the Deno sandbox. Easy peasy, and it makes it easier for app devs to interact with files in the appspace dir: they can just use relative paths instead of the helper functions that return the absolute path of the appsapce files.

I did run into a weird thing that caused me to spend a few days on this: Deno is a bit inconsistent with path normalization. When symlinks are involved, or when using different capitalizations of paths in a case-insensitive filesystem, Deno will fail to recognize that two paths are the same even if they are not character-for-character the same. I filed this issue.

Commit cec557.

What’s Next

I will work on a few more bugs and improvements before I release Dropserver 0.13.1. Beyond that I am not sure what comes next, but this is looming:

Deno 2.0 and Outbound Requests

I’m eagerly waiting for the arrival of the first public releases of Deno 2.0. Deno has been a boon for the Dropserver project, but it’s not openly developed so I wait patiently to see what I will have to contend with when it drops.

In particular, a sorely needed feature for Dropserver apps is the ability to make outbound requests. Letting an app do this opens up a number possible security issues, and Deno’s permission API plays a big role. As such, I want to see if they improve the net permissions before I do too much design work for outbound requests. See these issues for an idea of things that have been bugging me:

#2705: Current design of –allow-net (stale-botted 😡)
#21227: Deno should block access to local network unless explicitly permitted

Once I know what I’m dealing with, I’ll get to work enabling outbound requests from Dropserver apps.

In the meantime I’ll continue experimenting with IndieWeb, ActivityPub, and RSS, with an eye towards making Dropserver apps that demonstrate what can be built on the platform.

See you next time.

Dropserver Progress - January 2024

Olivier Forget — Fri, 02 Feb 2024 10:50:00 -0800

This is the monthly progress report for Dropserver for January 2024. The previous report is here.

The Big Picture

Dropserver 0.13 is out! 🎉 This is the release that lets Dropserver install apps from a 3rd party website. If you’ve been reading these progress reports, you know it’s been a long time coming.

Getting to a Release

Before releasing I had to add a few more features and close up some unfinished work.

Generate the App Listing

The goal of the 0.13 release is that a Dropserver app should be installable (and upgradeable) just by pointing your instance to a URL. I spent most of my time in the last few months on the ds-host side of this: opening package files, fetching manifests, displaying changelogs, etc… But now I needed to do the other side: generating the files that can be hosted on a static file server.

At the very least ds-dev should be able to generate the app-listing.json which is the starting point of any interaction with a URL from the ds-host side. The app listing contains a list of versions and relative paths to each version’s package, icon, and changelog.

This part was easy enough. Commit 85b99a.

Generate the Website

What about generating an HTML page? I resisted doing this. It felt so superficial compared to the actual core functionality of 0.13. And yet, imagine this: you have an app published, and you can point people to a URL and tell them that’s how they can install your app. But visiting that URL just disgorges some useless data. Shouldn’t there be a webpage? Something that a human can consume?

The more I thought about it, the more it seemed that not having some page that describes the app, its relation to Dropserver, and how to actually install it, the more the whole idea of apps from URLs would fall on its face.

So I banged it out. It’s crude, it’s low-tech, but it’s there. Now when you generate an app listing you also get an HTML page. It’s self-contained but references the app icon to make it more lively. Screenshot below:

All the data is sourced from the app itself so it’s missing a lengthy description and screenshots. It’s really mostly good as a minimal presence to accompany a blog post for example.

Commit 18e1e2.

Finalize Handling of Redirects

Up until this point I had punted on how to handle redirects from the site where the app is hosted.

For the time being I don’t want the site that hosts the files to redirect to other sites. I concluded that the user is likely putting some level of trust in the site that claims to host the app files. If redirects are followed transparently, the files may actually come from a completely different host without the user knowing.

For this reason I don’t allow redirects. In the case where the app has to be moved to a different host or a different address the site can return a permanent redirect. However this redirect is not followed by ds-host. Instead, it presents the user with a notice of a new location for the app, and the ability to change the URL if they approve.

This way the user is always aware where their app files are coming from.

Commit 14f77f and others.

Update the Docs

I update the docs to describe listing and website generation and the process to move your site to a new URL:

https://dropserver.org/docs/app-packaging-distributing

Post-Release Meandering

After shipping 0.13 (🎉🎉) I have been feeling a bit directionless. I know there is tons more to work on but after focusing so much on one outcome, the sense of purpose feels awfully thin once the outcome is achieved.

I am allowing myself a period of vagabonding through some ideas I have. In particular I don’t like how we build frontends for web-apps right now. Big messy SPAs are more trouble that they are worth. So I’m thinking about some possible ways of doing better there. I may write a bit of experimental code just to get it out of my system.

This idea isn’t entirely disconnected from Dropserver: DS apps are meant to be “home cooked meals” and as such get put on a shelf for 6 months or more and revisited only occasionally. If upon revisiting the build system demands to be upgraded and then fails to do so, the app may be abandoned.

I am also thinking about the IndieWeb, the way search engines are failing us, and how we have a lot of tech that already exists on websites (RSS, microformats) but is under-utilized and under-valued.

These projects are fun to think about, but I can’t launch into anything that will take too much time away from the core of Dropserver. One thing I can do is improve the apps for Dropserver that I have built for myself, and publish them to a website. That was the whole idea of 0.13, right?

Leftovers App and Site

I made a few improvements to the Leftovers app and released version 0.6.2. There are more small bugs I want to fix, and will do soon.

First though I wanted to put Leftovers into its deserved home online:

https://leftovers.olivierforget.net.

I didn’t want to settle for the automatically generated website. There really needs to be a description and some screenshots. I took the generated HTML and added what I felt was necessary. I link to the generated site for version details.

Other Work

IndieWeb

I added social media preview images to this site and polished some of its microformats. I have more work to do, but I’m excited to get involved with the IndieWeb. In particular I am thinking of ways that Dropserver can help. Some of the IndieWeb requires a server to work, which often falls on a centralized solution. I’d love to see if Dropserver can help break that pattern.

ds-dev on Windows

I spent a bit of time working on getting ds-dev to compile and run on Windows. I still want to do this but I am dialing back the time I spend on it.

A version of Dropserver that is very easy to install and run on any OS is going to be a long term goal. I realized that after installing, the whole networking/domain/TLS certificates problem is such a huge hurdle for self-hosters that it’s just as important to resolve.

Lume and npm on Dropserver

I played with Lume to see if it could eventually run as part of a Dropserver app. It won’t be easy because it depends on a lot of modules from npm, some of which expect to have full read permissions on the filesystem.

I think a lot of the issues I encountered running Lume can be resolved and it will be able to run in a Dropserver app. It will take some tweaks on the Dropserver side, but this is a good test. App developers will want to make use of many npm modules, and Dropserver should accommodate this pattern.

If Lume does run on Dropserver, that means you have a full fledged static site generator that can run in a Dropserver app. This opens up exciting possibilities: CMS for Lume inside a Dropserver app?

What’s Next

I’ll continue to improve the Dropserver apps I have, and publish them online. This will help people who are kicking the tires of Dropserver have something to play with. I also have ideas for more experimental “demo” apps (like a CMS+SSG as mentioned above) that give people a sense of what can be built.

I’ll continue exploring possibilities for making Dropserver easy to install and self-host. But it will be a long road, and I realize that there is an other side to this equation: there needs to be a reason for people to install DS in the first place. That’s why I’ll also spend time on apps and demos of what’s possible.

When I feel ready I’ll get to work on some bug fixes and improvements in view of a 0.13.1 release.

Thanks for reading, see you next time.

Dropserver Progress - December 2023

Olivier Forget — Tue, 09 Jan 2024 14:10:00 -0800

This is the monthly progress report for Dropserver for December 2023. The previous report is here.

The Big Picture

I made a big push to get version 0.13 out the door. While I didn’t succeed at actually cutting the release, the most challenging part is complete and I am now just adding a few more items. I also continued developing thoughts on how to take Dropserver to the next stage. I started work on getting ds-dev to work on the Windows operating system.

Work on Installing App from URL

Version 0.13 will make it possible to add an app to your Dropserver instance by pasting a URL. The app will be downloaded and installed, and Dropserver will periodically query that URL to determine if there are new versions of the app available. The user will see a callout when a new version is available to be installed.

ds-dev will generate all the necessary assets and JSON files necessary to distribute your app via a URL. All the developer needs to do is upload the generated files to a static site server.

Getting all these pieces to work took some doing. In this progress report I’ll spare you the play-by-play of all the refactorings I had to do. Below are a few notable thoughts about recent work:

The App Distribution Test Site Became a Dropserver App

Back in November I was using an entirely static site to test app distribution. I had created (by hand because the site generation code was not built) a static site that offered the necessary JSON and other files to distribute a Dropserver app. I uploaded them to Netlify and was able to test ds-host’s ability to fetch a remote app by pasting that URL in the UI.

However at the end of last month I got to the point where I needed to easily make changes to a published app in order to test how Dropserver displays the multitude of states it finds on the remote. For example: if I am writing the code that handles the appearance of a new app version on the distribution site, then ideally I should be able to set the distribution site to show that new version or not. That way I can repeatedly install the app, then “publish” the new version and see how ds-host behaves.

It is quite easy to update the site published on Netlify, but it’s still too slow and tedious for something I need to do dozens of times per day.

The solution was to create a Dropserver app that could serve whatever situation I could dream up. I didn’t want to spend much time on this bit of tooling, so it only lets me select from a menu of pre-coded situations.

The app is utterly basic (the main interface is an HTML form, with no styling or JS) and leverages some of Dropserver’s capabilities, like private routes for selecting the situation, and public file server route handlers for serving the JSON data files and app packages.

This worked really well. It was also a good exercise for me and for Dropserver: each time I thought of a new situation I wanted to serve, I coded it in the app, incremented the version, packaged and uploaded it to my instance, and it was live in a few seconds. This was accelerated dogfooding for app packaging, installation, and appspace migration.

With the ability to switch the responses served by the app distribution site, I was ready to grind through the rest of the project.

Detour into The Future

While doing the work described above it was hard to resist the temptation to create a full blown Dropserver app that serves as the app distribution site, instead of the rudimentary hard-coded thing. In the future, this is clearly how it will work: I will publish an app that you can use to publish your app.

The motivator for this is that static sites can be a bit limiting. For example perhaps I’d like to get changelogs for versions 1.3.0 to 2.1.1. With a static site I’d have to do a lot of requests to get versions one-by-one, or perhaps I can load the entire app changelog which could be a lot of data. With a server handling the request, this becomes a single request that returns just what is asked.

There are other reasons for the app distribution system to be Dropserver-based: it could include a support forum, reviews and comments, etc…

However the reason for sticking with the static site idea for now is that I think many devs will initially experiment with Dropserver locally, and may not expose their instance to the broader internet. As a result there should be a way to publish Dropserver apps without using Dropserver.

The Grind

A lot of the work went into correctly reflecting the data obtained from the remote app distribution site in the the local DB and in the UI. Some examples:

Dividing the UI flow between apps installed by uploading a package and apps installed from URL. When you add an app by uploading a package, newer versions must be obtained the same way. And vice versa, and app from a URL can only be updated when a new version is published at that URL.
Lots of refactors to support re-using the same code paths for different but similar use cases, like fetching a changelog for a new app and a new version, validating manifest prior to fetching package as well as during installation, etc..
Lots of head-scratching regarding error handling and reporting: the same error can be fatal for one process but informational in a different situation. This makes dividing up the code a bit tricky. I also improved how I report warnings to the user.
Reflecting as much of the app to the user before they even fetch the remote package. So lots of new routes, remote fetches, and UI components to display all this as needed.

Windows Version of ds-dev

Last month I decided to make Dropserver runnable on more common operating systems in an attempt to reduce the installation hurdles for self-hosters. As a first step I’m trying to get ds-dev to work on Windows.

So far I managed to get the test suite to run by eliminating the Linux/unix-specific code. At some point though I’ll have to figure out how get ds-dev and ds-host to quit nicely since Windows doesn’t support SIGTERM.

Now I’m just going through the many failing tests and fixing them for Windows. I don’t know if I’ll hit any show-stoppers. I found one reason to be optimistic: Windows supports AF_UNIX sockets! Dropserver communicates with its Deno sandboxes via unix sockets. Not having to re-write that is a plus!

Up Next

In January I’m shipping 0.13. That’s my focus. After that, we’ll see. See you on the other side!

Dropserver Progress - November 2023

Olivier Forget — Tue, 05 Dec 2023 19:10:00 -0800

This is the monthly progress report for Dropserver for November 2023. The previous report is here.

The Big Picture

Although my main coding focus was getting Dropserver to install and update an application that is hosted on a third party website I got side-tracked by big thoughts on other topics. These don’t help the release tempo but are crucial for steering Dropserver in the right direction in the long term.

A Light-Bulb Moment

I had this crazy idea: Dropserver should install and run on MS Windows and MacOS as a GUI application. It wouldn’t be terribly useful in its default configuration (localhost only), but it would allow potential users to get a taste of Dropserver. From there, configuration wizards would allow them to make their install more useful: like connecting other devices and users via a WireGuard tunnel, or leveraging 3rd party or Dropserver services for obtaining TLS certs and DNS management (think ngrok, Tailscale, NextDNS).

The big idea is that you have to go where the users are, and they are likely on a Win/Mac machine. When they get around to using a dedicated machine for Dropserver, the spare machines they have are also likely to be Windows and Mac. And if they were to buy a machine for this, they will be far more comfortable using a Mac Mini or an Intel NUC running Windows than a Linux box. That’s just the reality of it.

By choosing Deno as the sandbox for Dropserver, I effectively do not have any platform constraints. I should take full advantage of that.

But it’s not just the Win/Mac thing, it’s also the idea that most people are GUI users. So instead of an intimidating CLI and config file based installation, users should be able to run an installer then use a GUI to set things up. With that in mind I would also try to support a Linux GUI distribution for those users who made the jump to the penguin OS but would still rather avoid setting up complex configs using the command line.

There are a lot of questions to answer before making this idea a reality, in particular lots of thought will have to go into making this as secure as possible. Thankfully it seems mainstream OS are getting the hint about sandboxing apps: Windows will sandbox apps for better security, and MacOS already has a Sandbox. These may allow me to preserve the second layer of sandboxing that I achieved using Bubblewrap on Linux.

A nagging feeling I’ve had with Dropserver these last months has been that it’s fine to have a platform that allows users to install web-apps easily. But if they can’t install the platform itself, what have I achieved? I think supporting mainstream OS with a GUI application is maybe a way to crack that nut. We’ll see. At least it gives me hope.

Frontend Thoughts

I am spending a bit of time thinking about what tech stack I would use if I were to rebuild the ds-host frontend. I am not thrilled with what I have right now (Vue 3 SPA) given that it is the control panel for a server. Why is it an SPA?

SPAs are great at rendering a widget very fast, without a round trip to the server if you already have the widget data locally (like maybe the widget appeared in a list view earlier in the session). The Leftovers and ShoppingList Dropserver apps benefit from being locally rendered because these are the kinds of apps where any waiting time is painful for the user. But is this necessary for ds-host?

I originally chose Vue 3 and made it an SPA because that’s how I built the previous thing, whatever that was. It was less effort to just use the tech I knew. But now I’m seeing that it may not have been the right call. The SPA offers a good developer experience for me when I work on the components, but the data management story is poor. See below for an example.

There is also lot of talk these days about web components and other more HTML-first approaches to app frontends. Maybe it’s a case of the grass is greener on the other side, but I can’t help but think about a rebuild that is more HTML-centric and simpler to maintain, not to mention more accessible and it could work without JS. The challenge is preserving the developer experience.

Progress on Installing Apps From URLs

Although I was able to fetch and install an app from a remote URL at the end of last month, that was hardly the end of the story. This month I implemented the storage and updating of the remote app’s info, then worked to update the frontend with this data.

Note that if you don’t see any activity in the Github repo, it’s because I’m working in the “app-from-url” branch.

DB work

The DB work was fairly straightforward but lengthy. There are lots of possible situations that arise from updating data fetched remotely: any of a number of errors from various steps (fetching, validating), permanent redirects, data that has changed, data that hasn’t changed. Sorting through all that took some time, and the implementation and tests felt like a lot of work. Still it got done: commit 61489f9.

Frontend work

Fetching the app URL data from the DB and displaying it in the frontend was no big deal. Classic CRUD work. A challenge arose when I realized that a user may never know there is a new version of an app because of the way the frontend data model works.

Simply put: the frontend loads all the apps and appspace data when a list of these is viewed, and keeps that data around between page navigation. This means that rendering an app detail page is instant. Neat. But miserable. Holding on to data means you have to update it when it changes by pushing from the backend.

I started doing work to push events to the frontend: I implemented an event handler for app URL data on the backend, which ended up being my first contact with Go Generics. It took a bit of time but I found a satisfactory way of making backend events easy to implement thanks to Generics. I’ll call that a win. Commits 6e6f0d and e0e249.

In the end I am not pushing events to the frontend. I think the whole ds-host frontend is built on wrong-headed ideas given the type of data it presents. It just doesn’t need to be an SPA. So instead of adding complexity I just made it such that app URL data is reloaded when a user clicks through to a Manage App page. This solves the problem well enough and easily enough.

Refresh app URL data

In commit 0a7c31 I tied a bunch of things together (the DB data, in particular the stashed ETag, the frontend, and the fetch code with SSRF protection) to enable one-click refresh of the remote app URL data. Whew! It’s annoying that this took so long but it’s progress.

From there I shouldn’t have much trouble automatically refreshing the remote app URL data on a timer, which ds-host will do about once per day.

Next up

Next I’ll have to make my dummy remote app sites that I use to test all this stuff so that I can change the site dynamically and then fetch an update. The reason is even though I have all the pieces in place I need to implement a proper UI for when a new app version is available or other edge cases. Once I have a good UI for handling a new version of an app, I’ll make it so ds-dev can generate a static site for app distribution, and then I’ll ship 0.13.

I’ll also continue to research and develop my ideas about frontend technology and an easily-installable mainstream-OS GUI version of Dropserver.

Dropserver Progress - October 2023

Olivier Forget — Wed, 01 Nov 2023 09:07:00 -0800

This is the monthly progress report for Dropserver for October 2023. The previous report is here.

The Big Picture

My focus is on getting Dropserver to install and update an application that is hosted on a third party website. Most of the work is really just thinking about what the endpoints look like on the third party site, and all the different ways a user will proceed through the steps of installing an app and create an appspace, in particular thinking about how unattended upgrades will work.

I also worked on other things as they came up, which included releasing version 0.12 early.

Vagrant file for ds-host

Jacob Weisz submitted a PR for a Vagrant file to help potential developers build and run ds-host on their machine. I merged the PR of course but it made me wonder how Vagrant could be leveraged to really make it easy for developers to contribute to Dropserver.

Someone contributing to ds-host should be able to quickly rebuild and relaunch ds-host after making changes. Running ds-dev, compiled from source, should be easy as well.

A frontend developer should be able to launch this Vagrant machine and quickly get to a point where they are editing Vue components while enjoying the benefits of Hot Module Reloading.

I’ll have to dive into Vagrant a bit to make all this happen. I will have to dogfood this too: I will ditch the hand built VM I currently use to run ds-host and use the Vagrant development environment to do all the development of Dropserver.

Anyways, thanks Jacob!

Bug fix and 0.12 comes out early

Trevor Flowers reported a panic in ds-dev while working on the tutorial app. Never great to have a panic on what should be well trodden code paths of a “Hello World” app. It turns out Vim puts a file called “4913” and immediately deletes it. This caused a panic in my application code watcher loop.

It’s a classic case of a developer putting a panic in a branch of code because they think “this can’t ever happen” and a user comes along and says “oh yeah? watch this!”.

This was quickly fixed (explanation and resolution in this nice blog post) but unfortunately I had done quite a bit of work in the main branch of Dropserver, including a proper new feature (app changelogs) so that a patch increment (0.11.1) was inappropriate.

Thus was born Dropserver version 0.12, which frankly is no bad thing.

Going forwards I will be more careful not to mistreat the master branch so that I am always ready to release a patch if the need arises.

Thank you Trevor for taking the time to try working with ds-dev and reporting the bug.

Work on Apps from URLs

The early part of the month was spent figuring out how this should work. It’s not that getting an app from a URL is much harder than installing an app that was just uploaded. It’s that having an app available for installation at a URL opens up a whole new mode of operation for Dropserver:

👻👻👻💀💀💀 UNATTENDED APP UPGRADES 💀💀💀👻👻👻

Boo! Scary… 🫢

Since the base URL returns a listing of versions for an app, Dropserver could reload the URL periodically and install new versions of the app with no user interaction. This cascades further: unattended app upgrades lead to unattended appspace data migrations. Even more scary!

The big challenge with unattended app installs and data migrations is that it puts all the responsibility on the developer. It means doing things while the user is not present to be the ultimate guardian for what happens to their data. Naturally, we should never count on the end user to be the last-resort safety net. Users click “Yes” if they want the thing, so it should always be up to us to protect users as much as possible. But having a user click “yes” is a convenient cop-out for not doing everything we can to prevent a user getting into a bad situation.

Since apps and appspaces will eventually be upgraded without a user present, I need to do the work make it as safe as possible. Practically speaking, here are some ways I will try to do this:

Granular control

Granular control over automatic steps: user should be able to choose what they’re comfortable letting Dropserver do without them in attendance. Maybe fetching an app listing automatically is OK, and maybe installing the app unattended is fine. But migrating an appspace’s data? No way.

Guardrails

A system of warnings and good default guardrails. Since the user is not present to spot something “fishy” or that “looks weird”, Dropserver has to make those determinations. Some examples: if the license changes, if the app signature changes (not implemented yet), if the fetch URL redirects to a new, different domain, etc… would all cause the auto-upgrade to abort.

Usable Logging

What happened, and what was the outcome? A “log” of all operations, along with the results should be visible to the user and formatted in an easily understandable way. When operations are initiated by the user we can present the outcome of the op to them immediately and discard these results soon after. When things take place with the user absent, we have to stash all these operation results and make them available for perusal. This will not prevent problems of course, but can help the user what happened when something did go wrong.

Multi-step operations

In addition to all the above I want to lay the groundwork for a better UX when creating an Appspace. Currently, the user has to install an app, and then create an appspace for it. It’s really not good that these are two separate steps. In a future version I would really like for the app installation to be part of the appspace creation. The flow should be: drag an app URL into Dropserver, choose a subdomain, make a few selections for permissions if needed, and click “Create”.

This implies that the operation has to install the app and then create an appspace using the new app ID. Not a big problem but if you don’t think about it and plan it well, it’s a good way to end up with some serious spaghetti code.

So what code did I actually write?

After all that thinking I realized I needed to clean things up a bit in my appgetter code, so I refactored some of that to allow me to proceed more easily. Git commit b12d3c.

Then finally I wrote just enough code to be able to fetch and install an app from a URL. It’s incomplete but it’s functional as far as it goes. Git commit 35d6ae.

Currently I create the app website by hand. It’s utterly minimal, just enough to expose a link to a JSON and a few other files. I created bad sites too (missing manifest and package files) to check that error conditions are handled decently enough. I uploaded the sites to Netlify to allow me to test. You can see them here.

Note that part of the “apps from URLs” project is to make ds-dev generate a basic website capable of displaying app information to potential users as well as serve the right files to ds-host for the installation process.

SSRF protection

In the last few days of this month I started integrating an SSRF protection library to the fetch code.

I settled on Daenney’s SSRF library, instead of safeurl. This in spite of the fact that safeurl has 77 stars while Daenney’s has 1. I usually prefer to go with widely used libraries, so here is why:

Neither is really widely used apparently, so it’s a tossup in that category anyways.
Daenney’s SSRF is not a big library and I feel I understand the code well enough. I can patch it myself if need be.
I didn’t like that safeurl disables IPv6 by default. It makes me less confident about the library for some reason. SSRF embraces IPv6 from the get go and from what I can tell the developer did their homework.
I like that SSRF regenerates its list of default bad IP ranges from IANA sources twice a month. Neat!
The blog post accompanying Daenney’s SSRF is informative and confidence-inspiring.

I’m adding ds-host config fields to specify IPs (and ranges) that should be reachable by the instance despite being on a private network. Work is ongoing.

Work on Leftovers and other apps

While I work on Dropserver proper, I also try to spend time on the apps that I write for Dropserver. One reason is that I actually use these apps daily, and they all have papercuts that I am motivated to fix.

I’m currently working on the Leftovers app. I would like it to be in good shape for when ds-dev can create app distribution websites. I’ll use Leftovers as the primary example of that.

What’s Next?

I’ll be plowing ahead with apps from URLs. I’d like to get to the point where apps and appspaces get upgraded when a new app version is available (if user so desires, of course). But I may not be able to make app installation part of the appspace creation operation (see “Multi-Step Operations” above). I need to draw the line somewhere.

By the end of November I hope to have ds-dev generate app distribution sites too. We’ll see.

Dropserver Progress - September 2023

Olivier Forget — Tue, 03 Oct 2023 09:07:00 -0800

This is the monthly progress report for Dropserver for September 2023. The previous report is here.

The Big Picture

I worked on finishing app changelogs and started working on installing apps from a URL. This is all part of “Dropserver App Packaging and Distribution” project. I also spent some time thinking about how to present Dropserver.org, how to make it more easily usable, and whether working on app distribution is the best way to spend my time now.

Thinking and Questioning

When working on a large project over a long period of time there is always a risk that going on vacation will cause a rupture in the sense of progress that may trigger a slide into apathy. Luckily I don’t feel apathy, but the break has forced me to think about where Dropserver is at and how I get it to where I would like it to be. I went down these mental quests:

Low Friction Dropserver Offering

It’s no secret that I’m the only person who runs a Dropserver instance, and my wife is the only non-admin user in existence. That’s fine, for now. DS still has a lot of holes, and installing and running an instance is too demanding right now. But this can’t go on forever. I need to think about how I can make it easier for people (even technical users) to try and use Dropserver.

Ultimately the easiest way for someone to get on Dropserver is by signing up through a commercial hosting service. So that’s what I need to work towards. (I have known this for a long time of course, but every once in a while I have to sit and ponder every other option before returning to my starting point.)

But even with that, I would still be suffering from a lack of users because of a lack of apps.

Re-Orient Dropserver.org Website

A developer posted a cool project built on Glitch. Since this is the kind of project I’d like to see built for Dropserver, I wondered if a developer like Casey would consider developing an app for Dropserver if they even heard about it? Looking at the Dropserver.org home page with fresh eyes, it became immediately clear it wouldn’t do the trick. It’s focused towards the end-user, who scientifically speaking can not exist in this world right now. So what’s the point of pitching them?

Then, while looking around for mentions of Dropserver on various sites, I found this discussion on Reddit that shows that there are many home application server offerings. A comment by ocdtrekkie spells out exactly the problem: how do you differentiate yourself? The Dropserver.org doesn’t tell a developer why they are better off developing for Dropserver. Yet developer experience is one of the differentiating factors that I want to build into Dropserver. I need to rethink the Dropserver.org site with that in mind.

What to Work On

The problem with spending so much time thinking is that nothing gets built, and everything comes into question. For a period I thought I might punt on installing apps from distribution URLs. I decided to carry on due to the realization that the order of things I release doesn’t matter: it all needs to get built sooner or later. The worst thing I can do given my limited resources is to stop mid-project to focus on something else because I imagine somehow it will serve DS better. This is wasteful, and I should just build the thing that I can build most effectively right now.

After spending so much time on app installation and distribution, I should just knock it out and move on.

Some Code Progress, Finally

Finish Work on Changelogs

I talked about changelogs in the last update. On September 13 I committed the changes.

Work on Apps from Websites

The goal for Dropserver version 0.12 is that a user should be able to install an app by pasting a URL where that app can be found. Not only can the app be installed using this method, but it should be possible to fetch updates and install the latest version, and do so automatically if the user so desires.

A further constraint is I want developers to be able to publish their apps as static website. Static website hosts are widely available with a generous free tier, and developers routinely host their blogs and other sites on them. The idea is that if ds-dev can generate a set of files then the developer just has to upload them to their static host of choice, and the app is published.

All this presents me with a number of questions:

What is this URL? What does it look like?
What data resides at that URL?
What other files need to be generated?
How will internationalization and localization fit into all this?

The current design calls for the URL to point to a JSON file that contains an array of versions, each with URLs to fetch the complete manifest (JSON), the package file (tar.gz), the icon file, and the changelog file (plain text).

URLs

I would like for URLs to be clean, so a user should be able to paste olivierforget.net/leftovers/ to get the leftovers app. But that URL should serve a HTML page describing the app. I could use HTTP Headers to tell the server I want the data file, but remember I want this to be trivial to host. I don’t want the dev to fiddle with settings in their static host, which is usually required when you want to serve different content based on headers.

So I am thinking of using either a .well-known path like this:

olivierforget.net/leftovers/.well-known/dropserver/app-listing.json

My fear is that the interpretation of trailing slash in the original URL will cause some problems.

The other option would be to put a <meta> or <link> tag in the HTML page served that points to the data file, however it’s not clear either of these are appropriate for the task.

For now user will have to paste a complete URL to the JSON file.

SSRF Protection

The first thing I ran into when starting to actually implement fetching that URL is the potential for SSRF — Server-Side Request Forgery. Fetching apps remotely can’t be an avenue for probing your local network.

While mitigations for SSRF are well understood, they present a problem for home-lab / self-hosters: the main idea of the mitigation is to prevent connections to servers that have a non-public IP. But if you are hosting at home or on a private network, well, that’s all your IPs.

After asking on Mastodon it seems the solution is to block all connections to local addresses except those white-listed by the instance admin via a config. This is probably the direction I’ll take.

Here is a good post on preventing SSRF attacks in Go, and a library to go with it.

What’s Next?

In October I’ll continue working on fetching apps from URLs and on the generation of the files necessary to host an app.

I am still being pulled away from Dropserver by unrelated projects, so I don’t expect to be able to make leaps of progress, but at least I shouldn’t spend too much time thinking!

Dropserver Progress - August 2023

Olivier Forget — Fri, 01 Sep 2023 10:07:00 -0800

This is the third progress report for Dropserver. This one will be very short thanks to a much needed vacation during most of the month. See last month’s report

The Big Picture

After the release of version 0.11 I took some time for exploratory thinking. Upon my return from vacation I went to work on the “distribution” part of “app packaging and distribution”.

Exploratory Thinking

I18N

I collected links to tools and libraries that are used for translating apps with the goal of some day doing the work of making Dropserver localizable.

Tailscale

Tailscale has the potential to give users access to a self-hosted Dropserver instance without exposing that instance to the broader internet. I spent a bit of time trying to learn more about what it can do. I collected a few videos to watch during my vacation (I did not find the time to watch them 😑).

Back to coding

I’m back at work now writing code (sluggishly). The next release will make it possible to publish apps and Dropserver users will be able to get automated upgrades to the latest app version (opt-in of course).

App Changelogs

I’m working on integrating release notes aka “changelogs” in packaged apps.

I did a bit of research and found some interesting things about changelogs in major platforms.

I had punted on changelogs for v0.11 because I do not love the naïve approach I could take: simply include a markdown file with all release notes for shipped versions in reverse chronological order. I would like to be able to pick out the release notes for specific versions. A lot can be done to present things usefully for the user once you can pick individual version notes.

Another concern is the security implication of showing markdown content created by 3rd parties in the user’s admin panel. This can not be the source of a vulnerability. To show markdown (or HTML) safely I would have to filter the content through a library like bluemonday on the server side, and display it in an iframe to gain two layers of protection against potentially malicious content. This is more work than I’d like to do right now, so I’m sticking to plain text.

I decided to make plaintext file (not markdown) for changelogs, where the changelog for version x.y.z is preceded by a line consisting solely of “x.y.z”. With that, devs can easily write changelogs in a text file and the system can easily parse that file to extract the changes for a given version.

Having a structured changelog format allows me to do the following:

warn if no changes entered for latest version (each version should include a changelog)
present other metadata in listing of changes (release date, package size, API version, etc…)
show the changes made between the currently installed version and the version about to be installed (useful if user did not upgrade for a few versions)
if upgrade includes a major rev, highlight the changes for the major release (x.0.0) which are usually lengthy and describe meaningful changes
if some changes have been translated into the user’s preferred language, show these instead of the developer’s original version.

What’s Next?

August was mostly spent not working. This Fall I will be busy with a number of non-Dropserver things but I should still be able to make good progress.

I’ll be working on app distribution and automatic upgrades which will result in release of version 0.12.

Dropserver Progress - July 2023

Olivier Forget — Tue, 01 Aug 2023 09:07:00 -0800

This is the second progress report for Dropserver. This one will be modest unlike the previous epic.

The Big Picture

I continued to work on app packaging and managed to tag a release. To celebrate, I immediately went to work fixing some of the things that didn’t make it into the release.

Finishing Up App Packaging

App Name and Description

I had to create validations for app name, and decide how to deal with violations. The main factor is app name length.

App name length

I had to decide how to limit application names and short descriptions that are entered by the app developer in the manifest.

Every app platform I have come across limits the length of app names and other strings. Names should be limited in length because it makes it easier for users to scan when apps are displayed alongside other apps. But how does a hard limit square with the realities of responsive design? A name might be cumbersome to display on a phone screen but have lots of room to spare on a large display.

Let’s be honest: maximum app name length is completely arbitrary. For this reason I settled on an arbitrary value, but only give a warning and not a fatal error if the name is too long. I’ll deal with exceedingly long app names using CSS.

Don’t be naïve: count graphemes

The naïve approach to counting the number of characters in a string is to count the length of a string. This is quite wrong unless you’re dealing strictly with ASCII characters. The answers in this StackOverflow question cover a lot of ground.

Thankfully I found a good Go library that counts graphemes.

Commit e495dad.

Remove Bad URLs From Manifest

In earlier work bad URLs (like Javascript:) were marked with a warning, but that meant they were still displayed in the admin frontend, which is exactly what you don’t want. Ideally the bad urls would be visible but not rendered as a clickable link.

I don’t have the code paths to do this easily and reliably, so I do the easy thing and overwrite bad urls with an empty string.

Commit 8738eea.

Specify Package Base Filename

When you create a package file using ds-dev, you now specify a base filename. So:

ds-dev -create-package=/path/to/dist/ -app=/path/to/app -pagkage-name=leftovers

…creates leftovers-0.1.2.tar.gz.

I thought about putting the package name in the manifest, but it doesn’t make sense. It’s not a property of the app. It has no meaning. The package could be called anything and package file names can change from version to version, the system doesn’t care. For that reason it’s an argument to the packaging tool.

Commit 0222007.

Set Arbitrary Reasonable Manifest Size

While limiting app name length (as well as for other strings in the manifest) is mostly a UI issue, there remains the possibility that someone will abuse the manifest with absurdly long strings, hoping to cause the system to have to deal with all that data will cause a DDOS of some sort.

To prevent this I decided that manifests should only be so big. You can do anything you want, but beyond a certain size, the manifest will not be processed at all.

This works because there is a lot of room between the size of a reasonably large manifest and the amount of data needed to cause problems in ds-host.

The max size is 10kB. Commit bbbaf0c.

Validate Icon Image

I added validations for the app icon. Using a go library I check the icon size in pixels, I check that it’s square, and I warn if the icon size is too large (in kB).

Along the way I learned that you can compress PNGs a lot with tools like tinypng.com/ (which are probably wrappers aroudn open source tools).

I don’t allow gifs for app icons, but I learned that PNGs can be animated. I don’t want app icons to move in the admin panel, so I check that the PNG is not an animation.

Commit 8daf0f8.

Sometime after this something happened in the social media world: The Twitter app was rebranded to “X” and the icon changed. At least one app platform warned its users about the change:

See this post on Mastodon. This made me realize that I would probably eventually validate that the icon has not changed significantly from version to version. This will require doing a similarity analysis between icons, luckily this go library might work just fine.

Domain Change Tool

I wanted to test all the changes I made for App Packaging against my personal “production” instance of Dropserver, but I did not want to upset this instance at all, since I rely on it. Downloading all the data from that instance and sticking it on my dev VM is easy enough, but the problem is that there a lot of places in the DB and appspace data that stores domain names.

I decided to start work on a “domain change” tool in a branch (I won’t release it for a while) that would go through the DB and appspaces and change the domain name. That would allow me to take the data of any instance of ds-host and change the domain names so that it can run under a different domain.

This is useful for testing purposes. But it’s also handy if a user loses control of their domain and they want to reboot their instance under a different domain, for example.

Work is in the change-domain-tool branch.

After digging through to find all the spots where a domain is written, I had a tool that did the job. It worked well to test all the changes against my “prod” data. A nice boost in confidence for this release and a useful tool already showing value.

Documentation

I wrote a reference page for the app manifest.

I also wrote a high level overview of how to build a Dropserver app. It is meant to give potential devs a quick view of things that are different with Dropserver app development, along with some gotchas.

Finally I also added a page for packaging a Dropserver app. It’s a stub really for now.

I found myself struggling a bit writing these docs. On one hand I want things explained. On the other, each word of docs I write needs to be maintained. Outdated docs are worse than no docs. So I have to be concise and direct, so I can stand a chance to keep it all fresh.

Release v0.11.0 🥳🎉

Finally on July 19 I released App Packaging. Mastodon post.

Post-Release Fixes

First thing to do after releasing is to fix and tweak the things that were missed in the release.

Tweak Migrations Validations

In the released version of Dropserver, an app must provide migration functions in pairs (a down-migration for each up-migration). This is to encourage devs to give users a way to back out of an app upgrade that they are not happy with.

Unfortunately the mere presence of a down-migration function does not guarantee that it will do its job satisfactorily. I worried that a dev faced with a fatal error because of a missing migration would simply add a stub for the down migration. This is the worst possible outcome, because the system will assume the migration took place, leading to data corruption.

I decided a warning would be plenty good enough. Commits 3f19c30 and 502448e.

Display Migrations in a Grid

In both ds-dev and ds-host I wanted to show migrations that ship with an app as clearly as possible. I settled on a grid layout which I think is satisfactory.

Commits 71bd4b1 and 0dc3c20. Yes, there is an alignment problem, I know.

ds-dev UI and Code

I changed all Vue component code to use the <script setup> style. It’s a huge improvement in developer experience but refactoring each component is some very boring labor. This is where I wish I were a bit more adept at using AI to code because it would do quick work of this task. Oh well, it’s done.

What’s Next?

The big thing I’d like to work on next is app distribution and the ability to install and update an app automatically using just the URL at which the app is published. This is the “Distribution” part of “App Packaging and Distribution”.

But right now I’m headed for a vacation during a good chunk of the month of August so I am not going to start on that until September.

See you then!

Dropserver Progress - April to June 2023

Olivier Forget — Fri, 07 Jul 2023 16:07:00 -0800

I am going to try writing monthly updates on the progress of Dropserver development. Regardless of whether I have anything significant to share, I’ll post about the past month’s work. I’ll look at my commits and my notes (I take copious notes, arranged chronologically and in threads, but that’s the subject of a different post) and summarize what I worked on.

Since this is the first such post it will cover all the work I’ve done from my last release in mid April until the end of June 2023.

The Big Picture

I am working on application packaging. The goal is to make it easy for someone who wrote an app for Dropserver to make it available to others, and for non-technical people to make use of that app in their Dropserver instance.

The current version of Dropserver allows a user to upload a directory of app files. This is error prone and makes it hard to share apps around. Furthermore, the so-called “manifest” has very little useful information in it (basically just app name and version).

Here is what I had to do:

come up with a package format, including the manifest and the archive and compression format
create the tooling that creates the package and integrate it into ds-dev
adapt ds-host to accept these packages and install the app
update the frontend of ds-host to show app package values and warnings

This has been a big project. I haven’t released a new version of Dropserver since April 10th 2023, but hopefully I should be able to release this sometime in July.

Details

Note: I am developing this in the app-package-1 branch.

Exploratory thoughts

As is typical with how I work, I first had to explore the outer fringes of reasonable ideas before accepting that would I have to build is pretty straightforward and follows in line with prior art.

Do I even need a manifest?

What’s so necessary about a “manifest”? Dropserver already runs the code to get metadata about the app (such as available migrations) and perhaps putting everything in app code is the simplest thing to do for developers?

Unfortunately this doesn’t work for some fields: the entrypoint of the code can’t be set in code! And even if all data could be obtained by running code, Dropserver would still need a manifest format to stash that data. You do not want to run a sandbox every time you want to know something about an app. Once you have that, why not let the app dev write into it directly instead of cluttering their code?

“Single File Apps”

Wouldn’t it be great if you could write a single TS/JS file and have it become a web-app without doing any packaging work. You could share it around as a GitHub Gist or anywhere else. (Mastodon post)

That sounds cool but the reality is that most web-based things are inherently multi-file. If you serve a frontend, even a simple one, you’ll want HTML templates and a style sheet. I realized the value of “Single-File Apps” was low and the additional complexity in Dropserver was not worth it. Instead I am focusing on making packaging easy thanks to built-in tooling in ds-dev.

App distribution site

I don’t want Dropserver app devs and users to be beholden to a single centralized app store. It should be possible for an application developer to publish their app on a website that they control, and users should be able to install it from there. The independent and open web demands this.

I had originally thought that such an application distribution website would actually be driven by a Dropserver application, but at this stage it’s better to do something simpler. An app distribution site will just be a static site hostable on any static website host.

This is one idea I have not abandoned. I’m merely punting on it for a future version, but it’s still the plan.

Figure out the manifest

Having decided I needed a manifest, I had to decide what fields would go in it, what format these fields would be, and how they would be validated. I created a GH Project and an Ethercalc table to work through the scenarios and possible fields. (Note that these are outdated at this point and do not reflect all my latest thinking.)

I looked at manifests for Android (not very relevant), NPM’s package.json, Sandstorm, YUNoHost and others for inspiration. All were informative but I found some cool ideas in YUNoHost, like pre-defined tags for anti-features and a system for determining application quality levels. I’m not implementing these ideas yet, but I may in the future.

I had to decide on TOML versus JSON for the manifest file (mastodon link). I decided to use JSON. TOML is lesser known and still has detractors, while JSON is nothing new for JS devs, even if it’s not ideal. Additionally, the packaging process writes its own version of the manifest, so JSON is a bit easier to work with.

Package format

Zip or tar.gz or something else? See this Mastodon thread. I found tar.gz to be a better bet. It’s well supported and well understood. It’s fairly safe. No patent questions, unlike Zip.

Data backups of appspaces are saved in Zip format right now. But that’s because I expect end-users to want to open these locally. There is no reason for a non-dev to manually open the archive of an app, so it’s OK if it’s in a less popular archive format.

Packaging flow

I worked out that the app developer would write a partial manifest. Currently the only absolutely required field is the app version. They create their app package using ds-dev -create-package which runs the sandbox to obtain additional data like data schema, available migration functions, and others. These get added to the manifest, and this “augmented manifest” is placed in the package instead of the dev’s original. Additionally the augmented manifest is saved locally.

I do this because I want the dev to write a minimal manifest if possible, but some fields should be available without having to run a sandbox. So the augmented manifest serves as a cache for values determined at runtime.

Installation flow

A user can install an app by uploading an app package to their Dropserver account (wherever it may be). I wrote the necessary pieces to enable this flow:

ds-host unpacks the archive carefully to prevent archive-based attacks. In particular it has an expectation of a maximum size for a package, and bails on the process as soon as it hits that maximum. The max is hard-coded right now, but it will tie in to the user disk quota mechanism when that gets written.
It reads the manifest and runs the sandbox to obtain the values that are available at runtime. It validates all the values (note that it uses the values it gets from the sandbox instead of the values of the manifest where it can.)
If the user commits the new app, it saves the extracted files, and it places the augmented manifest in a JSON column in the DB. It also keep the original package file.

Add fields to the manifest

After I got rudimentary packaging and installation done, I added handling for more fields of the manifest:

App entrypoint: This is known as main in package.json. It’s the file that the sandbox runs. It used to be hard-coded as app.ts. Now it defaults to app.ts and app.js, or to whatever the manifest says. Here I had to be careful that it is not possible to make a path that climbs out of the app package.
App icon: This will be shown alongside the app name in ds-host UI and helps differentiate the apps. The app icon value in the manifest is a path relative to the package root to an image file. Naturally I had to take care that this would not be used to read files that are outside the package. I still have to put constraints on image size in pixels, as well as file size. Eventually expect to use something similar to Android’s Icon spec to guide icon design.
Accent color: A color that the dev chooses that complements the app icon or matches the UI of their app. This color is used to decorate a box that gives information about the app in the ds-host UI. It’s never used as a background or foreground color for text so it does not affect accessibility. To make things easy for the dev any CSS color is accepted. I found a CSS color parser in Go which made that nice and easy.
Authors: An array of author objects each with name, URL and email.
URLs: I added fields for app website, code repository and funding. Then I realized that there is such a thing as the javascript: protocol, and just displaying these URLs as links in the ds-host UI could be a terrible idea. So they get validated as proper http(s):// URLs. If they fail they get removed from the manifest upon app installation. I process author URLs the same.
Release date: A date in GMT that is automatically added to the manifest at package creation time. Its sole purpose is to let the user know whether the version was released recently or not.
License: I decided that the license field should preferably be an SPDX string. If it’s not you get a warning. The reason for using SPDX is that I may want to give users a quick synopsis of the license but I can only really do that if I can correctly identify it, and SPDX is really made for that. Furthermore in the unlikely event that a license is incompatible with Dropserver, then being able to positively identify it will be useful.

ds-dev frontend

I updated the UI of ds-dev, the local dev-oriented app runner, to show all the values and warnings about the app and the manifest that a user would get if they installed the app on ds-host. This should make it easy for the app dev to correct any issues before shipping.

ds-dev runs the same code to validate the app as ds-host does when it installs the app (as does the -create-package command) therefore the developer sees all the warnings the user will see (with the caveat that different versions of these programs may behave differently).

App version model

With so much metadata stashed in the manifest, I had to figure which fields were needed and in what circumstances. I don’t want to return the entire manifest every time some function needs to know something about the app version.

After spinning my wheels for a while I figured that there was an internal use case that is used to run the sandbox and do other operations, and only needs values related to that (entrypoint, and a few others). Then there is the basic “UI” case, where we are presenting the app to the user and need short-description, authors, urls, icon, color, etc… Finally we sometimes do need to get the entire manifest so there is a method for that.

In the first two cases I make use of sqlite’s JSON functions to cherry-pick desired values from the JSON manifest (which is stored in a JSON column in the db). I am happy with this. It means I have the whole manifest available, or any subset of it that I may need.

ds-host frontend

I did a lot of work on the ds-host frontend. With data like icon, color, license, authors now available as metadata for the app I was able to make the display of apps more rich and interesting.

See screenshots in this Mastodon thread.

Punting

I had to punt on a lot of things. It’s hard to do but it’s necessary to keep momentum. I’ll tackle these later:

signatures for app packages: needs to be done right, I want to spend more time thinking about it.
internationalization of manifest values: I need a Dropserver-wide strategy for I18N, which I’m thinking about but it’s not there.
release notes: surprisingly tricky to do well. I could take the easy way out (one md file with all release notes), but I want to see if I can improve on the status quo here.
get app from URL and auto-updating: that’s another huge chunk of work that I will do in a different release.

Looking Ahead

I’m at the point where I’m finishing up loose ends and doing testing. I still have a lot of docs to write but I hope to release this soon.

After that I’d like to go straight to work on installing an app from a URL, and everything that comes with that, but I may have to take a step back to see if that’s really what needs to happen at that point in time.

In the future I may add a number of other fields to the manifest, such as:

age appropriateness
anti-features (like YUNoHost)
features / capabilities listing (like ocdtrekkie suggested)
package size / installed size
whether additional resources are needed or if the package is self-contained

Other big ticket items that are in need of attention for Dropserver as whole:

I18N
Appspace user management and instance-to-instance data sharing for users
More powerful app capabilities: enable outbound requests (with permissions), support Server-Sent Events, and many more…
Working towards a hosted version of Dropserver
And countless other stuff…

Lots to do. See you next month!

The State of Sandboxing in Dropserver

Olivier Forget — Thu, 02 Mar 2023 13:07:00 -0800

One of the bigger challenges of developing Dropserver has been to somehow make it safe to run the user’s application code. In this long-ago post I relayed how I tried a number of different approaches, all of them being too difficult to make work until Deno arrived. Naturally that was not the end of the story.

But first…

Why Is Sandboxing Important in Dropserver?

The goal of Dropserver is to make it possible for regular users to run server-side code of their choosing. Currently, for all internet users except for a very slim minority, touching anything “online” means interacting with a server over which they have no control. Dropserver aims to change that by giving regular people the ability to do internet things without losing all control.

Dropserver users are able to upload and run “apps”, which are basically a pile of JS code written by a well-intentioned developer. While the “app” may include a frontend that will run in the browser, the real value of Dropserver is that the app can run server-side code.

An arrangement like that is rife for abuse if no guardrails are in place. That “well-intentioned” developer could actually be trying to gain control of the machine Dropserver runs on. Even if the developer is a goodie, compromised dependencies could seek to turn a nice app into a vector for an attack.

As a result Dropserver treats all app code as potentially malicious and gives it as little access as possible to protect the safety of users running apps, the admin hosting the instance, and everybody else on that instance.

My goal is that a Dropserver user should feel free to install apps just to kick the tires, even if they don’t know who the developer is and won’t be bothered to read through the code looking for vulnerabilities. Dropserver should feel safe. I think this is essential for the ecosystem to develop.

Deno is the Sandbox

Deno claims it is a secure runtime, and it’s likely fairly good at that. Through its use Dropserver can limit what an app can do, such as where on disk it can write and whether the app can make network requests.

However no software is without bugs, and even in the absence of bugs our current reality is that Spectre is a vulnerability that is pervasive across all running code. (Note there are some mitigations against Spectre attacks built into Deno: for example high-precision timers are behind a flag.)

In any case we should not assume Deno is and will remain airtight. For this reason a second layer of protection is essential. This is especially true since there is a good chance many Dropserver installations will not update Deno in time, and will therefore be vulnerable if any of their app code decides to take advantage of the vuln.

Bubblewrap is the Other Sandbox

Julia Evans aka “B0rk” has a neat writeup where she was looking at various sandboxing tools, and found Bubblewrap. Bubblewrap basically takes the challenging work of creating Linux namespaces for your sandbox in code and makes it accessible in a convenient command line utility.

Bubblewrap can set up Linux Kernel sandboxing features by specifying a few flags. It also helps make your sandbox useful by binding directories or files from the host system into the sandbox. Without Bubblewrap or another similar tool I’d be left with calling Linux system calls directly which would be more error-prone.

After some playing around, I feel so much better letting bwrap do this work rather than trying to code it myself.

It’s interesting to note that another app platform, Sandstorm (and now Tempest), roll their own C code to achieve a similar goal. There has been talk of leveraging Bubblewrap, but since they’ve already done the work and they trust their C code, they are sticking to it. More power to them (and more responsibility :))

Dropserver 0.10

In Dropserver v0.10.0 I added support for wrapping Deno instances in bwrap. Here are some notes on the current implementation:

Deno is dynamically linked and therefore needs access to /etc and some lib directories to run. Not great. I’m hoping we’ll get a statically linked version of Deno some day. This is being talked about.
I do not have an HTTP proxy for the sandbox yet so I have to give access to /run on certain systems so that it can do DNS lookups.
Deno requires /proc dir. With --unshare-pid that’s not a big problem since the processes in /proc will only be bwrap and deno.
Unfortunately I am not using --unshare-pid right now. It causes bwrap to give me the pid of the forked bwrap, not the deno command that I am running. I’ll spare you the details, but while this is something I can work around I decided it was time to ship.
I realized that DENO_DIR, where deno caches imported modules, etc… can be a vector for abuse. It has to be read-write in the sandbox, so if someone breaks out of Deno, they can alter cached code files and therefore compromise other apps without even touching them. Yikes. For this reason each appspace gets its own DENO_DIR. It slows the initial start, but after that it’s no different.
Since DENO_DIR is per-appspace, Deno makes many requests to the outside to obtain all the imported code the first time the appspace is run. For this reason in the current implementation the bwrap sandbox must have full net access.

As you can see there are a lot of caveats to Dropserver’s use of Bubblewrap for now. The good news is that there is a functional starting point that I can build on.

Roadmap

Here is how I see Dropserver’s sandbox evolving:

If/when a statically linked or musl-compatible version of Deno is available, stop binding all the libs and /etc in the sandbox.
A HTTP(S) proxy would make it possible to not bind the parts of the OS responsible for DNS lookups. (Such a proxy will happen alongside the feature that allows apps make outbound http requests.)
I spent some time experimenting with Linux virtual networking, such as following this post, but unfortunately I could not come up with a combination of veth and whatever that did what I needed. (Some networking stuff requires privileges, which my use of bwrap does not have). However, creating a unix socket that is accessible from in and out of the sandbox is very straightforward so I expect to make the proxy accessible through that. Note that for now Deno doesn’t understand proxies as unix sockets, but that may change. Deno leverages Reqwest, which is considering this feature. Just have to be patient: the reward will be turning off network access entirely for the bwrap sandbox.
Look deeper at user namespaces. I’m using --unshare-user-try, I need to look more at what happens when the user namespace does not get created.
Dig deeper into Spectre to ensure I’m doing everything I can to mitigate.
Seccomp. Bubblewrap has the ability to limit kernel calls via seccomp, but I have never tried that. This should make for a much tighter sandbox.
There is a new set of tools that can help isolate a process called Landlock. I would look into that however I’ll probably have to wait until Bubblewrap implements it, which for now is not planned.
Explore the possibility of putting Deno installation under ds-host’s control. That way it could automatically upgrade to latest patched versions as they become available.

My hope is that eventually Dropserver’s Bubblewrap sandbox will be lean and effective. Having this second layer of security around Deno makes me much more comfortable with the prospect of eventually running a Dropserver service and letting others upload and run anything they want (scary!)

Note that the work of sandboxing is never done. Witness the work that Cloudflare puts into sandboxing its Workers.

Sandboxing is, of course, only part of the picture, but it’s a critical part and I’m glad to be moving forward with it.

A Secret Santa App On Dropserver

Olivier Forget — Mon, 12 Dec 2022 13:07:00 -0800

Inspired by Simon Willison’s post on making a Secret Santa app as a Datasette plugin I decided to write a minimalist Secret Santa app for Dropserver.

Dropserver is my attempt at building a platform for hosting my own personal apps. A primary objective of DS is that it should be really easy for a developer, particularly a frontend developer, to write an app that can be hosted by Dropserver.

DS accomplishes this by using a JavaScript runtime (Deno) and by providing many of the hard parts of backend apps baked in. This means that in a few lines of business logic, written in a language you likely know well, you can build a useful app with many of the bells and whistles of a bigger SaaS app.

Writing the app

Anyways, let’s get going.

This project builds on the Dropserver app tutorial. We will use no new concepts, we will just add more business logic, more routes and more static files to serve (CSS, a font file and a santa-hat.png for cuteness).

Here is the GitHub repo if you want to follow along.

Users

In its current incarnation Dropserver does all the user management for the app. That means the app developer does not need to code any user registration login, password storage, etc… Users even have avatars, which helps make apps more friendly looking.

All the app needs to do is call the getUsers() method to get a list of users. Users are identified by a “proxy ID”, which is a string of alphanumeric characters that is stable and unique for each user. Personally identifying information, such as login email, is not accessible from the app.

As I began writing this app I remembered that I have not yet implemented the user permission system for Dropserver apps. That means that each user is at the same level: and there is no way to distinguish an admin user from a regular user. This is something I will add in a future version of Dropserver, but for now I’ll just have to be a bit clever about when pairings can be generated.

The app will detect whether Secret Santa pairings have been been made already and if the list of users in the pairings match the list of existing users for the app. If not, it will offer to generate the pairings. If a user tries to regenerate the pairings when they have already been generated and match the user list, they will get a message saying they can’t do that.

Once the pairings have been generated a user signing on to their Dropserver and clicking through to the appspace will see who they are paired with, as shown below:

Generating pairings

To generate the pairings, we will shuffle an array of users, then make a pair out of a user and the one preceding it in the array. This is simple and guarantees that you don’t pair a user with themselves.

We write the pairs of proxy ids as a json object to a file in the appspace. (An appspace is an app instance, an address where this app can be reached, and a dedicated directory where it can write files. For more details see Dropserver application model.)

While there are no routes that show all the pairings, the user who owns the appspace could download a backup and look at the json file and the user metadata to know what the pairings are. I think this is reasonable for such an app. I could encrypt each pairing like Simon did, but it means asking users to remember yet another pass code. I suspect the most likely outcome is someone not getting a gift because a user forgot their additional pass phrase.

The rest of the app

The entire server side code fits in a single 120 line TypeScript file, including comments and blank lines.

The browser side consists of plain HTML responses generated on the backend via Mustache templates. Hand coded CSS is served statically along with a few image assets and a fun little Christmasy font.

There is no build or packaging step for the frontend or for the Dropserver side.

Hosting your own Secret Santa App

If you have a Dropserver instance running, you just upload the app directory, and click through a confirmation. Then you choose an address (usually a subdomain on a domain you’ve already configured) for your appspace and start adding your users.

Note: you should be running ds-host v0.8.1 or later (I ran into a bug while writing this app and had to push a fix 😬). Documentation on running ds-host is sparse for now. For hints see the Readme.

Currently each user must have a Dropserver login to access an appspace, however it can be on any instance. In the future, users who don’t have a login will be able to access an appspace with just an email address where they will receive a secret login link.

Playing with this app locally

If you don’t have a Dropserver instance handy (and let’s face it, in December 2022 if your name is not Olivier Forget you probably do not) you can install ds-dev (v0.8.1 or later) and Deno to run it on localhost. Using ds-dev you can emulate users to see the different pairings show up. Read about how to use ds-dev here.

Go and fork it

The Secret Santa app is usable as-is, yet there is plenty of room for improvement. The good news is: there is no build step, it’s just plain HTML and CSS, so if you don’t like the background color, just fork it, open it in ds-dev, and tweak it to your liking.

I wrote this app to demonstrate that Dropserver is a platform that makes building simple, almost toy-like multi-user server apps approachable. You can make something basic but useful and fun and you can host it and use it with your friends. Others who want to use it can host their own copy with just a few clicks. And if you want to change something, it will only take a couple minutes to understand the code.

Happy Holidays! I hope your Secret Santa, whoever they are, and however they were selected, gets you something nice.

Has Deno Turned a Corner?

Olivier Forget — Fri, 21 Oct 2022 13:07:00 -0800

Deno is NodeJS reinvented. I like writing code for Deno, and I depend on it as a sandbox for Dropserver after trying many other approaches.

Unfortunately the JavaScript community at large does not seem to be embracing Deno as much as I am. It seems many people are simply sticking with Node for now even though Deno 1.0 has been out since Spring 2020.

I posted this on Mastodon last February: Deno’s mindshare in 2021 was in the single-digits and had only improved by one percentage point a year later. Yikes! Not a good place to be.

See State of JS 2020 survey results and State of JS 2021 survey results.

Why would people not make the switch? I think there are two reasons:

Most legacy code does not run on Deno. Deno is taking a clean sheet approach, and shunning the old Node Package Manager. It’s also all in on ESM modules, while most Node code is CommonJS. So the legacy code is (was) largely incompatible. For developers who are used to finding just about anything on NPM, looking for packages on deno.land/x/ must be a disappointment. For developers of large projects, such as Vite’s Evan You, doing the work to support Deno was simply unthinkable.

Furthermore, while Deno is an improvement over Node, it’s not revolutionary. I remember hearing Ryan Dahl (Node and Deno creator) talking about the early days of Node. He said people would just show up and write packages as they needed them, which quickly filled the void. However this time it’s different. When Node came out, it really stood out. There was nothing like it. Developer experience on most other platforms of the time was poor. So naturally developers did what developers do: they went to town and built it all.

Today there are plenty of fantastic platforms and Deno is not a sufficiently revolutionary step for developers to drop everything and write the libraries that are needed. It’s easier to just stick with Node.

What is Deno doing about it?

The Deno core team seem to have realized that the ecosystem is growing slowly and that the easiest way around that is to make it possible to use NPM dependencies in Deno projects. They’ve taken a multi-pronged approach to the problem: from transforming library CDNs to a compatibility mode in Deno itself. It’s a little messy but at least it’s now possible to use many of the code that assumes Node is the runtime.

Today I see reason for hope

I keep seeing hints that Deno is being embraced by the JS community. I don’t know if what I’m seeing now is the result of the NPM compatibility effort or if there is something else. It could just be that a good project that hangs around long enough eventually gets some uptake.

I saw a tweet the other day showing that Deno can build a Vite project. That’s a big deal. Remember the link above to a comment by Evan You saying it was too early to support Deno? It turns out it’s the Deno devs who made most of the effort to get Vite running, but that doesn’t matter. With Vite functional in Deno, all the dependent projects are suddenly much closer to supporting Deno. Many more good things can happen.
While researching the above, I found that Astro.js, a new better Static Site Generator with Server Side Rendering capabilities, officially supports Deno as an SSR runtime. This time it looks like it’s the Astro devs who put in the effort which shows devs backing Deno with their sweat and blood (OK, maybe not blood, not yet at least). It’s one thing to say “Deno looks cool”. It’s a different matter to release official support for Deno with libraries and docs.
I just learned that Supabase, a well funded and growing alternative to Firebase, chose Deno for its Edge Functions. Again, more developer awareness and commitment already on display from the Supabase devs. It adds to the growing list of platforms that have built-in integration with Deno, such as Slack and Netlify. Anybody who wants to write functions for these platforms becomes a Deno developer.

None of these are significant events, but to me they show a trend that will only build in the future. They make me optimistic. Will the 2022 JS survey show an uptick of Deno developers? Time will tell, but I expect it will. I think Deno is turning the corner.

Unpacking Moxie's "People don’t want to run their own servers, and never will"

Olivier Forget — Wed, 12 Jan 2022 14:27:00 -0800

Moxie Marlinspike’s article on web3 has resulted in huge amounts of conversation online about various aspects of the web and decentralization. One aspect that got a lot of attention, and one that I paid particular attention to given my current project is this statement:

“People don’t want to run their own servers, and never will.”

— Moxie Marlinspike

I want to unpack why I think it’s very challenging to get non-technical end-users to run a home server of some sort, yet it’s not something I’d qualify as “will never happen” either. Apart from the obvious folly of declaring that some tech thing will “never happen”, it already is happening in some ways, and we can learn a lot from that.

Anecdote: during the recent holiday gathering one family member treated us to festive music by streaming it from their NAS back at home in an other State. This clearly counts as hosting from home, yet our DJ was not a hobbyist sysadmin by any means.

So how does this happen? People can and will run a server at home, it’s just a matter of getting the package right and creating a product that is compelling, responds to a need, and lets them get on with their lives.

Let’s go over a few aspects of what can make home serving widely successful or not.

What are we serving?

First up, what is this server for? A “server” is a generic term that could imply a myriad services and many more protocols. All these are not created equal. If you create an offering that lets people serve from home, what you choose to serve will have a big impact on how successful your users are. Here’s an incomplete list of things you might want to serve:

files
eth
email
HTTP
chat
tor
ipfs
etc…

Given the comments I’m seeing it seems some people assume that just because you mention a server, that must mean a generic machine on which any of these protocols can potentially be served. This is one way to view things, but I think it’s more useful to realize some services and protocols lend themselves better to home serving than others, and winning this battle means picking the path of least resistance instead of fighting along all paths at once.

Consider email for example. There are software packages that include everything you need to host your own email server at home. However a common complaint is that emails sent from such a home server have a hard time landing in Gmail’s inbox. This is due to the overwhelming amount of email spam and email providers take any opportunity to label your email spam unless they can be sure it’s not. The burden of proof is on you, the sender, and its a big burden.

Compare that with HTTP: it’s much easier to successfully serve a web page. Sure, it’s possible to trigger various browser’s safe browsing filters, but if you manage to avoid serving anything nefarious there is no reason for a browser to block you. All you need is an IP, a domain name, and a TLS certificate (free from Letsencrypt) and all your friends can visit your site on any browser. Unlike email the burden of proof that allows your server to participate in the larger network is low.

Other protocols and services may be just as easy and possibly even easier than HTTP, but others may just lead to disappointment.

HX — “Hosting Experience”

I’d agree with Moxie if he’d written “Nobody wants to run Apache, or anything that requires hand-editing a config file.” A server intended for end-users needs to completely rethink what “running a server” means, and reconsider the relationship between the user and the software, which is entirely different than for something managed by a sysadmin.

“The server needs its iPhone moment”

— me on Twitter

Going further, such an end-user system would need to have all the characteristics of a modern wide-release computer product:

Good defaults and secure by default.
Foot-guns removed, or buried deep enough that most users will never get to them.
Assume bad intentions: users can’t know if an app is safe or not, so like a smartphone, limit what apps can do using permissions or capabilities and sandboxing.
Easy to set up: plug in power, plug in ethernet, visit some local web page and click through the setup wizard. 10 minutes tops.
Apps are installed with one click from an app catalog (or several, this doesn’t have to become a walled garden), and just work. They can auto-update too of course.

Good IOT hardware (yes I know there are a lot of bad ones), NAS and good home routers provide this experience already (to some degree at least) so it’s entirely within the realm of the possible, it just needs to be invented and executed well.

Why are we serving?

The reason a NAS can be bought at any store that sells computer electronics is not because they embody the freedom to run your own services from your home, but because they respond to a need. People buy these things because people have a lot of files, some quite large, and they need to put them somewhere. Sure they could use Dropbox but that’s slow for large videos and it limits the number of connected devices. All cloud services cap or start charging after a certain amount of data is stored, and a local NAS with a decently sized disk will store far more than that and at far greater speeds when you’re at home.

Buying another electronic box and taking the time to set it up is a high bar, so the value must be there. Sadly I have no clear answer on what services could run better from home than hosted by a 3rd party. But here are some general lines of thought:

Cloud-based services can get expensive. They are usually paid for monthly or yearly, so they are a constant drain on the user’s wallet. A one-time purchase of hardware can be attractive in such a case. The problem is consumers do not pay for online services that can be replicated in a home server (think Netflix, Spotify) so apart from file storage there are few options here.
On the flip side free services tend to have little respect for the user’s privacy. People are uncomfortable with what big tech companies do with their data, but they aren’t doing much about it. That could shift one day, particularly if equally convenient solutions are available.
Are services ruined by the constant search for growth? Most online services are VC-backed and are expected to show skyrocketing growth. This is often achieved in manipulative ways and is fatiguing to users. Again, it’s not enough to cause many to jump ship, but it’s another “con” to leverage against 3rd party services.
Does the service work reliably? It’s tough providing a service for millions, especially if it’s free. Trying to max out machine use, sometimes performance suffers. I just tried sending a tab to my laptop using the Firefox send-tab service, and the tab arrived minutes after I needed it. A little personal service on a home server, which would almost certainly be under-utilized, might have forwarded the tab in the blink of an eye.

As you can see I offer no slam-dunks, just vague possibilities.

How to Get People to Self-Host At Home

Find a real need. There has to be something that pulls in the potential self-hoster and makes them say “Yes, this is the better way to go for me”.
Serve something that can actually work given current realities. If their friends aren’t going to receive their emails, what’s the point?
Make it incredibly easy and safe. The iPhone of servers.

Each of these is hard on its own, yet you have to nail all three. Good luck!

This is the reason I expect most Dropserver users to rely on a cloud service to run Dropserver Apps. Later on, if it’s realistic, there could be a DS appliance, similar to what Home Assistant is doing. But that’s a long ways off.

Can An "App Store" Be The Solution to Funding Open Source Desktop Applications?

Olivier Forget — Fri, 19 Feb 2021 11:32:00 -0800

I’ve been thinking a lot about open source lately, and wondering if there are alternative business models that could make it sustainable. This “app store” idea is one possibility.

Motivation

I always prefer using an open source application if I can find one that meets my needs. I am uneasy spending money on closed source solutions to any of my problems.

It’s not the money that’s the issue. With an open source application I am more confident that there is no ill-intentioned code, and I can believe that even if development stalls, a sufficiently large user base will result in the application getting patches and other needed maintenance work, so that I can continue using it.

Closed source programs often use proprietary file formats to strengthen the moat they have around the market. This prevents me from moving across programs easily, and can result in loss of data in some cases. Open source projects don’t care about moats, and try to be compatible with as many formats as possible. This is a huge benefit for users.

Unfortunately, the flip side of open source is that it’s often “free as in beer” too, which means there are almost no funds for sustained development. The quality and usability of the application often suffers as a result. There is only so much unpaid or barely-paid open source developers and project maintainers can do.

One segment of software where open source projects can fund their development successfully is software that run on servers. In that case, offering a paid hosted version is natural and can result in a steady flow of cash for the developers. See Plausible Analytics and Taiga.io for example.

For environments where software is installed locally, such as desktop operating systems, an “application store” could provide such a system for generating funds for a project.

The Idea

The idea is to create an application store to significantly increase the appeal of open source applications for end users, and collect funds in exchange for some convenience and support.

Here’s how the store might work:

The store would present applications in a modern and user-oriented way.
The app store would have an integrated installer, so that an application can be installed in a single click.
Applications would be “sold” even though the license would remain unchanged from the original code.
The store would provide some level of support to buyers as a benefit.
The store would use part of the funds to promote the products in the app store.
The projects would receive funds to support their continuing development.

In a nutshell, the goal is to maximize the appeal of open source software to non-technical users by focusing on user experience. By doing this, we can expand the number of users (buyers) through promotion and PR, and therefore increase the funding for the projects.

By the same token, the application store would relieve the open source developers from some of the less enjoyable tasks, like providing support to end users, and self-promotion.

A Virtuous Feedback Loop

The hope with this idea is that better funded open source projects are more likely to respond to the needs of the users. This in turn brings more users and more funds, which opens the possibility of doing even more great things in FOSS.

However it may not be enough to hope. It’s possible the application store would have to be selective about which applications it offers. It’s important that users perceive the store as a purveyor of quality apps. A bad experience with a few apps and the user will not bother with the store anymore, nor will they recommend it to others.

Setting minimum user experience requirements puts a demand on projects, but I think it would result in a healthier open source ecosystem. Have a look at the Plausible site and product screenshots. Does Plausible look sharp and highly usable because it is well-funded, or is it that way because the developers know they won’t make as much money if they don’t “look the part”?

Like the chicken and the egg you can debate endlessly about which came first, the point is they go together. You want users to part with their money? Spend your resources so your application looks sharp, is easy to use, and doesn’t waste your user’s time.

This sounds harsh but the key to accepting this demand on projects is to think of how much funds could be available to open source projects if many non-technical users turned to FOSS for their needs. This can only happen with strong UX in the store and the applications.

With all of that, I can imagine a world where such an app store is well established, widely distributed among end users and large numbers of developers are working part-time or even full-time on open source applications.

What About “Cheaters”?

If the application store puts a price tag on open source licensed software, what prevents someone from simply bypassing the store and downloading the source or even the binaries on their own?

Nothing.

First, there are no “cheaters”. If someone can’t or won’t pay, it’s absolutely within their rights to obtain the software outside the store, and that’s perfectly fine.

I don’t think that means the idea can’t work. Remember that the key is to broaden the pool of FOSS users far outside the world of tech-savvy folks. This means many of these people may prefer to part with a few dollars than venture outside the safety of a well regarded store.

It will also be critical to make the benefits of using the store meaningful and the prices reasonable enough that a sufficient number of users don’t bother to get applications outside the store. For some users, knowing they’ll get some support from the store could be enough justification.

Can it Work?

I wrote above that I could imagine this working. The truth is my imagination is a well-oiled machine that can make many improbable things come true.

I do think FOSS on desktop is stuck in a local maxima, and donations are not enough to truly alter the landscape.

The only way to make open source sustainable, let alone thrive, is to make it attractive to regular people, yell about it on the rooftops, and stick a price tag on it.

Dark Mode: Not For Me Either

Olivier Forget — Fri, 12 Jun 2020 18:51:07 -0700

Kev Quirk managed to get himself on the front page of the orange site for his post on Dark Mode. The topic even drew 511 comments, some of which seemed to take this as a personal affront to their lifestyle choices. Now you’ve done it.

Kev’s post was well researched and I learned a few things from it. Interestingly none of the reasons he cites for using light mode are reasons why I go light.

Here are the my two main gripes with Dark Mode:

Reflections

Do you know how to make a mirror? You put a dark color behind a glass surface. Dark mode is your device trying to be a mirror, and the result is reflections galore.

For a while the Guardian app had a bug that caused all articles to render in dark mode even with it disabled. This meant I couldn’t read the news outside in the morning. The strongly lit surroundings would be reflected in the article I was trying to read, forcing me to work twice as hard to follow a line of text.

Painful.

This bug has mercifully been fixed, and while there are still reflections on the tablet screen, it’s far less distracting than before.

Even in my office I seem to always catch sight of something behind me when looking at a dark UI on my iMac. It’s distracting for no good reason.

It makes me think of those fancy computers in SciFi flicks that have a transparent screen. Cool for a movie prop, but terrible UX in reality.

Some will argue dark mode should be used when it’s, you know, dark. But unless you’re in a completely dark room I don’t see the point. And if you are you should probably put the phone down and go to sleep anyways. Sorry to preach.

Ghosts of Readings Past

There may be something wrong with my eyes but after reading light text on dark background the text lingers in my vision and dances around when I look at something else. It’s an uncomfortable feeling.

A lot of text editors come with dark themes enabled by default. I tried it for a while but I just can’t stand it. Ghosts of code everywhere, I feel like I’m going crazy.

I realize I may have sensitive eyes. My wife and I play dimmer wars all the time: she turns them up and I turn them down.

I dim the screen on my iMac when my office gets dark. It’s very effective at reducing strain and is far less jarring than inverting of the brightness spectrum.

Keep it Light

I don’t care if people like dark mode. They may be biologically predisposed to enjoy it, they may work in a cave, and I’ll give them some credit: they probably turn it off when reading in sunlight.

But for me it’s light mode all the way.

This was post 23 of the #100DaysToOffload challenge.

Proprietary Services Fulfill The Open Web's Dream

Olivier Forget — Tue, 02 Jun 2020 20:47:07 -0700

Last night, during the protests for #BlackLivesMatter, there was an urgent need to spread information to participants and others trying to help.

In an effort to get that information out, people reached for tools that they could use quickly and effectively.

One of these tools is Carrd, an online website builder that makes single-page websites or very simple multi-page sites. It is run by a solo founder-developer.

Carrd is great. If you want to put a single-page or other simple website online, it gets the job done fast and well. It’s freemium, which means you can put something online without reaching for your credit card.

When I look at the pages that were built using Carrd for the protests, like blacklivesmatters.carrd.co and blmprotest.carrd.co, I immediately think that this is what the open web is for: the ability for anybody to create and publish content that is accessible via a URL.

And yet, without the support of a service like Carrd, it’s oh so hard.

The Potential Problem

Protesters got lucky this time. Carrd creator AJ runs his service well (Billie Eilish put a link to a Carrd in her Instagram bio, and his servers are still standing) and doesn’t seem to have any desire to stop protesters from using his service.

Imagine an alternative scenario: someone builds a slick service that becomes popular at a time of crisis, except instead of the creator being on the right side of history, they are flatly in the wrong?

How much power would that person have over a movement? They could delete the accounts and pages, they could redirect the visitors to pages showing completely opposite views, they could even steal the login credentials of those users and see if maybe they reuse passwords on different services.

Who knows what an unscrupulous and upset service owner might do. The situation in the US for the last few years shows that saying “that will never happen” is not good enough. Some day, it will happen.

The Closed Writable Web

The web is an open technology. The communication protocols are owned by no-one, “free” to use by all. There are no licensing agreements to sign before making an HTTP request, no royalties to pay for that <HTML> tag.

This openness is freedom and independence. Any web browser can load any webpage and any accessible url, regardless of who wrote, and who approves or disapproves of it.

When it comes to consuming content online, we’re in a pretty good spot. In the US, 90% of the population can visit any site, from the largest internet properties to the smallest of independent web publishers.

But when it comes to creating content, participating in the read-write web, there is one big problem: the protocols are open, but the technology is hard to use.

HTML was supposed to be easy enough to write for anybody to do it. But that is 1990s thinking. There is no way someone is going to learn HTML and CSS enough to create a decent looking page by today’s standards unless they’re looking to learn the craft for hobby or professional advancement.

Then there is the hosting. You can fumble your way through the maze of AWS, or their behemoth brethren, or you can reach for something lighter like Netlify. However for any of these you choose, you have to learn how they work and adapt your stuff to their requirements.

You could also stick to an open source stack, and install a Linux server on a virtual machine and secure it, but now you’ve truly lost the non-tech folks.

The writable web is open for tech geeks, closed and proprietary for everyone else.

Some Exceptions

As I was writing this I realized there were some partial exceptions:

WordPress is open source and is a de-facto standard for online publishing. You can host your own or go with a service provider. It’s also rather heavy and not friendly to very occasional users.
Mastodon and other fediverse projects democratize online publishing, but there is a difference between “joining the fediverse” and just publishing something at a URL.
The Dat protocol is a work-in-progress, but they are espousing the read/write capabilities in their Beaker browser from the very beginning, a good sign.

None of these address the need to just be able to publish something simple online with a truly small investment in time and effort by a non-technical user.

A New Standard?

There could be a set of standards that address the need for writing and publishing websites, or at least static sites. There is a Read/Write Web Community Group in the W3C, but it’s basically dead at this point.

If this were revived, the focus should be on ease-of-use. The idea would be to bring static site publishing closer to how content consumption works: make it so 90% of a developed country can do it, and do it without lock-in or proprietary elements.

I suspect webpage builders would become phone and desktop apps, and hosts would serve static content. The standard would be the glue between the two: the protocol that lets the builder push new HTML and CSS and other assets to the host.

There would need to be a standard for DNS management since owning your domain is a key element of independence. Entering A and AAAA records and CNAMES and TXT records is not fun.

Something to think about. In the meantime, visit blacklivesmatters.carrd.co.

This was post 22 of the #100DaysToOffload challenge.

SpaceX Crew Dragon Launch and NASA's Gulfstream-IIIs

Olivier Forget — Sat, 30 May 2020 19:41:07 -0700

I was watching news coverage before SpaceX’s launch of NASA astronauts Bob Behnken and Doug Hurley when I noticed something that took me back to the very end of my aerospace career.

Astronauts usually live in Texas near NASA’s JSC, and fly to KSC (or Russia) when it’s time to launch. During the Shuttle era they would fly in T-38s from Texas. For this trip they took one of NASA’s Gulfstreams, though it’s unclear why. Perhaps it’s because of the pandemic, or maybe it’s because the Crew Dragon interior has been compared to that of a business jet, and you might as well fly in on something that feels like your spacecraft.

The thing that jumped out at me about Bob and Doug’s ride was the hard-point under the middle of the wing. Boy did that look familiar.

Hardpoint under the wing in the blue circle. Photo by Emre Kelly, from this article.

Rewind

Back in the mid-2000s I quit my job at XCOR Aerospace in Mojave, CA and moved to Los Angeles where I worked at an aircraft engineering firm. Our specialty was aircraft modifications. We did the engineering work and supporting documentation for certifying modifications of aircraft.

Our work ranged from simple mods: some rich private jet owner is upgrading the navigation avionics of their jet, so we would do the engineering work to modify the aircraft with a better external GPS antenna.

Other work was more demanding. We would usually be sub-sub-contractors on these, but they involved some rather drastic modifications of large aircraft for testing purposes. We helped modify a 737 into a F-35 avionics testbed, an A-3 into a radar testbed, etc…

Pin the Pod on the Gulfstream

One project involved hanging a pod underneath a Gulfstream G-III for NASA. For simple modifications a drive to the airport and a quick look at the aircraft armed with a digital camera, a ruler, and a drill was all we we needed to sort out a small antenna installation. (Yes, a drill. Sometimes you can’t know the skin thickness where you want your antenna installed, so the only solution is to drill a hole in the airframe where the antenna is going to be, and then do the engineering to fix it all up.)

But hanging a big pod under a the middle of the wing, which happens to be where the main fuel tank is takes a bit more work. Someone had to spend a large amount of time reclined in a low chair underneath the wing of that plane with a ruler and a notebook. That person measured and counted every line of rivets, and then re-measured again just to be sure. That person was me.

It sounds like hell but I loved it. Proximity to aircraft was my favorite perk of being an aerospace engineer. So I indulged in the task, and probably measured a third time just because it was better than being in the office. I also crawled all over the tail cone to prepare for the installation of a ventral tail fin to compensate for the lost stability of the hanging pod.

After all the wing measurements were done I sat at my workstation and cranked out an engineering model of the center of the wing. It was tedious work that had to be done as accurately as possible. We used the measurements of what was visible in the aircraft and cross-referenced engineering drawings to ensure that the model we were building represented as much as possible the details of the aircraft we were modifying. No drilling holes willy-nilly through the fuel tank!

After building the model we’d measure stresses in the structure under virtual loads for a standard configuration, and then measure forces with the pod hanging. The difference would tell us how much reinforcements were needed, and where.

Suffice it to say that aircraft was my professional obsession for a few months. It’s also the last aerospace engineering project I ever worked on and I wouldn’t see it to the end. I quit that Spring and started my own software biz.

NASA-Dryden’s G-III after the mod, with pod hanging and that big angular fin under the tail. NASA Photo.

Bob and Doug’s Excellent Ride

When I saw the hard-point under the G-III at KSC while Bob and Doug were giving their interview I thought for a while that it might be the one that I had worked on. How many G-IIIs could there be with a center hard point? It turns out there are at least two. I found out that JSC also has a G-III and it too was modified to carry a similar pod under its wing a few years after the mod I worked on.

JSC’s G-III with pod hanging. NASA photo from this page.

Apart from ferrying astronauts, this plane serves in NASA’s airborne science program. It’s similar to the Earth-observing stuff they do from space, but they do it from within the atmosphere.

The pod is made by JPL and is called UAVSAR. It consists of a synthetic aperture radar that can make a very detailed map of the ground below. A complete explanation is here. A big part of the instrument’s mission is to overfly natural disaster areas to learn as much as possible about the event.

When carried above a disaster area by one of NASA’s modified G-IIIs, the UAVSAR pod has helped better understand earthquakes, landslides, volcanos, hurricane damage, and oil spills among others. Here’s a video showing the NASA team doing the survey of the gulf oil spill.

Bob and Doug are floating in the SpaceX Crew Dragon as I write this, but the ride they took to get to the rocket shows a lesser-known but still cool and definitely important part of NASA’s mission. And I was glad to have been a tiny part of it.

This was post 21 of the #100DaysToOffload challenge.

Our Web Versus Search Engines

Olivier Forget — Mon, 25 May 2020 20:41:07 -0700

Sometimes it seems like search engines are amazing. And yes, for a lot of questions, they do get us an answer quickly.

But other times search engines fail me. For example I might search for pages that talk about two topics together, and I’ll get results for a popular page about the first topic that happens to have a link to a page on the second topic in its navigation menu.

For a long time Google would return pages that were written in PHP for queries about PHP, presumably because the URL ended in “.php” and that matched with my query: “PHP how to append an array”. Not useful, Google.

They have other quirks, like just today while trying to write proper asynchronous code in TypeScript I submitted dozens of queries, one of which was “javascript emit occasional result”. Not a great query but I was desperate. For some reason three of the first five results are for books about utterly unrelated topics. The first result is a book on nanoscience, another is on marine fauna. The fourth is on galaxy formations so I assume this is Google’s galaxy brain search results.

We all know search engines have limitations, but the bigger problems lie deeper.

Algorithms Are Abused

When the algorithm isn’t tripping over a simple query, other actors are determined to make it trip in their favor.

Everyone wants to be #1 on search results. Any aspect of any algorithm will be abused to get higher in the results pages.

Google is famous for inventing the PageRank algorithm. The number of pages linking to a page is a signal for how helpful that page is, or at least how popular.

But as soon as web properties figured this out, the bazaar of back-link bat-shittery began. How many times have you gotten (or sent 😲) an email asking for a link to a page? Are bots leaving random word salad comments in your forums with a few links over and over again? Thanks PageRank!

Google tries to penalize abusers, and it succeeds over time. But there are always new aspects to its algorithms, and the SEO crowd is a crafty bunch so the PhDs in Mountain View are always playing catch-up.

The reality today is results are littered with pages that should not be there: they tricked the algo and won.

Have you ever clicked on a promising search result, read through the content, and was left with the impression you just ingested the spilled contents of a word bag related to your query? There are sentences, and they make sense, yet so little meaning or insight is revealed? The site is often nice and usable and looks like every other helpful site out there, yet lacks a true personality? The only invariant is that there are a bunch of links sprinkled throughout, all pointing to the same commercial site. Uh huh. So much HTML theater for a measly click!

Thanks, search algorithms.

The Algorithm Shapes Our Web

It’s bad enough sifting through spammy search results to get a good one. Sometimes even the good ones are compromised by a desire to increase search visitors.

A few years ago Google updated its algorithm, and a bunch of people saw their search traffic drop. SEO folks figured out that pages that were a bit old but had a date on them weren’t being shown anymore. The way to recover traffic? Just remove the date from your posts. Easy peasy!

Have you ever landed on a technical blog, found the content promising but needed to know how old it was before committing to reading it? Tech moves fast, and reading a post that is many years old can be misleading. So you scroll all the way up and down to find the date, only to realize there is none on the page. Frustration! The reason? It’s likely the blogger heard that removing post dates would increase their search visitors.

Google has since corrected this, but not everybody is up to speed, so dates are still missing here and there. Sigh, and thanks again search algorithms.

The Algorithm Shapes Our View Of Our Web

Just the other day a question on the orange site ranked highly and got lots of positive feedback: Ask HN: Is there a search engine which excludes the world’s biggest websites?. It seems Google favors pages from large internet properties, and tends to leave small sites out of the results.

It’s clear that people (or HN users at least) feel they are missing out on part of the web. When every query is answered by one of very few search engines, we are viewing the web’s content through a filter that has its own stake holders who are not us.

There is the web and there is the web we see through Google. How much are we missing?

It’s like having an intergalactic spaceship that can take you anywhere in the universe in the blink of an eye, but its radar can only detect large planets with the proper arrangement of keywords.

We Need Something New

I want a new paradigm for finding stuff online. I want something that depends less on algorithms and more on humans. I want results that make sense to me and other humans like me, even if there are fewer and it takes a bit longer. I am no longer blown away by 60,000 results in 0.000003 seconds. Blow me away with five results that are outstanding. I’ll wait.

I know we can’t completely replace the might of Google, but just like I try searching on DuckDuckGo before I search on Google, I want to search on ? before I search on DDG and Google, whatever “?” is.

Let’s search for our “?”.

This was post 20 of the #100DaysToOffload challenge.

Thoughts on Sandstorm

Olivier Forget — Wed, 20 May 2020 18:41:07 -0700

When I tell people about my side-project to create a server that makes it safe and easy to run your own web apps, some say I should check out Sandstorm. I was a (small) backer of Sandstorm’s IndieGogo but I’ve been disappointed by how it worked out.

I think the idea of safely and easily hosting your own server-side applications and services is important for the internet to remain free. I thought this then and I still think it now.

Sandstorm is ambitious, and has a lot of good things going for it. However, development and adoption has been slow and the company behind it has shut down.

Some design choices in Sandstorm have made it harder for the platform to thrive. Here is how I see it:

Coerce Old Apps Into A New System

Sandstorm’s “Grains” make it technically possible to run just about any existing application inside Sandstorm, since they are using Linux container technology. But unlike Docker, which also uses Linux containers, apps on sandstorm usually need to be modified.

For example, a web-app on Sandstorm should not provide a login and user management service since the platform provides it. Since most web-apps include this as an essential part of the system, it means ripping it out or somehow bypassing it to work with Sandstorm.

It’s hard work going into an existing codebase to rip out one of its primary elements. Many open source web-apps are old, and old code is even harder to muck about with. Login is just one thing, Sandstorm also has its own data sharing paradigm and apps have to be altered for that too.

Another issue is that developers of typical standalone web-apps assume the app is started when the host starts, and runs continuously until the server needs to be rebooted. There is no reason to optimize startup time, and some apps take seconds to start. However on Sandstorm the grains are only started when the user actively summons them. The result: the user waits.

Not Easy to Create a New App

While it’s work to modify an app to run on Sandstorm, that same container tech makes it intimidating to create an app for Sandstorm in the first place. Have a look at this page that explains how do get started with a simple PHP app. Yikes! It’s Ok if you’re regular container slinger, but otherwise it seems kind of steep.

Sandstorm’s approach puts it in an odd spot: while it has the advantage of being able to run almost any app thanks to its container tech, it’s a lot more work for devs to make their web-based application code run on Sandstorm than on Docker (or any other container runtime), so few have done so. But it’s also hard to create an app from scratch.

So, steep hill to climb if you’re forking an app to run on Sandstorm and steep hill to climb if you’re creating one from scratch. As a result, Sandstorm’s app market had a very modest number of apps, most of which were a few (or many) releases behind their source, some of which were slow to start, and in my experience had a few bugs. Populating an app store for a new platform is always a hard problem, but Sandstorm’s design made it even harder.

Not Really Good At Services

This one is the kicker. Sandstorm works well with applications that you tend to sit in front of. It’s not great for situation where you regularly serve dynamic requests. See this page (the section at the bottom titled “Why only static content”).

Sandstorm also has an issue with URLs that are not clean looking, and hooking up a service to a nice sub-domain is still eluding the platform, apparently. It’s hard to feel like your service is “of the web” if it can’t simply be connected to a clean domain.

To me the very essence of an online service is to serve dynamic requests. Quite possibly a lot of them, coming from many different clients at any random time with potentially different authorizations. Services should be reachable at a nice clean URL, respond quickly even if not used often, not bring the host to its knees if under some load, and take up almost no resources when not in use.

The Future of Self-Hosting Web Apps

There are other pain points for Sandstorm: the user interface, the unfamiliar and complex concepts for users, etc… Some of this is typical of a very ambitious project: niceties like the UI and making it simple are left for later. This discussion illustrates a few of these problems, along with a large number of comments from people who were justifiably saddened that Sandstorm was not taking off.

Sandstorm is still evolving as a community project and may yet gain enough speed to take off. I certainly am not writing it off. But I think there is room for other approaches, with different trade-offs and different priorities.

This was post 19 of the #100DaysToOffload challenge.

The End of the Before-Times

Olivier Forget — Mon, 18 May 2020 12:50:07 -0700

Amused by this Guardian article, which draws from this tweet, which is inspired by this BBC piece, here is the last picture on my phone that represents normal life, or as we’ll soon know it: the before-time.

This is sunrise in Joshua Tree, California on March 8. We had met some friends for a one-nighter camping trip. The virus was in the news a lot by then, and we greeted each other by kicking boots. It was mostly for laughs though, we hugged too and the kids played together. Social distancing was not yet a thing.

Soon after, we moved into our freshly remodeled new home, just squeezing by as California and LA County announced lockdown measures. We got all our stuff into the house, shut the door, and haven’t been very far since.

Here’s another photo, this one is the sunset of the previous evening.

There is plenty of room to social-distance in the desert, but huddling by the campfire is better with friends. I hope we’ll get to do this again soon.

PS: I’m kidding about this “before-times” stuff. I think we’ll work our way out of this. Just last week the cycling trails and beaches of Los Angeles have reopened. While we’re still social-distancing, it feels nice to get out of the house.

This was day 18 of the #100DaysToOffload challenge.

100 Days Offload: Time For An Adjustment

Olivier Forget — Thu, 14 May 2020 18:41:07 -0700

This is day 17 of the 100DaysToOffload challenge for me.

I have to say it’s been taxing.

While I am very happy that I got this far, and I am happy with the posts I cranked out, I can tell it’s time to make an adjustment.

This pace of posting is too demanding for me and my family, and it’s also limiting to some extent how much I’m getting out of the challenge.

The Costs

We’re in a pandemic. Things are harder than they normally are to begin with. The kiddo isn’t going to daycare so my wife and I take turns watching over her. These moments with the little one at home are a gift, but they’re not the most restful. My wife spends more time with her than I do because my work hours are longer.

In the midst of all of this I decide to do a marathon blogging challenge. Brilliant move. I’m in front of my computer even more and I go to bed late, which is a big no-no in my book. When my wife suggests we should watch a movie I say no because I have to go back to the blog after dinner.

This is also hurting my rule about taking a true day off at least once a week. If the challenge is to blog every day, I have to somehow justify time in front of my laptop on my “rest” day.

Of course the rules of the challenge are that there are no rules, but I tend to latch on to trivial frameworks to accomplish things. I operate similarly when I work out: I’ll force myself to keep pedaling until I reach the top of the hill.

Most evenings, after I finally shut the lid on the laptop, there has been a thought nagging in the back of my head: how long can I keep this up? I’m not even close to 100 days. It turns out I’m not alone: Kev Quirk, creator of the challenge, is also feeling the costs and adjusting.

Blogger Unleashed

I am really grateful to the challenge for getting me to publish a bunch of posts to my blog. I know it’s only been 16½ but that’s like 5.5 times more posts than when I started.

The main thing this challenge has taught me so far is that I can get to the end of a post, in a complete and acceptably written way, and hit publish, and move on. In the past I would spend too much time thinking about the meaning of each post, as if somehow there was a significant meaning, and I would stall out in paralysis by analysis.

Posts are just a thought man! You can amend it later with another thought. Now you’ve got two posts and you’re a blogging machine. After some time you can forget you ever posted it and there is a decent chance nobody will ever remind you (just don’t ever run for office).

I am a perfectionist in my work, but I’m learning that with blogging you just have to let it go. It’s fine to publish and have it be “just OK”.

All this to say that in 16 posts I feel like I’ve unlocked that part of my blogging abilities. When I sit down to write I am much more confident that I’ll come up with something, and that publishing it will feel more good than bad.

In fact I’ve been surprised by the content that has poured out of the various recesses of my mind. Some days I have had a hard time thinking of what to write. But when I decide on something, it ends up coming out in droves. It gushes out and I end up with far too much editing work.

Each of these posts, even the hard ones, make me feel happy knowing they’re out there. There isn’t a single throw-away in there as far as I’m concerned (you may disagree, but this is a personal perspective). Each one was something that was on my mind, and it’s out there now. And it feels good.

The Next Phase

I want to work on a different aspect of blogging now. Clearly I can slam a post out in a day, day after day if I want to. Check. Now I want to work on quality.

I wrote earlier about how I had a hard time with drafts. Time to fix that. I want to take more time writing posts so that they go a bit deeper, are better researched and better written.

Hopefully this will free up some time. By not forcing myself to publish every day I can forego the writing session if it’s more appropriate to do something else.

This won’t make it easier though. I found the daily posting to be a big motivator. Without it, I’m worried I’ll spend too much time on drafts and end up with big gaps between my posts. I’m also worried that this will bring back the need to post something extra-meaningful and powerful, and failing that, to post nothing at all.

But I’ll learn to overcome. That’s why it’s called a challenge.

This was post 17 of the #100DaysToOffload challenge.

Deno Saved My Side-Project

Olivier Forget — Wed, 13 May 2020 12:15:22 -0700

My main side-project is a server that runs personal web-based applications. The idea is that web apps could be built and distributed more like the smartphone ecosystem than the current model of “everything online is a service”.

I’ll talk more about this project some other time, but for now suffice it to say that for the idea to work, a user should be able to install a web-app without putting themselves at risk.

Basically, whatever code runs and however it runs, it must have very limited and controlled access to the underlying system.

Examples of actions that must be controlled:

read/write to disk
outgoing web requests
create network listeners (the host takes care of that)
consume too much space on the disk
consume too many CPU cycles
consume too much memory

My project has been going on for close to two years now, and a lot of that time was spent trying to figure out how I can run code with such restrictions.

I want the apps to be written in a language that web developers are familiar with. Naturally this means JavaScript.

VM2

My first attempt was using a JS “sandbox VM” thing. I quickly saw that this wasn’t good enough. JS is like the gunk that comes off of pine trees: once you touch it, it gets everywhere and you just can’t be sure you’ve gotten rid of it. It’s nearly impossible with current JS to isolate one part of it. There is always some weird prototype chain trick that can be exploited to get out.

VM2 is an impressive project, but I fear the never ending fight against JavaScript escape artists.

JavaScript Realms

My second hope was for JS Realms. It addresses the above point directly: arm the language with the ability to segregate parts of itself. This seems like a good move for the JS ecosystem as a whole, and I thought the proposal was moving along nicely, but it appears stalled.

Containers

Next, I dove into containers. If JS can’t be tamed then I’ll tame it at the OS level. Or so I thought, but this was tough. First, I am not a Linux system expert by any means. I know just enough to get by. I got hung up on esoteric linux stuff. Second, JS isn’t a great language for this kind of work, so I needed an upgrade. A lot of container tech is written in Go, so that’s what I landed on.

It’s weird that container stuff is written in Go since it’s inherently multi-threaded, which causes problems. But hey whatever, I still like the language (especially now that I’ve moved on from containers).

Anyhoo I learned me some Go, and I am now quite dangerous with this language, and I got some custom built container orchestration Rube Goldberg machine going.

It worked, sort of. I want to be able to turn sandboxes on and off rapidly, but the latency and overhead of the system was not encouraging. I didn’t expect it to be great, but I was desperately in need of something.

Along the way I learned that containers are one thing, sandboxes are another. While creating containers is easy, making a truly robust sandbox is a different level of challenge. Any little mistake in the setup of the container can leave a door wide open for an escape. Jessie Frazelle can do it, but I should probably not try.

Deno

Sometime after I had sunk a lot of work into my container-dispatcher-axe-throwing thing I started noticing a project called Deno. It was in very early stages then, but its tagline was a promise that went straight to my heart: “A Secure Runtime For JS”. Oh baby.

I decided this had to be the solution. Each app could run in its own instance of Deno with its own permissions, and cgroups to control resources. Startup latency is good enough for now, and I can already think of ways to make it seem faster.

I created a branch and obliterated my container-juggling-circus-act code and went all-in on Deno. I figured I’d have time to write all the other parts of the system before Deno is ready (I was wrong, I’m nowhere near done).

Deno and Security

At first I wasn’t sure how serious the devs were about the security aspect of the project. But come on, “secure” is the fourth word on their homepage after “Deno is a…” so surely they’d be on it?

I filed a few issues (#2705, #2761, #3401) against early builds around security and permissions. Ryan Dahl, creator of Deno (and co-creator of Node) was receptive to my concerns, and most of my issues are now fixed. This gives me some confidence that Deno would eventually become pretty tight.

Sadly I couldn’t contribute actual code because I would have to know Rust. Having learned some Go the year before my enthusiasm for learning Rust was low. Besides I’d like to become less dangerous with Go instead of super-dangerous in two languages.

At this stage I wouldn’t run untrusted code in Deno, but since it’s reached 1.0 we can hope that the community will start poking at it and expose the flaws, and they’ll get fixed. The fundamentals of its design are sound so it’s a matter of finding and quashing bugs.

Thank You Deno

I’m really glad Deno is here. It’s important to be able to run code that you can’t necessarily trust in an environment that limits what an attacker should do. We should have that for JS.

I’m also glad because my project would be dead without it.

This was day 16 of the #100DaysToOffload challenge.

Thoughts on Hugo

Olivier Forget — Tue, 12 May 2020 09:10:21 -0700

This blog is generated using Hugo. A documentation site I’m rebuilding is also made with Hugo. These two projects have given me a chance to get to know the tool, to appreciate it at times, and to curse it otherwise.

Some of my complaints are inherent to static site generators, others are more specific to Hugo.

Your Directory Tree Is Configuration Code

In its simplest form, a static site generator takes a directory tree full of markdown files, and cranks out an identically structured directory tree full of HTML files.

It’s expected that the shape of your directories will affect how the generator works.

Hugo has accumulated such a vast array of capabilities that the simple model described above has multiplied to the point where it’s no longer easy to know what effect a file has, or why some file does not have the effect you think it should.

Have a look at the template lookup order “help page”. Hugo accomplishes a lot here, with its multi-language abilities, and its taxonomies, etc… But you can see from the number of examples and possible lookup places that the capabilities are paid for in complexity.

I don’t think the filesystem is the right tool to describe the complex relations between posts and languages and templates and output formats. It’s too easy to get lost.

Index.md Or _Index.md Or my-post.md

It took me a little while to understand that _index.md and index.md are not the same thing. One is used to showcase a list of posts, the other one is the post itself within a page bundle. Or something like that. I used the wrong one for a while and got no errors, but my table-of-contents didn’t seem to work. I spent a lot of time sorting that out.

I ran into a number of subtle behavior gotchas like that with the tool. For instance there are different “kinds” of pages and different “types” of pages, and it affects whether a page shows up in a collection of pages among other things. (Don’t ask me to explain, I can’t.) The point is there are lots of concepts, and these concepts don’t all work intuitively, and the difference between the tool behaving one way or the other can boil down to a single underscore in a file name.

I ran into another issue where I wanted to render markdown in content within a shortcode. Here again one little character makes a big difference. This {{< shortcode >}} is not the same as {{% shortcode %}}. This behavior has changed over time, causing confusion, and guess what? Though I read the docs, the behavior on my site is opposite what the docs indicate.

Of course it’s all my fault. Everything is documented. The thing is the documentation runs longer than your typical “learn a programming language” volume. I haven’t actually counted the words but it seems like there is a lot more than I should have to go through to render a blog or some docs site. The section on “Content Management” has 24 pages, each bringing in new concepts and new flags to tweak in your configuration.

You might say “it’s powerful, it’s up to you to learn it”. I won’t argue it’s not powerful, but something this powerful could be more sane to use if it relied on something other than the presence of an underscore to mean something about a file. Is the file tree truly the best abstraction for the user to accomplish their goals?

All Text No Tools

I understand the draw of using text files in a file tree: they can be worked on from the user’s favorite text editor, and it can all be checked into version control. Fine.

The problem is the tooling for text editors is weak. If something is this complex and powerful, and intended to be seen as “code” my expectations grow accordingly: where is the live error reporting? Where is the “debugger”? Autocomplete?

For instance, you can create a link to a page within your site using a “relref”. The cool thing is that the site will fail to build if you put a typo in the link. Guaranteed link correctness! I like. But the way it works is you type your stuff in your text editor, at some point hit save, then and only then your Hugo build will throw an error.

Does this remind you of anything? It takes me back to engineering school in the nineties where I was learning Fortran 77 (and the new hotness Fortran 90!) Write code. Save. Run compiler. Oops error. Find error file and line. Think some. Aha! Fix. Save. Run compiler. And on and on…

I don’t know about you but I don’t work like this anymore. Whether I write Go or Typescript, I get to see my mistakes as I type them, and I get suggestions for what is actually correct. The creators of those languages provide tools to help me because they feel my pain. I’m writing cryptic text but I’m getting a lot of help. When I do things in Hugo I get almost no help.

If a template doesn’t work as you expect and you want to step through it as if it were normal code, too bad. A helpful docs page recommends you put “printf” statements in your template. The nineties called, they want their debugger back.

Why would I not get an autocomplete when I create an internal link using relref? Shouldn’t I get a red squiggly underline if my relref has a typo? How about autocomplete when using a shortcode? (I tried installing this extension to my VSCode but it doesn’t seem to be doing much apart from causing an autocomplete popup for every single English word I type.)

I’d like to be able to click on any shortcode, or select any file, and have a tool tell me where it’s sourcing the templates that will be used to render it, and where the output will go.

But I don’t have any of that, so when something doesn’t work quite work like I expect I end up just playing a lengthy and frustrating guessing game.

Disenchantment

I started off really wanting to like Hugo. I got into it after I realized I was going to have to put my web content authoring tool on the backburner.

I figured if I can’t build something great, then I should pick the best of what exists and learn to live with the things I disliked.

It’s been over a year now, and I just can’t find love for Hugo. I use it, it’s effective when it’s all set up correctly. But does it feel like a wonderful powerful tool? A bicycle for my content-generating mind? Hardly.

There is a great talk by Alan Kaye dubbed “Is it complex? or did we make it complicated?” that applies here.

Static site generators have stepped over and beyond the bounds of what made them great: they were simple tools for simple problems. By making them do so much more they became complicated, and less effective.

PS: I’m sorry for not including proper code snippets. Turns out if I wrap Hugo short codes in a code block, Hugo tries to interpret them as actual shortcodes. If I escape the characters with \ it renders the slashes, and if I don’t it crashes with “Null pointer dereference error”. Yay.

This was day 15 of the #100DaysToOffload challenge.

Laravel Temptations

Olivier Forget — Mon, 11 May 2020 19:22:37 -0700

I need to modernize an old PHP web-app. I am facing down many hours of sifting through old messy code (written by me over many years) to figure out the essence and transform it into something more robust.

This is going to be painful, and I’m looking for anything that can help ease the burden.

One such helpful thing that comes up over and over in modern PHP chatter is Laravel.

Laravel in Context

Laravel is a modern PHP framework. By modern I mean it was first written in 2011. For reference PHP first appeared in 1995! PHP is an old language, but it’s still evolving for the better, which I find kind of endearing.

I started with PHP 5 some thirteen years ago, and built everything without a framework. My code suffered from multiple personality disorder as I tried different approaches to solve the same problem.

Now I need to put the whole thing into a blender, make a slurry of it and 3D-print a beauty of an application.

Using Laravel would represent a deep transformation not just of code but of attitude compared to when I started.

Laravel Attraction

I will need to rewrite my admin and backend code to accommodate new features. I’m really not looking forward to this. Is there a way to avoid doing that? Laravel Nova caught my attention. It’s a ready-baked admin panel, apparently, and if it’s half as good as it sounds, I’m in.

Nova would cost me a couple hundred bucks, but that’s peanuts compared to the pain of doing this myself.

That’s part of the attraction to Laravel: there are a bunch of these “add-ons”, some free some paid, and some as a service that I might leverage to remove some of my workload.

Other add-ons I find interesting (all are linked from Laravel’s home page):

Echo for real-time events
Cashier for subscription billing (although it’s for Stripe, which I am not using. It looks like they had Braintree support in 5.8?)
Passport for OAuth2
Scout for full-text search

I know many other frameworks offer similar solutions in various forms, but it’s comforting to see a neatly put together set of tools and services designed to make you more productive.

If Nova didn’t work out for me I could also use Backpack, but I like the idea of getting add-ons from the developer of the framework, it gives me hope that the add-on is seamlessly integrated with the main framework.

Laravel Concerns

I need to do more in-depth research on what it’s like to write an entire application using this framework. Like what it’s really like. Not a “Hello-World” type thing.

In fact I’ll probably have to code an example app just to get the feel of it. And you know what that app is going to be? My admin panel (or the start of it at least). Yup, you know it. This is how I am going to slip into this framework.

But seriously, I do have some concerns. My view of frameworks in general is that they make a lot of standard stuff very easy, but as soon as you get into an edge case you have to work harder to do what you need.

Another concern is breaking changes. One sweet advantage of “rolling your own” anything, is that you don’t change it if you don’t need to. Frameworks, libraries, all other code written to be reused by many others change a lot over time. There is always pressure to do things better, to stay relevant, to sell the latest add-on. The result is unwanted migrations that can lead to breakage and a lot of work for no real reason.

To Be Continued

If you’ve had experience with Laravel I’d like to hear your impressions.

If I go further with the framework I’ll let you know how it goes right here.

This was day 14 of the #100DaysToOffload challenge.

Web Of OK People

Olivier Forget — Sun, 10 May 2020 19:04:27 -0700

I’m terribly frustrated with common social networks, and by extension the spread of misinformation and the spread of hate.

This is a great time to be an a-hole, or a blowhard, or to have a desire to spread a viral idea to the world if you have no scruples.

We are in the golden age of BS.

Social media companies love to say they are bringing the whole world together. Zuck thinks that the world magically gets better if more and more people can get in direct contact with each other.

Unfortunately this is a naïve view of things. On social media, the things that spread are the things that trigger a quick reaction. Subtlety, nuance, and complex subjects don’t do as well.

This is why a–holes and blowhards thrive in this era. Anything that shocks people gets to spread. Insults, controversy, sewing division. All are much “liked” on social media.

It is not a surprise that people resort to making their point as “triggering” as possible, so that it gets re-shared. This leads to oversimplification, or leaving out mitigating factors, or simply making things up.

I don’t like any of this, so I’ve been asking myself the question: how could social media be designed such that it is much harder for jerks to be jerks.

People Of The Net

At a fundamental level, this shouldn’t be too hard. Most humans are cordial to each-other, and don’t start trash-talking to someone’s face for no reason.

The internet is different. People who don’t know each-other get into debates about the direction of their country. This is the promise of social media: connect with anybody about meaningful things.

In the physical world neighbors who know they aren’t on the same political usually prefer to just skip those conversations and instead talk about the kids soccer match, or any other matter they can talk about easily.

How can we bring civility to tough discussions?

Basic Civility

It should be possible to come up with purely objective “acceptable behavior” that every participant would always have to adhere to. A code of conduct that doesn’t tell you what you can or can’t say, but one that mandates decency.

Next, we need to figure out who decides whether a post or comment breaks the “acceptable behavior” rule. The typical approach is to use moderators. Personally, I don’t like this approach. Moderators are more powerful than other users, which is concentration of power which usually leads to problems.

I would like an approach where every user is a moderator. This of course sets up a situation where people use moderation as a tool to abuse other users (moderation tools are always used against users).

To counter that, moderation itself should be a moderated. If a user flags a post as “offensive” when it clearly is not, that contribution could be flagged as “bad moderation”.

Now of course we’re set up for an endless cascade of of users flagging flags. How does it end?

I’m thinking that maybe it doesn’t matter thanks to another concept:

Web Of OK People

There could be a web-of-trust type of thing, but it would be a web-of-OK-people.

Basically, every user would have a list of people who they consider to be “OK”, as in they are stand-up folks who aren’t going to throw feces around the room. A user would mark anybody they know, either in-person, or because you’ve witnessed their good behavior around the net for long enough to know they are stand-up folks, as OK-people.

Now when there is a controversy, any flag or moderation or comment is more likely to be showed to you if they are by one of your ok-people.

A score could be given to any post based on the reactions of your OK-people: if they “liked” it, the score goes up, flagged it and it goes down. If they flagged a flag, then any negative effect of that flag would be removed.

Unfortunately this would mean adding hundreds or thousands of ok-people just to get some coverage of the millions of posts in an active social network. Instead the system would find who the OK-people consider OK-people themselves, and consider those as OK-people too, but with less power. Then it would go another step and another and another until it amassed for each user a large list of OK to OKish people.

With a web-of-OK-people weighing moderations and flags on content, the content a user sees can be curated for them.

While users of current social media can curate their feed by following good people, things get ugly when you look at replies to a post, or look at the comments section on YouTube for example (yikes!).

With the web-ok-Ok-people eliminating the replies by those who don’t measure up, social media could be a decent place to be again.

The community would essentially do personalized distributed censorship of unacceptable posts, and if a user thought the censorship was wrong, they could adjust who is in their list of OK-people so that it better matches their tastes.

An army of bots wouldn’t get very far. A bad actor could create thousands of accounts, but with no track record (or a bad one if any) and no personal connections, most bots won’t be considered OK-people by anybody, eliminating their influence.

Open Questions

Look, I don’t know if this would ever work. It might be an OK theory but a disaster in practice. I have no idea.

Wikipedia experimented with something like this at some point, but are no longer using it. I’ll need to research why.

There are questions around the computational demands of such a system. It’s not clear it’s even feasible at scale.

I can’t help but wonder if this would just create highly tuned filter bubbles. I think users who would want more diverse views could get that by adding OK-people who see things differently but aren’t jerks.

I worry as well about onboarding new users. How does a fresh new user get anywhere on such a system if they don’t already know people? There would have to be a patronage system to help get people started.

Alright, enough questions. There is bound to be a lot more to this than what I wrote, but it felt good to offload it.

This was day 13 of the #100DaysToOffload challenge.

Google Services I'm Stuck With (For Now)

Olivier Forget — Sat, 09 May 2020 19:04:37 -0700

I used to merrily sign up to any new service that Google created. The company still espoused its “Don’t be evil” motto, and the concept of privacy had yet to feel like something we had taken for granted.

Things are different now. We’ve become more aware of how much of our private lives are floating from server to server, trading between data brokers to make a whole bunch of people – but not us – rich.

There is plenty of criticism of Google and it all affects how I feel about the company, but one of the big concerns is having all my eggs in one basket.

We’ve already seen the company change when it is financially secure, imagine what they’ll do when things start going bad for them (every empire goes down at some point, it’s just a matter of time).

De-Googling

I have a background process in the depths of my mind whose task is to cut away from Google, little by little.

The process wakes up once in a while, possibly triggered by something scary I read about the big company and its treatment of data, and it takes a step to cut another thread.

Here is where I made some progress:

Gmail

I got my own domain and a Fastmail account. I haven’t killed the gmail, I figure I’ll need it as long as I have and Android phone, but I have changed my email on many services and told all of my friends to write me there.

Search

Google is first and foremost a search engine, but its results are sometimes a letdown. I think search is due for a disruption, but that’s a different post. For now I use DuckDuckGo as my primary search engine, and fall back to Google when DDG fails me, as it does fairly regularly.

Browser

I tested out Brave for a while, though I am unsure about their plans for serving me ads. I still use it on my tablet for some reason, but all other devices have switched to FireFox.

FF works well. There are a few concerns about the way they treat private data as well, so I’m keeping an eye on them.

Stuck With Google

OK those three were big ones. But this next list is depressingly long: these are the services and devices I have made no progress replacing.

Android

I could wait for Linux phones (Librem, Pinephone) to become viable, but I’m afraid that’s going to take a long time.

A good camera and excellent camera software is important to me, so any off-brand thing is likely to disappoint.

The only reasonable option is an iOS device, probably. Hopefully Apple continues to differentiate itself from Google by valuing privacy, and hopefully it’s genuine and not a marketing pitch. Of course we can’t really know because everything is closed with them. Urgh.

I think my first move will be to replace my aging Lenovo with an iPad, despite how much I love that weird thing. Then I’ll decide how I feel about iOS and the Apple walled garden for a phone.

Photos

This is the other big one. Every photo I take on my Pixel goes into my Photos, so if I lose my phone none of the priceless memories of my growing daughter are lost. That’s huge.

Apart from jumping head first in Apple’s garden-of-loveliness-with-giant-walls, I don’t know what the other options are. I’m sure there are some, I just haven’t looked yet. The background process hasn’t perked up on this one yet.

Maps

I use Google Maps a lot. Their maps and navigation are pretty darn good. Google’s product integration here is enticing: if I get an email about a hotel reservation, that hotel gets a callout in Maps with the reservation. Now that, is a good reason to read my mail.

I looked for a few alternative navigation and map apps but none seemed to hold a candle to big G for that function.

Again if I jump into Apple’s planters I can easily solve the maps problem, but then I’d have to ask myself: am I too deep in Apple’s mulch? Will I have to write a post like this in a few years about how I need to pole-vault my way out of their yard?

Chromecast

This one is really convenient. Anything that is on any of our Android devices can instantly be played in the living room. I know there are alternatives, and among them, again, there is Apple. I think as long as I’m on Android, the Chromecast stays.

Play Music

I used to be on Spotify, which was OK, but I got on Play Music in large part because paying for it removes the ads from YouTube. I hate ads, and I am glad I can watch YT in the evenings and never hit an ad. This one is going to stick around for a while.

We’ll see how much Google is able to botch their music service though. It seems to be one of their most unloved products, and I am not looking forward to the impending migration to YouTube-Music.

Will I Ever

With the list above as long as it is, and with each service a struggle of compromise, or a head-first dive into some other mega-corp’s yard, I’m not terribly optimistic about my De-Google future.

For some services it doesn’t matter really: I don’t care if they know what music I listen to, and I am not going to quit YouTube as long as they have all the best content.

For others, I was able to make a change by doing some research and spending the time it takes.

Keep working, little background process.

This was day 12 of the #100DaysToOffload challenge.

Wfh Tip: Get Outdoors and Exercise

Olivier Forget — Fri, 08 May 2020 18:59:43 -0700

Los Angeles is starting to lift its stay-at-home restrictions, and hiking and cycling trails are among the things reopening.

Whew, just in the nick of time.

I haven’t really exercised since March (I don’t count doing a few pullups as “real exercise”), and my irritability is getting bad.

I need to get out and exhaust myself in some way at least once a week or I am prone to moments of anger and a general sense of angst.

So if you’re at home a lot and you’re feeling a little testy, try to get out.

This is the third in a series on tips for working from home. Previous posts were A Good Day Off, and Don’t Miss Your Sleep Train.

Get a Good Workout

I like a long, primarily cardio workout. It’s what I’m built for.

The key is to get your body moving. When you don’t even walk to your car to get to work, you end up with an abundant supply of energy waiting to get out. Most days you can ignore it, but keep it bottled up long enough, and it may get out on its own.

Apart from the positive effect this has on your mind, your body will end up fitter and healthier.

After a good workout you’re likely to sleep better, and we know that’s critical.

Get Outdoors

I used to swim laps, but after becoming self-employed I heard my own voice quite enough. Putting my head underwater to guarantee airtight isolation and staring at a line of tiles with nothing else to do but contemplate the same thoughts that have nibbled at my sanity all day is a “no thanks!”.

I find gyms claustrophobic. Apart from that period when I was into racquetball, I never enjoyed being at the gym. Climbing gyms are an exception, but even then I prefer climbing outdoors.

The goal here is to escape. The inside of your house is thoroughly mapped, you need to go some place where the walls are out on the horizon and the ceiling is the sky.

One of the best escapes for me is mountain biking. There are a lot of trails (well, fire-roads really) that run for miles in the hills that surround LA. The uphill parts are a great workout and the downhill is thrilling.

A typical weeknight ride would be 20-something miles and 2000 feet of climbing, while a weekend ride would almost double both numbers. But forget the workout, the reason I did it was because the views and the contact with nature cleaned out so many of my worries. It reset me.

Maybe just look at these pictures I took on rides:

That last one could be used on a pamphlet about the after-life.

And maybe that’s the point. To get out, to get up a hill, and to look out at a view like that is the best way to feel that whatever comes will come, and it will be fine.

This was day 11 of the #100DaysToOffload challenge.

To Draft Or Not To Draft

Olivier Forget — Thu, 07 May 2020 18:53:00 -0700

I wrote my first nine posts for #100DaysToOffload in one day without starting from a draft. Many bloggers leverage a well stocked drafts folder, and the really rich ones might be sitting on 30 post drafts and ideas!

I also have a folder full of partially written posts, and you would think I’d leverage that to get me through this nerdy take on Century Club. But that’s not how I’ve been doing it.

Most days I wake up not knowing what I’ll write about and wait for inspiration to strike. And then it’s a mad dash to get it fleshed out, prettied up, slimmed down, and finally published.

The Idea Folder

The idea folder doesn’t work for me.

When I look at my pile of ideas, I don’t perceive anything like the energy I got when I wrote them down. Even though each was a good blog post in my mind back then, I no longer see them that way at a later date.

The few times I tried to take an idea from the pile and make it blossom into a full post, it was an agonizing process. It seems I couldn’t rekindle with the original flame of the idea and labored to bring it back to life.

The problem is that writing is inspiration, and you can write an idea down but you can’t preserve the inspiration in a notebook.

Also, writing, to any degree, is a release for that moment of inspiration. Once you’ve written it down, even in part, it’s been consumed in some way.

The Never-Ending Draft

So writing ideas down doesn’t work, but what if I wrote more than an idea? What if I tried to capture the essence of the spark in imperfect written form? Then I could polish it up and publish when it’s ready. It seems that should work.

When I do this, I end up in an endless series of edits.

My inner perfectionist is a problem here. I know how I get when I try to do good work. It never ends. I’ll perpetually rethink the point I’m trying to make and then contemplate making this a series of posts instead of a one-off.

Then I’ll second-guess myself. I’ll lose my confidence in my idea and decide it’s trash, putting it on the backburner until I feel more ready to tackle it (never).

Drafts to me are like black holes: things only go in, nothing ever comes out.

Even during this series of daily posts I find myself reading and re-reading, tweaking sentences and trying to make it flow better. If it weren’t for bedtime and the need to write another one the next day, I’d still be on post number one when this pandemic ends.

Time Pressure

It all comes down to pressure, the need to get something out. It was my secret weapon in school. I always left essays to the last minute and knocked them out fast when I was low on time.

This is why #100DaysToOffload is working well for me: I can’t get ahead, so it forces me to write under pressure and it works out for me.

Each day I put my inner perfectionist in the corner, my ideas folder is out of sight, and I wait to catch a spark.

This was day 10 of the #100DaysToOffload challenge.

Mojave Planes From My XCOR Cubicle (Circa 2003)

Olivier Forget — Wed, 06 May 2020 19:08:00 -0700

A tweet about an Antonov heavy lift aircraft at LAX reminded me that one such beast had landed in Mojave when I was working there.

I set off to find pictures of the event in my old files, but we’re talking about early 2000s here, and I came up empty. It’s very possible I didn’t have a camera with me the day they let us look around the inside of the beast. It’s also possible there was a photographic record but it’s gone. I can’t find a single photo that predates 2003!

I did find a cool pic that proves I’m not crazy for remembering this event on JetPhotos. A couple more here.

What I did find in my old picture files was a small collection of pics I had taken from my cubicle window while employed at XCOR Aerospace.

An Office With a View

Mojave is not an ordinary airport, it’s a very active civilian flight test center. Countless unique experimental aircraft have flown and continue to push the limits at the desert outpost. And if they’re not unique and experimental, they are often in the upper atmosphere of coolness.

We had moved into the building on the flight-line during my time at the company. Prior to that, we were working out of an old shop with a trailer attached in the middle of blowing tumbleweeds, far from the runway. The move to the flight-line was like moving to the big leagues. Scaled Composites used the building to our right, and BAE Systems' QF-4 Drones operation the one to our left.

That is why Having a window on the flight-line was a treat. My office chair was a front-row seat to some fine aviation geekery.

I got this idea for a project to take a picture of every cool thing that taxied past or parked in front of my window. And I was going to tape it all up into some sort of mosaic of flying coolness.

Sadly back then I had to borrow the office’s digital camera every time I wanted to take a picture, so my output wasn’t a match for what would happen today with a smartphone camera in my pocket.

Still, I got a few, and for the project to live on I post them here.

Mosaic Of Flying Coolness

Here’s some random helicopter cruising by. Nothing special, but I just wanted to set the scene. You’ll notice the window is old. Well, our building was very old, possibly dating back to WWII. When I said we had graduated to the big leagues, I should have said we were the scrappiest team in the big leagues.

Next up, another mundane aircraft. There was a US Airways 737 parked outside my window for a while. Nothing special, but it was fun having a largish aircraft filling my view.

Speaking of airliners, there was a DC-10 too at one point. I remember when it arrived. It taxied and parked right in front of me one evening while I was working late. After it shut down I stepped out onto the ramp and walked around it, taking in its giant size. At one point a large forklift showed up and raised a pallet up to the door behind the cockpit. I guess there was no movable stairs handy, so that’s how they got the pilots down. The two clean-cut uniformed dudes took a hesitant but brave step from the aircraft onto the pallet and the forklift let them down slowly. Welcome to Mojave boys.

In the image above, beyond the tail of the DC-10, you can spot Scaled’s Proteus hanging out outside its nest, probably undergoing checks before a test flight.

As far as unique aircraft go, this one fits the bill. Only one built, fully experimental, and it’s still in service 22 years later, flying experimental payloads for any customer with good money.

I remember when it was built and rolled out. I was working inside of Scaled’s facilities as an employee from a different project that I’ll certainly blog about at some point.

Anyways, I remember the Scaled crew were nice enough to invite me to the rollout, and I probably took pictures back then (1998!) But do I have them? No, no I don’t.

Speaking of Scaled, here is White Knight, and if you look closely that’s its buddy SpaceShipOne tucked underneath.

It’s a pretty lousy photo I know, but the point is that it was shot from my office chair, and what’s lousy about that? These days I see SS1 when I visit the National Air and Space Museum in Washington DC.

I admit I’m kind of amazed, 17 years later, that I didn’t bother to raise the blinds for this one. Really? Maybe the explanation is we saw Scaled’s aircraft go past our windows so many times we didn’t exactly jump out of our chairs when they did.

Judging by the timestamp on the photo, this is early in captive-carry part of the WK+SS1 flight test program. Meaning the SpaceShip had not been dropped yet.

That might have been Pete Siebold flying that day. He was a regular WK pilot early in the program. Much later, he would survive a terrifying accident. He was among the nicer Scaled folks. Always pleasant to talk to, even with us scrappy underdogs.

RIP Mike A, who did not survive, and was a good guy too.

And finally, here’s our crew working on the EZ-Rocket. We never flew the EZ while I was employed at XCOR. They flew it before (I still remember where I was sitting when the whole town first heard that thing fly down the runway), and they flew it and other rocket powered aircraft after I left.

But just my luck, or lack of luck, somewhat characteristically of my time on various projects at Mojave, nothing great happened while I was there.

What I Have Missed

That’s a pretty weak set of photos compared to all the things I clearly remember seeing.

First, the obvious one: where are my F4s at? With BAE next door converting the venerable machines to target drones, there was no shortage of Phantoms going past my window. But I can’t find a picture of that right now.

Then there were all the times I could not have captured the moment because it was fleeting. My friend Mike did better than me, as with this F-117 doing a few flybys for example.

Mike also captured that time a vintage Douglas A-1 Skyraider came in for a pit stop. OK, it wasn’t exactly outside my window, I did have to walk a 100 yards or so but it was grand. Here’s a pic of me (at left) taking it all in.

Welcome to Mojave people.

This was day 9 of the #100DaysToOffload challenge.

WFH Tip #2: Don't Miss Your Sleep Train

Olivier Forget — Tue, 05 May 2020 14:18:00 -0700

Last night’s blog post on Home Automation took a lot of work to close up. I started writing before dinner then tried to polish it off quickly after our meal and episode 1 of Waco.

My goal was to finish quickly because I’ve learned that going to bed late, especially if I stay up in front of the computer with my mind engaged, is bad for my sleep and for the next day’s productivity.

The urge to edit didn’t subside until close to 11pm, and by then the damage was done. I slept poorly, and I am a total zombie today.

The bummer is I totally learned this lesson a long time ago. So here it is spelled out in blog post form.

This is the second in a series on tips for working from home. The first one was A Good Day Off.

Bed Time Not Wake Up Time

A common refrain among recent home workers is to claim that they’ll set their alarm to an early time and that’s when they’ll get out of bed and start their day.

Fine in theory, and easy to subscribe to since many normal daily job occupations revolve around dragging ourselves out of bed early enough to get there before the manager makes a point of obviously staring at his watch while you walk to your desk.

However, if you don’t need to start work at a specific time, then skip the alarm clock and worry instead about the time at which you go to bed.

This seems counterintuitive (“with no alarm clock I’ll spend the day in bed!") but it actually works out better, at least for me.

I haven’t used a daily alarm clock in over a decade. I naturally wake up at a reasonable time when my body is rested, which is all that matters.

Sleep Hours Not Work Hours

Sitting at home, disconnected from co-workers (if you even have co-workers) and pounding out work all day is taxing. It takes energy. Your internal drive isn’t free. Anything that can reduce your output will: headaches, fatigue, lack of exercise, too much exercise, etc…

Lack of sleep is very high on the list of things that cause me to have a lousy work day. Don’t get me wrong I can be slumped at my desk for a full 9 hours, but I’ll get little done if I’m not rested and ready, and what I will get done probably won’t be that great.

When you’re self-employed all that matters is that you get the work done and the work is good. Spending hours in front of the computer being unproductive has no value.

That’s why I focus on sleep hours and my overall wellbeing over a concern for what time I wake up. I rely on my body to tell me when it’s ready to tackle the day by waking up naturally. Most days I’m out of bed between 7:30 and 8:30.

There are also days when my body needs to rest, and rest it does. I wake up late, get started late and have to be efficient to get all my work done that day. I may not even get enough work done, but it’s still a win because I am well rested and will have the energy to make up for it later in the week.

Shutoff Time

In the early days of my career as a solo dev I had many sleepless nights. The uncertainty led to anxiety and long hours of turning over in my bed like an overcooked rotisserie chicken.

I’ve learned to deal with the uncertainty (you just get used to it) and I’ve learned to knock myself to sleep by using what turns out to be some form of self-hypnosis.

But the easiest thing to learn was to get away from my computer by 10pm at the very latest. It takes over an hour for my mind to relax enough to get into a good sleep. If I am still at work past 10, I will miss my natural sleep cycle and end up having a bad night, with the usual consequences.

This is the rule I broke last night trying to finish my blog post. The night was restless. This morning’s programming work was weak, labored, and not fun. I took forever dealing with email because my mind would completely stop focusing on the task at hand and wander pointlessly.

A bad night’s sleep is always followed by a bad day’s work.

Ease Into Sleep

After closing the laptop and putting the iMac to sleep the next step is a winddown period, which inevitably involves a screen, but that is not work related or overly mind-engaging. Social media, watching videos, movies… all those are fine.

After I get to bed I read a book until my eyes feel heavy. Here the idea is to avoid all screen time as I get closer to actually sleeping.

The only books I read at night are novels (no self-help, deep thoughts, or programming-related stuff, or anything that could bring to the fore current news events). I like books that make me travel to distant lands and faraway times.

It’s an escape, a dream before I dream.

Good Night

Tonight I’m finishing this post before dinner, and I plan on going to bed much earlier.

Tomorrow will be a better, more productive day.

This was day 8 of the #100DaysToOffload challenge.

Home Automation Thoughts

Olivier Forget — Mon, 04 May 2020 18:55:00 -0700

I’m intrigued by home automation but I haven’t taken the plunge yet. Here are some thoughts on where I stand.

Internet of 💩

I follow @internetofshit on twitter for a daily dish of how internet connected devices fail their users in absurd and painful (and sometimes expensive) ways.

Generally speaking, devices that connect to the manufacturer’s servers are iffy. Servers go down or companies pivot or go out of business. Others may just want to push their newest offerings by sabotaging their older products. Sonos raised eyebrows earlier this year for wanting to purposefully brick part of its product line.

Unfortunately most off-the-shelf home automation things are basically a proprietary device that talks to the manufacturer’s servers. And if you’re lucky enough that it works and stays up for a few years, you also have to contend with what this means for your privacy.

Internet of 🕵️‍♂️

This is the age of surveillance capitalism and any device we have that is connected to some cloud server, whether it’s that manufacturer’s or something else is likely leaking data about our private lives to corporations.

Most of this happens by silently shuttling bits of data gathered by the device in any way it can: microphone, camera, motion sensors, and we have only a vague notion of what gets sent, who gets it, and where it goes after that.

Personally this irks me and I want to have as few such devices as I can live with. The problem with most typical IoT devices is that the “I” stands for “internet”.

We have a Google Nest thermostat that was installed in the house before we bought it. I have yet to connect it to the wifi, and I may never. The idea of giving big-G still more data about me is irritating. Then again that company probably already knows the genders of our future children, so I may be overreacting, but whatever.

Do I Really Want Automation?

I find a lot of home automation frivolous. Maybe I’m old-school, maybe I just need to experience it to change my mind, but a lot of things I see seem like more effort than they’re worth.

Some people have an automatic “movie mode” that dims the lights and turns on the TV, etc… That’s fancy, but does it make my life better?

Add the potential for breakage of a probably brittle system and it sounds like it quickly turns into a nightmare. I imagine these systems are one bad software update from “automatically” turning the lights on in the bedroom at 3am.

These are just my impression. I might be wrong. And despite all that there are some things I would like.

Some Home Automation Things I Want

Thermostat

I can get up to change the temperature, but it would be nice not to have to run downstairs to do that. Even better, some “smart” thermostats let you add environment sensors in different rooms with the promise that the temperature will be ideal for the room you’re in, not the hallway downstairs where the thermostat is.

Google’s Nest has one of the dumbest ones on the market so I’m unlikely to burrow further into that ecosystem. Hey that’s a good thing.

Cameras

We need to add a few security cameras to our new home. Some for outside and some to keep an eye on the kiddo.

I want to view the footage from my phone, but I don’t want this potentially private stream going far into the cyber universe outside my control. I’m bothered by all my neighbor’s Ring cameras and where that footage goes.

I worry about hacks too. The manufacturer’s servers and the device itself are potential avenues for problems. I may even want to prevent the stream from ever leaving the home network, but will the device let me?

Problem Detectors

Yesterday morning I found a small puddle of water on the floor in the kitchen. Confused, I opened the nearest cabinet to figure out where the water was coming from. That cabinet, which happens to be under the sink, was full of water. No major damage, but it reminded me of a good use case for IoT: sensors to detect problems before they get expensive.

I can imagine putting water detection sensors under all the sinks, by the water heater in the garage, and perhaps near sliding doors in case it rains. Now that would be worth it.

Home Automation Soup

Dejected by the surveillance capitalists I turn my nose at their offerings and dive into the DIY, open source, home automation community. I find tons of projects and there are people doing amazing things.

There is just one problem: it’s a total soup. I have no idea what’s what. The other day I saw a this tweet by Home Assistant creator Paulus Schoutsen:

After @andreadonno showed off the pn532 working with esphome I had to try. Spend the evening and with a lot of me bugging him on Discord.. but it is working now! Also able to scan cards with the rdm6300 as that would work with my existing Magic Cards cards. – @balloob

I do not know what he’s talking about. All I know is some highly experienced HA hacker took a long time to get something to work. This does not bode well for me.

I did figure out that ESPHome refers to a software thing that helps you run things that are powered by an ESP8266 or ESP32 chip.

Cool. So I know about a chip. I’m sure I’m just a few web searches away from figuring out how to have my phone notify me of a leaky faucet.

Look, this is a hobby. I would love to use FOSS and open devices and assemble it all using my own ingenuity, but it will take time that I don’t have right now. (At least not for another 93 days.)

Living In A Manual World (For Now)

Home automation will have to wait as long as I don’t want to hand more data over to various companies.

I have denied my thermostat a connection to its mothership, and I get some exercise by going up and down the stairs to tweak the temperature setting on hot days.

I pray my plumbing holds and I don’t forget to close a window.

It’s fine, for now.

This was day 7 of the #100DaysToOffload challenge.

WFH Tip: A Good Day Off

Olivier Forget — Sun, 03 May 2020 17:08:00 -0700

Today is my day off for the week.

Unlike many people being at home all the time is normal for me. I’m used to it. Over the years I’ve learned techniques and patterns that make this life work well.

How many years you ask?

Thirteen years.

In the Spring of 2007 I stepped out of my cubicle at the small aerospace engineering office that no longer employed me, and have not been back to a “place of work” since. I also haven’t had a boss since then (other than myself).

Over the years I have assembled many little tips and tricks to get through the day. With 94 blog posts left to write I’ll get around to most of them.

One Day Off Every Seven

One day out of every seven has to be a full day off. If I don’t take a true day off regularly my fatigue grows and my mood drops as the days blend into each-other. Inevitably my productivity becomes worse than if I actually stopped working.

Yesterday I failed to accomplish all the work hours I set out to do for the week. Working at home with a baby doesn’t always work out the way you want. Despite that, it is more important to take today fully off than to meet my work goals.

“Why didn’t you do a bit of work in the morning and then take your day off?” you might ask. Nope. That’s not how it works. During a “day off” you must…

Do No Work At All

A day off has to be completely free of work, from the moment you wake up to the moment you go to sleep.

Ignore emails and everything else. Try not to think about it. Unless your servers are on fire, just ignore it.

Do Nothing At All

During the day off it’s important to have time to just lounge and feel like you’re doing nothing, and be OK with that.

During a work day from home, the pressure to get things done and the energy it takes to self-motivate can be draining.

That’s why you have to have moments where you deliberately do nothing, or nothing of any value. Be fully aware of your state of unproductivity, and bask in it. It’s a good thing.

I made the mistake in the past of doing something elaborate on a “day off”, like going on a day trip, or going sailing. These are tricky. It can feel very nice, but if you have to rush, or work hard planning and organizing it’s best to plan another real day off soon after.

Do Some Chores

Yes I know I said “day off” and “do nothing”, but here’s the thing. On typical work days, there is little time to do things around the house that need doing.

These end up weighing on your mind, and can distract while you’re trying to work.

Doing something useful can help you feel good about yourself. It’s the flip side of doing nothing: you’ll feel good for taking something off your to-do list.

A good day off is a careful balance of doing absolutely nothing part of the time, and checking some chores off your list.

Avoid Side Projects

If you have side-projects, only work on the ones that are distant enough from actual work. I avoid all forms of code.

I almost skipped writing my daily blog post for #100DaysToOffload (my 6th) but I think writing is relaxing enough, and I’m not writing about code.

However, I’ll keep it short.

Get More Done Next Week

Take a day off, a real one, regularly. Take two if you can. Enjoy doing nothing, and you’ll do more next week.

This Is a Bicycle For The Mind

Olivier Forget — Sat, 02 May 2020 19:22:00 -0700

I was working on a project today and found myself struggling to keep a mental model of the architecture. The project has been growing for many months and it’s gotten a bit hairy, particularly since I am constantly rethinking how it needs to be put together.

The code itself is fairly readable since Go lends itself well to that, but the high level view is not always clear in my head.

Today in particular I felt the need to give my mind a diagram that shows the relation between the different systems. I reached for my favorite tools for that kind of work: the Lenovo Yoga Book and DrawExpress.

The Yoga Book

Look, it’s flawed, it’s weird, it’s probably not the tablet you want to own, but to me it’s the bee’s knees (I don’t know what that means, but knees bend and so does this tablet).

The Lenovo Yoga Book is a tablet with a full-size digitizer attached. It feels like a slightly thick and slightly heavy tablet when the digitizer is folded against the back of the screen, but unfold it and you have an extremely light and thin device.

With the digitizer out the tablet becomes a magnificent drawing device. Here’s why:

The digitizer surface has a texture that makes the plastic stylus feel a bit like pencil on paper. When I draw on glass surfaces, as in a normal tablet, I find the “skating on glass” feel throws my drawing off.
With the digitizer to the side of the screen or below it, I always see all of my drawing. When drawing on the upper left side of a tablet (or a piece of paper!) my hand covers the rest of my drawing, and I find myself moving my hand a lot more.
No need to worry about my palm registering as an input. Some tablets have great palm rejection, and for others it’s necessary to wear weird gloves when drawing on touchscreens. I can’t relate. I can smother the digitizer surface with my palm and it registers nothing.
I use both hands: one to pinch-to-zoom, one to draw. Each hand is on its own half of the device, so I don’t need to move them much. It’s efficient and steady.

This tablet has plenty of flaws (have a look at the negative reviews on Amazon) but the magic of the drawing experience makes it one of my favorite devices ever.

So that’s the hardware.

DrawExpress

There are all kinds of drawing apps for mobile devices. Most simply retrace the movement of your stylus or finger using a brush of some sort. If you don’t have a steady hand you end up with horrifying squiggles. Hardly helpful.

I use these drawing apps too, but if I am out of practice my drawings are just painful to look at.

DrawExpress is different. It’s not a drawing app so much as it is gesture recognition interface with line-and-box drawing as its output.

DrawExpress doesn’t just log and retrace your trembly squiggles, instead it recognizes that what you drew looks kind of like a rectangle, so it draws a perfect rectangle for you. Trace a drunk circle that misses closing at its starting point by a mile, and it will draw a perfect one for you.

If you draw a line from more-or-less one rectangle to more-or-less another one, it draws a straight line connecting the two. Draw a arrowhead at the end of the line and guess what? The line becomes an arrow. And if the line is curved but bumpy, you’ll get a smooth curve. And so on.

Gestures aren’t just used for drawing, they’re for editing too. For example, tap a shape to select, then delete it by drawing an X on it.

I find this amazing. Think of how pretty much everything works on a computer: you find a button, and you hit the button. The button might be drawn in the UI, or it might be the del key on your keyboard, but it’s a button and your only interaction with it is to click it, tap it, or smash it. Future humans will find this amusingly primitive.

I find that when I draw an ‘X’ to remove something I am no longer communicating through a UI that I am aware of. It’s like my hand is the UI again, but now it’s incredibly stable and capable. It can make a perfect rectangle appear, and then disappear in a snap.

The video on DrawExpress’s site is broken, but it’s worth watching the one in the Play Store.

My Mind Went Bicycling

So this afternoon I pulled out the Lenovo, went all over the house looking for the stylus (I don’t use it that often, and where do you keep a pen-like thing that doesn’t write on paper?) and sat down at my desk.

With the code up on the iMac and the tablet where my keyboard normally is I set out to trace relations in my code. What object owns what? Who’s communicating with who?

I didn’t set out to do a formal diagram like UML or any such thing (though you can do that in DE), I just wanted to turn a mental hairball into something understandable.

From mental hairball, I drew squiggly ugly lines, and from those I ended up with this:

Looks simple doesn’t it? Yep, that’s the point.

This was day 5 of the #100DaysToOffload challenge.

Should I Write a New Protocol?

Olivier Forget — Fri, 01 May 2020 14:22:00 -0700

I’m finding myself in need of a protocol to communicate between two processes, and I can’t find what I want.

The two processes will communicate over a Unix Socket in a peer-to-peer fashion. This is a very straightforward situation that I thought I would be able to handle trivially.

It is, of course, very easy for two processes to exchange bytes over a unix socket. The web is full of tutorials on writing various “Hello World” and echo services over unix sockets.

There are also many data packaging and serialization protocols, such as the aptly named msgpack, or the tried and true JSON, and Protocol Buffers.

What I found to be missing is the bits in between. On a streaming unix socket, there are no inherent message frames, or message request-response paradigms, so it’s entirely up to you to work that out.

There are a few such protocols available, but most seem to be too heavy, or are less than ideal for one reason or another.

What I Am Looking For

Here is what I would like:

Lean, fast, lightweight. This needs to run on low power devices.
Message payloads should be binary or plain text. Not JSON. I want full flexibility to pass anything as payload, and I do not want to coerce binary data into a JSON string.
There are multiple different services at each end of the connection, so the protocol should include metadata that identifies the service that should receive the message without decoding the payload.
The protocol should pass metadata that tells the service how to handle the payload. Think of this as the function that the service has to call.
Protocol should support request/response, and request/multi-response (to get rows from a DB as individual messages instead of one giant lump).
All messages passed should get a response indicating whether its processing was successful or not.
Support multiple simultaneous asynchronous messages.
Go and Typescript implementations.

While I am itching to roll my own (being in over my head yet learning something along the way is an appealing cocktail of emotions), I did look into existing solutions.

Existing Protocols

NetStrings

NetStrings is the simplest of protocols. It fails to meet my needs but I absolutely love the minimalism.

When I first looked into this I thought I could just use NetStrings. But once I added the need to forward payloads to different services, and to pass other metadata (including message ids) I found myself creating “Netstrings With Metadata” and it got complicated.

MsgPack-RPC

MsgPack is appealing, so its RPC extension should be as well, right? Turns out it’s a bit underwhelming for my needs: it only supports request-response.

WAMP

WAMP is a websocket subprotocol, but it can be used over unix sockets as well. It looks like it is essentially what I need. The Go project that supports it is called Nexus, and it looks solid if maybe more than what I need.

If I fail at rolling my own this is a good candidate to rescue me.

Stomp

According to their home page STOMP “comes from the http school of design”, and taking a very brief look at their protocol, I do see the family resemblance. I’m looking for something that uses fewer messages. I don’t need a lot of connection stuff because I’m essentially hooking up two processes, that’s it.

Zero-MQ

ØMQ is rad, and venerable, and I’ve always been interested in it but never got to use it. Looking at all it can do, it seems overkill for my needs right now.

I should look more at it, I just have a resistance to “everything and the kitchen sink” type of solutions to small problems.

Hessian

I came across this so I’m including it, but it looks dated. No support for Go means it’s DOA as far as I’m concerned. Moving on.

BEEP

Dated, and way too verbose (it uses a subset of XML), but I found some interesting pages about protocol design on their site.

AQMP

This is an enterprise grade standard. Looks like it’s too much for what I need. The bare minimum message is already much bigger than what I have in mind.

MQTT

This caught my attention because the acronym comes up in home automation systems. But it’s a protocol for telemetry, so not what I’m looking for in this context.

What Did I Miss?

If you know of something I should look at please ping me.

What Now?

I started an implementation of something that might work in Go. It’s rough and I’m not very confident about it. So I keep going back to learning more about existing protocols to see if it’s less trouble to adapt to something non-ideal than to roll my own.

Expect more posts about this as I work through the #100DaysToOffload challenge. (Today was day 4.)

The Walled Wide Web

Olivier Forget — Thu, 30 Apr 2020 13:14:00 -0700

There is a situation developing on the World-Wide-Web that runs completely against its ethos.

I am talking about the growing number of news websites that require that you are signed-in, and possibly a paying member to read their articles.

From the web peruser’s point of view, it’s a terrible experience. You might be reading your feed on social media and decide to click on a link to read the actual article (instead of just piling on in the replies) but then, bam:

Paywall.

Instead of navigating effortlessly to an article you were interested in you just hit a wall. Your flow is disrupted. No login? No subscription? You’re out. Go back to where you came from.

The problem here is not that news sites should not get paid (they should), or that I don’t want to pay (I’m willing, and I currently do for a couple outlets), it’s that putting paywalls around entire sites goes completely against the grain of the WWW.

The World Wide Web

The promise of the web is that you can read a document, and in that document there are “links” to further information that expand and go deeper on whatever text was around that link.

It’s an incredibly powerful concept that accelerates the development of knowledge. A link can serve as a path to “more info”, or as “proof is here”, or as “you might also be interested in”, or as “here is where I got the original idea”, etc…

To click on a link is to effortlessly float from world to world because the mind yearns and I made the smallest of physical efforts.

It’s such a powerful thing that people refer to “going down the rabbit hole” when they set out to follow links and get engrossed in a topic that may or may not have been what originally made them curious.

In fact one of three reactions to my post on Elon Musk yesterday was a request that I add links to the things I was talking about! (I wanted to but ran out of time, such is the life of a #100DaysToOffload blogger.)

I highly recommend perusing the original proposal for the web by Tim Berners-Lee. It’s good to be reminded of the original vision for the web once in a while.

The Walled Wide Web

In these pandemic times, there is news coming out of everywhere, lots of which is important and I find myself clicking through to more news sites than usual. Unfortunately, it hardly feels like the golden age of information availability.

During my morning reading time, I might need a dozen different logins to read what I clicked on. What are the odds that I have a login for some news site in Nebraska? Am I going to sit there and type in all my info to create a login just to read one article? No way.

So I hit the back button, disappointed, cheated out of the information I was teased, and that much less confident I know what I need to know.

In the original proposal for the web there is the idea that any page author can create a link to another page, without seeking authorization from that other page’s author. The implication here is that all pages are part of one system, and they can all cross-link without concern for boundaries.

The web is meant to enable the free movement of information travelers. That’s the point. The whole point. Paywalls are like walls along borders: they keep people out, break their flow of travel.

The Solution

I don’t have one, sorry. This is a difficult problem that involves a lot more than just some opinion on what the web “is supposed to be”. News costs money to make, and that’s a huge constraint. Still, a few things are being tried:

No Paywall

The Guardian rejects paywalls and instead make a plea to those who can afford to subscribe. They also have some lower level app-only subscriptions. This is one of the ones I pay for.

MicroPayments

The idea here would be to allow people to pay per-article in some way. It’s probably a terrible idea (warning: Medium link).

Bundles

My friend Josh tweeted this today (and set me on course to write this post):

“I end up paying for no media subscriptions because I’m always flipping through headlines and sites but would almost certainly pay $20/MO for deeper all-in-one access.”

– @joshuakrafchin

Apparently that’s what Apple News+ is. But now you’ve jumped into a different walled garden.

Keep It Open

Like I said I don’t have a solution. I just know you can link to this page if you want to, and I won’t throw a paywall up at you.

PS: Today is the 27th anniversary of TBL putting the www in the public domain.

This was day 3 of the #100DaysToOffload challenge.

Bad Elon

Olivier Forget — Wed, 29 Apr 2020 13:15:00 -0700

I checked Twitter briefly last night before going to sleep. Bit of a mistake. I saw something I did not like and it caused me to spend some time feeling uncomfortable as I was trying to sleep.

I’m talking about Elon Musk suddenly aligning fully with the “Freedom” protesters.

Let’s backtrack a bit in case this gets read at some future date.

We Are In A Pandemic

It’s 2020, we’re in the middle of a pandemic courtesy of the COVID19 virus. In the US, things aren’t going great. The US has more deaths than any country. We’re all under some degree of “stay-at-home” orders from various authorities. According to experts this is the only way to keep the number of cases low enough to not overwhelm the healthcare system until we come up with a vaccine or a cure. It could also give them time to gain enough of an understanding of the virus to know how to tame it with a less blunt hammer than “everybody stay home”.

The economy is taking a huge hit, that’s undeniable, and people would like to work. I don’t envy those in charge. There are a lot of unknowns, research is hastily published with contradictory results, and consequences of getting it wrong can be dire.

These are tough times, but most of the country seems to agree that we need to do the right thing to prevent deaths of those who are vulnerable: the old, those with other medical conditions, and of course healthcare workers.

In the midst of this debate some people have decided that the right course of action is to flaunt all public orders, all health recommendations, and even all logic by going out onto the streets without masks, and sometimes with guns. In some cases, medical professionals have showed up to counter-protest, setting up for news photographers to capture some painful imagery: angry people, decked out to go to war, yelling at nurses and doctors.

Do I need to type an argument defending the role and value of healthcare workers during a pandemic? I hope this is a low point in American culture and politics, because it’s just painful to watch.

All these protesters rallied around a call for freedom. It’s usually written in caps (“FREEDOM”) because they’re yelling in anger, possibly at a nurse. They were encouraged by our Dear Leader, always the promoter of divisive causes.

So that’s the context.

Back To Elon

I’m a big fan of SpaceX and Tesla. Both companies have done so much to upend their respective industries. These successes pull at my heart strings as an aerospace engineer and environmentally conscious Californian (we own a Tesla and love it). Unlike some people I can’t imagine these companies would be as successful (and in the case of SpaceX, even exist) if the man at the top was not Elon Musk.

Elon gets a lot of crap for his attempts to help when the world is rallying around a cause. In most cases I think he’s genuinely trying to do good.

So last night as I took a last look at Twitter I saw this gut punch of a tweet:

“FREE AMERICA NOW” – @ElonMusk

I imagine this wild cry into the night by a man who owns many factories has something to do with his personal finances.

Most of the time I feel Ok brushing off his more unsavory tweets. I know he’s a flawed character, there is ample evidence of that. I don’t defend his BS, and I don’t make excuses. He’s a dude with lots of capabilities and plenty of flaws too. When tweets come, I know sometimes I’ll have to shake my head or sigh.

This one just stings.

This was day 2 of the #100DaysToOffload challenge.

100 Days To Offload

Olivier Forget — Tue, 28 Apr 2020 13:29:00 -0700

I came across a post on Mastodon with the hashtag “#100daysToOffload”.

Having seen a lot of #100DaysOfCode posts on Twitter, I was wondering what this “offloading” was all about.

The Challenge

100 Days To Offload is a challenge to post in your personal blog every day for 100 days.

Hahahaha like that’s going to happen.

I set up this Hugo thing over a year ago because I wanted a place to recall my experience seeing a SpaceX Falcon Heavy launch. Then guess what happened? I got one more post published in December 2019, and it only saw the light of day after an agonizingly long series of drafts and reframings.

Not Doing It Man

To think I could post every day is just hilarious, but that’s too bad. The first line in my “Hello World” post was “I’ve been meaning to blog”. Guess what pal? You’re still meaning to.

I have ideas, man. Just because my blog is blank doesn’t mean my mind is. Promise!

But like heck I’m going to join a challenge to blog every day for 100 days. I can barely keep up with daily work with this pandemic. We have the kiddo at home and my work is still going 100%. Now you want me to dig even deeper and do the thing I couldn’t make myself do in the before-times?

It’s clear I need to be pushed if I am going to unlock a spell of writing on here, but come on.

Out of curiosity I checked out some other blogs who are on the challenge’s blogroll, like this one by @basil. It’s another year-old Hugo install with three posts, and an anticipation of failing this challenge after four posts.

Sounds exactly like me right now. And they’re doing it.

OK I’m Doing It Man

I decided I am doing this. At worst I will fail in good company.

See you tomorrow.

My Personal Adventure Through Note-Taking Services, 2008 - Present

Olivier Forget — Mon, 02 Dec 2019 19:04:44 -0700

I love taking notes. It might be the only effective way for my brain to work. Regardless of the technology involved I always gravitate towards a blank slate of some kind and spill thoughts onto the medium.

I used to buy paper notebooks and filled them with barely readable words and other scratches. There were arrows, diagrams, and strike-throughs on every page, and exclamation points inside triangles for when I really meant it. The notes were pure expressions of my thoughts.

Then one day I decided the physical paper medium was too cumbersome (I had a pile eight inches high) and I switched to digital notes. I knew I would miss the free-wheeling nature of paper notes but I believed I would gain in searchability, portability, and durability.

So how did that work out?

Luminotes (2008-2010)

Once upon a time there was a web-based note-taking service called Luminotes. It was brilliant. It was like a wiki but it didn’t require remembering tricky syntax to link related pages.

When I was taking notes on paper I could let my hand write, I just had to think. I wanted to stay as close to that feeling in the digital realm. Any kind of syntax necessary to create a note would force me to engage my coding brain (are the brackets matched? do I need to escape this underscore?) and break my flow. In fact I valiantly resisted using markdown-based systems because they make me feel like I’m writing code, not notes.

I dove in and started organizing all thoughts related to my projects in a bunch of linked notes. The Wiki was the organizational paradigm that I could not get on paper: I could now record relations between notes in addition to the notes themselves.

I made heavy use of this service and depended on it. I was a big fan, an early and dedicated user, and I exchanged a number of emails with the developer.

Then one day the developer said he would no longer maintain the project. He had moved on and didn’t want to deal with the overhead of maintaining the server. He was willing to open source the code, but he would not participate in its development.

To be clear I don’t blame the dev. Maintaining things is hard, and when you’re providing a service it’s even harder. Babysitting servers when your heart is no longer in the project is even more challenging, and shutting things down is be the only reasonable thing to do.

I really wanted to keep using Luminotes in some way or another, so I created a Google Group to organize the efforts of the few Luminotes die-hards who would band together to keep the app alive through the magic of open source collaboration.

We got nowhere.

I never managed to install the darned thing on a development server. It was written in a language I didn’t know and depended on outdated dependencies that I failed to get installed. Others in the group had similar issues. Our meager efforts fizzled then died, the hurdles too numerous to justify staying involved.

It was time to look for another note-taking application.

Catch Notes (2010-2013)

I settled on Catch Notes after a vain search for a wiki system that had the same ease of use as Luminotes.

I examined plenty of wiki authoring tools but none of them had the same degree of “thought-free operation” as Luminotes did. I tried using some but they felt finicky and I just couldn’t write comfortably with them.

So Catch Notes it was. The problem with Catch Notes is that it wasn’t a wiki at all. It used a system of tags to keep notes organized. The result: after migrating my data my notes degraded from a carefully crafted personal knowledge graph to a pile of disconnected notes. I could no longer just click around from note to note to follow my thought process at the time. My painstaking work had lost a good chunk of its value.

Still, I knew that would happen and accepted my fate. Over time I started thinking in terms of tags and little by little found it to be a decent way to go. I found peace with CN and tags and enjoyed productive note-taking.

Then – shocker – Catch Notes shut down.

Evernote (2013-2016)

This time I decided to go with a big established player: Evernote. I had fully adopted the tags system of Catch Notes, but unfortunately Evernote’s support for tags was flimsy. My notes lost some of their value again as they got processed through another migration.

My experience with Evernote was never stellar. I was there because I felt they wouldn’t die and not because their offering was compelling to me. Their desktop app on Mac was slow and badly designed and they seemed to be developing new products in dozens of new directions while their core offering was underwhelming. Not a good place to be.

Then Evernotes started showing signs of not being around forever. There was talk of financial troubles and of an exodus of users.

But who would be surprised by this turn of events? This was a note-taking app that also sold socks on their website for crying out loud.

This time I decided not to wait until the app I used went belly-up to spread my wings and fly away.

OneNote (2016-present)

I don’t recall exactly how I ended up in Redmond’s grip. It might have been because Satya was charming the tech world by reinventing Microsoft.

Or it might have had something to do with my new Windows laptop with a touchscreen and digital pen. I got excited about taking freehand notes again, and OneNote’s tools seemed well suited for that.

That excitement was nothing short of the tantalizing feeling that technology was catching up to fantasy. Could I get the best of both worlds? The freedom of hand-written notes with the weightlessness of the digital medium?

No. The answer is no. The fantasy did not survive contact with reality. I wouldn’t take any notes by drawing with the stylus or my finger. I tried, but it didn’t work. A 15" laptop is very awkward to use as a tablet to draw on, and OneNote’s tools for drawing leave a lot to be desired.

So I got duped by a bunch of kids apparently sketching cool mods for their fixie bikes. Can a corporation pander to me any harder than that? I should have known.

So now I’m using big clunky OneNote and it’s not a party. I don’t sketch with it like I wanted, and its UI is pointlessly cumbersome. Their phone apps aren’t good, syncing takes too long and is unreliable, and their desktop Mac app always throws some sort of authentication error so I’ve given up on it. Also its search function is terrible.

I desperately want to spread my wings again.

Where To Now?

There are some new players in the space and people rave about them on the socials. I read about these shiny new note-taking apps and I feel a tinge of envy and a desire to try them out. But I’ve been let down too many times. I want to do one more migration and then that’s it.

Open Source

Clearly I should go with something open source. Looking at my trajectory from Luminotes to OneNote, I have gravitated towards closed systems. In retrospect it’s possible much of my troubles are due to that.

However remember that Luminotes was open source, and that didn’t do me any good. It’s not enough to be open, there has to be a sufficiently active community for the project to survive the departure of the original creator.

Another factor is that open source note taking apps are often developer-centric, which usually means they use markdown or some other syntax, which I am not a fan of. Frankly I just haven’t found an open source note taking system that suits me.

Finally there is the issue of self-hosting. I might jot something down on any of 4 devices I use regularly so I don’t want my notes to live on a single device. If there is no service available I’d have to run a server myself. This is a pain. I spend enough time dealing with servers, I really don’t view tinkering with a Linux server as a hobby. To me it’s a chore, so self-hosting comes with its own set of problems.

New Note Taking Paradigm?

I’ve greatly evolved how I take notes and how I think of my notes. I don’t think the Wiki approach works for me anymore. As the number of notes grows it becomes a lot of work to keep that graph organized.

Categories don’t cut it either, they just don’t scale. And tags can scale but are too limiting. I’m rethinking the very concept of a note too.

To be continued…

PS: Thanks to the Internet Archive for making it possible to get screenshots of defunct services 10 years on! If you’re able please donate to the Internet Archive.

What it's Like to View a SpaceX Falcon Heavy Launch

Olivier Forget — Sun, 14 Apr 2019 10:04:44 -0500

I flew to Florida to witness in person the awesome power of the Falcon Heavy.

My viewing location for the launch of Arabsat 6a was Banana Creek within the Kennedy Space Center, also known as “Feel The Heat” in KSC’s launch viewing nomenclature. It is 3.9 miles due West of the launch pad, as close as one can be for such a launch. I do not want my recollection of this experience to become clouded with time, so here are my notes.

The Ambiance

I traveled alone for this event and I expected to be alone most of the time. Boy was I wrong. I had many wonderful conversations with other space fans. I met a lot of people from different walks of life, and from many places further than my home in Nashville. Some I met in person after communicating on a Slack channel connected to the /r/spacex subreddit, but most were other excited space geeks who were waiting in line for a bus for the umpteenth time.

Apart from the FH launch, other big space events were happening: the first photo of a black hole was revealed and Beresheet was scheduled to land while we were waiting for launch. We all followed the progress of the Israeli lander while sitting in the bleachers and commiserated when word of its demise came through. The bleachers are so stocked with space nerds that when I wondered what Beresheet had launched on I asked the person nearest me and got an immediate answer: it was a Falcon 9. Where else can you ask that question to anybody and get an answer?

As the countdown clock ticks past milestones, the crowd begins to react more and more excitedly. At the 1 hour mark big cheers erupt. Each time the announcer mentioned even the vaguest possibility of a scrub, he would elicit boos from the crowd. We’re all passionate, excited and jittery in the bleachers. It makes for swift and audible reactions every time something happens. A puff of oxygen emanates from the rocket? Wild cheers. The announcer reminds us that they will scrub if they notice anything wrong during the last second of the countdown? Merciless booing.

The History

It is hard to sit across the water from pad 39A and not think about its history.

Every person who walked on the moon launched from that same chunk of concrete. It’s remarkable to think that humans boarded a ship in this location and sailed across the vast expanses of space to reach another rock to call home. I noticed the moon hanging directly over 39A and the symbolism was perfect. A vast expanse of featureless sky between a rocket pad and its destination.

Apollo 11, July 16 1969. Every person who walked on the moon launched from Pad 39A. *Photo credit: NASA*

Eighty-two Space Shuttles were launched from this same pad, among them the very first and last. *Photo credit: NASA*

What I saw: Falcon Heavy, 2019, same pad. *Photo credit: SpaceX*

Original idea for this sequence of photos from this tweet.

Soon, if all goes according to plan, US astronauts will launch into space from US soil aboard US rockets for the first time since the end of the Space Shuttle. They will do so from 39A.

The Sound

This is the thing most people wonder about. Home speakers or even good headphones can’t replicate the sound in your living room, so what is it really like?

It begins with silence. It takes 18 seconds for the sound to travel from the pad to your ears. When the engines are lit, an immense angry cloud jets out from the side of the launch pad signaling the beast has awakened. However there is no sound. The Heavy lifts off and climbs. Still no sound. A rocket can travel a long way in 18 seconds, and I began to feel a real disconnect between what I was seeing and what I was hearing.

We had been told it was possible to see the sound waves coming across the water. I saw no such thing. My first indication of sound was this sense of deep motion in the air, the lowest deepest notes I can imagine hearing. It’s a soft, pleasant experience that soon gives way to an ever increasing roar. Finally, things sound like they are supposed to.

The roar sounds just like just what it is: an immense amount of energy unleashed all at once. It builds gradually in intensity while dropping in pitch (due to the Doppler effect.) It’s a roar that you can feel in your chest, and that is why headphones can’t replicate the experience.

It then begins to crackle and pop. I used to think the crackling of rockets in launch videos was due to limitations of the recording equipment. It turns out it’s the physics of the atmosphere. Imagine a loud thunderclap. Now imagine that sound in a continuous loop. You’re getting close.

Then, very quickly, it becomes quiet again. The Heavy accelerates away so fast that it doesn’t take long for it to go out of earshot.

The Light

I was not prepared for this.

Once the rocket cleared the tower and the plume was in full view, I felt like I was looking into the sun. It was so bright my eyes instinctively wanted to look away. But we had waited 6 hours for the launch to go off the day before, and 4 more hours today in addition to all the bus lines, so when that rocket finally made itself visible I was not going to look away. Too bad for my retinas, let ‘em burn. I am not taking my eyes off that rocket.

The Great Arcs

I have watched many webcasts of SpaceX launches and they are a great way of following the action and getting a taste for the excitement from home. However to see it in real life brings a new dimension to the event. It becomes less about zoomed in views and performance data, and more about the vast expanses, the palpable speeds and dizzying altitudes. We were able to see the different elements play on the vast stage that is the sky in one continuous take.

I threw together a couple of graphics based on the few pics I took with my phone to show the sequence as we saw it:

Full size original photo.

Launch! Much excitement. Once it clears the tower the rocket plus its giant plume makes for a satisfyingly large and shockingly bright thing to look at from Banana Creek.
Sound is finally getting to our ears. The rocket is accelerating and is starting to turn downrange, meaning we see less of its side and more its tailpipe. It’s an impressive sight, like a three-eyed Sauron.
Aw shoot. We launched into a perfectly clear sky, but the rocket created a contrail that grew into a cloud, hiding itself from our view. Last we saw it, it was headed straight away from us. It’s quiet in the bleachers. Surely the party’s not already over, is it?
Falcon Heavy reappears below the cloud as a wide orange triangle. Wild cheers from the crowd. It is below the cloud because it is now heading more away from us than up, like an airplane headed for the horizon. The plume has grown so much in the rarified air that we still see it as a distinct shape. It quickly gets dimmer as the rocket accelerates away from us.
Booster separation! By this point we’re really straining to see clearly, but I did see the boosters separate as two little sticks that fell off to each side. After a few more seconds we could see the two additional smoky plumes from the boost-back burn. As soon as that burn ended we lost sight of the side boosters.
By the time the photo was taken the center core was accelerating fast away from us and about to disappear for good.

Full size original photo.

At 6 minutes and 11 seconds after launch the re-entry burns were going to start so we all trained our eyes to the sky, taking our best guess as to where the two boosters would appear. Somebody yelled "there!" and we all looked to see where that person was pointing. Two bright yellow dots were plainly visible to the right of the cloud. The reentry burns are short but they allowed us to spot the returning boosters. I was able to follow them all the way to the ground. They were mere specks at first but became more discernable as they dropped.
Holy moly, these things are cooking. I remember being awed by the speed of the two objects I was looking at. Seeing them plummet from such an altitude really brings home why rocket landings are such an extraordinary feat.
The landing zones are much further away from Banana Creek than the launch pad so the landing burns appear as points of light. This is too bad, but you can either be close to the launch or close to the landing. I lucked out: my spot in the bleachers allowed me to follow the boosters as they squeezed just between the VAB and the tan building. Fifty two seconds after the landing, a quadruple thud of sonic booms hit the crowd at Banana Creek, who were all too happy to erupt in cheers yet again.

Some Tips

Here are a few tips for viewing a SpaceX launch and landing at Banana Creek:

Download the press kit PDF for your launch and have it ready on your phone. It spells out the exact times of all the events of the launch. Use that and the countdown clock (which counts up after the launch) to prepare to spot events as they happen. You don’t want to be looking away when boosters separate! Here is the Arabsat Press Kit.
To see as much of the landing(s) as possible, sit at the top of the bleachers South of the Saturn V Center. To avoid having your view obstructed by the tan building, don’t sit in the Northern most bleacher. I was at the South side of the second bleacher and saw as much as you can see from Banana Creek. Trees and other things will obscure the view as you sit further South.
Ignore the livestream that they are showing on screens. It’s a distraction. It’s as much as 30 seconds behind and buffers a lot.

So What’s it Like?

The sound, sights, the ambiance are each thrilling and together they are more than the sum of their parts. Viewing a launch live is unlike anything I’ve experienced. And like any good experience, there is no substitute. No blog post can convey the complete experience. You have to go for yourself.

The First Post

Olivier Forget — Sat, 06 Apr 2019 10:04:44 -0500

I’ve been meaning to start a blog for a while now.

I suppose I took so long to start because I much prefer building things than talking about building things. When I was an aerospace engineer I worked in small experimental shops that routinely produced hardware. We lived by the classic saying “Hardware talks and bullsh– walks” and we had little respect for office-dwelling engineers who only produced theoretical papers.

Now as a software developer I live by the same idea. As a result I rarely talk about what I’m working on. I’d rather show you when I have something usable. Unfortunately, the nature of projects is that some never reach that stage, and as a result I never talk about them.

Why Even Blog

I want to blog about the things that I am working on and my motivations for working on them. I want to do this before projects reach releasable state if only to get it out of my system. Some projects never reach that state and they languish on my hard disk without a single publicly visible trace.

I also want to relate some of my past experiences in my winding career. If only for myself so that I can come back and refresh my memory before it all evaporates.

The Stack

I’m generating this blog with Hugo. It was a natural choice given my recent adoption of Go. I moved the olivierforget.net site from AWS to Netlify to simplify deployment.

I like simple readable blogs so I am modifying the Indigo theme to my taste.

Taking the leap to using an existing static site generator is an admission to myself that Cicerone is not usable for blogging after all the work I put into it. But Hugo’s complexity reminded me — yet again — why I was developing Cicerone in the first place.

Maybe I should blog about that.