Dropserver Progress - March 2024
This is the monthly progress report for Dropserver for March 2024. The previous report is here.
This happens to be the twelfth month for which I am publishing an update (the first progress update covered April-June 2023). A full year! Yay me.
It’s also time for me to change how I write these posts. I think I’ll make them shorter and more to the point and I will dedicate complete blog posts when a topic deserves deeper examination. For example, The State of Sandboxing in Dropserver is a post I keep going back to and that stands out in my list of posts, while the series of “progress updates” are just so much water under the bridge.
The Big Picture
I’m plugging away despite having less time to spend on Dropserver due to other projects. This reduction in time results in a slight decrease in time spent on the core of Dropserver, and a significant curtailing of peripheral work like experimenting with Lume and other side-quests.
I started work on the next big feature of Dropserver, but before that I did some maintenance work and pushed a minor version.
Prepare for Deno 2.0
The big 2.0 for Deno keeps getting pushed back. At this point I don’t know when it’s coming but it sounds like the previous timeframe of “April” will not happen.
Dropserver uses Deno’s sandboxing features to prevent app code from doing bad things, so a big change like whatever might be coming in Deno 2 keeps me on my toes.
Thankfully Deno published a guide for migrating to Deno 2 that describes the APIs that are definitely going away in D2. I made changes to the sandbox code to use the latest server APIs and std lib (commits 283b48 and b603e5.)
I am still struggling with a mistake I made long ago: I included the now deprecated Deno.RequestEvent TypeScript type as part of a type in dropserver_lib_support, which is kind of a bummer since that library is not supposed to change to preserve backwards compatibility. I may write a separate post about how that works and lessons learned there.
Inotify watchers fix
Back in December I experienced a (thankfully rare) anomaly on my personal ds-host instance. It turns out it was this bug that was causing ds-host to create and not clean up inotify watchers. I fixed it in commit 0580e5.
(I just looked at my personal instance which has been running this fix for a few weeks, and can confirm the inotify watchers are not growing unbounded. Pfew.)
Release 0.13.1
With these improvements committed, why not release a new version of Dropserver? I tagged 0.13.1 on March 13.
Next Feature: Outgoing Net Requests
Dropserver apps are currently not allowed to make outbound network requests. This is a huge limitation that I am keen to eliminate. I could of course add a --allow-net to the Deno sandbox and call it a day. But that would go against all the principles of security that I’ve established for this project.
I am spending time trying to figure out how to allow network access such that it is convenient for users and developers, and such that it does not pose a security risk to the user.
There is lots to consider, including how Deno does network sandboxing (not great currently, maybe better in the elusive 2.0?), the kinds of things I want to protect against (SSRF among others), etc… I have run across gotchas everywhere and it’s making my head hurt.
If I start writing about this here I won’t be able to publish this update any time soon. So it will go in a separate post some day.
Proxying outgoing requests
Here is something tangible I spent time on: I worked on adding an HTTP proxy in Dropserver that sits between the Deno sandbox and the outside world. The idea is that for HTTP fetches at least, I can see what the app is requesting, I can log it, I can even deny it if it doesn’t pass my SSRF filters.
I learned along the way the MITMing HTTPS connections is very computationally expensive. If an app needs to download modules from deno.land/x/ its startup time increases five-fold! Without MITM I can still log the destination host and run the connection through my SSRF filter, so that’s good enough.
This proxy is not a good security solution because it’s only effective if the app uses the “fetch” call, but it’s a starting point of something. While I suspect a proxy like this will play a role in a fully implemented secure outbound request solution, it’s not what it is now.
Still, I may include it in a minor point release along with the ability to see fetch history, purely as an informational thing.
Work On Leftovers App
I made some small improvements to the Leftovers app. I use this app daily at home so its little UX paper cuts got to me all the time. Finally it’s starting to feel quite pleasant to use.
A true home cooked app for home cooked meals.
This was also an opportunity to run through the release + publish flow for a Dropserver app. It worked, but it was a bit fiddly. I may need to create a Github Action that takes care of everything for the app developer. Push a git tag and it cuts a release and publishes an updated version of the website to GH pages. Boom done.
It was also an opportunity to experience the app upgrade process from the user’s point of view. Here again, more work needed 😔.
What’s Next
While I want to make progress on outgoing net requests, there are a few other things I should improve.
In particular with JSR out, and the realization that it uses import maps, I should probably support them in Dropserver apps. There are other reasons too: I’d like to support vendoring dependencies so that apps can be published with all code included, but that depends on import maps too. (And guess what? Vendoring affects outbound requests too: see the bit above about hitting deno.land/x/ through a MITM proxy!)
There are many other things I could work on, and I’m not sure what I will do… so see you next month!
