Winding down Bad Gateway – Evert Pot

In 2019 I started Bad Gateway as a software development agency. Last year we
grew all the way to 7 people. It was crazy challenging, especially with Covid
in the mix; but ultimately could not get the company into a good financial
state to be able to carry on.

Big thanks to my co-workers and partners Ju, Becky, Phil,
Michael, Siep, Richard and Syed.
I’m incredibly grateful you came on this journey with me. We shared some
tears, exchanged some words but mostly had lots of laughs. Despite the
challenges I feel my relationship with you has only strengthened and I wish
you well in the next steps of your career.

Also thank you to our customers and especially Underknown who’ve stuck
with us since the start.

Despite this outcome, I have a hard time seeing the last few years as a
failure. It’s been hella fun, and I feel we stuck to our values even in times
of crisis. Off to the next adventure.

Image of person standing in front of a gateway.

Building a simple CLI tool with modern Node.js – Evert Pot

I’m a maintainer of several dozen open source libraries. One thing I’ve
always done is maintain a hand-written changelog.

Here’s an example from a12n-server

0.22.0 (2022-09-27)
-------------------

Warning note for upgraders. This release has a database migration on the
`oauth2_tokens` table. For most users this is the largest table, some
downtime may be expected while the server runs its migrations.

* #425: Using a `client_secret` is now supported with `authorization_code`, and it's read from either the request body or HTTP Basic Authorization header.
* The service now keeps track when issuing access tokens, whether those tokens have used a `client_secret` or not, which `grant_type` was used to issue them and what scopes were requested. This work is done to better support OAuth2 scopes in the future, and eventually OpenID Connect.
* Fixed broken 'principal uri' in introspection endpoint response.
* OAuth2 service is almost entirely rewritten.
* The number of tokens issued is now displayed on the home page.
* Large numbers are now abbreviated with `K` and `M`.
* #426: Updated to Curveball 0.20.
* #427: Typescript types for the database schema are now auto-generated with `mysql-types-generator`.

These are all written in Markdown. You might think: isn’t Git also a log? Why
bother hand-writing these?

The reason is that the audience for these is a bit different. I want to bring
attention to the things that are the most important for the end-user, and
focus on the impact of the change to the user.

I thought it would be handy to write a CLI tool that makes it a bit easier to
maintain these. So, I did!. If you are curious what kind of technology
choices went into this, read on.

Goals and features

The tool should be able to do the following:

  • Reformat changelogs (a bit like prettify) (changelog format)
  • Add an entry via the command line (changelog add --minor -m "New feature").
  • Automatically set the release date (changelog release)
  • Pipe a log of a specific version to STDOUT, so it can be used by other tools
    (like integrating with github releases).

I also had a bunch of a non-functional requirements:

  • Use the latest Node features.
  • Use up to date Javascript standards and features (ESM).
  • Avoid dependendencies unless it’s unreasonable to do so.
  • Make it low maintanance.

Want to find the finished tool right now? It’s open source so just go to Github.

The implementation

ESM & Typescript

Ecmascripts modules worked really well here. It’s a small change of habits,
but the general recommendation I would have is to just save your files
as .mjs and start using it.

Here’s the first few lines of parse.mjs:

// @ts-check
import { Changelog, VersionLog } from "./changelog.mjs";
import { readFile } from 'node:fs/promises'; /** * @param {string} filename * @returns {Promise<Changelog>} */
export async function parseFile(

Truncated by Planet PHP, read more at the original (another 27805 bytes)

Xdebug Update: January 2023 – Derick Rethans

Xdebug Update: January 2023

In this monthly update I explain what happened with Xdebug development in this past month. These are normally published on the first Tuesday on or after the 5th of each month.

Patreon and GitHub supporters will get it earlier, around the first of each month.

You can become a patron or support me through GitHub Sponsors. I am currently 41% (4% less than last month) towards my $2,500 per month goal. If you are leading a team or company, then it is also possible to support Xdebug through a subscription.

In the last month, I spend 18 hours on Xdebug, with 26 hours funded. Sponsorships, especially through Patreon, are continuing to decline, which makes it harder for me to dedicate time for maintenance and development.

Xdebug 3.2

I have continued to triage new bug reports in Xdebug 3.2, and most notably trying to find the bug that people reported with regards to the xdebug.mode setting not sticking, or being wrong. So far I have not managed to reproduce this in a reliable environment. If you run into this bug, please get in contact so that I can figure out what the cause is, by being able to reproduce this.

During a support call with one of my supporters, we discussed an issue that prevented correct breakpoints from being set through the PHP Debug Adapter for Visual Studio Code for virtual file systems, such as with the SSH FS plug-in. The debug adaptor did not know how to use a path mapping with this specific schema to work. There is now a new release (1.13.0) of the plug-in (https://marketplace.visualstudio.com/items/xdebug.php-debug/changelog) to make the following work:

"pathMappings": { "/home/derick/dev": "ssh://singlemalt/home/derick/dev"
}, 

Which maps the server path /home/derick/dev/ to the virtual file path ssh:://singlemalt/home/derick/dev, which I have added to my workspace.

Mastobot: For your Fediverse PHP posting needs – Larry Garfield

Mastobot: For your Fediverse PHP posting needs

Like much of the world I’ve been working to migrate off of Twitter to Mastodon and the rest of the Fediverse. Along with a new network is the need for new automation tools, and I’ve taken this opportunity to scratch my own itch and finally build an auto-posting bot for my own needs. And it is, of course, available as Free Software.

Announcing Mastobot! Your PHP-based Mastodon auto-poster.

Continue reading this post on PeakD.

Larry
23 January 2023 – 10:13pm

Knex (with MySQL) had a very scary SQL injection – Evert Pot

Knex recently released a new version this week (2.4.0). Before this version,
Knex had a pretty scary SQL injection. Knex currently has 1.3 million weekly
downloads and is quite popular.

The security bug is probably one of the worst SQL injections I’ve seen in recent
memory, especially considering the scope and popularity.

If you want to get straight to the details:

My understanding of this bug

If I understand the vulnerability correctly, I feel this can impact a very
large number of sites using Knex. Even more so if you use Express.

I’ll try to explain through a simple example. Say, you have MySQL table structured
like this:

CREATE TABLE `users` ( `id` int NOT NULL AUTO_INCREMENT, `name` varchar(100) DEFAULT NULL, PRIMARY KEY (`id`)
)

And you have a query that does a SELECT using Knex:

const lookupId = 2; const result = await knex('users') .select(['id', 'name']) .where({ id: lookupId });

You’d expect the query to end up roughly like this

SELECT `id`, `name` FROM `users` WHERE `id` = 2

The issue is when the user controls the value of lookupId. If somehow they
can turn this into an object like this:

const lookupId = { name: 'foo'
}

You might expect an error from Knex, but instead it generates the following query:

SELECT `id`, `name` FROM `users` WHERE `id` = `name` = 'foo'

This query is not invalid. I don’t fully understand fully understand MySQL’s behavior,
but it causes the WHERE clause to be ignored and the result is equivalent to:

SELECT `id`

Truncated by Planet PHP, read more at the original (another 8765 bytes)

Xdebug Update: December 2022 – Derick Rethans

Xdebug Update: December 2022

In this monthly update I explain what happened with Xdebug development in this past month. These are normally published on the first Tuesday on or after the 5th of each month.

Patreon and GitHub supporters will get it earlier, around the first of each month.

You can become a patron or support me through GitHub Sponsors. I am currently 45% towards my $2,500 per month goal. If you are leading a team or company, then it is also possible to support Xdebug through a subscription.

In the last month, I spend 25 hours on Xdebug, with 21 hours funded. Sponsorships are continuing to decline, which makes it harder for me to dedicate time for maintenance and development.

Xdebug Videos

I have published two new videos:

I have continued writing scripts for videos about Xdebug 3.2’s features, and am also intending to make a video about “Running Xdebug in Production”, as well as one on using the updated “xdebug.client_discovery_header” feature (from Xdebug 3.1).

You can find all previous videos on my YouTube channel.

I wish JSON5 was more popular – Evert Pot

As developers we write a lot of code, but we also deal with a lot of
configuration files.

The three major formats I tend to use day to day are:

  • JSON
  • YAML
  • .env

And, they all kinda suck. JSON feels like it should
never have become a format that people hand-write. So many quotes, and
and configuration files need comments to tell users why certain decisions
were made. .env has a specific purpose (and it’s ok at that), but it’s not a
great universal format, and YAML has always been difficult to read and write to me.
I can somehow never retain the syntax and end up copy-pasting things from examples.

Why YAML is difficult for me

A small example from Github workflows/actions:

steps: - uses: actions/checkout@v2 - uses: actions/setup-node@v2 with: node-version: 14 registry-url: https://registry.npmjs.org/ - run: npm ci - run: npm publish

I couldn’t tell you why uses has a dash in front, and node-version does
not. If there’s a difference in how a YAML reader outputs them, I’m not sure
how I would be able to retain this while writing YAML.

I also use/love home assistant, which lets you write some pretty cool
automations using YAML. I wanted to play with
this but it’s been a barrier I’ve not been able to overcome. I don’t know
if it’s me. I’m been working as a programmer for 22 years. I’m decent at it,
but when when I chat with some of my peers (hi mhum!) they did not share my
sentiment.

YAML can also have very surprising behavior, with casting types:

From the linked article, this:

- country1: ca
- country2: no

Becomes:

- country1: ca
- country2: false

I’m sure there’s YAML linters out there that help avoid the pitfalls, but in
my mind configuration files should be simple.

There’s some configuration formats I like, such as TOML and JSON5.
They strike the right balance to me with being easy to read and
write, unambigious, supporting comments, strictness and not being incredibly
hard to write a parser for.

TOML is like ini files on steroids, and JSON5 is JSON but with fewer quotes,
comments and multi-line strings.

I could write my NPM configuration file as package.json5 and automatically
convert it to package.json but that feels too surprising. My projects are
already kind of eclectic, so I want the ‘plumbing’ to be unsurprising. Plus
there’s the whole chicken and egg thing with needing a JSON5 parser before we
have dependencies.

I’d love the NPM project to adopt JSON5. It seems like a great fit. JSON and
YAML can’t be the final word for human-maintained data formats. It’s so
obviously sub-optimal.

If NPM adopted JSON5, I would annotate so much in my package.json. I’d
document why a dependency is needed, why we are stuck using a previous major
version of a dependency and what the purpose is of each script.

I wouldn’t know what format would be ideal for Github Actions. Maybe the
answer is ‘nothing’ and they need a good DSL.

And while we’re at it, stop polluting my projects root directory! Can’t we
all agree on a .meta directory for finding configuration files?