PHP Internals News: Episode 97: Redacting Parameters – Derick Rethans

PHP Internals News: Episode 97: Redacting Parameters

In this episode of “PHP Internals News” I chat with Tim Düsterhus (GitHub) about the “Redacting Parameters in Back Traces” RFC.

The RSS feed for this podcast is https://derickrethans.nl/feed-phpinternalsnews.xml, you can download this episode’s MP3 file, and it’s available on Spotify and iTunes. There is a dedicated website: https://phpinternals.news

Transcript

Derick Rethans 0:00

Before we start with this episode, I want to apologize for the bad audio quality. Instead of using my nice mic I managed to use to one built into my computer. I hope you’ll still enjoy the episode.

Derick Rethans 0:30

Hi, I’m Derick. Welcome to PHP internals news, a podcast dedicated to explaining the latest developments in the PHP language. This is episode 97. Today I’m talking with Tim Düsterhus about Redacting Parameters in Backtraces RFC that he’s proposing. Tim, would you please introduce yourself?

Tim Düsterhus 0:50

Hi, Derick, thank you for inviting me. I am Tim Düsterhus, and I’m a developer at WoltLab. We are building a web application suite for you to build online communities.

Derick Rethans 0:59

Thanks for coming on this morning. What is the problem that you’re trying to solve with this RFC?

Tim Düsterhus 1:05

If everything is going well, we don’t need this RFC. But errors can and will happen and our application might encounter some exceptional situation, maybe some request to an external service fails. And so the application throws an error, this exception will bubble up a stack trace and either be caught, or go into a global exception handler. And then basically, in both cases, the exception will be logged into the error log. If it can be handled, we want to make the admin side aware of the issues so they can maybe fix their networking. If it is unable to be handled because of a programming error, we need to log it as well to fix the bug. In our case, we have the exception in the error log. And what happens next? In our case, we have many, many lay person administrators that run a community for their hobby, they’re not really programmers with no technical expertise. And we also have a strong customers help customers environment. What do those customers do? They grab their error log and post it within our forums in public. Now in our forum, we have the error log with the full stack trace, including all sensitive values, maybe user passwords, if the Authentication Service failed, or something else, that should not really happen. In our case, it’s lay person administrators. But I’m also seeing that experienced developers can make this mistake. I am triaging issues with an open source software written in C. And I’ve sometimes seeing system administrators posting their full core dump, including their TLS certificates there, and they don’t really realize what they have just done. That’s really an issue that affects laypersons, and professional administrators the same. In our case, our application attempts to strip those sensitive information from this backtrace. We have a custom exception handler that scans the full stack face, tries to match up class names and method names e.g. the PDO constructor to scrub the database password. And now recently, we have extended this stripping to also strip anything from parameters that are called password, secret, or something like that. That mostly works well. But in any case, this exception handler will miss sensitive information because it needs to basically guess what parameters are sensitive values and which don’t. And also our exception handler grew very complex because to match up those parameters, it needs to use reflection. And any failures within the exception handler cannot really be recovered from, if the exception handler fails, you’re out of luck.

Derick

Truncated by Planet PHP, read more at the original (another 17418 bytes)

Exposing webhooks via mezzio-swoole – Matthew Weier O’Phinney

I was first introduced to the concept of webhooks via a 2009 blog post by John Herren, a former colleague at Zend.
At the time, they were in their infancy; today, they’re ubiquituous, as they provide a mechanism for a service to notify interested parties of events.
This saves traffic; instead of consumers polling an API for event changes, the service notifies them directly.
It also means that the consumer does not need to setup things like cronjobs; they instead setup a webhook endpoint, register it with the service provider, and their application takes care of the rest.

The thing is, handling a webhook can often lead to additional processing, and you are expected to send an immediate response to the provider indicating you received the event.

How can you achieve this?

Offloading processing

It’s likely no secret that I’m a fan of Mezzio and OpenSwoole1.
Running PHP in a persistent process forces me to think about state in my applications, which in turn generally forces me to be more careful and explicit in how I code things.
On top of that, I get the benefit of persistent caching, better performance, and more.

One feature I pushed into mezzio-swoole (the Swoole and OpenSwoole bindings for Mezzio) was functionality for working with swoole task workers.
There’s a variety of ways to use the functionality, but my favorite is by using a PSR-14 EventDispatcher to dispatch an event to which I attach deferable listeners.

What does that look like?

Let’s say I have a GitHubWebhookEvent, for which I have associated a GitHubWebhookListener2 in my event dispatcher.
I would dispatch this event as follows:

/** @var GitHubWebhookEvent $event */
$dispatcher->dispatch($event);

The nice part about this is that the code dispatching the event does not need to know how the event is processed, or even when.
It just dispatches the event and moves on.

To make the listener deferable, in Mezzio applications, I can associate a special delegator factory provided by the mezzio-swoole package with the listener.
This is done with standard Mezzio dependency configuration:

use Mezzio\Swoole\Task\DeferredServiceListenerDelegator; return [ 'dependencies' => [ 'delegators' => [ GitHubWebhookListener::class => [ DeferredServiceListenerDelegator::class, ], ], ],
];

This approach means that my listener can have any number of dependencies, and be wired into the container, but when I request it, I’ll be returned a Mezzio\Swoole\Task\DeferredServiceListener instead.
This class will create a swoole task from the listener and event, which defers execution to the task workers, offloading it from the web workers.

Event state

Task workers receive a copy of the event, not the original instance.
Any state changes your listener makes in the event instance will not be reflected in the instance present in your web workers.
As such, you should only defer listeners that do not communicate state back to the dispatching code via the event.

Sharing an event dispatcher with the web server

mezzio-swoole defines a marker interface, Mezzio\Swoole\Event\EventDispatcherInterface.
This interface is used to define an event-dispatcher service consumed by Mezzio\Swoole\SwooleRequestHandlerRunner for the purpose of dispatching swoole HTTP server events, getting around the “one event, one handler” rule swoole follows.
However, that can mean that you end up with two different dispatchers in your application: one used by the swoole web server, and one by the application, and that means you cannot delegate tasks.

To get around this, alias the Mezzio\Swoole\Event\EventDispatcherInterface service to the Psr\EventDispatcher\EventDispatcherInterface service:

use Mezzio\Swoole\Event\EventDispatcherInterface as SwooleEventDispatcher;
use Psr\EventDispatcher\EventDispatcherInterface as PsrEventDispatcher; return [ 'dependencies' => [ 'alias' => [ SwooleEventDispatcher::class => PsrEve

Truncated by Planet PHP, read more at the original (another 6872 bytes)

Running cronjobs via an Openswoole timer – Matthew Weier O’Phinney

Sites I build often utilize cronjobs to periodically pull in data from other sources.
For example, I might want to poll an API once a day, or scrape content from another website once a month.
Cronjobs are a perfect fit for this.

However, cron has a few problems:

  • If the job is writing information into the file tree of your web application, you need to ensure permissions are correct, both at the filesystem level, and when writing the cronjob (e.g., running it as the same user, or changing permissions on completion).
  • If you are running console tooling associated with your PHP application, you may need to worry about whether or not particular environment variables are in scope when you run the job.
  • In containerized environments, usage of cron is strongly discouraged, as it means running another daemon.
    You can get around this with tools such as the s6-overlay, but it’s another vector for issues.

Since most sites I build anymore use mezzio-swoole, I started wondering if I might be able to handle these jobs another way.

Task workers

We introduced integration with Swoole’s task workers in version 2 of mezzio-swoole.
Task workers run as a separate pool from web workers, and allow web workers to offload heavy processing when the results are not needed for the current request.
They act as a form of per-server message queue, and are great for doing things such as sending emails, processing webhook payloads, and more.

The integration in mezzio-swoole allows you to decorate PSR-14 EventDispatcher listeners in mezzio-swoole Mezzio\Swoole\Task\DeferredListener or DeferredServiceListener instances; when that happens, the decorator creates a task with the Swoole server, giving it the actual listener and the event.
When the schedule process the task, it then calls the listener with the event.

The upshot is that to create a task, you just dispatch an event from your code.
Your code is thus agnostic about the fact that it’s being handled asynchronously.

However, because tasks work in a separate pool, this means that the event instances they receive are technically copies and not references; as such, your application code cannot expect the listener to communicate event state back to you.
If you choose to use this feature, only use it for fire-and-forget events.

I bring all this up now because I’m going to circle back to it in a bit.

Scheduling jobs

Swoole’s answer to scheduling jobs is its timer.
With a timer, you can tick: invoke functionality each time a period has elapsed.
Timers operate within event loops, which means every server type that Swoole exposes has a tick() method, including the HTTP server.

The obvious answer, then, is to register a tick:

// Intervals are measured in milliseconds.
// The following means "every 3 hours".
$server->tick(1000 * 60 * 60 * 3, $callback);

Now I hit the problems:

  • How do I get access to the server instance?
  • What can I specify as a callback, and how do I get it?

With mezzio-swoole, the time to register this is when the HTTP server starts.
Since Swoole only allows one listener per event, mezzio-swoole composes a PSR-14 event dispatcher, and registers with each Swoole HTTP server event.
The listeners then trigger events via the PSR-14 event dispatcher, using custom event types internally that provide access to the data originally passed to the Swoole server events.
This approach allows the application developer to attach listeners to events and modify how the application works.

To allow these “workflow” events to be separate from the application if desired, we register a Mezzio\Swoole\Event\EventDispatcherInterface service that returns a discrete PSR-14 event dispatcher implementation.
I generally alias this to the PSR-14 interface, so I can use the same instance for application events.

I use my own phly/phly-event-dispatcher implementation, which provides a number of different listener providers.
The easiest one is Phly\EventDispatcher\AttachableListenerProvider, which defines a single listen() method for attaching a listener to a given event class.

On top of that, Mezzio and Laminas have a concept of delegator factories.
These allow you to “decorate” the creation of a service.
One us

Truncated by Planet PHP, read more at the original (another 14608 bytes)

Xdebug Update: December 2021 – Derick Rethans

Xdebug Update: December 2021

In this monthly update I explain what happened with Xdebug development in this past month. These will be published on the first Tuesday after the 5th of each month.

Patreon and GitHub supporters will get it earlier, around the first of each month.

You can become a patron or support me through GitHub Sponsors. I am currently 46% towards my $2,500 per month goal. If you are leading a team or company, then it is also possible to support Xdebug through a subscription.

In December, I worked on Xdebug directly for only about 26 hours, with funding being around 21 hours. Please become a supporter of Xdebug through Patreon or GitHub.

Xdebug 3.1 and further

On the first of the month, I released Xdebug 3.1.2.

It addresses a few crash bugs related to PHP 8.1 fibers, a crash bug when Xdebug can’t write a profiler file, and an issue with Xdebug’s var_dump() not using the magic __debugInfo method.

The full list of changes can be found on the updates page on the Xdebug website.

Since Xdebug 3.1.2 I have fixed a few more bugs, which are not yet in a released version. One fix pertains to the debugger generating not-well-formed XML, and another one improves performance with long strings in the debugger.

I spend most of my time in December to investigate issues that have not yet been solved. One of them turned to be a change in PHP behaviour between PHP 8.0 and 8.1. In PHP 8.0 and earlier, the $_GLOBALS[] superglobal also had a key GLOBALS, which PHP 8.1 no longer has. PhpStorm was reading values for its watch feature, from the GLOBALS context, but also added the extra (unnecessary) GLOBALS array element to read out the real variables. Fixing this in Xdebug is complex, so hopefully this will be addressed in the next version of PhpStorm itself.

The second issue turned out to be an issue with PHP-FPM, which is not strictly following PHP’s processing model. This can cause a discrepancy between PHP-FPM’s control and worker processes, where they do not agree what the value of the xdebug.mode INI setting is. Ideally this should get fixed in PHP-FPM, but there are further issues that both PHP-FPM and Xdebug probably need changes for.

Xdebug Videos

I did not create any new Xdebug videos this month on my YouTube channel. But I am working on a more thought out set of instructional videos. Stay Tuned!

If you would like to suggest a topic for a 5 to 15 minute long video, feel free to request them through this Google Form.

Truncated by Planet PHP, read more at the original (another 582 bytes)

Hello 2022! – Evert Pot

Yesterday I received an email from a reader asking ‘Are you ok?’.

It’s been nearly 8 months since the last time I wrote here. In that
last post I celebrated blogging on this website for 15 years with
some consistency, so perhaps it’s a bit ironic for that to be immediately
followed by complete silence.

The last big gap in blogging for me was in 2017, the year I joined Yelp.
This experience was so depressing, every day I was done work I had no
creative energy left for anything else.

2021 was a bit different though. 2 years back I started a software
development agency
, which grew from 2 to 5 people in the last year.
The stakes have increased quite a bit, and it’s taken up a lot of my
emotional reserves.

I’ve also made the mistake of not taking any vacation all year. There was
just not much gas left in the tank. This is so stupid. Less time off doesn’t
result in more productivity. I know this, but the last 2 years there’s been
little travel or activities due to lockdowns and restrictions. Every day
looks the same and it kind of just flew by.

Over the last holidays I’ve taken an actual break though, and have since
started several new projects and buzzing with new ideas. I’m still motivated
to work on Curveball and Ketting (we use it every day for almost
every customer!), and I’ve also started a series of live streams in which
I build a Time Tracking application with Hypermedia on twitch.tv/evrt3.

If this sounds interesting, the first few episodes are up on
my youtube channel, but I’ll share more on this blog later.

I’m also preparing for a tech talk on January 19th for Toronto JS. It’s
online and free!

So am I ok? I think I am? This year is off to a good start. I just have to
make sure I don’t forget to take it easy.

Happy stupid new year! I hope it sucks less!

Advent of Functional PHP: Review – Larry Garfield

Advent of Functional PHP: Review

Over the last few weeks, I’ve been following Advent of Code 2021, using Functional PHP as an approach. It’s been a fun and educational process, at least for me and apparently for a few other people, at least given how popular the articles have been.

For reference, the full list of articles in this series is here:

Larry
29 December 2021 – 7:10pm

Advent of Functional PHP: Day 10 – Larry Garfield

Advent of Functional PHP: Day 10

For the 10th Day of Advent of Code, we’re asked to solve a matching braces problem. This is a common parser exercise, but it’s made a bit more complex in this case by using multiple types of braces. Specifically, we’re handling a series of lines that contain ( and ), but also < and >, [ and ], and { and }.

The story jazzes it up as being the code of our submarine's navigational computer, which consists entirely of braces in a sort of eldritch horror version of brainfuck, but that's mostly just a distraction.

Larry
24 December 2021 - 7:14pm

Advent of Functional PHP: Day 9 – Larry Garfield

Advent of Functional PHP: Day 9

Submitted by Larry on 23 December 2021 – 11:21am

Day 9 of this year’s Advent of Code revolves around grid interpretation. Specifically, we are given a grid of numbers and want to find the low points, that is, the numbers that are smaller than any of their orthogonal neighbors. (We’re told to ignore diagonals in part 1.)

After finding the low points, we need to do a bit of math on each one, and add them up. As usual, this last step is mostly just to produce a single verification number at the end. That part is easy as usual, but how do we find the low points?

Continue reading this post on PeakD.

PHP 8.1 lays new ground on Platform.sh – platform.sh

Just days after its official release, we are thrilled to announce the immediate availability of PHP 8.1 for all Grid plans projects. The new PHP foundation releases a new main version every year at the very end of November, and It’s some kind of early Christmas for us developers and application makers.
Sparks and acceleration PHP 8.1 comes with many new and long-expected features such as Enums, Readonly properties, First-class callable syntax, and new in initializers.