Commit your code as if it could be accidentally deployed – Matthias Noback

The one simple trick to do a better job as a programmer is to git commit as if your commit could be accidentally deployed (and it wouldn’t break the production environment…)

Why would this improve your work? Because it pushes for improvements in several areas.

If a commit needs to leave the project in a working state, you need to:

  1. Be able to prove that after making your changes, the project is in a working state. This forces you to put automated tests in place.
  2. Be able to make changes that don’t break existing code, or that only add new features. This forces you think about first improving the existing code design to allow for your change to happen.

To practice with the latter, I recommend learning about the Mikado method. It makes it easier to recognize prerequisites for the change you want to make, and then forces you to enable the real goals by implementing the prerequisites first.

This process is also known as “making the change easy, then making the easy change”. An amazing result of applying this process is that after some practice you’ll be able to make many, much smaller commits during the day. Each of those commits will leave the project in a working state. To me this always feels great because:

  1. I’m not worried that my change breaks something.
  2. I only make small jumps between stepping stones and I’m safe on each stone.
  3. I can switch tasks when I need to.

Consultancy secrets

As a consultant I apply this “trick” to trigger improvement of the development process. I like to work with development teams on one of their real programming tasks, as an ensemble. We use a pomodoro timer, and we establish a goal for the “pomodoro” (25 minutes). Reflecting at the end of the pomodoro we often conclude two things:

  • We didn’t reach the goal, it was too ambitious
  • We won’t be able to commit anything until several pomodoro’s later

During all this time we don’t feel safe at all. We don’t have a sense of accomplishment either. Realizing this turns out to be a great starting point for improving the development process.

Request bodies in GET requests – Evert Pot

12 years ago I asked on Stack Overflow: Are HTTP GET requests allowed to
have request bodies?
. This got a 2626 upvotes and a whopping 1.6 million
views, so clearly it’s something lots of people are still curious about, and
in some cases disagree with the accepted answer.

Stack overflow screenshot

Because it keeps popping up in my Stack Overflow notifications (and
I compulsively visit the site), the question has lived in my head
rent-free. I keep adding context in my head, and I’ve been meaning to write
some of this down for a few years now and hopefully evict it.

Anyway, if you’re just looking for a quick answer, it’s ‘No, you shouldn’t do
this.’, you should probably use QUERY.

Undefined behavior

A number of people (most famously ElasticSearch) have gotten this wrong,
but why? I think it’s because of this sentence in the HTTP Spec:

A payload within a GET request message has no defined semantics

That sentence could easily suggest that there’s no specific behavior associated
to request bodies with GET requests, and that the behavior is left up to the
implementor.

The reality is that this is more like Undefined behavior from languages
like C/C++. My understanding is that leaving certain aspects of the C language
undefined (instead of for example requiring an error to be thrown) leaves room for
compiler implementations to make certain optimizations. Some compilers also
have fun with this; GCC hilariously starts a video game in a specific case
of undefined behavior which really brings home this point.

If you were to write a C program that relies on how a compiler dealt with
specific undefined behavior, it means your program is no longer a portable
C program, but it’s written in variant of C that only works on some compilers.

The same applies for HTTP as well. It’s true that undefined behavior means
that you as a server developer can define it, but you are not an island!

When working with HTTP, there’s servers but also load balancers, proxies,
browsers and other clients that all need to work together. The behavior isn’t
just undefined server-side, a load balancer might choose to silently drop
bodies or throw errors. There’s many real-world examples of this. fetch()
for example will throw an error.

This hasn’t stopped people from doing this anyway. OpenAPI removed
support for describing GET request bodies in version 3.0 (and DELETE,
which has the same issue!), but was quitely added back in 3.1 to not
prevent people from documenting their arguably broken APIs.

Why it’s not defined

The best source I have is this quote from Roy Fielding in 2007. Roy Fielding
coined REST is and is one of the main authors of the HTTP/1.1 RFCs.

Yes. In other words, any HTTP request message is allowed to contain a message body, and thus must parse messages with that in mind. Server semantics for GET, however, are restricted such that a body, if any, has no semantic meaning to the request. The requirements on parsing are separate from the requirements on method semantics. So, yes, you can send a body with GET, and no, it is never useful to do so.
This is part of the layered design of HTTP/1.1 that will become clear again once the spec is partitioned (work in progress).

….Roy

(His message was originally sent to the now-dead rest-discuss
group on Yahoo Groups, but I found an archive in JSON format
)

However, I always found this answer unsatisfying. I understand that you might
want a low-level protocol design that just passes messages containing headers,
bodies,

Truncated by Planet PHP, read more at the original (another 9288 bytes)

PHP Internals News: Episode 97: Redacting Parameters – Derick Rethans

PHP Internals News: Episode 97: Redacting Parameters

In this episode of “PHP Internals News” I chat with Tim Düsterhus (GitHub) about the “Redacting Parameters in Back Traces” RFC.

The RSS feed for this podcast is https://derickrethans.nl/feed-phpinternalsnews.xml, you can download this episode’s MP3 file, and it’s available on Spotify and iTunes. There is a dedicated website: https://phpinternals.news

Transcript

Derick Rethans 0:00

Before we start with this episode, I want to apologize for the bad audio quality. Instead of using my nice mic I managed to use to one built into my computer. I hope you’ll still enjoy the episode.

Derick Rethans 0:30

Hi, I’m Derick. Welcome to PHP internals news, a podcast dedicated to explaining the latest developments in the PHP language. This is episode 97. Today I’m talking with Tim Düsterhus about Redacting Parameters in Backtraces RFC that he’s proposing. Tim, would you please introduce yourself?

Tim Düsterhus 0:50

Hi, Derick, thank you for inviting me. I am Tim Düsterhus, and I’m a developer at WoltLab. We are building a web application suite for you to build online communities.

Derick Rethans 0:59

Thanks for coming on this morning. What is the problem that you’re trying to solve with this RFC?

Tim Düsterhus 1:05

If everything is going well, we don’t need this RFC. But errors can and will happen and our application might encounter some exceptional situation, maybe some request to an external service fails. And so the application throws an error, this exception will bubble up a stack trace and either be caught, or go into a global exception handler. And then basically, in both cases, the exception will be logged into the error log. If it can be handled, we want to make the admin side aware of the issues so they can maybe fix their networking. If it is unable to be handled because of a programming error, we need to log it as well to fix the bug. In our case, we have the exception in the error log. And what happens next? In our case, we have many, many lay person administrators that run a community for their hobby, they’re not really programmers with no technical expertise. And we also have a strong customers help customers environment. What do those customers do? They grab their error log and post it within our forums in public. Now in our forum, we have the error log with the full stack trace, including all sensitive values, maybe user passwords, if the Authentication Service failed, or something else, that should not really happen. In our case, it’s lay person administrators. But I’m also seeing that experienced developers can make this mistake. I am triaging issues with an open source software written in C. And I’ve sometimes seeing system administrators posting their full core dump, including their TLS certificates there, and they don’t really realize what they have just done. That’s really an issue that affects laypersons, and professional administrators the same. In our case, our application attempts to strip those sensitive information from this backtrace. We have a custom exception handler that scans the full stack face, tries to match up class names and method names e.g. the PDO constructor to scrub the database password. And now recently, we have extended this stripping to also strip anything from parameters that are called password, secret, or something like that. That mostly works well. But in any case, this exception handler will miss sensitive information because it needs to basically guess what parameters are sensitive values and which don’t. And also our exception handler grew very complex because to match up those parameters, it needs to use reflection. And any failures within the exception handler cannot really be recovered from, if the exception handler fails, you’re out of luck.

Derick

Truncated by Planet PHP, read more at the original (another 17418 bytes)

Exposing webhooks via mezzio-swoole – Matthew Weier O’Phinney

I was first introduced to the concept of webhooks via a 2009 blog post by John Herren, a former colleague at Zend.
At the time, they were in their infancy; today, they’re ubiquituous, as they provide a mechanism for a service to notify interested parties of events.
This saves traffic; instead of consumers polling an API for event changes, the service notifies them directly.
It also means that the consumer does not need to setup things like cronjobs; they instead setup a webhook endpoint, register it with the service provider, and their application takes care of the rest.

The thing is, handling a webhook can often lead to additional processing, and you are expected to send an immediate response to the provider indicating you received the event.

How can you achieve this?

Offloading processing

It’s likely no secret that I’m a fan of Mezzio and OpenSwoole1.
Running PHP in a persistent process forces me to think about state in my applications, which in turn generally forces me to be more careful and explicit in how I code things.
On top of that, I get the benefit of persistent caching, better performance, and more.

One feature I pushed into mezzio-swoole (the Swoole and OpenSwoole bindings for Mezzio) was functionality for working with swoole task workers.
There’s a variety of ways to use the functionality, but my favorite is by using a PSR-14 EventDispatcher to dispatch an event to which I attach deferable listeners.

What does that look like?

Let’s say I have a GitHubWebhookEvent, for which I have associated a GitHubWebhookListener2 in my event dispatcher.
I would dispatch this event as follows:

/** @var GitHubWebhookEvent $event */
$dispatcher->dispatch($event);

The nice part about this is that the code dispatching the event does not need to know how the event is processed, or even when.
It just dispatches the event and moves on.

To make the listener deferable, in Mezzio applications, I can associate a special delegator factory provided by the mezzio-swoole package with the listener.
This is done with standard Mezzio dependency configuration:

use Mezzio\Swoole\Task\DeferredServiceListenerDelegator; return [ 'dependencies' => [ 'delegators' => [ GitHubWebhookListener::class => [ DeferredServiceListenerDelegator::class, ], ], ],
];

This approach means that my listener can have any number of dependencies, and be wired into the container, but when I request it, I’ll be returned a Mezzio\Swoole\Task\DeferredServiceListener instead.
This class will create a swoole task from the listener and event, which defers execution to the task workers, offloading it from the web workers.

Event state

Task workers receive a copy of the event, not the original instance.
Any state changes your listener makes in the event instance will not be reflected in the instance present in your web workers.
As such, you should only defer listeners that do not communicate state back to the dispatching code via the event.

Sharing an event dispatcher with the web server

mezzio-swoole defines a marker interface, Mezzio\Swoole\Event\EventDispatcherInterface.
This interface is used to define an event-dispatcher service consumed by Mezzio\Swoole\SwooleRequestHandlerRunner for the purpose of dispatching swoole HTTP server events, getting around the “one event, one handler” rule swoole follows.
However, that can mean that you end up with two different dispatchers in your application: one used by the swoole web server, and one by the application, and that means you cannot delegate tasks.

To get around this, alias the Mezzio\Swoole\Event\EventDispatcherInterface service to the Psr\EventDispatcher\EventDispatcherInterface service:

use Mezzio\Swoole\Event\EventDispatcherInterface as SwooleEventDispatcher;
use Psr\EventDispatcher\EventDispatcherInterface as PsrEventDispatcher; return [ 'dependencies' => [ 'alias' => [ SwooleEventDispatcher::class => PsrEve

Truncated by Planet PHP, read more at the original (another 6872 bytes)

Interview with Eric Van Johnson & John Congdon – Voices of the ElePHPant

Join host Cal Evans as he talks to the new owners of PHP Architect magazine about The magazine, conferences, PHP, and much more.

This episode is sponsored by
RingCentral Developers

The post Interview with Eric Van Johnson & John Congdon appeared first on Voices of the ElePHPant.

Running cronjobs via an Openswoole timer – Matthew Weier O’Phinney

Sites I build often utilize cronjobs to periodically pull in data from other sources.
For example, I might want to poll an API once a day, or scrape content from another website once a month.
Cronjobs are a perfect fit for this.

However, cron has a few problems:

  • If the job is writing information into the file tree of your web application, you need to ensure permissions are correct, both at the filesystem level, and when writing the cronjob (e.g., running it as the same user, or changing permissions on completion).
  • If you are running console tooling associated with your PHP application, you may need to worry about whether or not particular environment variables are in scope when you run the job.
  • In containerized environments, usage of cron is strongly discouraged, as it means running another daemon.
    You can get around this with tools such as the s6-overlay, but it’s another vector for issues.

Since most sites I build anymore use mezzio-swoole, I started wondering if I might be able to handle these jobs another way.

Task workers

We introduced integration with Swoole’s task workers in version 2 of mezzio-swoole.
Task workers run as a separate pool from web workers, and allow web workers to offload heavy processing when the results are not needed for the current request.
They act as a form of per-server message queue, and are great for doing things such as sending emails, processing webhook payloads, and more.

The integration in mezzio-swoole allows you to decorate PSR-14 EventDispatcher listeners in mezzio-swoole Mezzio\Swoole\Task\DeferredListener or DeferredServiceListener instances; when that happens, the decorator creates a task with the Swoole server, giving it the actual listener and the event.
When the schedule process the task, it then calls the listener with the event.

The upshot is that to create a task, you just dispatch an event from your code.
Your code is thus agnostic about the fact that it’s being handled asynchronously.

However, because tasks work in a separate pool, this means that the event instances they receive are technically copies and not references; as such, your application code cannot expect the listener to communicate event state back to you.
If you choose to use this feature, only use it for fire-and-forget events.

I bring all this up now because I’m going to circle back to it in a bit.

Scheduling jobs

Swoole’s answer to scheduling jobs is its timer.
With a timer, you can tick: invoke functionality each time a period has elapsed.
Timers operate within event loops, which means every server type that Swoole exposes has a tick() method, including the HTTP server.

The obvious answer, then, is to register a tick:

// Intervals are measured in milliseconds.
// The following means "every 3 hours".
$server->tick(1000 * 60 * 60 * 3, $callback);

Now I hit the problems:

  • How do I get access to the server instance?
  • What can I specify as a callback, and how do I get it?

With mezzio-swoole, the time to register this is when the HTTP server starts.
Since Swoole only allows one listener per event, mezzio-swoole composes a PSR-14 event dispatcher, and registers with each Swoole HTTP server event.
The listeners then trigger events via the PSR-14 event dispatcher, using custom event types internally that provide access to the data originally passed to the Swoole server events.
This approach allows the application developer to attach listeners to events and modify how the application works.

To allow these “workflow” events to be separate from the application if desired, we register a Mezzio\Swoole\Event\EventDispatcherInterface service that returns a discrete PSR-14 event dispatcher implementation.
I generally alias this to the PSR-14 interface, so I can use the same instance for application events.

I use my own phly/phly-event-dispatcher implementation, which provides a number of different listener providers.
The easiest one is Phly\EventDispatcher\AttachableListenerProvider, which defines a single listen() method for attaching a listener to a given event class.

On top of that, Mezzio and Laminas have a concept of delegator factories.
These allow you to “decorate” the creation of a service.
One us

Truncated by Planet PHP, read more at the original (another 14608 bytes)

Xdebug Update: December 2021 – Derick Rethans

Xdebug Update: December 2021

In this monthly update I explain what happened with Xdebug development in this past month. These will be published on the first Tuesday after the 5th of each month.

Patreon and GitHub supporters will get it earlier, around the first of each month.

You can become a patron or support me through GitHub Sponsors. I am currently 46% towards my $2,500 per month goal. If you are leading a team or company, then it is also possible to support Xdebug through a subscription.

In December, I worked on Xdebug directly for only about 26 hours, with funding being around 21 hours. Please become a supporter of Xdebug through Patreon or GitHub.

Xdebug 3.1 and further

On the first of the month, I released Xdebug 3.1.2.

It addresses a few crash bugs related to PHP 8.1 fibers, a crash bug when Xdebug can’t write a profiler file, and an issue with Xdebug’s var_dump() not using the magic __debugInfo method.

The full list of changes can be found on the updates page on the Xdebug website.

Since Xdebug 3.1.2 I have fixed a few more bugs, which are not yet in a released version. One fix pertains to the debugger generating not-well-formed XML, and another one improves performance with long strings in the debugger.

I spend most of my time in December to investigate issues that have not yet been solved. One of them turned to be a change in PHP behaviour between PHP 8.0 and 8.1. In PHP 8.0 and earlier, the $_GLOBALS[] superglobal also had a key GLOBALS, which PHP 8.1 no longer has. PhpStorm was reading values for its watch feature, from the GLOBALS context, but also added the extra (unnecessary) GLOBALS array element to read out the real variables. Fixing this in Xdebug is complex, so hopefully this will be addressed in the next version of PhpStorm itself.

The second issue turned out to be an issue with PHP-FPM, which is not strictly following PHP’s processing model. This can cause a discrepancy between PHP-FPM’s control and worker processes, where they do not agree what the value of the xdebug.mode INI setting is. Ideally this should get fixed in PHP-FPM, but there are further issues that both PHP-FPM and Xdebug probably need changes for.

Xdebug Videos

I did not create any new Xdebug videos this month on my YouTube channel. But I am working on a more thought out set of instructional videos. Stay Tuned!

If you would like to suggest a topic for a 5 to 15 minute long video, feel free to request them through this Google Form.

Truncated by Planet PHP, read more at the original (another 582 bytes)

Hello 2022! – Evert Pot

Yesterday I received an email from a reader asking ‘Are you ok?’.

It’s been nearly 8 months since the last time I wrote here. In that
last post I celebrated blogging on this website for 15 years with
some consistency, so perhaps it’s a bit ironic for that to be immediately
followed by complete silence.

The last big gap in blogging for me was in 2017, the year I joined Yelp.
This experience was so depressing, every day I was done work I had no
creative energy left for anything else.

2021 was a bit different though. 2 years back I started a software
development agency
, which grew from 2 to 5 people in the last year.
The stakes have increased quite a bit, and it’s taken up a lot of my
emotional reserves.

I’ve also made the mistake of not taking any vacation all year. There was
just not much gas left in the tank. This is so stupid. Less time off doesn’t
result in more productivity. I know this, but the last 2 years there’s been
little travel or activities due to lockdowns and restrictions. Every day
looks the same and it kind of just flew by.

Over the last holidays I’ve taken an actual break though, and have since
started several new projects and buzzing with new ideas. I’m still motivated
to work on Curveball and Ketting (we use it every day for almost
every customer!), and I’ve also started a series of live streams in which
I build a Time Tracking application with Hypermedia on twitch.tv/evrt3.

If this sounds interesting, the first few episodes are up on
my youtube channel, but I’ll share more on this blog later.

I’m also preparing for a tech talk on January 19th for Toronto JS. It’s
online and free!

So am I ok? I think I am? This year is off to a good start. I just have to
make sure I don’t forget to take it easy.

Happy stupid new year! I hope it sucks less!