PHP: Code Quality with Custom Tooling Extensions

After many years of using PHPStan, PHP-CS-Fixer, PHP_CodeSniffer, … I will give you one advice: add your own custom code to extend your Code-Quality-Tooling.

Nearly every project has custom code that procures the real value for the product / project, but this custom code itself is often not really improved by PHP-CS-Fixer, PHPStan, Psalm, and other tools. The tools do not know how this custom code is working so that we need to write some extensions for ourselves.

Example: At work, we have some Html-Form-Element (HFE) classes that used some properties from our Active Record classes, and back in the time we used strings to connect both classes. :-/

Hint: Strings are very flexible, but also awful to use programmatically in the future. I would recommend avoiding plain strings as much as possible.

1. Custom PHP-CS-Fixer

So, I wrote a quick script that will replace the strings with some metadata. The big advantage is that this custom PHP-CS-Fixer will also automatically fix code that will be created in the future, and you can apply / check this in the CI-pipline or e.g. in a pre-commit hook or directly in PhpStorm.

<?php

declare(strict_types=1);

use PhpCsFixer\Tokenizer\Analyzer\ArgumentsAnalyzer;
use PhpCsFixer\Tokenizer\Analyzer\FunctionsAnalyzer;
use PhpCsFixer\Tokenizer\Token;
use PhpCsFixer\Tokenizer\Tokens;

final class MeerxUseMetaFromActiveRowForHFECallsFixer extends AbstractMeerxFixerHelper
{

/**
* {@inheritdoc}
*/
public function getDocumentation(): string
{
return 'Use ActiveRow->m() for "HFE_"-calls, if it is possible.';
}

/**
* {@inheritdoc}
*/
public function getSampleCode(): string
{
return <<<'PHP'
<?php

$element = UserFactory::singleton()->fetchEmpty();

$foo = HFE_Date::Gen($element, 'created_date');
PHP;
}

public function isRisky(): bool
{
return true;
}

/**
* {@inheritdoc}
*/
public function isCandidate(Tokens $tokens): bool
{
return $tokens->isTokenKindFound(\T_STRING);
}

public function getPriority(): int {
// must be run after NoAliasFunctionsFixer
// must be run before MethodArgumentSpaceFixer
return -1;
}

protected function applyFix(SplFileInfo $file, Tokens $tokens): void
{
if (v_str_contains($file->getFilename(), 'HFE_')) {
return;
}

$functionsAnalyzer = new FunctionsAnalyzer();

// fix for "HFE_*::Gen()"
foreach ($tokens as $index => $token) {
$index = (int)$index;

// only for "Gen()"-calls
if (!$token->equals([\T_STRING, 'Gen'], false)) {
continue;
}

// only for "HFE_*"-classes
$object = (string)$tokens[$index - 2]->getContent();
if (!v_str_starts_with($object, 'HFE_')) {
continue;
}

if ($functionsAnalyzer->isGlobalFunctionCall($tokens, $index)) {
continue;
}

$argumentsIndices = $this->getArgumentIndices($tokens, $index);

if (\count($argumentsIndices) >= 2) {
[
$firstArgumentIndex,
$secondArgumentIndex
] = array_keys($argumentsIndices);

// If the second argument is not a string, we cannot make a swap.
if (!$tokens[$secondArgumentIndex]->isGivenKind(\T_CONSTANT_ENCAPSED_STRING)) {
continue;
}

$content = trim($tokens[$secondArgumentIndex]->getContent(), '\'"');
if (!$content) {
continue;
}

$newContent = $tokens[$firstArgumentIndex]->getContent() . '->m()->' . $content;

$tokens[$secondArgumentIndex] = new Token([\T_CONSTANT_ENCAPSED_STRING, $newContent]);
}
}
}

/**
* @param Token[]|Tokens $tokens <phpdoctor-ignore-this-line/>
* @param int $functionNameIndex
*
* @return array<int, int> In the format: startIndex => endIndex
*/
private function getArgumentIndices(Tokens $tokens, $functionNameIndex): array
{
$argumentsAnalyzer = new ArgumentsAnalyzer();

$openParenthesis = $tokens->getNextTokenOfKind($functionNameIndex, ['(']);
$closeParenthesis = $tokens->findBlockEnd(Tokens::BLOCK_TYPE_PARENTHESIS_BRACE, $openParenthesis);

// init
$indices = [];

foreach ($argumentsAnalyzer->getArguments($tokens, $openParenthesis, $closeParenthesis) as $startIndexCandidate => $endIndex) {
$indices[$tokens->getNextMeaningfulToken($startIndexCandidate - 1)] = $tokens->getPrevMeaningfulToken($endIndex + 1);
}

return $indices;
}
}

To use your custom fixes, you can register and enable them: https://cs.symfony.com/doc/custom_rules.html  

Example-Result:


$fieldGroup->addElement(HFE_Customer::Gen($element, 'customer_id'));

// <- will be replaced with ->

$fieldGroup->addElement(HFE_Customer::Gen($element, $element->m()->customer_id));

Hint: There are many examples for PHP_CodeSniffer and Fixer Rules on GitHub, you can often pick something that fits 50-70% for your use-case and then modify it for your needs.

The “m()” method looks like this and will call the simple “ActiveRowMeta”-class. This class will return the property name itself instead of the real value.

/**
* (M)ETA
*
* @return ActiveRowMeta|mixed|static
* <p>
* We fake the return "static" here because we want auto-completion for the current properties in the IDE.
* <br><br>
* But here the properties contains only the name from the property itself.
* </p>
*
* @psalm-return object{string,string}
*/
final public function m()
{
return (new ActiveRowMeta())->create($this);
}
<?php

final class ActiveRowMeta
{
/**
* @return static
*/
public function create(ActiveRow $obj): self
{
/** @var static[] $STATIC_CACHE */
static $STATIC_CACHE = [];

// DEBUG
// var_dump($STATIC_CACHE);

$cacheKey = \get_class($obj);
if (!empty($STATIC_CACHE[$cacheKey])) {
return $STATIC_CACHE[$cacheKey];
}

foreach ($obj->getObjectVars() as $propertyName => $propertyValue) {
$this->{$propertyName} = $propertyName;
}

$STATIC_CACHE[$cacheKey] = $this;

return $this;
}

}

2. Custom PHPStan Extension

In the next step, I added a DynamicMethodReturnTypeExtension for PHPStan, so that the static code analyze knows the type of the metadata + I still have auto-completion in the IDE via phpdocs.

Note: Here I’ve also made the metadata read-only, so we can’t misuse the metadata.

<?php

declare(strict_types=1);

namespace meerx\App\scripts\githooks\StandardMeerx\PHPStanHelper;

use PhpParser\Node\Expr\MethodCall;
use PHPStan\Analyser\Scope;
use PHPStan\Reflection\MethodReflection;
use PHPStan\Type\Type;

final class MeerxMetaDynamicReturnTypeExtension implements \PHPStan\Type\DynamicMethodReturnTypeExtension
{

public function getClass(): string
{
return \ActiveRow::class;
}

public function isMethodSupported(MethodReflection $methodReflection): bool
{
return $methodReflection->getName() === 'm';
}

/**
* @var \PHPStan\Reflection\ReflectionProvider
*/
private $reflectionProvider;

public function __construct(\PHPStan\Reflection\ReflectionProvider $reflectionProvider)
{
$this->reflectionProvider = $reflectionProvider;
}

public function getTypeFromMethodCall(
MethodReflection $methodReflection,
MethodCall $methodCall,
Scope $scope
): Type
{
$exprType = $scope->getType($methodCall->var);

$staticClassName = $exprType->getReferencedClasses()[0];
$classReflection = $this->reflectionProvider->getClass($staticClassName);

return new MeerxMetaType($staticClassName, null, $classReflection);
}
}
<?php

declare(strict_types=1);

namespace meerx\App\scripts\githooks\StandardMeerx\PHPStanHelper;

use PHPStan\Reflection\ClassMemberAccessAnswerer;
use PHPStan\Type\ObjectType;

final class MeerxMetaType extends ObjectType
{

public function getProperty(string $propertyName, ClassMemberAccessAnswerer $scope): \PHPStan\Reflection\PropertyReflection
{
return new MeerxMetaProperty($this->getClassReflection());
}

}
<?php

declare(strict_types=1);

namespace meerx\App\scripts\githooks\StandardMeerx\PHPStanHelper;

use PHPStan\Reflection\ClassReflection;
use PHPStan\TrinaryLogic;
use PHPStan\Type\NeverType;
use PHPStan\Type\StringType;

final class MeerxMetaProperty implements \PHPStan\Reflection\PropertyReflection
{

private ClassReflection $classReflection;

public function __construct(ClassReflection $classReflection)
{
$this->classReflection = $classReflection;
}

public function getReadableType(): \PHPStan\Type\Type
{
return new StringType();
}

public function getWritableType(): \PHPStan\Type\Type
{
return new NeverType();
}

public function isWritable(): bool
{
return false;
}

public function getDeclaringClass(): \PHPStan\Reflection\ClassReflection
{
return $this->classReflection;
}

public function isStatic(): bool
{
return false;
}

public function isPrivate(): bool
{
return false;
}

public function isPublic(): bool
{
return true;
}

public function getDocComment(): ?string
{
return null;
}

public function canChangeTypeAfterAssignment(): bool
{
return false;
}

public function isReadable(): bool
{
return true;
}

public function isDeprecated(): \PHPStan\TrinaryLogic
{
return TrinaryLogic::createFromBoolean(false);
}

public function getDeprecatedDescription(): ?string
{
return null;
}

public function isInternal(): \PHPStan\TrinaryLogic
{
return TrinaryLogic::createFromBoolean(false);
}
}

Summary

Think about your custom code and how you can improve it, use your already used tools and extend it to understand your code. Sometimes it’s easy, and you can add some modern PHPDocs or you need to go down the rabbit hole and implement some custom stuff, but at last it will help your software, your team and your customers.

Timeout Problems: Web Server + PHP

What?

First there is an HTTP request and that will hit your Web server, then it will pass the request via TCP- or UNIT-Socket via FastCGI to your PHP-FPM Daemon, here we will start a new PHP process and in this process we will connect e.g. to the database and run some queries.

PHP-Request

The Problem!

There are different timeout problems here because we connect different pieces together and this parts need to communicate. But what if one of the pieces does not respond in a given time or, even more bad, if one process is running forever like a bad SQL-query.

Understand your Timeouts.

Timeouts are a way to limit the time that a request can run, and otherwise an attacker could simply run a denial-of-service with a simple request. But there are many configurations in several layers: Web server, PHP, application, database, curl, …

– Web server

Mostly you will use Apache or Nginx as Web server and in the end it makes not really a difference, there are different timeout settings, but the idea is almost the same: The Web server will stop the execution and kills the PHP process, now you got a 504 HTTP error (Gateway Timeout) and you will lose your stack trace and error-tracking because we killed our application in the middle of nothing. So, we should keep the Web server running as long as needed.

“`grep -Ri timeout /etc/apache2/“`

/etc/apache2/conf-enabled/timeout.conf:Timeout 60

/etc/apache2/mods-available/reqtimeout.conf:<IfModule reqtimeout_module>

/etc/apache2/mods-available/reqtimeout.conf: # mod_reqtimeout limits the time waiting on the client to prevent an

/etc/apache2/mods-available/reqtimeout.conf: # configuration, but it may be necessary to tune the timeout values to

/etc/apache2/mods-available/reqtimeout.conf: # mod_reqtimeout per virtual host.

/etc/apache2/mods-available/reqtimeout.conf: # Note: Lower timeouts may make sense on non-ssl virtual hosts but can

/etc/apache2/mods-available/reqtimeout.conf: # cause problem with ssl enabled virtual hosts: This timeout includes

/etc/apache2/mods-available/reqtimeout.conf: RequestReadTimeout header=20-40,minrate=500

/etc/apache2/mods-available/reqtimeout.conf: RequestReadTimeout body=10,minrate=500

/etc/apache2/mods-available/reqtimeout.load:LoadModule reqtimeout_module /usr/lib/apache2/modules/mod_reqtimeout.so

/etc/apache2/mods-available/ssl.conf: # to use and second the expiring timeout (in seconds).

/etc/apache2/mods-available/ssl.conf: SSLSessionCacheTimeout 300

/etc/apache2/conf-available/timeout.conf:Timeout 60

/etc/apache2/apache2.conf:# Timeout: The number of seconds before receives and sends time out.

/etc/apache2/apache2.conf:Timeout 60

/etc/apache2/apache2.conf:# KeepAliveTimeout: Number of seconds to wait for the next request from the

/etc/apache2/apache2.conf:KeepAliveTimeout 5

/etc/apache2/mods-enabled/reqtimeout.conf:<IfModule reqtimeout_module>

/etc/apache2/mods-enabled/reqtimeout.conf: # mod_reqtimeout limits the time waiting on the client to prevent an

/etc/apache2/mods-enabled/reqtimeout.conf: # configuration, but it may be necessary to tune the timeout values to

/etc/apache2/mods-enabled/reqtimeout.conf: # mod_reqtimeout per virtual host.

/etc/apache2/mods-enabled/reqtimeout.conf: # Note: Lower timeouts may make sense on non-ssl virtual hosts but can

/etc/apache2/mods-enabled/reqtimeout.conf: # cause problem with ssl enabled virtual hosts: This timeout includes

/etc/apache2/mods-enabled/reqtimeout.conf: RequestReadTimeout header=20-40,minrate=500

/etc/apache2/mods-enabled/reqtimeout.conf: RequestReadTimeout body=10,minrate=500

/etc/apache2/mods-enabled/reqtimeout.load:LoadModule reqtimeout_module /usr/lib/apache2/modules/mod_reqtimeout.so

/etc/apache2/mods-enabled/ssl.conf: # to use and second the expiring timeout (in seconds).

/etc/apache2/mods-enabled/ssl.conf: SSLSessionCacheTimeout 300

Here you can see all configurations for Apache2 timeouts, but we only need to change etc/apache2/conf-enabled/timeout.conf`` because it will overwrite `/etc/apache2/apache2.conf` anyway.

PS: Remember to reload / restart your Web server after you change the configurations.

If we want to show the user at least a custom error page, we could add something like:

ErrorDocument503 /error.php?errorcode=503
ErrorDocument 504 /error.php?errorcode=504

… into our Apache configuration or in a .htaccess file, so that we can still use PHP to show an error page, also if the requested PHP call was killed. The problem here is that we will lose the error message / stack trace / request etc. from the error, and we can’t send e.g. an error into our error logging system. (take a look at sentry, it’s really helpful)

– PHP-FPM

Our PHP-FPM (FastCGI Process Manager) pool can be configured with a timeout (request-terminate-timeout), but just like the Web server setting, this will kill the PHP worker in the middle of the process, and we can’t handle the error in PHP itself. There is also a setting (process_control_timeout) that tells the child processes to wait for this much time before executing the signal received from the parent process, but I am uncertain if this is somehow helpfully here? So, our error handling in PHP can’t catch / log / show the error, and we will get a 503 HTTP error (Service Unavailable) in case of a timeout.

Shutdown functions will not be executed if the process is killed with a SIGTERM or SIGKILL signal. :-/

Source: register_shutdown_function

PS: Remember to reload / restart your PHP-FPM Daemon after you change the configurations.

– PHP

The first idea from most of us would be maybe to limit the PHP execution time itself, and we are done, but that sounds easier than it is because `max_execution_time` ignores time spent on I/O (system commands e.g. `sleep()`, database queries (SELECT SLEEP(100)). But these are the bottlenecks of nearly all PHP applications, PHP itself is fast but the external called stuff isn’t.

Theset_time_limit()function and the configuration directive max_execution_time only affect the execution time of the script itself. Any time spent on activity that happens outside the execution of the script such as system calls using system(), stream operations, database queries, etc. is not included when determining the maximum time that the script has been running. This is not true on Windows where the measured time is real.

Source: set_time_limit

– Database (MySQLi)

Many PHP applications spend most of their time waiting for some bad SQL queries, where the developer missed adding the correct indexes and because we learned that the PHP max execution time did not work for database queries, we need one more timeout setting here.

There is the MYSQLI_OPT_CONNECT_TIMEOUT and MYSQLI_OPT_READ_TIMEOUT (Command execution result timeout in seconds. Available as of PHP 7.2.0. – mysqli.options) setting, and we can use that to limit the time for our queries.

In the end you will see a “Errno: 2006 | Error: MySQL server has gone away” error in your PHP application, but this error can be caught / reported, and the SQL query can be fixed, otherwise the Apache or PHP-FPM would kill the process, and we do not see the error because our error handler can’t handle it anyway.

Summary:

It’s complicated. PHP is not designed for long execution and that is good as it is, but if you need to increase the timeout it will be more complicated than I first thought. You need for example different “timeout”-code for testing different settings:

// DEBUG: long-running sql-call
// Query(‘SELECT SLEEP(600);’);

// DEBUG: long-running system-call
// sleep(600);

// DEBUG: long-running php-call
// while (1) { } // infinite loop

Solution:

We can combine different timeout, but the timeout from the called commands e.g. database, curl, etc. will be combined with the timeout from PHP (max_execution_time) itself. The timeout from the Web server (e.g. Apache2: Timeout) and from PHP-FPM (request_terminate_timeout) need to be longer than the combined timeout from the application so that we still can use our PHP error handler.

e.g.: ~ 5 min. timeout

  1. MySQL read timeout: 240s ⇾ 4 min.
    link->options(MYSQLI_OPT_READ_TIMEOUT, 240);
  2. PHP timeout: 300s ⇾ 5 min.
    max_execution_time = 300
  3. Apache timeout: 360s ⇾ 6 min.
    Timeout 360
  4. PHP-FPM: 420s ⇾ 7 min.
    request_terminate_timeout = 420

 

Links: