Why Agile Won’t Fix Your Project

We’ve all heard the promises of Agile: faster delivery, more flexibility, and a better team dynamic. But let’s face it—Agile won’t save your project if the foundation isn’t right. Frameworks are tools, not magic wands, and success depends on one key factor: your people. Without commitment, purpose, and vision, even the best methodology falls flat.


Commitment Is the Foundation

In every successful project I’ve seen, there’s one constant: the team cares. They’re passionate about the project, the technology, or the customer. When people are engaged, they bring their A-game. Without that commitment, Agile rituals become hollow motions—stand-ups where no one stands up for quality and retrospectives that only scratch the surface.

But how do you ignite that spark?

  • Give Purpose: Teams perform better when they know why their work matters. Whether it’s a product that changes lives or a system that transforms workflows, connect the dots between their efforts and the bigger picture.
  • Empower Ownership: Nothing kills commitment faster than micromanagement. Give your team the autonomy to solve problems their way, and they’ll take pride in their work.
  • Set the North Star: Someone needs to have the big picture in mind. This isn’t about controlling every detail; it’s about providing clarity and direction. A team without a vision is just a group of people checking tasks off a backlog.

Agile: A Framework, Not a Fix

Agile isn’t inherently bad—it’s just not a silver bullet. A poorly implemented Agile process can feel like busy work, creating friction instead of flow. You’ve seen it: endless sprints with no tangible value, misaligned priorities, and teams that are Agile in name only.

The truth is, Agile is just a tool, and like any tool, its effectiveness depends on how you use it. Without the right culture, mindset, and leadership, Agile can’t thrive.


The Power of Flow

When your team is committed and has a clear purpose, they enter the elusive “flow” state. Flow is where the magic happens: work feels both challenging and enjoyable, and everything just clicks. Achieving flow requires more than process—it demands an environment that fosters focus, removes blockers, and aligns everyone toward shared goals.


Common Pitfalls in Agile Projects

  • Lack of Vision: If the goals are unclear, the team loses direction. Agile iterations can’t compensate for a missing strategy.
  • Token Empowerment: Teams are told they’re empowered, but their decisions are constantly overridden by management.
  • Process Over People: Agile becomes a checkbox exercise, focusing on rituals instead of outcomes.
  • Burnout Culture: Passion doesn’t mean working endless hours. A healthy pace is critical for long-term success.

Building the Right Environment

Here’s what makes the difference:

  1. Clarity and Direction: Define a clear vision and align the team’s work with it. Avoid vague objectives—make the goals tangible and achievable.
  2. Purposeful Leadership: Leaders need to support, not control. Their role is to remove obstacles, align efforts, and amplify strengths.
  3. Empowerment: Trust your team to make decisions. Give them the tools, autonomy, and confidence to innovate.
  4. Continuous Reflection: Agile emphasizes improvement, but that requires honesty. Create a safe space for open feedback and genuine introspection.

Final Thoughts: Commitment Over Process

At the end of the day, success isn’t about following Agile to the letter. It’s about creating an environment where your team thrives. Agility is just one piece of the puzzle. Your team’s commitment, passion, and purpose are what truly drive results.

So don’t just “go Agile.” Build a culture that values people over processes, outcomes over rituals, and vision over micromanagement. Agile might not save your project, but your team will.

🚀 Because in the end, your people are your greatest asset.

LLM Prompt Optimizations: Practical Techniques for Developers

Optimizing inputs for LLMs ensures better, more consistent outputs while leveraging the full potential of the model’s underlying capabilities. By understanding core concepts like tokenization, embeddings, self-attention, and context limits, you can tailor inputs to achieve desired outcomes reliably. Below, you’ll find fundamental techniques and best practices organized into practical strategies.

PS: You can use this content to automatically improve your prompt by asking as follows: https://chatgpt.com/share/6785a41d-72a0-8002-a1fe-52c14a5fb1e5


🎯 1. Controlling Probabilities: Guide Model Outputs

🧠 Theory: LLMs always follow probabilities when generating text. For every token, the model calculates a probability distribution based on the context provided. By carefully structuring inputs or presenting examples, we can shift the probabilities toward the desired outcome:

  • Providing more examples helps the model identify patterns and generate similar outputs.
  • Clear instructions reduce ambiguity, increasing the probability of generating focused responses.
  • Contextual clues and specific phrasing subtly guide the model to prioritize certain outputs.

⚙️ Technology: The model operates using token probabilities:

  • Each token (word or part of a word) is assigned a likelihood based on the input context.
  • By influencing the input, we can make certain tokens more likely to appear in the output.

For example:

  • A general query like “Explain energy sources” might distribute probabilities evenly across different energy types.
  • A more specific query like “Explain why solar energy is sustainable” shifts the probabilities toward solar-related tokens.

⚙️ Shifting Probabilities in Prompts: The structure and wording of your prompt significantly influence the token probabilities:

  • For specific outputs: Use targeted phrasing to increase the likelihood of desired responses: Explain why renewable energy reduces greenhouse gas emissions.
  • For diverse outputs: Frame open-ended questions to distribute probabilities across a broader range of topics: What are the different ways to generate clean energy?
  • Few-Shot Learning: Guide the model using few-shot learning to set patterns: Example 1: Input: Solar energy converts sunlight into electricity. Output: Solar energy is a renewable power source. Example 2: Input: Wind energy generates power using turbines. Output: Wind energy is clean and sustainable. Task: Input: Hydropower generates electricity from flowing water. Output:

💡 Prompt Tips:

  • Use clear, direct instructions for precise outputs: Write a PHP function that adds two integers and returns a structured response as an array.
  • Use contextual clues to steer the response: Explain why PHP is particularly suited for web development.

💻 Code Tips: LLMs break down code and comments into tokens, so structuring your PHPDocs helps focus probabilities effectively. Provide clarity and guidance through structured documentation:

/**
 * Adds two integers and returns a structured response.
 *
 * @param int $a The first number.
 * @param int $b The second number.
 * 
 * @return array{result: int, message: string} A structured response with the sum and a message.
 */
function addIntegers(int $a, int $b): array {
    $sum = $a + $b;

    return [
        'result' => $sum,
        'message' => "The sum of $a and $b is $sum."
    ];
}
  • Include examples in PHPDocs to further refine the probabilities of correct completions: /** * Example: * Input: addIntegers(3, 5) * Output: [‘result’ => 8, ‘message’ => ‘The sum of 3 and 5 is 8’] */

✂️ 2. Tokenization and Embeddings: Use Context Efficiently

🧠 Theory: LLMs break down words into tokens (numbers) to relate them to each other in multidimensional embeddings (vectors). The more meaningful context you provide, the better the model can interpret relationships and generate accurate outputs:

  • Tokens like “renewable energy” and “sustainability” have semantic proximity in the embedding space.
  • More context allows the model to generate richer and more coherent responses.

⚙️ Technology:

  • Tokens are the smallest units the model processes. For example, “solar” and “energy” may be separate tokens, or in compound languages like German, one long word might be broken into multiple tokens.
  • Embeddings map these tokens into vectors, enabling the model to identify their relationships in high-dimensional space.

⚙️ Optimizing Tokenization in Prompts: To make the most of tokenization and embeddings:

  • Minimize irrelevant tokens: Focus on core concepts and avoid verbose instructions.
  • Include context-rich phrases: Relevant terms improve the embedding connections.
  • Simplify Language: Use concise phrasing to minimize token count: Solar energy is renewable and reduces emissions.
  • Remove Redundancy: Eliminate repeated or unnecessary words: Explain why solar energy is sustainable.

💡 Prompt Tips:

  • Include only essential terms for better embedding proximity: Describe how solar panels generate electricity using photovoltaic cells.
  • Avoid vague or verbose phrasing: Explain solar energy and its uses in a way that a normal person can understand and provide details.
  • Use specific language to avoid diluting the context: Explain why solar energy is considered environmentally friendly and cost-effective.
  • Avoid vague instructions that lack actionable context: Explain me solar energy.

💻 Code Tips: Write compact and clear PHPDocs to save tokens and improve context:

/**
 * Converts raw user input into a structured format.
 *
 * @param string $input Raw input data.
 * 
 * @return array{key: int, value: string} Structured output.
 */
function parseInput(string $input): array {
    $parts = explode(":", $input);

    return [
        'key' => (int)$parts[0],
        'value' => trim($parts[1])
    ];
}
  • Use compact and descriptive documentation to maximize token efficiency: /** * Example: * Input: “42:Hello” * Output: [‘foo’ => 42, ‘bar’ => ‘Hello’] */

🧭 3. Self-Attention and Structure: Prioritize Context

🧠 Theory: LLMs work with the principle of self-attention, where the input tokens are interrelated with each other to determine the relevance and context. This mechanism assigns importance scores to tokens, ensuring that the most relevant words and their relationships are prioritized.

⚙️ Technology:

  • Self-attention layers: Compare each token with every other token in the input to generate an attention score.
  • Multi-head attention: Allows the model to consider multiple perspectives simultaneously, balancing relevance and context.
  • Pitfall: Too many irrelevant tokens dilute the attention scores, leading to distorted outputs.

⚙️ Optimizing Structure in Prompts:

  • Structure Your Inputs: Use lists, steps, or sections to emphasize relationships: Compare the benefits of solar and wind energy: 1. Environmental impact 2. Cost-efficiency 3. Scalability
  • Minimize Irrelevant Tokens: Keep prompts focused and free from extraneous details.

💡 Prompt Tips:

  • Well-Structured: Organize tasks into sections: Explain the environmental and economic benefits of renewable energy in two sections: 1. Environmental 2. Economic
  • Unstructured: Avoid asking everything at once: What are the environmental and economic benefits of renewable energy?

💻 Code Tips: In PHPDocs, organize information logically to enhance clarity and guide models effectively:

/**
 * Calculates the cost efficiency of renewable energy.
 *
 * Steps:
 * 1. Evaluate savings-to-investment ratio.
 * 2. Return a percentage efficiency score.
 *
 * @param float $investment Initial investment cost.
 * @param float $savings Annual savings.
 * 
 * @return float Efficiency percentage.
 */
function calculateEfficiency(float $investment, float $savings): float {
    return ($savings / $investment) * 100;
}

🧹 4. Context Management and Token Limits

🧠 Theory: LLMs operate within a fixed token limit (e.g., ~8k tokens for GPT-4), encompassing both input and output. Efficiently managing context ensures relevant information is prioritized while avoiding irrelevant or redundant content.

⚙️ Technology:

  • Chunking: Break long inputs into smaller, manageable parts: Step 1: Summarize the introduction of the report. Step 2: Extract key arguments from Section 1. Step 3: Combine summaries for a final overview.
  • Iterative Summarization: Condense sections before integrating them: Summarize Section 1: Solar energy’s benefits. Summarize Section 2: Wind energy’s benefits. Combine both summaries.
  • Pitfall: Excessive context can truncate critical data due to token limits.

💡 Prompt Tips:

  • For large inputs, use step-by-step processing: Step 1: Summarize the introduction of the document. Step 2: Extract key arguments from Section 1. Step 3: Combine these points into a cohesive summary.
  • Avoid presenting the full text in a single prompt: Summarize this 20-page document.
  • Focus on specific sections or tasks: Summarize the introduction and key points from Section 1.

💻 Code Tips: Divide tasks into smaller functions to handle token limits better:

function summarizeSection(string $section): string {
    // Summarize section content.
}

function combineSummaries(array $summaries): string {
    // Merge individual summaries.
}

🎨 5. Reasoning and Goals: Strengthen Prompt Direction

🧠 Theory: LLMs generate better results when the reasoning behind a task and its intended goal are explicitly stated. This guides the model’s probabilities toward meaningful and relevant outcomes.

⚙️ Technology:

  • Explicit reasoning provides semantic depth, helping the model focus on the task’s purpose.
  • Explaining the goal improves alignment with user expectations and narrows token probabilities.

💡 Prompt Tips:

  • State the reason for the task and its goal: Explain renewable energy because I need to create an introductory guide for high school students.
  • Avoid generic prompts without a clear goal: Describe renewable energy.

💻 Code Tips: Use PHPDocs to explain both the reasoning and expected outcomes of a function:

/**
 * Generates a detailed user profile report.
 *
 * This function is designed to create a comprehensive profile report based on user data inputs. 
 * It is useful for analytical dashboards requiring well-structured user insights.
 *
 * @param array{name: string, age: int, email: string} $userData The user data array.
 * 
 * @return string A formatted profile report.
 */
function generateProfileReport(array $userData): string {
    return sprintf(
        "User Profile:\nName: %s\nAge: %d\nEmail: %s\n",
        $userData['name'],
        $userData['age'],
        $userData['email']
    );
}

🛠️ 6. Iterative Refinement: Simplify Complex Tasks

🧠 Theory:
Breaking down complex tasks into smaller, manageable steps improves accuracy and ensures the model generates focused and coherent outputs. This method allows you to iteratively refine results, combining outputs from smaller subtasks into a complete solution.

⚙️ Technology:

  • Chunking: Split large tasks into multiple smaller ones to avoid overwhelming the model.
  • Validation: Intermediate outputs can be validated before moving to the next step, minimizing errors.
  • Recombination: Smaller validated outputs are merged for the final result.

💡 Prompt Tips:

  • For multi-step tasks, provide clear, incremental instructions: Step 1: Summarize the environmental benefits of solar energy. Step 2: Describe the cost savings associated with solar energy. Step 3: Combine these summaries into a single paragraph.
  • Avoid handling complex tasks in a single step: Explain the environmental benefits and cost savings of solar energy in one response.

💻 Code Tips: Ask the LLM to create the code step by step and ask for confirmation after each step so that the LLM can focus on one aspect of the implementation at a time. Focus on one aspect of the implementation at a time.


🔗 7. Cross-Contextual Coherence: Maintain Consistency

🧠 Theory:
LLMs lack persistent memory between interactions, making it essential to reintroduce necessary context for consistent responses across prompts. By maintaining cross-contextual coherence, outputs remain aligned and relevant, even in multi-step interactions.

⚙️ Technology:

  • Use context bridging: Reference key elements from previous responses to maintain relevance.
  • Store critical details in persistent structures, such as arrays or JSON, to reintroduce when needed.
  • Avoid overloading with irrelevant details, which can dilute coherence.

💡 Prompt Tips:

  • Reintroduce essential context from previous interactions: Based on our discussion about renewable energy, specifically solar power, explain the benefits of wind energy.
  • Summarize intermediate outputs for clarity: Summarize the main benefits of renewable energy. Then expand on solar and wind energy.

💻 Code Tips: Use seperated files for Code-Examples that we can provide e.g. Custom GPTs so it can learn from learnings/findings this way.


🌍 8. Style and Tone: Adapt Outputs to the Audience

🧠 Theory: LLMs generate better responses when the desired style and tone are explicitly stated. By matching the tone to the audience, you can make content more engaging and effective.

⚙️ Technology:

  • The model uses semantic cues in the prompt to adjust style and tone.
  • Specific words and phrases like “formal,” “casual,” or “technical” help steer the model’s output.

💡 Prompt Tips:

  • Specify the tone and audience: Write a technical explanation of solar panels for an engineering audience.
  • Adjust the style for different contexts: Explain solar panels in a simple and friendly tone for kids.

💻 Code Tips: In PHPDocs, define the intended audience and tone to guide LLM-generated documentation:

/**
 * Calculates the total energy output of a solar panel system.
 *
 * Intended Audience: Engineers and technical experts.
 * Tone: Formal and technical.
 *
 * @param float $panelArea The total area of solar panels in square meters.
 * @param float $efficiency The efficiency rate of the solar panels (0-1).
 * @param float $sunlightHours Daily sunlight hours.
 * 
 * @return float Total energy output in kilowatt-hours.
 */
function calculateSolarOutput(float $panelArea, float $efficiency, float $sunlightHours): float {
    return $panelArea * $efficiency * $sunlightHours;
}

🔍 9. Fine-Tuning and Domain Expertise

🧠 Theory: Fine-tuning allows LLMs to specialize in specific domains by further training them on domain-specific datasets. This enhances their ability to generate accurate, relevant, and nuanced outputs tailored to specialized tasks or fields.

⚙️ Technology:

  • Fine-tuning adjusts the weights of a pre-trained model by using a curated dataset that focuses on a specific domain.
  • This process requires labeled data and computational resources but significantly improves task performance in niche areas.

💡 Prompt Tips:

  • Use fine-tuning to simplify prompts for repeated tasks: Generate a legal brief summarizing the key points from this case.
  • Without fine-tuning, include detailed instructions and examples in your prompt: Write a summary of this legal case focusing on liability and negligence, using a formal tone.

💻 Code Tips: When fine-tuning is not an option, structure your PHPDocs to include domain-specific context for LLMs:

/**
 * Generates a compliance report for renewable energy projects.
 *
 * This function creates a detailed compliance report tailored for regulatory agencies. It checks for adherence to
 * energy efficiency standards and sustainability guidelines.
 *
 * @param array<string, mixed> $projectData Details of the renewable energy project.
 * @param string $region The region for which the compliance report is generated.
 * 
 * @return string The compliance report in a formatted string.
 */
function generateComplianceReport(array $projectData, string $region): string {
    // Example report generation logic.
    return sprintf(
        "Compliance Report for %s:\nProject: %s\nStatus: %s\n",
        $region,
        $projectData['name'] ?? 'Unnamed Project',
        $projectData['status'] ?? 'Pending Review'
    );
}

The Wild West of Coding: Why We’re Still Burning Digital Cities


The year 2050 is closer than the year 1990, yet we’re still writing code like it’s the 1800s.

It’s 2025, and while we’ve made incredible strides in software development—automated memory management, static analysis tools, refactoring IDEs, and AI copilots like ChatGPT—it still feels like the Wild West. Sure, the tools are better, but the way we approach software remains chaotic, inefficient, and overly reliant on custom solutions.

Every day, thousands of developers solve the same problems repeatedly. Companies roll out their own authentication systems, file upload handlers, and error trackers. Many of these are flawed. Vulnerabilities creep in because we’re not building resilient systems—we’re building digital bonfires.

This isn’t progress. This is the time before the First Industrial Revolution of Software Development.


Lessons from History: What the Past Teaches About Our Digital Fires

1. The Great Fire of Hamburg (1842): Building Without Safety

In 1842, a quarter of Hamburg burned to the ground. The city’s lack of fire safety standards—wooden buildings, narrow streets, no prevention systems—made disaster inevitable.

Software today mirrors this recklessness:

  • File upload systems without malware checks.
  • APIs with vulnerabilities because “it’s faster to skip security.”
  • Custom-built logging systems without consistency or reliability.

After the fire, Hamburg rebuilt with fireproof materials and strict regulations. We need the same shift in software development: adopting universal standards, secure-by-design frameworks, and centralized tools to prevent disaster before it happens.


2. Electrical Sockets in the Early 1900s: Chaos Without Standards

Before standardization, electrical sockets were a mess. Every region had its own plug type, making interoperability nearly impossible. Plugging in a device often meant wasting time finding the right adapter.

Software development today is no different:

  • APIs lack consistent patterns.
  • Libraries solve the same problem in incompatible ways.
  • Developers reinvent logging, error handling, and authentication with every project.

The solution? Standardized, language-agnostic tools—the equivalent of universal plug designs. Imagine APIs and services that integrate seamlessly across languages and frameworks:

  • Centralized logging and error tracking APIs, similar to Sentry but designed for internal use with cross-language compatibility.
  • High-performance Unix socket APIs for tasks like logging, monitoring, and file scanning.
  • Shared SDKs for foundational needs like security or metrics collection.

By building shared infrastructure, we could eliminate redundant work and improve reliability across projects.


3. The Facade Problem: Buying Software for Looks, Not Stability

Imagine buying a house because the facade looks amazing but never checking the foundation. That’s how most software is evaluated today:

  • Buyers focus on flashy UIs and marketing demos.
  • Security, scalability, and maintainability are often ignored.

This approach leads to brittle, insecure systems. What’s missing? Inspectors for digital bridges—specialized roles that assess software foundations, enforce accountability, and ensure systems are solid beneath their shiny exteriors.


Moving Toward the First Industrial Revolution of Software Development

The Industrial Revolution replaced handcrafting with standardization and mass production. Software development is still stuck in its pre-industrial phase:

  • Custom solutions are built repeatedly for the same problems.
  • Accountability is rare—no one ensures the “bridge” (software) is stable before it’s deployed.

To mature as an industry, we need:

1. Universal Blueprints

Developers today still create bespoke solutions for common problems. We need standardized tools and APIs, such as:

  • A unified SDK for antivirus scanning, accessible via /usr/sbin/antivirus or unix:///var/run/antivirus.
  • Centralized APIs for error tracking, metrics, and monitoring, with cross-language support.

2. Specialized Roles

In the 19th century, collapsing bridges led to the creation of specialized roles: architects for design, builders for execution, and inspectors for safety. Software teams need similar specialization:

  • System Inspectors to evaluate software for security, scalability, and maintainability.
  • Digital Firefighters to enforce standards and proactively address vulnerabilities.

3. Accountability

When a bridge collapses, someone is held responsible. In software, failures are often patched silently or ignored. We need:

  • Transparency: Bugs and vulnerabilities must be documented openly.
  • Retrospectives: Focused on systemic improvements, not just quick fixes.

Building Digital Cities That Don’t Burn

Imagine a future where software is built like modern cities:

  • Fireproof Systems: Universal standards for security, maintainability, and testing.
  • Digital Firefighters: Publicly funded teams safeguarding critical infrastructure.
  • Inspectors for Digital Bridges: Specialized roles ensuring software is built to last.

AI tools like GPT can help accelerate this process, but they are not the solution. AI is like the steam engine of programming—amplifying productivity but requiring skilled operators. If we don’t lay the right foundations, AI will only magnify our inefficiencies.

This future isn’t about writing more code—it’s about creating resilient, scalable systems that stand the test of time. The tools to build fireproof digital cities are already here. The question is: are we ready to use them?


Let’s move beyond the Wild West of coding and into the Industrial Revolution our industry desperately needs. It’s time to stop building bonfires and start building something that lasts.

Legacy Codebase: A Love Story

After some years, working with a > 10 years old legacy PHP codebase, I can truly say: you can escape the legacy codebase and introduce whatever is helpful, in a well-maintained system.

Here are 5 important steps that I have done:

  • Custom error handling: Reporting notices for developers, report bad “assert” calls in the dev container, report bad indexes, reporting wrong code usage, …
  • Autocompletion for everything: classes, properties, SQL queries, CSS, HTML, JavaScript in PHP (e.g. via /* @lang JavaScript */ in PhpStorm), …
  • Static-Code Analysis: Preventing bugs is even better than fixing bugs, so just stop stupid bugs and use types in your code.
  • Automate the refactoring: With tools like PHP-CS-Fixer or Rector you can not only fix your code one time, you can fix any future wrong usage of the code.
  • Do not use strings for code: Just use constants, classes, properties, … use something that can be processes by your static-code analysis and something where you will have autocompletion.

Here are 5 additional steps that I already introduce:

  • Sentry: External error collecting (aggregating) tool + custom handler to see e.g. IDs of every Active Record object.
  • Generics: via PHPDocs + autocompletion via PhpStorm
  • No “mixed” types: Now we use something like, e.g. “array<int, string>” instead of “array”.
  • PSR standards: e.g. PSR-15 request handler, PSR-11 container, PSR-3 logger, …
  • Code Style: One code style to rule them all, we use PHP-CS-Fixer and PHP-Code-Sniffer to check / fix our code style for all ~ 10,000 PHP classes.

Here is what helped me mostly while working with old existing code.

First rule, first: 🥇 think or / and ask someone in the team

Analyzing: Here are some things that helped my analyzing software problems in our codebase.

  • Errors: Better error handling / reporting with a custom error handler, with all relevant information.
  • Understandable logging: Hint, you can just use syslog for medium-sized applications.
  • Grouping errors: Displaying and grouping all the stuff (PHP / JS / errors + our own custom reports) into Sentry (https://sentry.io/), now you can easily see how e.g. how many customers are effected from an error.
  • git history: Often new bugs were introduced with the latest changes (at least in often used components), so that good commit messages are really helpful to find this changes. (https://github.com/voku/dotfiles/wiki/git-commit-messages)
  • Local containers: If you can just download the application with a database dump from yesterday, you can analyze many problems without touching any external server.
  • Linux tools: mytop, strace, htop, iotop, lsof, …
  • Database tools:  EXPLAIN [SQL], IDE integration / autocompletion, …

Fixing: Here are some tricks for fixing existing code more easily.

  • IDE: PhpStorm with auto-completion and suggestions (including results from static analysis)
  • auto-code-style formatter: (as pre-commit hook) is also helpful because I do not need to think about this anymore while fixing code 
  • git stuff: sometimes it can also be helpful to understand git and how to revert or cherry-pick some changes

Preventing: Here are some hints how you can prevent some bugs.

  • custom static analysis rules: http://suckup.de/2022/07/php-code-quality-with-custom-tooling-extensions/
  • root cause: fixing the root cause of a problem, sometimes this is very hard because you need to fully understand the problem first, bust mostly spending this time is a good investment
  • testing: writing a test is always a good idea, at least to prevent the same problem

Job: If you now would like to work with this codebase (PHP 8 | MySQL 8 | Ubuntu), please contact me and take a look at this job offer: https://meerx.de/karriere/softwareentwickler-softwareentwicklerin/

What have I learned so far in my job?

I will start a new job next month (02-2023), so time to recap, I’m going to describe what I’ve learned so far.

me: Lars Moelleken |
> Assistant for business IT
> IT specialist for system integration
> IT specialist for application development

What did I learn as IT specialist for system integration?

– You only learn as much as you want to learn.

In contrast to school / technical college, I could and had to teach and work on many things myself during the training. And you quickly realize that you only learn as much as you want. Once you’ve understood this, you’re happy to sit down and learn whatever you want. For example, local or online courses, go to meetups or conferences. Worry about your skill because if you do something, you should do it right.

“An investment in knowledge still pays the best interest.” – Benjamin Franklin

– No panic!

What you learn pretty quickly as a sysadmin is “keep calm” and think first – then act. Hasty actionism does not help and usually even damages. Before you act, you should first obtain some information yourself (information from log files, hardware status, system status, …) so that you really know how to fix the error.

– Unix & command line <3

If you haven’t worked with a Unix operating system before, you unfortunately don’t know what you’re missing out on. If you want or have to use Windows for whatever reason, you can nowadays still use some of the advantages of Linux via WSL (Windows Subsystem for Linux). Leave your own comfort zone and trying out new operating systems helps to understand your computer better overall. At this point, I would have recommended “Arch Linux” as sysadmin, but today I would choose something that needs less maintaining.

One should also become familiar with the command line if you want to increase your productivity rapidly. For example, you should take a closer look at the following commands: e.g. find / grep / lsof / strace

– Read the official documentation.

It is often better to read the official documentation of the thing (hardware || software) you are currently using. While you start programming or learn a new programming language / framework, we often use stackoverflow.com and quickly finds answers and code examples, but the “why” and “how” is usually neglected. If you look at the specification / documentation first, you not only solve this problem, you also understand the problem, and maybe you will learn how to solve similar problems.

– Always make a backup (never use “sudo” drunken).

Some things you have to learn the hard way, apparently installing “safe-rm” was part of it for me!

apt-get install safe-rm

“Man has three ways of acting wisely. First, on meditation; that is the noblest. Secondly, on imitation; that is the easiest. Thirdly, on experience; that is the bitterest.” – (Confucius)

– Be honest with customers, employees and yourself.

Be honest with customers, especially when things go wrong. If the customer’s product (e.g. server) fails, then offer solutions and no excuses and solve the problem, not the question of blame. No one is helped by pointing the finger at colleagues or customers, not at the server, not at the customer and ultimately not at yourself.

– Ask questions if you don’t understand something.

Don’t talk to customers about something you don’t understand, not knowing something (especially in training) is fine, but then ask a colleague before you talk to a customer about it!

– Think about what you are doing (not only at work).

If you question things and think about your work and what you do, then you can develop personally. Question critical, for example, whether you should really order from Amazon, or should you rather order the book directly from the publisher? What is the advantage for the author and do I have any disadvantages? Should we use Nagios or rather Icinga directly? Question your work and critically evaluate whether this is really a good / safe / future-oriented solution.

If you are not sure yourself or your perspective is too limited (because you only know this one solution, for example), then you should acquire new knowledge, research other solutions or “best practices” and discuss the topic with others.

– Use Google correctly …

1. When searching an issue, look for the error message in quotes: “Lars Moelleken”

2. You can limit the result to special URLs: inurl:moelleken.org

3. Sometimes it’s helpful to find only specific files: filetype:txt inurl:suckup.de

> There are even more tricks that can help you in your daily work, just Google for it:

Full example: intitle:index.of mp3 “Männer Sind Schweine” -html -htm -php

What did I learn as IT specialist for application development?

– You never stop learning…

When you start to deal with web programming (HTML, CSS, JS, PHP, …), you don’t even know where to start. There is so much to learn, and this feeling accompanies you for some time (years) until you recognize recurring concepts. However, the advantage in web programming is that many different things have common APIs or at least can be combined. I can write a class in PHP that creates data attributes for my HTML, which I can read out with JavaScript to design them accordingly using CSS classes. But it stays the same, you never stop learning, and that’s also what’s so exciting about this job.

– Try to write code every day. (but set yourself a LIMIT)

Even if the boss isn’t in the office today, and you don’t have any more tasks, then write some code, if you’re sitting on the couch at home (and no new series is running on Netflix), then write or read some code and if you’re on vacation, you’re on vacation!

Here is an interesting link from “John Resig” (jQuery):
http://ejohn.org/blog/write-code-every-day/

– Think in modules and packages…

If you write software every day, you don’t want to code the same functionality (e.g. database connection, send e-mail or error logging …) multiple times for different projects (and remember, mostly you don’t want to code standard functions yourself). Therefore, every programmer should think in modules and design an application in different, preferably interchangeable parts. Often, the system can then also be expanded better, since there is already an interface for modules that can be used. The independence and decoupling of program parts also has the advantage that side effects from different places in the source code are further avoided. In addition, one should minimize the coupling of the corresponding modules, otherwise one gains nothing from modules.

There are already package managers for almost everything in web development. e.g.:
– Frontend (css, js): npm
– Backend (php): composer

– Open-Source-Software

If you are already working with modules and packages, you can publish them as an OSS project + run tests via GitHub actions + code coverage + at least a small amount of documentation. For these reasons alone, publishing as open source is worthwhile. The code quality also increases (in my experience), since the source code is released to the public and therefore more or less conscious attention is paid to the code quality.

The moment when you get your first pull request for your software or code together with someone from Israel, Turkey and the USA is priceless.

At some point, you would like to have the same advantages of OSS in your daily work because often, there is no package (code is directly added into the existing code), no tests (even not for bugs) and no documentation. So, possibly, you have now collected enough arguments to convince your boss to publish some package from your code at work.

– Git & Good

I don’t know how people can work with the PC without version control. I even use “git” for smaller private projects or for configuration files. The following are some advantages, but the list can certainly be extended with a few more points:

– changes can be traced (git log, git blame)
– changes can be undone (git revert, git reset)
– changes can be reviewed by other employees
– employees can work on a project at the same time (git commit, git pull, git push)
– development branches (forks) can be developed simultaneously (git checkout -b , git branches)

– Use GitHub and learn from the best.

GitHub itself is not open-source, but there has been an unofficial agreement to use the platform for open-source projects. You can therefore find many good developers and projects there, and you can learn a lot just by reading the source code / changes. Especially because you can understand and follow the development of the projects: How do others structure your code? How do others write their “commit” messages? How much code should a method contain? How much code should a class contain? Which variable names are better to avoid? How to use specific libraries / tools? How do others test their code? …

– try { tests() }

Especially when you write tests for your own code, you catch yourself testing exactly the cases that you have already considered, so you should test the corresponding functionality with different (not yet considered) input. Here are some inputs for testing: https://github.com/minimaxir/big-list-of-naughty-strings

Hint: We should add another test whenever an error occurred, so that already fixed error does not come back to use.

– Automate your tests.

Unit tests, integration tests and front-end tests only help if they are also executed, so you should deal with automated tests at an early stage and also run them automatically when the source code changes. Where and when these tests are run also determines how effective these tests ultimately are. As soon as you have written a few tests, you will understand why it is better not to use additional parameters for methods and functions, since the number of tests increases exponentially.

– Deployment is also important.

As soon as you work with more than one developer on a project, or the project will become more complex, you want to use some kind of deployment. As in application development, often the simplest solution is also here a good starting point: e.g. just pull the given changes into a directory and change the symlink from the existing source directory so that you can switch or rollback all files easily. PS: And you properly never need to write database-migration rollbacks, I never used them.

– Understanding concepts is more important than implementing them.

Understanding design patterns (programming concepts) not only helps in the current programming language, but can mostly be applied to other programming languages ​​as well.

Basic concepts (classes, objects, OOP, functions, ORM, MVC, DDD, unit tests, data binding, router, hooks, template engine, …) can be found in many frameworks / programming languages ​​and once you have understood the terms and concepts, it is no longer that difficult to use new / different frameworks. And you can see different strengths and weaknesses of these frameworks and tools: “If you only have a hammer as a tool, you see a nail in every problem.”

– Solving problems also means understanding customers.

Design patterns are part of the basic equipment, but you should always ask yourself: Which problem is actually supposed to be solved with the given solution? If necessary, you can find an even more elegant / simpler solution. And sometimes the customer actually wants something thoroughly different, he just doesn’t know it yet or someone has misunderstood the customer.

– Solving problems also means understanding processes.

But it is just as important to understand why a certain feature is implemented, and otherwise you are programming something that is either not needed or used at all. One should therefore understand the corresponding task before implementation and even before planning in the overall context.

project-swing-tree

– Spread code across multiple files.

Use one file for a class, use one file for CSS properties of a module or a specific page, use a new file for each new view. Dividing the source code into different files / directories offers many advantages, so the next developer knows where new source code should be stored and you can find your way around the source code faster. Many frameworks already provide a predefined directory structure.

– Readability comes first!

The readability of source code should always come first, since you or your work colleagues will have to maintain or expand this code in the future.

YouTube’s videos about “Clean Code”: https://www.youtube.com/results?search_query=%22Clean+Code%22&search_sort=video_view_count

Best Practices: http://code.tutsplus.com/tutorials/top-15-best-practices-for-writing-super-readable-code-net-8118

– Good naming is one of the most difficult tasks in programming.

It starts with the domain name / project name, goes through file names, to directory names, class names, method names, variable names, CSS class names. Always realize that others will read this and need to understand it. Therefore, you should also avoid unnecessary abbreviations and write what you want to describe.

We want to describe what the function does and not how it is implemented.

⇾ Incorrect: sendMailViaSwiftmailer(), sendHttpcallViaCurl(), …
⇾ Correct: mail->send(), http->send(), …

Variables should describe what they contain and not how they are stored.

⇾ Incorrect: $array2use, $personsArray, …
⇾ Correct: $pages, $persons, …

Summary: Describe what the variable/method/function/class is, not how it is implemented: https://github.com/kettanaito/naming-cheatsheet

Programming less bad PHP

– Save on comments (at least inline)…

Good comments explain “why” and not “what” the code is doing, and should offer the reader added value that is not already described in the source code.

Sometimes it makes sense to add some “what” comments anyway, e.g. for complicated regex or some other not optimal code that needs some hints.

Examples of inline comments:

bad code:

// Check if the user is already logged in if ( isset ( $_SESSION['user_loggedin']) && $_SESSION['user_loggedin'] > 1 ) { ... }
       

slightly better code:

// check if the user is already logged-in 
if ( session ( 'user_loggedin' ) > 1 ) { ... }
      

better code:

if ( $user->isLoggedin === true ) { ... }     

… and another example …

bad code:

// regex: email 
if (! preg_match ( '/^(.*<?)(.*)@(.*)(>?)$/' , $email ) { ... }
    

better code:

define ( 'EMAIL_REGEX_SIMPLE' , '/^(.*<?)(.*)@(.*)(>?)$/' ); 

if (! preg_match ( EMAIL_REGEX_SIMPLE , $email ) { ... }    

– Consistency in a project is more important than personal preferences!

Use the existing code and use given functions. If it brings benefits, then change / refactor the corresponding code, but then refactor all places in the project which are implemented in this way.

Example: If you have formerly worked without a template system and would like to use one for “reasons”, then use this for all templates in the project and not just for your current use case; otherwise, inconsistencies will arise in the project. For example, if you create a new “Email→isValid()” method, then you should also replace all previous RegEx attempts in the current project with the “Email” class; otherwise inconsistencies will arise again.

Read more about the topic:

– “Be consistent [and try to automate this process, please]!” http://suckup.de/2020/01/do-not-fear-the-white-space-in-your-code/

– “Why do we write unreadable code?”
http://suckup.de/2020/01/do-not-fear-the-white-space-in-your-code/

– A uniform code style has a positive effect on quality!

As in real life, if there is already rubbish somewhere, the inhibition threshold to dump rubbish there drops extremely. But if everything looks nice and tidy, then nobody just throws a “randumInt() { return 4; }” function on the floor.

It also helps to automate some refactoring because the code looks everywhere the same, you can also apply the same e.g. PHP-CS-Fixer and you do not need to think about every different coding style.

– Use functional principles & object-oriented concepts.

A pure function (“Pure Functions”) only depends on its parameters and with the same parameters it always returns the same result. These principles can also be considered in OOP and create so-called immutable classes (immutable class).

https://en.wikipedia.org/wiki/Pure_function
https://de.wikipedia.org/wiki/Object-oriented_programming
https://en.wikipedia.org/wiki/Immutable_object

– Please do not use global variables!

Global variables make testing difficult because they can cause side effects. Also, it’s difficult to refactor code with global variables because you don’t know what effects these changes will have on other parts of the system.

In some programming languages ​​(e.g. JavaScript, Shell) all variables are global and only become local with a certain keyword (e.g. in the scope of a function or a class).

– Learn to use your tools properly!

For example, if you’re writing code with Notepad, you can dig a hole with a spoon, which is just as efficient. Learn keyboard shortcuts for your programs and operating system! Use an IDE, e.g. from JetBrains (https://www.jetbrains.com/products.html) and use additional plugins and settings.

Modern IDEs also give hints/suggestions on how to improve your code. For example, for PHP, you can use PhpStorm + PHPStan and share the global IDE Inspections settings in the team.

– Performance?

In nearly every situation you don’t have to worry too much about performance, as modern programming languages ​​/ frameworks support us and with common solutions; otherwise the motto is “profiling, profiling… profiling”!

– Exceptions === Exceptions

You should not use exceptions to handle normal errors. Exceptions are exceptions, and regular code handles the regular cases! “Use exceptions only in exceptional circumstances” (Pragmatic Programmers). And nearly under no circumstances you should “choke off” exceptions, e.g. by trivially catching several exceptions.

– Finish your work

You should finish what you started. For example, if you need to use “fopen()” you should also use “fclose()” in the same code block. So, nobody in the team needs to clean up your stuff after he / she uses your function.

– Source code should be searchable [Ctrl + F] …

The source code should be easy to search through, so you should avoid using string nesting + “&” with Sass, for example, and also avoid using PHP functions such as “extract()”. Whenever variables are not declared, but created as if by magic (e.g. using magic methods in PHP), it is no longer so easy to change the source text afterward.

Example in PHP: (bad)

extract ( array ( 'bar' => 'bar' , 'lall' => 1 )); 
var_dump ( $bar ); // string 'bar' (length=3)      

Example in Sass: (bad)

. teaser { 
  font - size : 12px ; 

  & __link { 
    color : rgb ( 210 , 210 , 22 ); } }
  

Sass Nesting (code style): https://github.com/grvcoelho/css#nesting

– Program for your use case!

A big problem in programming is that you have to try to think and program in a generalized way so that you can (easily) expand the source code if new requirements are added or you can (easily) change it.

What does project sometimes look like? → A customer orders 10,000 green apples from a farm, changes his order to 10,000 red apples the morning before delivery and when these are delivered, the customer would prefer 10,000 pears and would like to pay for them in 6 months.

And precisely for this reason you should only write the source code that is really required for the current use case because you can’t map all eventualities anyway and the source code is unnecessarily complicated.

– KISS – Keep it simple, stupid.

One should always keep in mind that the source code itself is not that valuable. The value only arises when other developers understand it and can adapt / configure / use it for the customer or themselves. This should be kept in mind during programming so that a solution can be implemented as comprehensibly and “simply” as possible. And if I don’t need to use a new class or nice design pattern for the current problem, I probably shouldn’t. However, this does not mean that you should throw all good intentions overboard and use global variables / singletons everywhere. However, if a simple solution already does the job, go for that one.

A good example of what not to do is the JavaScript DOM Selector API. Not exactly nice to read or write…

Bad: (DOM Selector via JS)

document.getElementsByTagName ( "div" ) 
document.getElementById ( "foo" ) 
document.getElementsByClassName ( "bar" ) 
document.querySelector ( ".foo" ) 
document.querySelectorAll ( "div.bar" )

Better: (DOM Selector via jQuery)

$( "div" ) 
$( "#foo" ) 
$( ".bar" ) 
$( ".foo" ) 
$( "div.bar" )

– DRY – Don’t Reap Yourself.

Repetitions / redundancies in the source text or in recurring work arise relatively quickly if people do not communicate with each other. But also unintentionally due to errors in the software design because you don’t have a better idea or don’t think you have time for it.

improve

To avoid repetition, make your solution easy to find and easy to use. So that other devs will use it instead of re-creating a solution.

– The will to learn and understand something new is more important than previous knowledge.

If you can already program ActionScript (Flash), for example, but are not willing to learn something new, then previous knowledge is of no use because “The only constant in the universe is change.” – Heraclitus of Ephesus (about 540 – 480 BC).

– Read good books and magazines.

Books I have read: https://www.goodreads.com/user/show/3949219-lars-moelleken
Free Books: https://github.com/vhf/free-programming-books/blob/master/free- programming-books.md
books for programmers: http://stackoverflow.com/questions/1711/what-is-the-single-most-influential-book-every-programmer-should-read

– Follow other programmers on Twitter / GitHub / dev.to / YouTube / Medium / …

It sometimes helps to motivate yourself, to write e.g. a blog post or testing some new stuff, if you know some people how has the same interest, so just follow some of them online, there are many excellent developers out there, and they share their knowledge and tricks mostly for free. :)

– Listen to podcasts & subscribe to RSS feeds / newsletters & watch videos, for example from web conferences

To find out about new technologies, techniques, standards, patterns, etc., it is best to use different media, which can be consumed in different situations. An interesting podcast on “Frontend Architecture” before falling asleep or a video on “DevOps” while preparing lunch, reading a book on the tram in the morning entitled “Programming less badly” … to name just a few examples.

Podcasts: https://github.com/voku/awesome-web/blob/master/README.md#-audio-podcast
github is awesome: https://github.com/sindresorhus/awesome
and there is more: https://github.com/jnv/lists

– Attend Meetup’s & web conferences and talk to other developers.

Meetups are groups of people who meet regularly and talk about things like Python, Scala, PHP, etc. Usually, someone gives a lecture on a previously agreed topic.

⇉ http://www.meetup.com/de-DE/members/136733532/

Web conferencing is fun. Point. And every developer / admin should visit them because you get new impressions and meet wonderful people. Some conferences are expensive, but here you should contact your employer, if necessary, the company will take care of it. And there are also really cheap conferences.

– Post answers at quora.com || stackoverflow.com || in forums || your blog…

To deal with a certain topic yourself and to really understand it, it is worth doing research and writing a text (possibly even a lecture) that others can read and criticize and thus improve.

– Don’t stay at work for so long every day; otherwise nobody will be waiting for you at home!

With all the enthusiasm for the “job” (even if it’s fun), you shouldn’t lose sight of the essential things. Again, something I had to learn the hard way. :-/

PHP: Code Quality with Custom Tooling Extensions

After many years of using PHPStan, PHP-CS-Fixer, PHP_CodeSniffer, … I will give you one advice: add your own custom code to extend your Code-Quality-Tooling.

Nearly every project has custom code that procures the real value for the product / project, but this custom code itself is often not really improved by PHP-CS-Fixer, PHPStan, Psalm, and other tools. The tools do not know how this custom code is working so that we need to write some extensions for ourselves.

Example: At work, we have some Html-Form-Element (HFE) classes that used some properties from our Active Record classes, and back in the time we used strings to connect both classes. :-/

Hint: Strings are very flexible, but also awful to use programmatically in the future. I would recommend avoiding plain strings as much as possible.

1. Custom PHP-CS-Fixer

So, I wrote a quick script that will replace the strings with some metadata. The big advantage is that this custom PHP-CS-Fixer will also automatically fix code that will be created in the future, and you can apply / check this in the CI-pipline or e.g. in a pre-commit hook or directly in PhpStorm.

<?php

declare(strict_types=1);

use PhpCsFixer\Tokenizer\Analyzer\ArgumentsAnalyzer;
use PhpCsFixer\Tokenizer\Analyzer\FunctionsAnalyzer;
use PhpCsFixer\Tokenizer\Token;
use PhpCsFixer\Tokenizer\Tokens;

final class MeerxUseMetaFromActiveRowForHFECallsFixer extends AbstractMeerxFixerHelper
{

/**
* {@inheritdoc}
*/
public function getDocumentation(): string
{
return 'Use ActiveRow->m() for "HFE_"-calls, if it is possible.';
}

/**
* {@inheritdoc}
*/
public function getSampleCode(): string
{
return <<<'PHP'
<?php

$element = UserFactory::singleton()->fetchEmpty();

$foo = HFE_Date::Gen($element, 'created_date');
PHP;
}

public function isRisky(): bool
{
return true;
}

/**
* {@inheritdoc}
*/
public function isCandidate(Tokens $tokens): bool
{
return $tokens->isTokenKindFound(\T_STRING);
}

public function getPriority(): int {
// must be run after NoAliasFunctionsFixer
// must be run before MethodArgumentSpaceFixer
return -1;
}

protected function applyFix(SplFileInfo $file, Tokens $tokens): void
{
if (v_str_contains($file->getFilename(), 'HFE_')) {
return;
}

$functionsAnalyzer = new FunctionsAnalyzer();

// fix for "HFE_*::Gen()"
foreach ($tokens as $index => $token) {
$index = (int)$index;

// only for "Gen()"-calls
if (!$token->equals([\T_STRING, 'Gen'], false)) {
continue;
}

// only for "HFE_*"-classes
$object = (string)$tokens[$index - 2]->getContent();
if (!v_str_starts_with($object, 'HFE_')) {
continue;
}

if ($functionsAnalyzer->isGlobalFunctionCall($tokens, $index)) {
continue;
}

$argumentsIndices = $this->getArgumentIndices($tokens, $index);

if (\count($argumentsIndices) >= 2) {
[
$firstArgumentIndex,
$secondArgumentIndex
] = array_keys($argumentsIndices);

// If the second argument is not a string, we cannot make a swap.
if (!$tokens[$secondArgumentIndex]->isGivenKind(\T_CONSTANT_ENCAPSED_STRING)) {
continue;
}

$content = trim($tokens[$secondArgumentIndex]->getContent(), '\'"');
if (!$content) {
continue;
}

$newContent = $tokens[$firstArgumentIndex]->getContent() . '->m()->' . $content;

$tokens[$secondArgumentIndex] = new Token([\T_CONSTANT_ENCAPSED_STRING, $newContent]);
}
}
}

/**
* @param Token[]|Tokens $tokens <phpdoctor-ignore-this-line/>
* @param int $functionNameIndex
*
* @return array<int, int> In the format: startIndex => endIndex
*/
private function getArgumentIndices(Tokens $tokens, $functionNameIndex): array
{
$argumentsAnalyzer = new ArgumentsAnalyzer();

$openParenthesis = $tokens->getNextTokenOfKind($functionNameIndex, ['(']);
$closeParenthesis = $tokens->findBlockEnd(Tokens::BLOCK_TYPE_PARENTHESIS_BRACE, $openParenthesis);

// init
$indices = [];

foreach ($argumentsAnalyzer->getArguments($tokens, $openParenthesis, $closeParenthesis) as $startIndexCandidate => $endIndex) {
$indices[$tokens->getNextMeaningfulToken($startIndexCandidate - 1)] = $tokens->getPrevMeaningfulToken($endIndex + 1);
}

return $indices;
}
}

To use your custom fixes, you can register and enable them: https://cs.symfony.com/doc/custom_rules.html  

Example-Result:


$fieldGroup->addElement(HFE_Customer::Gen($element, 'customer_id'));

// <- will be replaced with ->

$fieldGroup->addElement(HFE_Customer::Gen($element, $element->m()->customer_id));

Hint: There are many examples for PHP_CodeSniffer and Fixer Rules on GitHub, you can often pick something that fits 50-70% for your use-case and then modify it for your needs.

The “m()” method looks like this and will call the simple “ActiveRowMeta”-class. This class will return the property name itself instead of the real value.

/**
* (M)ETA
*
* @return ActiveRowMeta|mixed|static
* <p>
* We fake the return "static" here because we want auto-completion for the current properties in the IDE.
* <br><br>
* But here the properties contains only the name from the property itself.
* </p>
*
* @psalm-return object{string,string}
*/
final public function m()
{
return (new ActiveRowMeta())->create($this);
}
<?php

final class ActiveRowMeta
{
/**
* @return static
*/
public function create(ActiveRow $obj): self
{
/** @var static[] $STATIC_CACHE */
static $STATIC_CACHE = [];

// DEBUG
// var_dump($STATIC_CACHE);

$cacheKey = \get_class($obj);
if (!empty($STATIC_CACHE[$cacheKey])) {
return $STATIC_CACHE[$cacheKey];
}

foreach ($obj->getObjectVars() as $propertyName => $propertyValue) {
$this->{$propertyName} = $propertyName;
}

$STATIC_CACHE[$cacheKey] = $this;

return $this;
}

}

2. Custom PHPStan Extension

In the next step, I added a DynamicMethodReturnTypeExtension for PHPStan, so that the static code analyze knows the type of the metadata + I still have auto-completion in the IDE via phpdocs.

Note: Here I’ve also made the metadata read-only, so we can’t misuse the metadata.

<?php

declare(strict_types=1);

namespace meerx\App\scripts\githooks\StandardMeerx\PHPStanHelper;

use PhpParser\Node\Expr\MethodCall;
use PHPStan\Analyser\Scope;
use PHPStan\Reflection\MethodReflection;
use PHPStan\Type\Type;

final class MeerxMetaDynamicReturnTypeExtension implements \PHPStan\Type\DynamicMethodReturnTypeExtension
{

public function getClass(): string
{
return \ActiveRow::class;
}

public function isMethodSupported(MethodReflection $methodReflection): bool
{
return $methodReflection->getName() === 'm';
}

/**
* @var \PHPStan\Reflection\ReflectionProvider
*/
private $reflectionProvider;

public function __construct(\PHPStan\Reflection\ReflectionProvider $reflectionProvider)
{
$this->reflectionProvider = $reflectionProvider;
}

public function getTypeFromMethodCall(
MethodReflection $methodReflection,
MethodCall $methodCall,
Scope $scope
): Type
{
$exprType = $scope->getType($methodCall->var);

$staticClassName = $exprType->getReferencedClasses()[0];
$classReflection = $this->reflectionProvider->getClass($staticClassName);

return new MeerxMetaType($staticClassName, null, $classReflection);
}
}
<?php

declare(strict_types=1);

namespace meerx\App\scripts\githooks\StandardMeerx\PHPStanHelper;

use PHPStan\Reflection\ClassMemberAccessAnswerer;
use PHPStan\Type\ObjectType;

final class MeerxMetaType extends ObjectType
{

public function getProperty(string $propertyName, ClassMemberAccessAnswerer $scope): \PHPStan\Reflection\PropertyReflection
{
return new MeerxMetaProperty($this->getClassReflection());
}

}
<?php

declare(strict_types=1);

namespace meerx\App\scripts\githooks\StandardMeerx\PHPStanHelper;

use PHPStan\Reflection\ClassReflection;
use PHPStan\TrinaryLogic;
use PHPStan\Type\NeverType;
use PHPStan\Type\StringType;

final class MeerxMetaProperty implements \PHPStan\Reflection\PropertyReflection
{

private ClassReflection $classReflection;

public function __construct(ClassReflection $classReflection)
{
$this->classReflection = $classReflection;
}

public function getReadableType(): \PHPStan\Type\Type
{
return new StringType();
}

public function getWritableType(): \PHPStan\Type\Type
{
return new NeverType();
}

public function isWritable(): bool
{
return false;
}

public function getDeclaringClass(): \PHPStan\Reflection\ClassReflection
{
return $this->classReflection;
}

public function isStatic(): bool
{
return false;
}

public function isPrivate(): bool
{
return false;
}

public function isPublic(): bool
{
return true;
}

public function getDocComment(): ?string
{
return null;
}

public function canChangeTypeAfterAssignment(): bool
{
return false;
}

public function isReadable(): bool
{
return true;
}

public function isDeprecated(): \PHPStan\TrinaryLogic
{
return TrinaryLogic::createFromBoolean(false);
}

public function getDeprecatedDescription(): ?string
{
return null;
}

public function isInternal(): \PHPStan\TrinaryLogic
{
return TrinaryLogic::createFromBoolean(false);
}
}

Summary

Think about your custom code and how you can improve it, use your already used tools and extend it to understand your code. Sometimes it’s easy, and you can add some modern PHPDocs or you need to go down the rabbit hole and implement some custom stuff, but at last it will help your software, your team and your customers.

Timeout Problems: Web Server + PHP

What?

First there is an HTTP request and that will hit your Web server, then it will pass the request via TCP- or UNIT-Socket via FastCGI to your PHP-FPM Daemon, here we will start a new PHP process and in this process we will connect e.g. to the database and run some queries.

PHP-Request

The Problem!

There are different timeout problems here because we connect different pieces together and this parts need to communicate. But what if one of the pieces does not respond in a given time or, even more bad, if one process is running forever like a bad SQL-query.

Understand your Timeouts.

Timeouts are a way to limit the time that a request can run, and otherwise an attacker could simply run a denial-of-service with a simple request. But there are many configurations in several layers: Web server, PHP, application, database, curl, …

– Web server

Mostly you will use Apache or Nginx as Web server and in the end it makes not really a difference, there are different timeout settings, but the idea is almost the same: The Web server will stop the execution and kills the PHP process, now you got a 504 HTTP error (Gateway Timeout) and you will lose your stack trace and error-tracking because we killed our application in the middle of nothing. So, we should keep the Web server running as long as needed.

“`grep -Ri timeout /etc/apache2/“`

/etc/apache2/conf-enabled/timeout.conf:Timeout 60

/etc/apache2/mods-available/reqtimeout.conf:<IfModule reqtimeout_module>

/etc/apache2/mods-available/reqtimeout.conf: # mod_reqtimeout limits the time waiting on the client to prevent an

/etc/apache2/mods-available/reqtimeout.conf: # configuration, but it may be necessary to tune the timeout values to

/etc/apache2/mods-available/reqtimeout.conf: # mod_reqtimeout per virtual host.

/etc/apache2/mods-available/reqtimeout.conf: # Note: Lower timeouts may make sense on non-ssl virtual hosts but can

/etc/apache2/mods-available/reqtimeout.conf: # cause problem with ssl enabled virtual hosts: This timeout includes

/etc/apache2/mods-available/reqtimeout.conf: RequestReadTimeout header=20-40,minrate=500

/etc/apache2/mods-available/reqtimeout.conf: RequestReadTimeout body=10,minrate=500

/etc/apache2/mods-available/reqtimeout.load:LoadModule reqtimeout_module /usr/lib/apache2/modules/mod_reqtimeout.so

/etc/apache2/mods-available/ssl.conf: # to use and second the expiring timeout (in seconds).

/etc/apache2/mods-available/ssl.conf: SSLSessionCacheTimeout 300

/etc/apache2/conf-available/timeout.conf:Timeout 60

/etc/apache2/apache2.conf:# Timeout: The number of seconds before receives and sends time out.

/etc/apache2/apache2.conf:Timeout 60

/etc/apache2/apache2.conf:# KeepAliveTimeout: Number of seconds to wait for the next request from the

/etc/apache2/apache2.conf:KeepAliveTimeout 5

/etc/apache2/mods-enabled/reqtimeout.conf:<IfModule reqtimeout_module>

/etc/apache2/mods-enabled/reqtimeout.conf: # mod_reqtimeout limits the time waiting on the client to prevent an

/etc/apache2/mods-enabled/reqtimeout.conf: # configuration, but it may be necessary to tune the timeout values to

/etc/apache2/mods-enabled/reqtimeout.conf: # mod_reqtimeout per virtual host.

/etc/apache2/mods-enabled/reqtimeout.conf: # Note: Lower timeouts may make sense on non-ssl virtual hosts but can

/etc/apache2/mods-enabled/reqtimeout.conf: # cause problem with ssl enabled virtual hosts: This timeout includes

/etc/apache2/mods-enabled/reqtimeout.conf: RequestReadTimeout header=20-40,minrate=500

/etc/apache2/mods-enabled/reqtimeout.conf: RequestReadTimeout body=10,minrate=500

/etc/apache2/mods-enabled/reqtimeout.load:LoadModule reqtimeout_module /usr/lib/apache2/modules/mod_reqtimeout.so

/etc/apache2/mods-enabled/ssl.conf: # to use and second the expiring timeout (in seconds).

/etc/apache2/mods-enabled/ssl.conf: SSLSessionCacheTimeout 300

Here you can see all configurations for Apache2 timeouts, but we only need to change etc/apache2/conf-enabled/timeout.conf`` because it will overwrite `/etc/apache2/apache2.conf` anyway.

PS: Remember to reload / restart your Web server after you change the configurations.

If we want to show the user at least a custom error page, we could add something like:

ErrorDocument503 /error.php?errorcode=503
ErrorDocument 504 /error.php?errorcode=504

… into our Apache configuration or in a .htaccess file, so that we can still use PHP to show an error page, also if the requested PHP call was killed. The problem here is that we will lose the error message / stack trace / request etc. from the error, and we can’t send e.g. an error into our error logging system. (take a look at sentry, it’s really helpful)

– PHP-FPM

Our PHP-FPM (FastCGI Process Manager) pool can be configured with a timeout (request-terminate-timeout), but just like the Web server setting, this will kill the PHP worker in the middle of the process, and we can’t handle the error in PHP itself. There is also a setting (process_control_timeout) that tells the child processes to wait for this much time before executing the signal received from the parent process, but I am uncertain if this is somehow helpfully here? So, our error handling in PHP can’t catch / log / show the error, and we will get a 503 HTTP error (Service Unavailable) in case of a timeout.

Shutdown functions will not be executed if the process is killed with a SIGTERM or SIGKILL signal. :-/

Source: register_shutdown_function

PS: Remember to reload / restart your PHP-FPM Daemon after you change the configurations.

– PHP

The first idea from most of us would be maybe to limit the PHP execution time itself, and we are done, but that sounds easier than it is because `max_execution_time` ignores time spent on I/O (system commands e.g. `sleep()`, database queries (SELECT SLEEP(100)). But these are the bottlenecks of nearly all PHP applications, PHP itself is fast but the external called stuff isn’t.

Theset_time_limit()function and the configuration directive max_execution_time only affect the execution time of the script itself. Any time spent on activity that happens outside the execution of the script such as system calls using system(), stream operations, database queries, etc. is not included when determining the maximum time that the script has been running. This is not true on Windows where the measured time is real.

Source: set_time_limit

– Database (MySQLi)

Many PHP applications spend most of their time waiting for some bad SQL queries, where the developer missed adding the correct indexes and because we learned that the PHP max execution time did not work for database queries, we need one more timeout setting here.

There is the MYSQLI_OPT_CONNECT_TIMEOUT and MYSQLI_OPT_READ_TIMEOUT (Command execution result timeout in seconds. Available as of PHP 7.2.0. – mysqli.options) setting, and we can use that to limit the time for our queries.

In the end you will see a “Errno: 2006 | Error: MySQL server has gone away” error in your PHP application, but this error can be caught / reported, and the SQL query can be fixed, otherwise the Apache or PHP-FPM would kill the process, and we do not see the error because our error handler can’t handle it anyway.

Summary:

It’s complicated. PHP is not designed for long execution and that is good as it is, but if you need to increase the timeout it will be more complicated than I first thought. You need for example different “timeout”-code for testing different settings:

// DEBUG: long-running sql-call
// Query(‘SELECT SLEEP(600);’);

// DEBUG: long-running system-call
// sleep(600);

// DEBUG: long-running php-call
// while (1) { } // infinite loop

Solution:

We can combine different timeout, but the timeout from the called commands e.g. database, curl, etc. will be combined with the timeout from PHP (max_execution_time) itself. The timeout from the Web server (e.g. Apache2: Timeout) and from PHP-FPM (request_terminate_timeout) need to be longer than the combined timeout from the application so that we still can use our PHP error handler.

e.g.: ~ 5 min. timeout

  1. MySQL read timeout: 240s ⇾ 4 min.
    link->options(MYSQLI_OPT_READ_TIMEOUT, 240);
  2. PHP timeout: 300s ⇾ 5 min.
    max_execution_time = 300
  3. Apache timeout: 360s ⇾ 6 min.
    Timeout 360
  4. PHP-FPM: 420s ⇾ 7 min.
    request_terminate_timeout = 420

 

Links:

Prepare your PHP Code for Static Analysis

Three years ago I got a new job as PHP developer, before that I called myself web developer because I build ReactJS, jQuery, CSS, HTML, … and PHP  stuff for a web agency. So now I am a full-time PHP developer and I converted a non typed  (no PHPDoc + no native types) project with ~ 10.000 classes into a project with ~ 90% type coverage. Here is what I have learned.

1. Write code with IDE autocompletion support.

If you have autocompletion in the IDE most likely the Static Analysis can understand the code as well. 

Example:

bad:

->get('DB_Connection', true, false);

still bad:

->get(DB_Connection::class);

good:

getDbConnection(): DB_Connection

2. Magic in Code is bad for the long run!

Magic methods (__get, __set, …) for example can help to implement new stuff very fast, but the problem is nobody will understand it, you will have no autocompletion, no refactoring options, other developers will need more time to read and navigate in the code and in the end it will cost you much more time than you can save with it.

3. Break the spell on Magic Code …

… by explaining to everyone (Devs > IDE > Tools) what it does.

Example 1:

We use a simple Active Record Pattern, but we put all SQL stuff into the Factory classes, so that the Active Record class can be simple. (Example) But because of missing support for Generics we had no autocompletion without adding many dummy methods into the classes. So one of my first steps was to introduce a “PhpCsFixer” that automatically adds the missing methods of the parent class with the correct types via “@method”-comments into these classes. 

Example 2:

Sometimes you can use more modern PHPDocs to explain the function. Take a look at the “array_first” function in the linked Post.

Example 3:

/**
* Return an array which has the Property-Values of the given Objects as Values.
*
* @param object[] $ObjArray
* @param string $PropertyName
* @param null|string $KeyPropertyName if given uses this Property as key for the returned Array otherwise the keys from the
* given array are used
*
* @throws Exception if no property with the given name was found
*
* @return array
*/
function propertyArray($ObjArray, $PropertyName, $KeyPropertyName = null): array {
// init
$PropertyArray = [];

foreach ($ObjArray as $key => $Obj) {
if (!\property_exists($Obj, $PropertyName)) {
throw new Exception('No Property with Name ' . $PropertyName . ' in Object Found - Value');
}

$usedKey = $key;
if ($KeyPropertyName) {
if (!\property_exists($Obj, $KeyPropertyName)) {
throw new Exception('No Property with Name ' . $PropertyName . ' in Object Found - Key');
}
$usedKey = $Obj->{$KeyPropertyName};
}

$PropertyArray[$usedKey] = $Obj->{$PropertyName};
}

return $PropertyArray;
}

Sometimes it’s hard to describe the specific output types, so here you need to extend the  functions of your Static Code Analyze Tool, so that it knows what you are doing. ⇾ for example here you can find a solution for PHPStan ⇽  but there is still no support for the IDE and so maybe it’s not the best idea to use magic like that at all. And I am sure it’s more simple to use specific and simple methods instead:  e.g. BillCollection->getBillDates()

4. Try to not use strings for the code.

Strings are simple and flexible, but they are also bad for the long run. Mostly strings are used because it looks like a simple solution, but often they are redundant, you will have typos everywhere and the IDE and/or Static Analysis can’t analyze them because it’s just text.

Example:

bad: 

AjaxEditDate::generator($bill->bill_id, $bill->date, 'Bill', 'date');
  • “Bill” ⇾ is not needed here, we can call e.g. get_class($bill) in the “generator” method
  • “date” ⇾ is not needed here, we can fetch the property name from the class
  • “$bill->bill_id” ⇾ is not needed here, we can get the primary id value from the class

good:

AjaxEditDate::generator($bill, $bill->m()->date);

5. Automate stuff via git hook and check it via CI server.

Fixing bad code is only done if you disallow the same bad code for the future. With a pre-commit hook for git it’s simple to run these checks, but you need to check it again in the CI server because the developers can simply skip these checks.

Example:

I introduced a check for disallowing global variables (global $foo && $GLOBALS[‘foo’]) via “PHP_CodeSniffer”. 

Links:

6. Use array shapes (or collections) if you need to return an array, please.

Array shapes are like arrays but with fixed keys, so that you can define the types of each key in the PHPDocs.

You will have autocompletion for the IDE + all other devs can see what the method will return + you will notice if you’re better retuning an object because it will be very hard to describe the output for complex data structures and Static Analysis can use and check the types. 

Example:

/**
* @return array{
* missing: array<ShoppingCartLib::TAB_*,string>,
* disabled: array<ShoppingCartLib::TAB_*,string>
* }
*/
private static function getShoppingCartTabStatus(): array {
...
}

Generics in PHP via PHPDocs

If you did not know that you can use Generics in PHP or you do not exactly know how to use it or why you should use it, then the next examples are for you.

Type variables via @template

The @template tag allows classes and functions to declare a generic type parameter. The next examples starts with simple functions, so that we understand how it works, and then we will see the power of this in classes.


A dummy function that will return the input.

https://phpstan.org/r/1922279b-9786-4523-939d-dddcfd4ebb86

    <?php    

    /**
     * @param \Exception $param
     * @return \Exception
     *
     * @template T of \Exception
     * @psalm-param T $param
     * @psalm-return T
     */
    function foo($param) { ... }

    foo(new \InvalidArgumentException()); // The static-analysis-tool knows that 
                                          // the type is still "\InvalidArgumentException" 
                                          // because of the type variable.

@template T of \Exception // here we create a new type variable, and we force that it must be an instance of \Exception

@phpstan-param T $param // here we say that the static-analysis-tool need to remember the type that this variable had before (you can use @psalm-* or @phpstan-* both works with both tools)

@phpstan-return T // and that the return type is the same as the input type 


A simple function that gets the first element of an array or a fallback. 

In the @param PHPDocs we write “mixed” because this function can handle different types. But this information is not very helpful if you want to understand programmatically what the code does, so we need to give the static-analysis-tools some more information. 

https://phpstan.org/r/1900a2af-f5c1-4942-939c-409928a5ac4a

    <?php
     
    /**
     * @param array<mixed> $array
     * @param mixed        $fallback <p>This fallback will be used, if the array is empty.</p>
     *
     * @return mixed|null
     *
     * @template TFirst
     * @template TFirstFallback
     * @psalm-param TFirst[] $array
     * @psalm-param TFirstFallback $fallback
     * @psalm-return TFirst|TFirstFallback
     */
    function array_first(array $array, $fallback)
    {
        $key_first = array_key_first($array);
        if ($key_first === null) {
            return $fallback;
        }

        return $array[$key_first];
    }

    array_first([1, 2, 3], null); 

    if ($a === 'foo') { // The static-analysis-tool knows that 
                        // === between int|null and 'foo' will always evaluate to false.
	    // ...
    }

@template TFirst // we again define your typed variables

@template TFirstFallback // and one more because we have two inputs where we want to keep track of the types

@psalm-param TFirst[] $array // here we define that $array is an array of TFirst types

@psalm-param TFirstFallback $fallback // and that $fallback is some other type that comes into this function

@psalm-return TFirst|TFirstFallback // now we define the return type as an element of  the $array or the $fallback type 


 Very basic Active Record + Generics

The IDE support for generics is currently not there, :-/ so that we still need some hacks (see @method) for e.g. PhpStorm to have autocompletion.

https://phpstan.org/r/f88f5cd4-1bb9-4a09-baae-069fddb10b12

https://github.com/voku/phpstorm_issue_53352/tree/master/src/Framework/ActiveRecord

<?php

class ActiveRow
{
    /**
     * @var ManagedFactory<static>
     */
    public $factory;

    /**
     * @param Factory<ActiveRow>|ManagedFactory<static> $factory
     * @param null|array                                $row
     */
    public function __construct(Factory $factory, array $row = null) {
        $this->factory = &$factory;
    }
}

/**
 * @template T
 */
abstract class Factory
{
    /**
     * @var string
     *
     * @internal
     *
     * @psalm-var class-string<T>
     */
    protected $classname;

    /**
     * @return static
     */
    public static function create() {
        return new static();
    }
}

/**
 * @template  T
 * @extends   Factory<T>
 */
class ManagedFactory extends Factory
{
    /**
     * @param string $classname
     *
     * @return void
     *
     * @psalm-param class-string<T> $classname
     */
    protected function setClass(string $classname): void
    {
        if (\class_exists($classname) === false) {
            /** @noinspection ThrowRawExceptionInspection */
            throw new Exception('TODO');
        }

        if (\is_subclass_of($classname, ActiveRow::class) === false) {
            /** @noinspection ThrowRawExceptionInspection */
            throw new Exception('TODO');
        }

        $this->classname = $classname;
    }

    // ...
}

final class Foo extends ActiveRow {

    public int $foo_id;

    public int $user_id;

    // --------------------------------------
    // add more logic here ...
    // --------------------------------------
}

/**
 * @method Foo[] fetchAll(...)
 *
 * @see Foo
 *
 * // warning -> do not edit this comment by hand, it's auto-generated and the @method phpdocs are for IDE support       
 * //         -> https://gist.github.com/voku/3aba12eb898dfa209a787c398a331f9c
 *
 * @extends ManagedFactory<Foo>
 */
final class FooFactory extends ManagedFactory
{
    // -----------------------------------------------
    // add sql stuff here ...
    // -----------------------------------------------
}

A more complex collection example.

In the end we can extend the “AbstractCollection” and the static-analysis-tools knows the types of all the methods. 

https://github.com/voku/Arrayy/tree/master/src/Type

/**
 * @template TKey of array-key
 * @template T
 * @template-extends \ArrayObject<TKey,T>
 * @template-implements \IteratorAggregate<TKey,T>
 * @template-implements \ArrayAccess<TKey|null,T>
 */
class Arrayy extends \ArrayObject implements \IteratorAggregate, \ArrayAccess, \Serializable, \JsonSerializable, \Countable
{ ... }

/**
 * @template TKey of array-key
 * @template T
 * @template-extends \IteratorAggregate<TKey,T>
 * @template-extends \ArrayAccess<TKey|null,T>
 */
interface CollectionInterface extends \IteratorAggregate, \ArrayAccess, \Serializable, \JsonSerializable, \Countable
{ ... }

/**
 * @template   TKey of array-key
 * @template   T
 * @extends    Arrayy<TKey,T>
 * @implements CollectionInterface<TKey,T>
 */
abstract class AbstractCollection extends Arrayy implements CollectionInterface
{ ... }

/**
 * @template TKey of array-key
 * @template T
 * @extends  AbstractCollection<TKey,T>
 */
class Collection extends AbstractCollection
{ ... }

Links:

https://phpstan.org/blog/generics-in-php-using-phpdocs

https://psalm.dev/docs/annotating_code/templated_annotations/

https://stitcher.io/blog/php-generics-and-why-we-need-them

❤️ Simple PHP Code Parser

It based on code from “JetBrains/phpstorm-stubs” but instead of Php-Reflection we now use nikic/PHP-Parser, BetterReflection, phpDocumentor and PHPStan/phpdoc-parser internally. So, you can get even more information about the code. For example, psalm- / phpstan-phpdoc annotations or inheritdoc from methods.

Install:

composer require voku/simple-php-code-parser

Link:

voku/Simple-PHP-Code-Parser

More:


Example: get value from define in “\foo\bar” namespace

$code = '
  <?php
  namespace foo\bar;
  define("FOO_BAR", "Lall");
';

$phpCode = PhpCodeParser::getFromString($code);
$phpConstants = $phpCode->getConstants();

$phpConstants['\foo\bar\FOO_BAR']->value; // 'Lall'

Example: get information about @property phpdoc from a class

$code = '
  <?php
  /** 
   * @property int[] $foo 
   */
  abstract class Foo { }
';

$phpCode = PhpCodeParser::getFromString($code);
$phpClass = $phpCode->getClass('Foo');

$phpClass->properties['foo']->typeFromPhpDoc); // int

Example: get classes from a string (or from a class-name or from a file or from a directory)

$code = '
<?php
namespace voku\tests;
class SimpleClass {}
$obja = new class() {};
$objb = new class {};
class AnotherClass {}
';

$phpCode = \voku\SimplePhpParser\Parsers\PhpCodeParser::getFromString($code);
$phpClasses = $phpCode->getClasses();

var_dump($phpClasses['voku\tests\SimpleClass']); // "PHPClass"-object