In my opinion this new project will be very important to the ASF. Apache only hosts two other PHP projects, Log4PHP and Shindig. Since PHP is becoming more and more popular for big business (think on Facebook) I think its necessary to attract more developers and bring them into the Apache community. Many PHP projects out there use the GPL license, and that is sometimes very difficult to combine with commerical work. Good that we now have a component framework under ASL! Together with Log4PHP it’s a good start for much proprietary work.

More in future, I think that even the PIWI Framework will benefit of this change. I already had some trouble with dependent GPL code in it and hopefully Zeta can address most of my wishes. PIWI built upon Log4PHP and Zeta would be charming. But as already said, this will need a while.

At the moment, Zeta Components do already have a mailinglists and status page on the incubator. Jira will follow and after finally the CLAs of the initial developers have been processed, work can begin. However, if you are interest in helping this project subscribe to the user and to the dev list mentioned in the status page.

Lucky me I have been elected into the Apache Incubator PMC and can now serve as a mentor to this project. It’s not only honour to me, it’s also a good chance to work with obviously very nice people and excellent technicians. Also it makes it easy to me to step into this project and learn all I need.

So, welcome Zeta Components!! I am glad you arrived at the Apache Software Foundation!

Now Log4PHP needs to succeed 3 votes. One for the Log4PHP team to vote for graduating; one for Apache Logging to accept the podling as a subproject. And finally – after the first two have succeeded – a vote on the incubator list to release the podling to its final destination.

The first two votes are already running and it looks very good so far. Votes need to be open another day then the next step can be done. For those who are interested – there is a detailed document about graduation available.

Let’s see how it works out – I think everything could be in place in quite less time, maybe the next two or three weeks.

It’s an exciting time for all involved, and is the result of a culmination of many like-minded individuals. Everyone’s worked hard and as the initial test community seem to have responded positively and in such detail they could test anything from Word to Think Bingo (http://www.thinkbingo.com/) with their eyes closed. The people behind the development had a few words on the announcement of the first Log4PHP.

Here is the original statement:

The Log4PHP community is pleased to introduce the Apache Log4PHP 2.0.0 (Incubating) release [1]. It’s the first Log4PHP release since 2004 and tons of changes have been done. Finally Log4PHP has become a well tested framework made for PHP 5. Many thanks to all the contributors who made this release possible. Please download [2] Log4PHP and enjoy

The Log4PHP team

[1] http://incubator.apache.org/log4php/changes-report.html
[2] http://incubator.apache.org/log4php/download.html

Time for me to think about the different options to write to log files.

I figured out, that I have to compare the following options:

1) Not closing the file while the whole request is running. This is not an option in a live system, but will give me a good idea whats currently the case

$fp = fopen($file, 'a+');
while($count < $loop) {
   fwrite($fp, $text);
}
fclose($fp);

2) Closing the file directly after fwrite is called

while($count < $loop) {
   $fp = fopen($file, 'a+');
   fwrite($fp, $text);
   fclose($fp);
}

3) Use file_put_contents, which is known as an alias to fopen, fwrite and fclose

while($count < $loop) {
   file_put_contents($file, $text, FILE_APPEND);
}

4) Leave the file open while the whole request, but unlock it with flock and flock it again, when the next log event occurs

$fp = fopen($file, 'a+');
flock($fp, LOCK_UN);
while($count < $loop) {
   if (flock($fp, LOCK_EX)) {
      fwrite($fp, $text);
   }
   flock($fp, LOCK_UN);
}
fclose($fp);

5) Use a nonblocking stream for this and flock

$fp = fopen($file, 'a+');
stream_set_blocking($fp, 0);

while($count < $loop) {
   if (flock($fp, LOCK_EX)) {
       fwrite($fp, $text);
   }
   flock($fp, LOCK_UN);
}
fclose($fp);

6) Use the error_log method, which my friend Kevin Horst brought up

while($count < $loop) {
   error_log($text, 3, $file);
}

For each of this options I wrote a simple function which wrote 10000 times 100 characters in  a freshly created log file. I measured before opening and after closing. Additionally I tried out with 2 seperated threads if the write access is nonblocking. Good thing is, option 2 to 6 are actually nonblocking. And here are the timing results:

1) Execution with NOT closing the log file took 0.0668561458588 seconds
2) Execution with CLOSING the log after each write file took 30.1630220413 seconds
3) Execution with file_put_content took 30.153963089 seconds
4) Execution with leaving the file open, but LOCKING and UNLOCKING it took 0.148998975754 seconds
5) Execution with nonblocking stream took 0.149605989456 seconds
6) Execution with the error_log method took 30.069578886 seconds

Let’s see what it means.

Not closing the file until the 10000 fwrite calls are handled as actually the fastest. No surprise. It just took 0.0668 seconds but this one is not really an option, cause other threads have to wait until this request has been finished.

If you close the file after each fwrite and make it available to other threads and then reopen it, is hell. For 10000 calls of this kind I needed 30 seconds! It’s insane to have this in a productive system. The same goes with file_put_contents. Well, I allready knew (its in the php docs) that this method is nothing else then a wrapper for fopen, fwrite and fclose. Times are so similar that I say it’s exactly the same. Sometimes the one is some millis faster, sometimes the other.

If you open the file, unlock it with flock and flock it again it works very well. Just 0.14 ms for the 10000 fwrite calls. Thats the double amount of option 1, but yeah, here we do some more stuff. The interpreter cares about who is allowed to write, together with the OS. flock works that way, that a call to this function blocks until the requestor gets the actual lock. You can be sure that only one thread is actually writing.

Same goes to the nonblocking stream. This works with stream_set_blocking($fp, 0);. The file stream is nonblocking, means each thread could write at it the same time. That no mess happens, we need an flock here too. That brings us to the nearly same results as fopen, flock, fwrite, flock, fclose option above. But looking at the logfiles of this one and option 4, this one looks more nice to me. This is just subjective, but it looks like the lock is shared more nicely between both threads.

Last one is the error_log method. It didn’t had any idea what to expect, but… 30 seconds! This one behaviours like a wrapper for fopen, fwrite and fclose, like file_put_content. No guys, this is not really a method enabled for logging! If one would use this in a framework like Log4PHP, that would be hell to performance. I would think that this should better removed out. The name suggests a good logging method, but this is not the case.

Having that all said, Log4PHP will get the lock and unlock option number 4. I feel good with it, since it’s quite straightforward. I don’t have too much expierence with the non blocking stream and don’t want to have this in a framework like log4php is.

However, Logging must be used carefully at all. I thnk on a system with 10000 request a second. Enabling logging into one file could bring the system down. I think a live system should have the option to log exactly one request. Maybe triggered by an url param. Think carefully what you log and how you configure your live system.

The complete script can be found here.

I think I will use a similar script for pushing stuff with Log4PHP. I will propose that today on the mailinglist. Additionally I think about making a WordPress plugin for my own blog. We’ll see how fast I am

Your comments are appreciated!

Here is the wave script. Its released under Apache SL 2.0. Credits to the Wave team, I based everything on their examples.

All the worlds speaks about testdriven development and unit testing. But most people finishing there exams don’t have a clue about what this means. To be honest, I was surprised that people could get out of a university just with one or two Junit tests implemented. Well, here are some words about testing.

What is a unit test?

A unit test is nothing else than some lines of test code which tries to check out if the lines of business logic you wrote actually works. To help you with this, tools like JUnit and PHPUnit (or SimpleTest in PHP enviroment) have been written. They are frameworks which give you some methods to work with. To be clear: a unit test is some kind of class which instantiates another class and checks if the parameters you put in bring the correct results.

What does testdriven development mean?

Hardcore people say, before you write the business logic, you’ll have to write a test case. This is test driven development: write your test, then write the actual business logic. For me it works usually like this: I write my business logic and as soon as I have something cool working, I write a testcase. Sometimes it behaves a bit different. Depends on. But thats a matter of taste. As usual I recommend not to be to extreme about everything and just try out what is best for you.

What and how much to test?

Everything. Well, not really. A good testing ratio is at 70%. More does mean that you even test your exceptions, less means you have forgotten some classes. There are tools available which help you with checking out how much you have tested. In Javaworld its Cobertura, in PHP its Spike PHP Testcoverage, for example.

How does a testcase look like?

Usually you have to extend your tester class from a class called TestCase or similar. In PHPUnit its PHPUnit_Framework_TestCase. They make your classes executable in the testing enviroment. Then implement methods – all methods which implement an actual test is prefixed with test.

An example from Log4PHP:

class LoggerLayoutHtmlTest extends PHPUnit_Framework_TestCase {
    public function testErrorLayout() {

The testing frameworks usually look for methods like this and execute them one by one. In newer JUnit Versions you have to annotate testing methods. OK, and this is a valid PHP testcase:

public function testErrorLayout() {
        $event = new LoggerLoggingEvent("LoggerLayoutHtmlTest", new Logger("TEST"), LoggerLevel::getLevelError(), "testmessage");
        $layout = new LoggerLayoutHtml();
        $v = $layout->format($event);
        $e = "blub";
        self::assertEquals($v, $e);
    }

Again, its Log4PHP. I instantiate a class which I am intending to test and call a method on it. The result is stored in $v. What I expect is stored in $e. All testing frameworks provide you so called assert methods, which enable you to compare or check otherwise if the expectation meets the actual result. In my case, “blub” is expected. If this assertion fails, my tool shows me that error.

Why is it necessary to keep old tests?

If you change business logic or refactor something you can simply execute old tests and recognize if something goes wrong. People say, before putting an echo or System.out.prinln somewhere, write an test. I agree here. A good testing ratio makes your software stable and you take care of side effects, even when software grows. And before you deliver you run all your tests. This way you make sure that everything is fine.

Test Data and dependend Tests

To make it short, test methods should never depend on the success of  test methods. Each test method can be executed stand alone. Otherwise your life will be hell – think on thousand methods depending on each other. Same goes to test data. A testcase must not depend on data another test method created. This will cause you hell.

I know that this is very difficult  with databases. Testframeworks usually give you “tearUp” or “tearDown” methods which are called before a test method starts. You can create your testdata in the databases in these methods. But this is very time consuming. Test executions of this kind can easily take hours. Best is, keep your tables short, make a SQL file for each package or even each test case. There are no best practices which fits on each project.

In my last project we have used Excel (=Manager aware) to take care of the data and then generated SQL files out of it. Time consuming, complex, but it worked. The relations in this tables where to strong, we couldn’t keep the data care up otherwise.

Testing helps coding

Having said the above, you can imagine that you should think about testability BEFORE you code. Spaghetthi code isn’t testable. Make short methods. Keep in mind what “Separation of Concerns” means. Same goes to databases. Make them plugable. Not at all time foreign keys are good. Think about it twice – they are mighty but can be evil too. I have worked in projects where it was very heavy duty or even impossible to delete any data. If you write your application testable, its going to be good designed in much cases.

This doesn’t mean you should put all methods to public scope! Test cases should be in the same package as their Test classes are, so you can try to work with package scoped methods aswell. However, whatever you do, testable code is good, but it never should break encapsulation just for beeing testable.

In most cases private methods can be tested with the test of public methods. Test coverage tools help to identify test lacks.

What to test? Again!

Try to test all public methods! If you need to create files, use the tempfolder. Set up databases for the test. Use Jetty and HSQL if need it embedded – in PHP its PDO and SQLite. Don’t waste your time with testing exceptions which usually never happen. Just use 20% of your time to make 80% of tests! Try to test business logic – if that means the creation of complex objects, so be it. Maybe you can share the logic. But don’t get bored with testing getter and setters. Those are called thousand of times within normal tests or procedures, no need to do that.

How much time can test development need??

Calculate the same time as implemeting the testcase. Too much? Not really! Imagine you are writing business code for 5 hours. Thousands of outputs on the console you read manually. You delete it, check it in and later want to fix again. Put those outputs in again? Have commented it out? Code is ugly? Put all your stuff in a testcase. Nobody claims if you need 5 hours for it, if its done properly. Sometimes testcases need 3 days but the fix just 1 hour. This can happen in EAI enviroments, where multiple systems communicate. No problem here – these is business critical, automatic tests are the best you can do.

Log4PHP is an api for logging within PHP applications. As the name suggests, it’s a port from Log4J which is todays standard in Java applications. Log4PHP has had several attempts to get running, but always went dry. It was february 2007 where lads kicked off the project again.

However, since I need this  Log4PHP as a dependency on PIWI I really don’t want to let it go. It has so much benefits to PIWI. And finally I don’t know a better logging tool than Log4J or its port Log4PHP is. Beside that Log4PHP is a great api, I think it will help to get PHP developers more attracted to Apache. This is important since there are so less PHP guys here. Apache Incubator Shindig is the only project I know with a regular contribution for PHP. Well, hopefully we get a community back the next days and hopefully some of those users turn into fullfledget committers too. We’ll see.

It’s some stuff todo. Nothing critical, just some design issues, small errors and some todos. I guess, we can work on a 1.0 once those issues are resolved. We’ll see

Here are some links for further reading:

http://incubator.apache.org/log4php/

https://issues.apache.org/jira/browse/LOG4PHP

http://incubator.apache.org/projects/log4php.html