|
|
Archive for the ‘Software Development’ Category
Friday, January 8th, 2010
Readers of my blog maybe have already noticed it: Jason found out, that somebody has taken my code, removed the copyright stuff and uploaded it to the Wordpress directory. I was quite surprised – the content-thief is telling on his blog that he had two weeks holiday while he came across Wave and implemented this plugin. I mean, what the f***? Are people really thinking that nobody will notice that theft in the case of beta software? I just can say: don’t use tecinfor-wave. This is outdated code and the distributor didn’t write it – I doubt he can help in error cases.
But everything has its good sides. I decided to put my development on that plugin more into public and created a new google code project: Wavediver. You can find the original Wordpress plugin there. I will try to update the API and make it more usable. At the moment it’s a little bit, well, ugly Let’s hope that it will reach the status to be included in the PIWI framework later.
Already included is a cli.php file, which enables you to send Adobe Wave message from the command line, just with: php -f cli.php. That can help testing.
If there is anybody who wants to join up – don’t hesitate.
Tags: Adobe Wave, PHP, Wavediver, Wordpress Posted in Open Source, Software Development | No Comments »
Wednesday, December 9th, 2009
I recently got a copy of Apache Geronimo 2.1: Quick Reference . This book should help me with my first steps with this container. Here is a little review for this brand new release of Packt publishing.
My first impression was: wow, is that really a quick reference? The book has a good format and comes with roundabout 370 pages. I hoped for a short book in these times I don’t have too much time for huge novels in the kind of “Gone with the wind”. However, I jumped through the content and figured out that this is actually a quick reference - very good for the impatient developers like me!
In fact, this book covers all relevant topics I can imagine: it starts with an introduction and a description of Geronimos architecture, which gives an brief (sometimes a bit short) overview of how Geronimo works with its plugins and its deployment strategy. I would have loved to read a bit more about the “hot deployment” feature of Geronimo. In JBoss world, this has brought me some headache. In this book it’s only covered with roundabout one page, just saying that it exists, were you need to place your files and how you can monitor that activity. Even later it’s not mentioned in special. Well, but that’s ok for a Quick Reference.
After this, one gets in touch with the most important knowledge of JMS, Database connectivity and JPA. Then there is an extraordinary good chapter about Security. Its one of the biggest chapters in this book and one can feel quite well how expierenced these guys are. Topics are handled in some kind of How-To way, like “Creating a new keystore” or “Changing a private key password”. These guys know what they speak about, probably the best chapter in this book.
Then it comes to CORBA, which I found also interesting, but to short in general. JNDI was reduced to the most important “put your hands there” information. Then it comes to Geronimo Plugins. This one was very interesting too and I wished to read more about it, but well, again, it’s short reference. You can find much Listings in this chapter, but I cannot say that they helped me too much without digging in the containers documentation.
In the administration chapter, the authors show the different portlets Geronimo provides for monitoring actions. It’s basically a walk through of the different pages of the Geronimo console. I think, this chapter could have been improved much more. For example, the Thread Pools Portlet is described with just one sentence: “…lists the thread pools defined in the server, and lets you monitor the thread pools.”. Ok, I don’t need a book for that information. Some other portlets are described a bit more in detail, but nothing which makes me to a Geronimo Guru.
Later you will have a How-To use the Geronimo Eclipse Plugin. It contains lots of screenshots which helps you to create a project specifically for the Geronimo. More impressive was the Cluster chapter. Clustering is a difficult topic and I would use this section for doing the job. It’s a good mix between explanation and reference.
Last chapter I was eager to see was the one with the Geronimo Internals. At some parts it reads like a smaller version of an API. It should help to develop own GBeans and it does, but not so much that I would have a huge benefit compared to the docs.
Final words: this one is really a reference and not a teaching book. If you would like to buy it, you should have some knowledge about JEE and about Containers in general. This book will not help you to understand the technologies behind. It’s more a collection of How-Tos, and that is what I expected from a Quick Reference. The authors made this point clear on the cover, were they are stating that you need to know about JEE5 concepts. At some points I would have preferred some more information. Sometimes I would have wished that they put not so much unnecessary listings (I don’t need import statements in java listing nor do I need XML-Comments in a 10 line XML file). I think, if you are developing an application, this book is a nice to have. If you are more an administrator and need to develop, package or cluster Geronimo, this book will probably give you some benefits. Especially the security chapter is well done.
However, thanks for this book, I like it, but it’s not one of my all-time faves.
Tags: Apache, Bookreview, Geronimo, Java Posted in Books, Open Source, Software Development | No Comments »
Wednesday, November 18th, 2009
The last weeks I struggled with JavaScript. I hated it quite soon, but with knowledge came some comfort and finally I am now able to use jQuery without too much pain. Let’s hope that I don’t have requirements which need a deep knowledge of that stuff However, that brought me to the idea (as odd as it sounds) to try out some server side javascript. It was exactly the right time when Packt Pub. contacted me about their new book on Apache Geronimo.
I wanted to step into Geronimo before a few years, but JBoss, Weblogic and even Websphere were used on my projects. No time for playing with Geronimo. Well, but with this book I will start finally and check out whats possible with this little beast
From the table of contents, server side javascript seems not to be on the list. But thats not to bad – I think I have a new idea for a blog post.
However, I am looking forward to this book. Packt already informed that it’s on the way to me. So far a very good impression of this publisher. Polite people, quick delivery. Let’s see how good the book is.
Tags: Geronimo, Java Posted in Books, Open Source, Software Development | No Comments »
Friday, August 21st, 2009
This is not too easy. At Log4PHP we have exactly that problem right now. Somebody is using the FileAppender and figured out, that one Apache process was waiting looong time before it could write. Reason: the logger locked the Logfile for the whole time of the request. If you have lots of requests, you can think what it means. Performance is past, in the case.
Time for me to think about the different options to write to log files.
I figured out, that I have to compare the following options:
1) Not closing the file while the whole request is running. This is not an option in a live system, but will give me a good idea whats currently the case
$fp = fopen($file, 'a+');
while($count < $loop) {
fwrite($fp, $text);
}
fclose($fp);
2) Closing the file directly after fwrite is called
while($count < $loop) {
$fp = fopen($file, 'a+');
fwrite($fp, $text);
fclose($fp);
}
3) Use file_put_contents, which is known as an alias to fopen, fwrite and fclose
while($count < $loop) {
file_put_contents($file, $text, FILE_APPEND);
}
4) Leave the file open while the whole request, but unlock it with flock and flock it again, when the next log event occurs
$fp = fopen($file, 'a+');
flock($fp, LOCK_UN);
while($count < $loop) {
if (flock($fp, LOCK_EX)) {
fwrite($fp, $text);
}
flock($fp, LOCK_UN);
}
fclose($fp);
5) Use a nonblocking stream for this and flock
$fp = fopen($file, 'a+');
stream_set_blocking($fp, 0);
while($count < $loop) {
if (flock($fp, LOCK_EX)) {
fwrite($fp, $text);
}
flock($fp, LOCK_UN);
}
fclose($fp);
6) Use the error_log method, which my friend Kevin Horst brought up
while($count < $loop) {
error_log($text, 3, $file);
}
For each of this options I wrote a simple function which wrote 10000 times 100 characters in a freshly created log file. I measured before opening and after closing. Additionally I tried out with 2 seperated threads if the write access is nonblocking. Good thing is, option 2 to 6 are actually nonblocking. And here are the timing results:
1) Execution with NOT closing the log file took 0.0668561458588 seconds
2) Execution with CLOSING the log after each write file took 30.1630220413 seconds
3) Execution with file_put_content took 30.153963089 seconds
4) Execution with leaving the file open, but LOCKING and UNLOCKING it took 0.148998975754 seconds
5) Execution with nonblocking stream took 0.149605989456 seconds
6) Execution with the error_log method took 30.069578886 seconds
Let’s see what it means.
Not closing the file until the 10000 fwrite calls are handled as actually the fastest. No surprise. It just took 0.0668 seconds but this one is not really an option, cause other threads have to wait until this request has been finished.
If you close the file after each fwrite and make it available to other threads and then reopen it, is hell. For 10000 calls of this kind I needed 30 seconds! It’s insane to have this in a productive system. The same goes with file_put_contents. Well, I allready knew (its in the php docs) that this method is nothing else then a wrapper for fopen, fwrite and fclose. Times are so similar that I say it’s exactly the same. Sometimes the one is some millis faster, sometimes the other.
If you open the file, unlock it with flock and flock it again it works very well. Just 0.14 ms for the 10000 fwrite calls. Thats the double amount of option 1, but yeah, here we do some more stuff. The interpreter cares about who is allowed to write, together with the OS. flock works that way, that a call to this function blocks until the requestor gets the actual lock. You can be sure that only one thread is actually writing.
Same goes to the nonblocking stream. This works with stream_set_blocking($fp, 0);. The file stream is nonblocking, means each thread could write at it the same time. That no mess happens, we need an flock here too. That brings us to the nearly same results as fopen, flock, fwrite, flock, fclose option above. But looking at the logfiles of this one and option 4, this one looks more nice to me. This is just subjective, but it looks like the lock is shared more nicely between both threads.
Last one is the error_log method. It didn’t had any idea what to expect, but… 30 seconds! This one behaviours like a wrapper for fopen, fwrite and fclose, like file_put_content. No guys, this is not really a method enabled for logging! If one would use this in a framework like Log4PHP, that would be hell to performance. I would think that this should better removed out. The name suggests a good logging method, but this is not the case.
Having that all said, Log4PHP will get the lock and unlock option number 4. I feel good with it, since it’s quite straightforward. I don’t have too much expierence with the non blocking stream and don’t want to have this in a framework like log4php is.
However, Logging must be used carefully at all. I thnk on a system with 10000 request a second. Enabling logging into one file could bring the system down. I think a live system should have the option to log exactly one request. Maybe triggered by an url param. Think carefully what you log and how you configure your live system.
The complete script can be found here.
Tags: Apache Log4PHP, PHP Posted in Apache Log4PHP, Open Source, Software Development | 12 Comments »
Tuesday, August 4th, 2009
Well, the term finished is a bit overrated. But I wrote a Wordpress plugin which sends a message to your Adove Wave feed, after you have published a blog post. The message will contain the title of your post as text and is linked the latest blog post.
Installation is quite simple: just download the Wordpress plugin, unpack it, and upload it via FTP to your blog. You need to store it in the wp-content/plugins folder. Then log into you wordpress installation, choose “Plugins” and activate the “Christian Grobmeier Wave Plugin”. Sorry for the name dudes, but Wordpress usually needs something unique and I hadn’t had a name which sounds more cool. Well, you have to deal with it.
Once that is done you need to configure your plugin. This can be done in “Settings / Grobmeier Wave Plugin”. Please provide the topic you want to post to, your wave Username and your Wave password. If you don’t have such things, you’ll probably need a Adobe Publisher Account.
That’s all for the plugin. Once you finished the steps above, Wordpress will send notifications to your users when ever you post something.
If you come into unexpected trouble, then please check if you are running Wordpress with PHP5. I figured out this morning that my installation was running on PHP4 for a long time… this brought up several syntax exceptions. To be clear: you’ll need PHP5 to run this plugin.
Last thing beeing said: this is licensed with Apache License 2.0. No guarantees, use at own risk.
Your feedback is appreciated. If you have any more questions, drop me some lines too.
Tags: Adobe Wave, PHP, Wordpress Posted in Open Source, Software Development | 14 Comments »
Monday, August 3rd, 2009
For a while Adobe catched my attention with their new product Adobe Wave. It’s basically Growl, but for websites. Means one can subscribe and a website publisher can notify you if some update happens. I realized that I like AIR, the enviroment of the Wave client. Looking at the examples I put together a simple Wave script, which lets you publish news on your feed.
I think I will use a similar script for pushing stuff with Log4PHP. I will propose that today on the mailinglist. Additionally I think about making a WordPress plugin for my own blog. We’ll see how fast I am
Your comments are appreciated!
Here is the wave script. Its released under Apache SL 2.0. Credits to the Wave team, I based everything on their examples.
Tags: Adobe Wave, Apache Log4PHP, PHP, Software Posted in Apache Log4PHP, Open Source, Software Development | No Comments »
Sunday, July 5th, 2009
You know I am working on several open source projects, and lately one of my comrades told me I should stop coding asap and:”please correct your client configs I don’t want to fix the stuff anymore.”. This shout made me think about my bad behaviour and to be honest, I didn’t know why I should do this and how.
After some small googling, I figured out that the most cool feature of the SVN properties is to ensure that if you check out something from repository, you’ll get the line endings you need.
“In other words, if a user on a Windows machine checks out a working copy that contains a file with an svn:eol-style property set to native, that file will contain CRLF EOL markers. A Unix user checking out a working copy which contains the same file will see LF EOL markers in his copy of the file.”
Quote from: http://svnbook.red-bean.com/en/1.1/ch07s02.html#svn-ch-7-sect-2.3.5
So, this is client config. That means YOU have to make sure that you add your files with the correct properties. Let’s assume you want to add a file named test.txt. You would do:
svn add test.txt
After adding you need to perform:
svn propset -R svn:eol-style native test.txt
to set the properties correctly.
Having that beeing said. most people will complain now that this is a difficult task and very error proven. Well it is. But you don’t have to take care if you are sure that only one operating systems (means: one development enviroment) is set up. This is true for most projects I guess.
If it is NOT the case, you can force your developers to create a default config setup. On windows it just some entries in the registry (please try this, I am not using windows for open source stuff anymore), on Mac OS X and Linux the file ~/.subversion/config must be edited.
In my case, this did do the trick:
[miscellany]
enable-auto-props = yes
[auto-props]
*.java = svn:mime-type=text/plain;svn:eol-style=native
*.css = svn:mime-type=text/plain;svn:eol-style=native
or at least mostly. SVN told me, after I wanted to change properties to native, that my css files were binary, which this wasn’t the case, of course! I couldn’t change it, until Sebastian Bazley came up with the following:
svn pl -v
showed:
Properties on 'maven.css':
svn:mime-type
application/octet-stream
He further told me to do:
svn pd svn:mime-type maven.css
And that was it – SVN is forced to use mime type I want.
Last but not least, here is the link to the apache recommendation or SVN config.
Tags: Apache, Java, OS X, Software Posted in Open Source, Software Development | No Comments »
Wednesday, May 27th, 2009
Unit Testing with Junit and PHPUnit
All the worlds speaks about testdriven development and unit testing. But most people finishing there exams don’t have a clue about what this means. To be honest, I was surprised that people could get out of a university just with one or two Junit tests implemented. Well, here are some words about testing.
What is a unit test?
A unit test is nothing else than some lines of test code which tries to check out if the lines of business logic you wrote actually works. To help you with this, tools like JUnit and PHPUnit (or SimpleTest in PHP enviroment) have been written. They are frameworks which give you some methods to work with. To be clear: a unit test is some kind of class which instantiates another class and checks if the parameters you put in bring the correct results.
What does testdriven development mean?
Hardcore people say, before you write the business logic, you’ll have to write a test case. This is test driven development: write your test, then write the actual business logic. For me it works usually like this: I write my business logic and as soon as I have something cool working, I write a testcase. Sometimes it behaves a bit different. Depends on. But thats a matter of taste. As usual I recommend not to be to extreme about everything and just try out what is best for you.
What and how much to test?
Everything. Well, not really. A good testing ratio is at 70%. More does mean that you even test your exceptions, less means you have forgotten some classes. There are tools available which help you with checking out how much you have tested. In Javaworld its Cobertura, in PHP its Spike PHP Testcoverage, for example.
How does a testcase look like?
Usually you have to extend your tester class from a class called TestCase or similar. In PHPUnit its PHPUnit_Framework_TestCase. They make your classes executable in the testing enviroment. Then implement methods – all methods which implement an actual test is prefixed with test.
An example from Log4PHP:
class LoggerLayoutHtmlTest extends PHPUnit_Framework_TestCase {
public function testErrorLayout() {
The testing frameworks usually look for methods like this and execute them one by one. In newer JUnit Versions you have to annotate testing methods. OK, and this is a valid PHP testcase:
public function testErrorLayout() {
$event = new LoggerLoggingEvent("LoggerLayoutHtmlTest", new Logger("TEST"), LoggerLevel::getLevelError(), "testmessage");
$layout = new LoggerLayoutHtml();
$v = $layout->format($event);
$e = "blub";
self::assertEquals($v, $e);
}
Again, its Log4PHP. I instantiate a class which I am intending to test and call a method on it. The result is stored in $v. What I expect is stored in $e. All testing frameworks provide you so called assert methods, which enable you to compare or check otherwise if the expectation meets the actual result. In my case, “blub” is expected. If this assertion fails, my tool shows me that error.
Why is it necessary to keep old tests?
If you change business logic or refactor something you can simply execute old tests and recognize if something goes wrong. People say, before putting an echo or System.out.prinln somewhere, write an test. I agree here. A good testing ratio makes your software stable and you take care of side effects, even when software grows. And before you deliver you run all your tests. This way you make sure that everything is fine.
Test Data and dependend Tests
To make it short, test methods should never depend on the success of test methods. Each test method can be executed stand alone. Otherwise your life will be hell – think on thousand methods depending on each other. Same goes to test data. A testcase must not depend on data another test method created. This will cause you hell.
I know that this is very difficult with databases. Testframeworks usually give you “tearUp” or “tearDown” methods which are called before a test method starts. You can create your testdata in the databases in these methods. But this is very time consuming. Test executions of this kind can easily take hours. Best is, keep your tables short, make a SQL file for each package or even each test case. There are no best practices which fits on each project.
In my last project we have used Excel (=Manager aware) to take care of the data and then generated SQL files out of it. Time consuming, complex, but it worked. The relations in this tables where to strong, we couldn’t keep the data care up otherwise.
Testing helps coding
Having said the above, you can imagine that you should think about testability BEFORE you code. Spaghetthi code isn’t testable. Make short methods. Keep in mind what “Separation of Concerns” means. Same goes to databases. Make them plugable. Not at all time foreign keys are good. Think about it twice – they are mighty but can be evil too. I have worked in projects where it was very heavy duty or even impossible to delete any data. If you write your application testable, its going to be good designed in much cases.
This doesn’t mean you should put all methods to public scope! Test cases should be in the same package as their Test classes are, so you can try to work with package scoped methods aswell. However, whatever you do, testable code is good, but it never should break encapsulation just for beeing testable.
In most cases private methods can be tested with the test of public methods. Test coverage tools help to identify test lacks.
What to test? Again!
Try to test all public methods! If you need to create files, use the tempfolder. Set up databases for the test. Use Jetty and HSQL if need it embedded – in PHP its PDO and SQLite. Don’t waste your time with testing exceptions which usually never happen. Just use 20% of your time to make 80% of tests! Try to test business logic – if that means the creation of complex objects, so be it. Maybe you can share the logic. But don’t get bored with testing getter and setters. Those are called thousand of times within normal tests or procedures, no need to do that.
How much time can test development need??
Calculate the same time as implemeting the testcase. Too much? Not really! Imagine you are writing business code for 5 hours. Thousands of outputs on the console you read manually. You delete it, check it in and later want to fix again. Put those outputs in again? Have commented it out? Code is ugly? Put all your stuff in a testcase. Nobody claims if you need 5 hours for it, if its done properly. Sometimes testcases need 3 days but the fix just 1 hour. This can happen in EAI enviroments, where multiple systems communicate. No problem here – these is business critical, automatic tests are the best you can do.
Tags: Java, PHP, Software Posted in Apache Log4PHP, Software Development | No Comments »
Friday, May 22nd, 2009
The Commons Compress team is pleased to announce the commons-compress-1.0 release!
Commons Compress is a component that contains Ar, Cpio, Jar, Tar, Zip and BZip2 packages
Source and binary distributions are available for download from the Apache Commons download site:
http://commons.apache.org/compress/download_compress.cgi
When downloading, please verify signatures using the KEYS file available at the above location when downloading the release.
For more information on Apache Commons Compress, visit the Commons Compress home page:
http://commons.apache.org/compress/
Changes in this version include:
New features:
o Initial release
Have fun!
-Commons Compress team
Tags: Commons Compress, Java, Software Posted in Commons Compress, Open Source, Software Development | No Comments »
Sunday, May 10th, 2009
On my Mac Mini box running with OS X 10.4 aka Tiger I had once a Lacie harddisk mounted with the name ANDREW. I plugged it off and connected later another one under the same name. Unfortunatly the mountpoint of this disk wasn’t:
/Volumes/ANDREW
as expected, but rather:
/Volumes/ANDREW 1
which gave me hell when running my Junit Test for Apache Commons Compress due to the space in the harddisks name. I was wondering why OS X created the name ANDREW 1. Looking with my bash into /Volumes i recognized that there already was a folder with the name ANDREW. This happened cause when I removed the disk i started eclipse and the IDE couldn’t find a specific folder. Instead of telling me that, Eclipse simply created the folder. This could happen since the /Volumes folder has write access to everybody on OS X.
I deleted the unnecessary folder. After that I disconnected and reconnected my harddisk named ANDREW. The mountpoint was correct again and life is going normal
Tags: OS X Posted in Software Development | No Comments »
|