Welcome to my blog and the Surfsoft Consulting web site. Feel free to have a look around, read the blog, leave comments, and look through the articles here and in the wiki. For more information about me and the company, see the about page.
Welcome to Surfsoft Consulting
June 20th, 2009Build Your Software on Firm Foundations…
March 23rd, 2009
What makes for a successful software development project? I was mulling over a number of projects I had been involved with in the past, some very successful, some not so, and thinking about the key differences between the two. There are any number of aspects that can be considered – the quality of project management, clearness of requirements, thoroughness of the analysis, quality of execution and technical leadership to name but a few. Here I am going to focus on two key technical aspects, not being a career project manager, and explain why they are both important and also both frequently neglected.
From my experience there are two main aspects to a technically successful software development project, the Software Architecture, and the Software Engineering Process (I am taking it as read that your teams have the basic capabilities to develop software using your chosen technical platform, which in my case is Java).
Hidden Files in Mac OS X
December 1st, 2008Having finally got around to uploading the original rss4jsp projects into SourceForge this evening I found myself with a problem – I was uploading all sorts of stuff that was not applicable. For example, Apple creates a .DS_Store file in every folder viewed by the Finder, to store information about how the folder looks in the Finder. Not essential to rss4java. For that matter, neither did I want to add my Eclipse files, but in they went…
A quick Google led me to Shane Duffy’s blog post on showing hidden files (in Unix this is anything that starts with a full stop). However this did not show folders (like the Eclipse .metadata folder). This turned out to be even easier than using Google – I just tried the obvious extrapolation.
rss4jsp: A Tag Library for Project Rome
November 24th, 2008One of the features being introduced to Fantasy-F1.Net in the next few weeks is an RSS Feeds page. During the closed season it is useful to have alternative sources of material to keep people coming back to the site which would otherwise be all but dormant. The RSS Feeds page will provide access to material without the user leaving the site, and with new material being indexed by the search engines, a wider range of search terms have the potential to drive traffic to the site.
The first part of this exercise was originally to identify a suitable tag library to support this functionality. The site already uses the Project Rome library to generate RSS feeds from the site but a search quickly showed that there were no libraries available that either already performed the tasks required, or that could be extended to do so.
The rssutils project from Rodrigo Oliveira provided a useful introduction to doing some of what was required, but the code was old and unfortunately flawed, so the decision was taken to develop a tag library, in a similar vein to the rssutils project, but on top of Project Rome. Having written an initial set of tags the idea to share them for the benefit of others quickly followed and the rss4jsp project was born.
Having perfected the basics of a library I have written a tutorial here and set up the rss4jsp project on SourceForge.net. The distributions (there are two, the library itself and a samples distribution) are now available for download, and the source code (also included in the library donwload) will be uploaded in the next couple of evenings. As I think of ways to extend the library I will add feature requests to the project. Please read the tutorial, try the library, and give me feedback, it will all be appreciated.
rss4jsp is open source software licenced under the Apache Licence, Version 2.0
Looking Forward to JavaFX
November 14th, 2008Sun introduced JavaFX to the world at JavaOne 2007, touting it as a new way to build rich internet applications, and putting it head-to-head with Microsoft Silverlight and Adobe Flex. Coming to the party last, can Sun produce a winner and make lasting inroads into the rich internet applications space?
Apple TV: Nice hardware, shame about the limitations
October 22nd, 2008Recent weeks have seen me busy spinning, i.e. achieving very little on the technology front outside of work. A nasty cold and related sinus issues has kept me away from computer screens as much as I can get away with but now at last the problems are receding and I’m starting to get traction again.
As somebody who tends to fall into the trap of over-committing themselves I have a number of projects in a variety of states…
My Formula One site is approaching the winter maintenance window where I’d like to implement OpenId support. Which first requires me to pick up Maven properly. That all needs finishing by the end of the year. At the same time I need to move the site to new hardware provided by me, but before I can do that I need to sort out how to configure the DNS and mail server.
I’ve recently started working on another project which is called the Java Library Search Project. This is my first collaborative effort and I’ll be writing more about that in the near future. To save cost that project is being hosted on a Mac Mini via my home ADSL connection and so far I’ve got the domain sorted, the wiki up and running, subversion installed though not, yet, as a service and again I have email to sort out. And thats before we start any real work on the project itself.
Now add in one or two other real back burner projects relating to consolidating my external hosting into one account with one provider and you can see I have enough to keep me busy for some time to come. Obviously this is the time to start yet another project. Step forth Apple TV.
When the Apple TV was launched I liked what I saw. A quiet, slim box that will sit discretely under the TV and provide access to all my media. Except of course there are a few teeny weeny restrictions. The box only supports a couple of video formats (H264, MPEG-4) while almost all of my video is MPEG2 or DivX. Oh, and to access material not already on the device’s hard drive requires iTunes to be running on the machine that holds the media. All of my audio and video is on a NAS device and I don’t want to run another machine just to get access to it.
The Apple TV is a good starting point for a home media centre because it is in essence a small, low power, Intel based PC but as always in the Apple world, there is a tie-in to iTunes thereby providing Apple with a revenue stream once you realise that all video has to come from them, or be converted first.
Under the covers the Apple TV is simply an underpowered Mac OS X and a specialised user interface, and this has given hackers a way in to ‘customise’ the device (which will, of course, void the warranty). However it is a relatively simple process to provide remote SSH access and from there move on to install additional CODECs and indeed other software to extend the functionality. Apple TV comes with either a 40Gb or 160Gb hard drive and it is possible to change those as well…
So I’ve bought a nearly new 40Gb Apple TV on eBay and after cloning the hard drive (I want to keep the original drive in its unmodified form) I will crack it and then upgrade it to be the machine it always should have been. About the only thing it won’t be able to do is record. The things I am currently considering:
- Adding a range of additional CODECs so that I can play non-MPEG4 format videos
- Adding support for accessing media across the network without the need for iTunes to intervene
- Changing the CPU, if feasible, to something with a little more grunt so that I can…
- Run background processes to perform other useful tasks like downloading podcasts and other media resources from the internet.
I’ve already found one good source for achieving at least some of theses goals at appletvhacks.net.
My delivery is due any day, and I’ll post updates as my customisation progresses.
Effective Java Second Edition
September 16th, 2008Apologies for the large gap – work has been busy and I’ve been on holiday too, as well as setting up my company web site. I now find myself with time to post once more.
I recently picked up a copy of the second edition of Effective Java having owned a copy of the first edition for several years. Its a book I have always recommended to anybody new to Java because whereas many books on Java discuss the letter of the language, this book discusses the spirit, and puts it into clear context, leaving the reader in no doubt of (a) how to use particular language features, but (b) where and why to do so.
Having worked with some Java 5 features for the last couple of years I have picked up autoboxing, generics and enums in a casual way. Taking the new edition on holiday proved a boon when the whole family fell ill – I had the chance to spend a couple of days reading in. Very quickly I learnt more about the right way to use the Java language, partly because I never read the original from cover to cover, but referred to it when I was puzzling over an implementation detail. Mostly, though, because the book is a very effective training course on the new areas of the Java language.
I had been picking up the new features around generics – converting Java 1.4 code to Java 1.5 had left warnings that I had no urgency to address – but having read about half of the new edition I have been able to quickly eradicate most of the outstanding warnings in my code, either by fully utilising generics, or through annotations to suppress the warnings that occur when using legacy code from Java 5/6 code (not the right approach, but it will suffice for now).
Mr Bloch has done an excellent job of rewriting his book and whether you have a copy of the first edition or not, I strongly recommend you purchase a copy of the second edition if you are serious about producing the best quality Java code you can.
Method Contracts! Method Contracts! Method Contracts!
July 14th, 2008I recently spent a few days back-filling test code to help a team meet one of its its quality objectives – namely unit test coverage – and the scany level of documentation within the code re-enforced my views on properly documenting the contract of a method. The methods I wrote tests for had the minimum of JavaDoc, enough to ensure no warnings from Eclipse, but nothing beyond that. So I had an idea of what the method was supposed to do, and roughly what the arguments were for – but no indication if null was a valid value for a given argument, or what would happen if an argument value was non-null but invalid.
To write tests for the method contract I had to pick through the underlying code – far from ideal, as I’m now inferring the contract from the code, and the code may not be correct. And this particular organisation hadn’t adopted the de facto approach to validating method arguments as documented in the excellent Effective Java (recently re-issued for Java 6) by Josh Bloch.
Briefly, the method contract consists of two component parts: the explicit contract – the method name, arguments and their types, and the implicit contract – which arguments can be null, the valid range of values for a numeric argument, etc. The degree of documentation for a method and its contract should be proportional to its complexity and the level of exposure the method is likely to receive – and it doesn’t get much more exposed than a component interface, so these should be very well documented indeed. The objective is to leave no ‘wriggle room’ in the specification – either for those developing an implementation or those who might want to use it. Or indeed, test it.
By providing thorough documentation for a method you are in effect crystalising the purpose of that method. As a developer it clarifies what you are trying to achieve; for a user of the method it helps them to understand what the method does and how to use it correctly. It should also tell them what error conditions exist and therefore allow them to be catered for.
This may seem overkill for a one or two man project, and you may be justified in short-circuiting the process to an extent. However it is vital in larger projects, and putting the documentation with the method ensures that future maintainers and users of the code do not have to hunt around for supplementary documentation. Or the original author, to explain what is going on. And even for a one man project, coming back to code cold after, say, 18 months, you might just find that the documentation refreshes your memory…
Briefly, the method contract consists of two component parts: the explicit contract – the method name, arguments and their types, and the implicit contract – which arguments can be null, the valid range of values for a numeric argument, etc. The degree of documentation for a method and its contract should be proportional to its complexity and the level of exposure the method is likely to receive – and it doesn’t get much more exposed than a component interface, so these should be very well documented indeed. The objective is to leave no ‘wriggle room’ in the specification – either for those developing an implementation or those who might want to use it. Or indeed, test it.
By providing thorough documentation for a method you are in effect crystalising the purpose of that method. As a developer it clarifies what you are trying to achieve; for a user of the method it helps them to understand what the method does and how to use it correctly. It should also tell them what error conditions exist and therefore allow them to be catered for.
Configuring Tomcat for Multiple Instances
July 7th, 2008This is the second of three posting discussing how to configure Tomcat, on Max OS X. This and the third post deal with working in a hosting environment. I am building a Mac Mini to act as a hosting solution for several web sites, at least two of which will be Tomcat based. The sites should have independent configurations so that they can be started, stopped and configured separately.
This isn’t a big overhead, my most popular site has about 2,000 active users (out of 8,000 registered) with 50% visiting every month and runs happily with a Tomcat heap size of 64Mb for all but the most busy times. So on a Mac Mini already running Apache HTTP server and MySql 5 there is 400Mb of 1Gb free – enough for several Tomcat instances.
In the previous post I installed Tomcat; now I am going to document a step-by-step process that will get you into a situation where you have one or more Tomcat applications with their own dedicated cofiguration files, log and temp directories, ready to be installed as OS services.
I’ve made it a step-by-step tutorial to ensure that anyone who uses this can understand the changes that have been made, thereby learning more about Tomcat along the way. I could present some configuration files and a short set of instructions, but problems will be hard to diagnose and fix if you are unfamiliar with Tomcat. Better instead to introduce and test incremental changes. Right, lets get started.
I’ll start by discussing the directory structure for the applications. All my Apache and Tomcat hosting files will be stored in /Hosting. an Apache subdirectory will hold all the Apache based web sites while a Tomcat subdirectory will hold all the Tomcat based web sites.
Inside /Hosting/Tomcat I will create a directory for each application, containing the web application, configuration, log files and Catalina temporary directory. For sake of example I’m going to call my Tomcat application/site phils-hosting.com, which means that all the files relating to this site will be in /Hosting/Tomcat/phils-hosting.com.
OK, enough about structure, lets configure Tomcat to run the application in /Hosting/Tomcat/phils-hosting.com/webapp.
First the application. I run mine unpacked – the application has the ability to upload files – images – so I just unpack my war file. This also makes it simpler to patch the application when live – for example fixing a defect in a JSP page.
Start by creating your directory structure (at this stage we are keeping all the file ownerships as ‘root’ – we’ll change that later):
sudo mkdir /Hosting
cd /Hosting
sudo mkdir Tomcat
cd Tomcat
sudo mkdir phils-hosting.com
cd phils-hosting.com
sudo mkdir webapp
cd webapp
Now copy your war file into the webapp directory and then use tar to unpack it:
sudo cp
sudo tar -xvf
Perform any other application-specific configuration you need – for example setting up a schema in MySql and now you are ready to run your application with Tomcat in its default configuration.
Now to get the application running. We’ll start simple by just getting the application running inside the default Tomcat installation. Edit /usr/local/apache-tomcat-5.5.26/conf/server.xml (make the conf directory and server.xml files writable first, and back up server.xml). I actually rename server.xml and copy server-minimal.xml which contains all you need.
Find the line
<Host name=”localhost”
appBase=”/Hosting/Tomcat/phils-hosting.com”>
<Context path=”/”
docBase=”webapp”
debug=”0″
reloadable=”true”/>
<Host>
Now start the Tomcat server manually. Change directory to /usr/local/apache-tomcat-5.5.26/bin and run startup.sh:
sudo ./startup.sh
At this stage we’re still running as root, and using the configuration and log directories in /usr/local/apache-tomcat-5.5.26. Now open a browser and enter the URL http://localhost:8080 – and if all is well you will see the home page of your application. Note that there is no application name in the URL – we are only going to run one application per Tomcat instance, so the context path for the application is “/”.
If you have any problems then double-check your server.xml file and have a look in the log files in the logs directory for any useful messages. The catalina.log file will have an error message in it if the application failed to start; if it started but isn’t working properly, try the localhost log file for the current date.
OK, having got the application working we will now migrate the configuration and log files into our hosting area so that it is ready to be run stand-alone.
Stop Tomcat – change directory to /usr/local/apache-tomcat-5.5.26/bin and run shutdown.sh:
sudo ./shutdown.sh
Now in the /usr/local/apache-tomcat-5.5.26 directory, copy the configuration files into the /Hosting/Tomcat application directory:
cp conf /Hosting/Tomcat/phils-hosting.com/conf
You can trim back the files in he new ‘conf’ directory quite considerably. We also need to create the ‘logs’ and ‘temp’ folders:
cd /Hosting/Tomcat/phils-hosting.com
sudo mkdir logs
sudo mkdir temp
Now to tell Tomcat where to find its configuration we use two Tomcat environment settings – CATALINA_HOME, which points to the original Tomcat installation, and CATALINA_BASE, which points to /Hosting/Tomcat/phils-hosting.com. When Tomcat starts, anything it can’t find in CATALINA_BASE will be picked up from CATALINA_HOME:
export CATALINA_HOME=/usr/local/apache-tomcat-5.5.26
export CATALINA_BASE=/Hosting/Tomcat/phils-hosting.com
CATALINA_HOME should be a system-wide setting defined on syste start-up, while CATALINA_BASE will be configured on a per-application basis.
Remove the log files from /usr/local/apache-tomcat-5.5.26/logs, and rename /usr/local/apache-tomcat-5.5.26/conf/server.xml so that it can’t be picked up in error. Now restart Tomcat as above, and you should observe two things. First, your application is running (check the browser) and second, your log files are now in /Hosting/Tomcat/phils-hosting.com/logs. If they are not, check your environment settings.
In the next post I will explain how to configure multiple Tomcat instances, how to run them under a non-privileged user, and how to ensure your Tomcat instances start up when your machine boots.
Hosting on a Mac Mini
July 7th, 2008One of the problems of running several web sites is that of hosting costs. As long as you stick to LAMP (Linux, Apache, Mysql and PHP) the costs can be quite reasonable, but as soon as you want to do things like (legitimate) mass-mailing to your users, or run a Tomcat server, the costs start to escalate.
Specifically, I pay about £60 a quarter for my Tomcat based site, £15 a quarter for my osCommerce based site, and another £50 a year for an admittedly rather expensive ‘email only’ service for a third domain. I have a further three domains that I’d like to start using, but for the cost. Currently I pay out £350 a year (it would be more, but I’ve saved money by using some US hosts) and it struck me that for £400 I could buy a Mac Mini and do all my own hosting. Any site that got too big/popular could be migrated back to a commercial hosting service fairly easily, and in the meantime my hosting costs would fall to a fraction of what they are now.
The decision was made for me when my Tomcat hosting company told me I could no longer send large quantities of email once a fortnight. To do that I’d need to rent an entire physical server, something that would cost me at least £750 a year.
Hosting is not for the faint hearted, but provided you follow a disciplined and thorough approach it is possible! When I thought about it seriously here is a short list of the main issues:
- Configuring Apache to run multiple domains.
- Running multiple copies of Tomcat, and routing HTTP traffic via the Apache HTTP server.
- Providing remote administration capabilities – FTP and SSH as a minimum, but ideally using some kind of web based admin package (you never know when you’ll need to restart a Tomcat server with no direct access to the server).
- Configuring a mail server that is capable of serving mail for all the domains.
- Installing and configuring a database and associated web based administration.
What do you need to host? Well in very simple terms, hosting for yourself works best if you can dedicate an IP address just to hosting – and make sure that your ISP terms and conditions allow you to host. Yes, you can host with a dynamic IP address but but this isn’t a sound solution if you need your sites to be available 24×7.
You need a server too. The Mac Mini appeared made for the job for several reasons:
- It is very small, quiet and cool running.
- It consumes only 110W even when fully loaded (that’s 19KWh a week), a regular PC consumes at least twice that at idle.
- The OS provides many of the features I need already installed.
If you don’t take the Mac Mini route then any PC with a flavour of Unix is equally as good – but don’t underestimate power consumption (typically 42KWh a week). I don’t consider Windows a good base for hosting when I’m not using any .Net capabilities, but you may prefer to go that way.
You should also add an external hard drive to automate backups to – Time Machine! Other things you might want/need to consider are:
- A cheap UPS to at least allow the server to shut down cleanly in the event of a power failure.
- A dedicated keyboard, mouse and monitor for maintenance purposes.
- Additional networking hardware to allow your home network to sit on the other side of a firewall to your server.
Now the Mac Mini provides much of the required functionality for free – Apache 2.x server pre-installed with lots of modules, Firewall, FTP and SSH supported out of the box together with remote management tools and built-in backup solution (Time Machine). To this I need to add Tomcat, MySQL and a mail server.
I thought it would be a useful exercise to blog the set-up of the machine as a full-blown web server in some detail – first it means I have reference material to go back to if I need to change or fix something, and second it means that other people can follow these steps should they decide to try it for themselves.
I shall (or in some cases, already have) produce posts on the following aspects of the configuration:
- MySQL Server Installation – documented here.
- Tomcat Installation and Configuration – part 1 (basic installation), part 2 (multiple independent configurations) and part 3 (running as a service).
- Apache Configuration for virtual domains and reverse proxy – to be documented.
- FTP, SSH, Firewall and Remote Admin – to be documented.
- Mail Server Installation and Configuration – to be documented.
- Remote administration facilities – to be documented.
Now I’ve not done the network set-up, because the server will not be in my house but a friends – I’ll try and get him to write a few words on the network set-up. Anyway, feel free to post questions and suggestions about the set-up and I’ll answer them as best I can. Check back regularly because I will be updating the links as I configure the box over the next few weeks.