Sorcerer's Tower

Welcome to the Tower, programming and web development blog. It consists of articles on software which I have released, as well as thoughts about technologies, techniques and ideas that I feel like writing about.

Most articles are currently CFML focused, but going forward there will be more written about general software development topics.

I have an aggregated blog which includes articles from my other blogs, along with the ability to filter particular combinations of tags.

(i) .)) O

[ all categories ]

Configuring Jetty for HTTPS with Let's Encrypt

The Jetty documentation for Configuring SSL/TLS is long and daunting, but makes no mention of how to work with the EFF's Let's Encrypt certificate authority, which provides free automated certificates with the aim of having the entire web available over HTTPS.

This article provides the steps for obtaining a Let's Encrypt certificate, importing it into Jetty, enabling HTTPS using the certificate, and handling renewals.

It assumes you have Jetty setup in a home/base configuration, serving over HTTP for one or more Internet-facing domain names.

As with all such guides, it is recommended to read all steps before making any changes, and ensure you have backups for any existing files you may modify.

Continue.

Getting the Original URL in Apache

There are various situations where one might want to know the full URL sent over HTTP by the user agent, before any rewriting has occurring.

Depending on the situation and setup, it can be as simple as using CGI variables such as path_info, redirect_url or request_uri, and within a JVM servlet getRequestUrl() may prove useful - but none of those are guaranteed to be the URL which Apache received, nor are any of Apache's other documented variables.

Fortunately there is a workaround, because one variable provided is the first line of the HTTP request, which contains the desired request URL nestled between the method and protocol, i.e: "GET /url HTTP/1.1" - meaning all that needs doing is to chop the ends off.

It is relatively simple to extract the URL, and at the same time provide it to later scripts, by using the RequestHeader directive from mod_headers to set and modify a header, like so:

RequestHeader set X-Original-URL "expr=%{THE_REQUEST}"
RequestHeader edit* X-Original-URL ^[A-Z]+\s|\sHTTP/1\.\d$ ""

The first line creates a header named X-Original-URL with the full value of the variable.

The second line performs a regex replace on the specified header, matching both the request method and its trailing space (^[A-Z]+\s) then the protocol plus its preceding space (\sHTTP/1\.\d$) and replacing with an empty string to leave just the URL.

The * after edit is what makes the replace occur multiple times - without it only the first match would be replaced. (i.e. the * is equivalent to a g/Global flag.)

The name X-Original-URL is used for compatibility with the equivalent header set by the IIS URL Rewrite Module - both that module and the above solution provide the full request URL, including query string, and encoded in whatever manner the user agent sent, but one difference is that the above config always sets the header, whilst the IIS version only sets it when the URL has been rewritten.

Why Test-Driven Development?

Test-Driven Development - TDD - is a great way to catch bugs before they go live, to ensure fixes stay fixed, and to prove the functionality of software. This stability alone is reason enough to ensure tests exist for as much of every application as is feasible, but is not the only benefit.


When you write the all the key tests for a piece of functionality in advance of writing the rest of the code, you are defining when that functionality will be complete, how much progress has been made, and specifically what tasks are next to be done.

For anyone practising Agile development, the parallels should be blatant. There is similarity in the benefits too, particularly with respect to keeping focused on the task being worked on and knowing when you're done and ready to move on.


With Agile development, each piece of functionality is broken down in a couple of ways - as business-focused acceptance criteria, and development-focused tasks, which is (in part) about making large pieces of work less daunting. Likewise, TDD provides a way for an otherwise overwhelming piece of work to be described step-by-step by someone who understands the matter at hand, and (with appropriate software structure and functionality) can then be divided up amongst multiple developers to increase overall velocity.

Incomplete functionality can be more easily passed from one developer to another, simplifying the explanations of what has been done and still needs to be done; the tests do the talking.


TDD means all newly written code has tests to describe what it should do, which helps to build trust in the application - a full set of passing tests proves a particular change has not altered the tested functionality, and allows you to deploy it with far more confidence than otherwise.

TDD means writing the tests happens first - before making the changes that the tests apply to, not afterwards. Writing tests afterwards still has advantages compared to no tests at all, but it does not guarantee that all code has tests - indeed, it makes it more likely that tests will be rushed or dropped if time is limited; precisely the situation when bugs are more likely to be introduced - and thus can give false confidence in what the result of a set of changes may be.


Test-Driven Development increases productivity. As with most things, TDD has a learning curve - it's not something a programmer will instantly be completely efficient with - but some programmers will still complain (erroneously) that writing automated tests slows them down.

It's obvious that writing tests is additional work compared to not doing so, but this additional upfront cost comes with the saving of not having to manually step through performing repetitive tasks with minor variations each time - each test is written once, and is executed easily each time the code progresses. When automated, tests are not forgotton - the full test suite can be run at any time to check for regressions or unexpected side-effects. Automated testing doesn't remove the need to use an application as a user, but it far reduces the time it takes to check that changes are working as intended, and helps reduce the space for human error.

Of course, practising TDD does mean that pesky bugs are identified at the time of development - not days, weeks, or months down the line - and this can also appear to make things slower; in reality identifying bugs sooner generally makes the issue easier to fix.


Finally, with TDD you know when you're done - when all the tests pass. Without tests written up-front, there may not be a clear finishing point. It's easy for developers to become side-tracked, or to unconciously introduce scope creep, and thus slow down the completion of a task which an appropriate set of test cases would have identified as already implemented.


This article hopefully explains how Test-Driven Development is not just about code quality, reducing application errors and increasing stability. It enables developers to work smarter and faster, give clearer progress updates, know what they're doing next and not become disheartened on bigger chunks of work.

TDD is a critical methodology which any competent professional developer should want to use.

Introduction to Password Security

Earlier this year I gave a talk on Password Security at Digital Croydon #5, and didn't want to simply put up the slides without any commentary, so I've written this article to accompany them. It is primarily for the benefit of those that attended - by which I mean it wont explain every slide or go into detail on everything the talk covered, but will explain the key points, provide links, etc.

Similarly, as mentioned in the talk, it didn't cover everything there is to know, and this article wont do that either - it's an introduction, not a comprehensive guide.

(At some point I do hope to write up more detailed articles, both on the points covered and on related subjects - if/when that happens I will add the relevant links to this article.)

Continue.

Introducing Lucee on Jetty

Lucee on Jetty bundles the Jetty server with the Lucee CFML engine, cleanly configured using the home/base functionality in Jetty 9, and extensively documented.

The main aim is to provide a Lucee package that is a simple unzip-and-run to get started, whilst providing a fully functional and capable web server, and also making it clear how everything works to allow it to be adapted as needed.

This first release is v0.5 because I don't consider it sufficiently complete yet - it all works fine, but doesn't yet contain everything I feel it should - for example, HTTPS has not been configured and documented, and whilst Jetty itself does support HTTPS and there's nothing stopping anyone consulting the Jetty docs and setting it up, this project is about reducing that work.

So for development use it's fine, if you don't need HTTPS or are willing to configure it yourself, go ahead - otherwise I hope to get what I consider a complete v1.0 ready as soon as time permits, but didn't want to delay releasing what I've done so far.

Downloads are available from the Lucee on Jetty project page; there's a documentation wiki at GitHub, and the template for building bundles in the GitHub repo .

As ever, I welcome any feedback or questions you might have - please use the issue tracker.