Hosting servlets via Apache, mod_jk, Tomcat, mod_ssl and a few rewrite rules.
Hosting multiple Java Web-enabled applications with Apache/SSL in combination with Tomcat is potentially highly detailed. Separating the dynamic from the static content requires URL rewriting and aliases. This article discusses one viable configuration.
I describe the basics of how to host multiple Java Web applications using a pure Apache project approach. In other words, I explain how to apply Apache, mod_ssl, some rewrite rules and the Tomcat Servlet container to gain control of a consistent and viable production environment. In real life, I am a more-than-a-little-busy developer, and one of my more-recent tasks was to define and implement a structure to host a complex database-intensive Web-enabled searchable publication mechanism through the life cycle. I condense the experience gained and explain the most relevant details here.
The basics of placing an Apache Web server in front of multiple Tomcat servers is explained in an article by Daniel McCarthy on the Linux Journal Web site (see Resources). I take this article somewhat further by adding the ability to provide secure communication via SSL and show how to optimize performance by separating dynamic content, such as JSP pages, from static content, such as HTML and images. Further security issues also are nodded at briefly.
The following preparations are for those who want to generate a working instance of the infrastructure mentioned. This infrastructure involves a locally configured Apache server running with two Tomcat instances, all being referenced from a Web browser via different loopback (127.0.0.x) addresses. This article is still worth reading without following through with the recipe.
I assume that the following have been installed: Apache 1.3x Web server, mod_ssl, mod_jk and two instances of a Tomcat 5.5.x server, one running the ajp1.3 connector on the standard port of 8009 and the shutdown port of 8005, and the other on port 8019 and 8015. I have chosen a plain-old stable and reliable Apache 1.3.x server over an Apache 2.x version on the principle that you shouldn't fix what isn't broken. At the Institutes for which I have been responsible, during the past few years they have run Apache 1.3.x without issue, the system administrators have built up their knowledge, and the systems are maintained and patched to the highest levels and snuggly sit in the maturity section of the Web server's life cycle. The same applies for the choice of mod_jk over mod_jk2. In fact, mod_jk2 development has been discontinued due to the complexity of configuration.
If you have a Debian-based Linux distribution, to install the Apache server without compiling, try the following:
sudo apt-get install apache sudo apt-get install libapache-mod-jk sudo apt-get install libapache-mod-ssl
You should now have a running Apache instance with the configuration files sitting under /etc/apache.
For the Tomcat servers, you have two choices. The first is to use one instance of the binary and then two instances of the configuration, and then run a startup script that applies the unique instance of the binary with different configurations. The second choice is to use two copies of the Tomcat server and modify the server.xml file. The advantage of the first approach is the avoidance of replication of executable code. However, this is nearly always a false economy. The second approach has advantages for complex environments where you want to host different versions of Tomcat servers. The second approach is more relevant for Application Service Providers that have multiple customers. A division exists between code that is written for Java 1.5 that runs natively in Tomcat 5.5 (without installing the 1.4 compatibility package) and Java 1.4 that runs in Tomcat 5. Furthermore, the Servlet implementation is more up to date the newer the Tomcat version. Due to the current velocity of change, software that is hosted for more than a year can be considered legacy, so there always will be a demand for the use of older but still reliable servers.
Next, we want to test only on the loopback addresses with no packets reaching the network. This can be achieved by modifying the /etc/hosts file to something similar to:
127.0.0.10 bronze_a 127.0.0.11 silver 127.0.0.12 gold
Therefore, every time you type https://bronze_a, no DNS lookups are necessary. The packets from the browser never will reach the Internet and will stay local to 127.0.0.10.
In the main Apache configuration file, httpd.conf, you will find an include line that tells Apache to look under the conf.d directory for further configuration. For example:
Include /etc/apache/conf.d
Every time a package is installed that requires configuration changes for Apache, you will find an extra configuration file within the conf.d directory. In fact, if you want (for a nice aside), try to install Drupal and read the Drupal.conf file that is dumped.
I want to keep our work separate from the rest of the world's. No doubt, we will generate mistakes during playtime. Add a second line to include a directory for our virtual hosting files:
Include /etc/apache/vhosts
Then, make the directories /etc/apache/ssl and /etc/apache/vhosts. Later, we will place our certificates and server keys in the SSL directory, one set per virtual host.
Next, check the httpd.conf file to see whether the SSL engine is turned on. I want to turn the engine off until enabled per virtual host. So, the line SSLEngine On should change to SSLEngine Off.
Now we have an Apache 1.3.x server that is ready for action.
If you have not set up your Tomcat servers yet, you need to modify the following lines under the tomcat_root/conf/server.xml file for the second instance. Change the port numbers to 8015 for the shutdown command and port 8019 for the AJP/1.3 connector:
<Server port="8005" shutdown="SHUTDOWN">
<Connector port="8080"
<Connector port="8009"
enableLookups="false" protocol="AJP/1.3" />
For the sake of security, change the shutdown attribute from the value SHUTDOWN to some randomly long string. Otherwise, perhaps on the worst day under a badly defended system, a cracker can Telnet in and type SHUTDOWN, and then your server is down. Also, I would comment out the 8080 connector. There is no need to expose Tomcat directly to the Internet.
Only one task is left—to create two Web applications. Under the webapps directory of the first Tomcat instance, create a bronze_a directory, and then under that directory, create a WEB-INF directory. Place the following web.xml file in WEB_INF:
<?xml version="1.0" encoding="ISO-8859-1"?>
<web-app xmlns="http://java.sun.com/xml/ns/j2ee"
xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee
http://java.sun.com/xml/ns/j2ee/web-app_2_4.xsd"
version="2.4">
<display-name>BRONZE_A</display-name>
<description>
BRONZE_A Dynamic
</description>
<welcome-file-list>
<welcome-file>
index.jsp
</welcome-file>
</welcome-file-list>
</web-app>
Notice the mention of web-app_2_4.xsd. This web.xml file will not work under Tomcat 5, which uses the 2.3 standard. Under the webapps/bronze_a directory, place the following index.jsp file. This is our poor yet relevant example of dynamic content:
<%String mess="Hello World from Bronze_a"; %> <%=mess%> <br><%=request.getRequestURI()%>
Follow the same procedure for the second instance, but replace the string bronze_a with silver under the webapps/silver directory of the second Tomcat instance.
Making the Apache and Tomcat servers talk with each other is surprisingly straightforward. If this doesn't already exist somewhere within the httpd.conf file, add the following lines to the end of the file:
JkWorkersFile /usr/local/apache/conf/workers.properties JkLogFile /usr/local/apache/logs/mod_jk.log JkLogLevel error
The exact location of the worker.properties file is left to your discretion. The JkLoggFile and JkLogLevel values are not necessary, as we will override them within the virtual host files. However, for peace of mind, I like to place default values in case of misconfiguration later. The worker property defines how the connections behave. The first line defines the list of workers—in this case, bronze and silver. The next lines are for the details of configuration for each worker set. bronze attaches itself to port 8008 and silver to port 8019, with both sets talking the AJ1.3 protocol. These two worker sets are mentioned later in the virtual host files:
worker.list=bronze,silver worker.bronze.port=8009 worker.bronze.host=localhost worker.bronze.type=ajp13 worker.silver.port=8019 worker.silver.host=localhost worker.silver.type=ajp13
Virtual hosting is the hosting of multiple servers on one machine by listening for either incoming hostnames or IP addresses. Using multiple virtual hosts with SSL works only for IP-based virtual hosting. Let me explain by example. First, say I want to view a normal transaction between a Web browser and a server. To achieve this, I use the rather excellent Apache SOAP tool TcpTunnelGui. To do this, first download the current archive from the Apache SOAP Web site (see Resources). On expanding it, you will see a directory called lib. Perform the following actions, and if all goes well, you will have Java installed locally and have brought up the GUI:
cd lib java -cp ./soap.jar org.apache.soap.util.net.TcpTunnelGui ↪9001 localhost 80
The GUI displays the text from any TCP connection going through port 9001 and redirects the input back to localhost 80. Feel free to change localhost to point to your own test Web server. In your browser, type http://localhost:9001. Expect to see the following type of transaction:
Accept: */* Referer: http://localhost:9001 Accept-Language: en-us Accept-Encoding: gzip, deflate User-Agent: Mozilla/4.0 (compatible; ↪MSIE 6.0; Windows NT 5.1; CHWIE_NL70; ↪SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322) Host: localhost:9001 Connection: Keep-Alive
As you can see, the browser sends information about itself and also the host and referrer variable. Apache uses the host variable in name-based virtual hosting to work out which configuration file to apply. By typing https://localhost:9001, you will get a garbled response similar to:
?L^A^C?+^A
The Host variable is not available until the SSL encryption is complete. Therefore, having a different SSL certificate per virtual host requires that the SSL process occurs before configuration. Yes, it's the proverbial chicken-and-egg problem. Why do we need multiple SSL certificates and, thus, IP-based configuration in the first place? The answer has to do with the cn attribute in the SSL certificate. For a certificate to be accepted as a valid server certificate by browsers, the cn attribute has to be defined with the value of the hostname of the target server. So, for the IP address 127.0.0.10, we need a certificate with cn=bronze_a, and for the IP address 127.0.0.11, a certificate with cn=silver.
To generate self-signed certificates with your own local CA, you need to have OpenSSL installed:
sudo apt-cache search openssl sudo apt-get install openssl
Use these three commands to generate a self-signed certificate:
openssl req -new -out silver.csr openssl rsa -in privkey.pem -out silver.key openssl x509 -in silver.csr -out silver.cert ↪-req -signkey silver.key -days 365
The first command generates the certificate request. Remember, the cn attribute must be the same value as the hostname contained within your virtual host—for example, silver or bronze_a. The other attributes can be of any text value you consider reasonable.
The second command moves the password from the newly generated server's private key to silver.key, removing the password protection. This is needed; otherwise, every time you restart Apache, you will be asked to type in the password at the command line. The final line generates a relevant certificate based on the certificate request. Place both the cert and key files in the /etc/apache/ssl directory. Perform the same action for bronze_a. Remember to defend the ssl directory with the least permissions possible.
To activate both port 80 and 443 for bronze, add the following virtual host under vhosts:
Listen 127.0.0.10:443
<VirtualHost 127.0.0.10:443>
ServerName bronze_a
Alias /static/ /var/www/customers/bronze_a/content/
RedirectMatch ^/$ https://bronze_a/bronze_a/
SSLEngine On
SSLCertificateFile ssl/bronze_a.cert
SSLCertificateKeyFile ssl/bronze_a.key
JkMount /bronze_a/* bronze
JkMount /bronze_a bronze
JkLogFile /usr/local/apache/logs/bronze_a_mod_jk.log
JkLogLevel info
JkLogStampFormat "[%a %b %d %H:%M:%S %Y] "
RewriteEngine on
RewriteRule ^/(.*):SSL$ https://%{SERVER_NAME}/$1 [R,L]
RewriteRule ^/(.*):NOSSL$ http://%{SERVER_NAME}/$1 [R,L]
</VirtualHost>
Listen 127.0.0.10:80
<VirtualHost 127.0.0.10:80>
ServerName bronze_a
Alias /static/ /var/www/customers/bronze_a/content/
RedirectMatch ^/$ https://bronze_a/bronze_a/
JkMount /bronze_a/* bronze
JkMount /bronze_a bronze
JkLogFile /usr/local/apache/logs/80_bronze_a_mod_jk.log
JkLogLevel info
JkLogStampFormat "[%a %b %d %H:%M:%S %Y] "
RewriteEngine on
RewriteRule ^/(.*):SSL$ https://%{SERVER_NAME}/$1 [R,L]
RewriteRule ^/(.*):NOSSL$ http://%{SERVER_NAME}/$1 [R,L]
</VirtualHost>
For silver, create a similar virtual host but with the 127.0.0.11 IP address, and replace the string bronze_a with the string silver. Double-check that your SSL certificate and private key are pointed to correctly.
Turn on SSL with these three little commands:
SSLEngine On SSLCertificateFile ssl/bronze_a.cert SSLCertificateKeyFile ssl/bronze_a.key
The rewrite rules are culled from the mod_ssl FAQ. What is happening is that you have control over the relative URLs, so you can switch between the SSL and non-SSL port easily. When you use /url:NOSSL as a URL, the URL is rewritten to HTTP instead of HTTPS, and the same is true for HTTP to HTTPS using /url:SSL.
Enable the mounting of the Tomcat server by the workers with the following:
JkMount /bronze_a/* bronze JkMount /bronze_a bronze
It is good practice to separate log files used potentially for debugging—for example:
JkLogFile /usr/local/apache/logs/80_bronze_a_mod_jk.log
Apache is better than Tomcat for delivering static content, security and URL reshaping. Therefore, it is of global benefit to separate the static and dynamic content of your site and allow Apache to deal with the static content via the filesystem and the dynamic content via mod_jk. One instance of the URL remapping is the top-level redirect. We had mounted mod_jk at /bronze_a. If a user had typed http://bronze_a/, he or she would either have found an empty page or seen a pretty file listing. You can resolve this issue by placing an index.html page at the top-level location or by redirecting down. The redirection is achieved via:
RedirectMatch ^/$ https://bronze_a/bronze_a/
To make sure the right page is picked up by the uri /bronze_a/, the following lines exist in the web.xml file:
<welcome-file-list> <welcome-file> index.jsp </welcome-file> </welcome-file-list
A simple method to link to the static content is to use an alias within the virtual host. For example, https//bronze_a/static/:
Alias /static/ /var/www/customers/bronze_a/content/
Developing Java Web applications tends to be a team sport. Static content, such as images (at least in my environment), tend to change more than the application itself. Therefore, you may consider doing the obvious and setting an FTP root above the static content, but not above the more sensitive dynamic content. Then, you can force the Web application to go through a full series of tests before placing any new version in production. In fact, you may even consider a hybrid solution. Developers like to work through CVS. By placing both static and dynamic content within a war file, you keep all your code and content together and have a synchronized deployment via the re-installation of the war file. This simplifies deployment, and system administrators have to perform the same repetitive task only when new property files or content is approved. Next, you would need to add some AliasMatch rules to treat certain URLs as file locations, dishing the files up directly rather than through mod_jk, thus avoiding potential performance hits. For example:
AliasMatch /web/customers/(.*)/javascript/(.*) /usr/local/tomcat6/webapps/$1/javascript/$2 AliasMatch /web/customers/(.*)/images/(.*) /usr/local/tomcat6/webapps/$1/images/$2 AliasMatch /web/customers/(.*)/css/(.*) /usr/local/tomcat6/webapps/$1/css/$2
This would map files in the CSS, JavaScript or image directories in the Web application as static content. For example, https://xxxxx/web/customers/little.com/javascript/editor.js translates to /usr/local/tomcat6/webapps/little.com/javascript/editor.js.
There are many ways to kill a cat, which, for cats, is most unfortunate. This article has shown one approach to hosting Web applications. I do not pretend that this is the only approach; it's simply one that has worked for me. At great speed I have mentioned mod_ssl, mod_jk and one approach to separating static and dynamic content. I hope this article has given you enough information to have a go at testing your hosting concepts yourself. With some basic configuration, it is relatively straightforward to control your SSL-enabled virtual hosts.
