This section works through a few more examples to show how many of the concepts we've talked about in this chapter come together in the real world. For detailed discussions of the packet filtering characteristics of particular protocols, see Chapter 8.
This section is designed to demonstrate the process of developing a filter set; filters are elaborated as we go on, rather than being produced in final form. We aren't attempting to show a complete filter set for any site. Every site is different, and you can get burned by packet filtering if you don't understand all the details and implications of its use in your particular environment. We want people to carefully consider and understand what they're doing - not blindly copy something out of a book (even ours!) without a careful consideration of how relevant and appropriate it is for their own situation. In any case, a full solution for a site requires considering packet filtering, proxying, and configuration issues. That process is illustrated in Chapter 9, Two Sample Firewalls.
Let's start with a simple example: allowing inbound and outbound SMTP (so that you can send and receive electronic mail) and nothing else. You might start with the following rule set.
NOTE: We assume in this example that, for each packet, your filtering system looks at the rules in order. It starts at the top until it finds a rule that matches the packet, and then it takes the action specified.
Rules A and B allow inbound SMTP connections (incoming email).
Rules C and D allow outbound SMTP connections (outgoing email).
Rule E is the default rule that applies if all else fails.
Now, let's consider some sample packets to see what happens. Let's say that your host has IP address 172.16.1.1, and that someone is trying to send you mail from the remote host with IP address 192.168.3.4. Further, let's say the sender's SMTP client uses port 1234 to talk to your SMTP server, which is on port 25. (SMTP servers are always assumed to be on port 25; see the discussion of SMTP in Chapter 8):
Figure 6.10 shows this case.
In this case, the packet filtering rules permit your incoming email:
Rule A permits incoming packets from the sender's SMTP client to your SMTP server (represented by packet number 1 above).
Rule B permits the responses from your server back to the sender's client (represented by packet number 2 above).
What about outgoing email from you to them? Let's say that your SMTP client uses port 1357 to talk to their SMTP server:
Figure 6.11 shows this case.
Again, in this case, the packet filtering rules permit your outgoing email:
Rule C permits outgoing packets from your SMTP client to their SMTP server (represented by packet number 3 above).
Rule D permits the responses from their server back to your client (represented by packet number 4 above).
Now, let's stir things up. What happens if someone in the outside world (for example, someone on on host 10.1.2.3) attempts to open a connection from port 5150 on his end to the X11 server on port 6000 on one of your internal systems (for example, 172.16.3.4) in order to carry out an attack? (See Chapter 8 for a discussion of X11 and some of its vulnerabilities.)
Figure 6.12 shows this case.
The rule set shown above allows this connection to take place! In fact, the rule set shown above allows any connection to take place as long as both ends of the connection are using ports above 1023. Why?
Rules A and B together do what you want to allow inbound SMTP connections.
Rules C and D together do what you want to allow outbound SMTP connections.
But Rules B and D together end up allowing all connections where both ends are using ports above 1023, and this is certainly not what you intended.
There are probably lots of vulnerable servers listening on ports above 1023 at your site. Examples are X11 (port 6000), OpenWindows (port 2000), databases (Sybase, Oracle, Informix, and other databases commonly use site-chosen ports above 1023), and so on. This is why you need to consider a rule set as a whole, instead of assuming that if each rule or group of rules is OK, the whole set is also OK.
What can you do about this? Well, what if you also looked at the source port in making your filtering decisions? Here are those same five basic rules with the source port added as a criterion:
And here are those same six sample packets, filtered by the new rules:
As you can see, when the source port is also considered as a criterion, the problem packets (numbers 5 and 6, representing an attack on one of your X11 servers) no longer meet any of the rules for packets to be permitted (rules A through D). The problem packets end up being denied by the default rule.
OK, now what if you're dealing with a slightly smarter attacker? What if the attacker uses port 25 as the client port on his end (he might do this by killing off the SMTP server on a machine he controls and using its port, or by carrying out the attack from a machine that never had an SMTP server in the first place, like a PC), and then attempts to open a connection to your X11 server? Here are the packets you'd see:
Figure 6.13 shows this case.
As you can see, the packets would be permitted, and the attack would likely succeed (X11 security being as weak as it is).
So what can you do? The solution is to also consider the ACK bit as a filtering criterion. Again, here are those same five rules with the ACK bit also added as a criterion:
Now, packet 7 (the attacker attempting to open a connection to your X11 client) will fail:
The only difference in this rule set are in rules B and D. Of these, rule D is the most important, because it controls incoming connections to your site. Rule B applies to connections outgoing from your site, and sites are generally more interested in controlling incoming connections than outgoing connections.
Rule D now says to accept incoming packets from things that are supposedly SMTP servers (because the packets are coming from port 25) only if the packets have the ACK bit set; that is, only if the packets are part of a connection started from the inside (from your client to his server).
If someone attempts to open a TCP connection from the outside, the very first packet that he sends will not have the ACK bit set; that's what's involved in "opening a TCP connection." (See the discussion of the ACK bit in the "TCP" section of "Protocols above IP" earlier in this chapter.) If you block that very first packet (packet 7 in the example above), you block the whole TCP connection. Without certain information in the headers of the first packet - in particular, the TCP sequence numbers - the connection can't be established.
Why can't an attacker get around this by simply setting the ACK bit on the first packet? If he does, the packet will get past the filters, but the destination will believe the packet belongs to an existing connection (instead of the one with which the packet is trying to establish a new connection). When the destination tries to match the packet up with the supposed existing connection, it will fail because there isn't one, and the packet will be ignored.