Xelerance has been working on making transport mode work when there are
two clients behind two NAT's that have the same IP.
It has other benefits as well: we can probably make a transport-mode OE
work through NATs, even when the clients have the same IP addresses.
That means that the clients do not have to be assigned unique IPs, but
we can't do gateway mode OE. Maybe that's okay, if we can do OE for
NAT-ed clients.
Also, BTNS is going towards transport-mode only SAs.
How does this all work.
1) we augment ipsecX with "mastX"
Of the three mastX modes that we discussed years ago, only one has
been implemented. That is the mode where if the nfmark is set to
0x80000000 | (SAref&0x7fff << 16)
then we extract the SAref and use it to lookup an SA in the SA table
that RGB wrote years ago.
Some changes to it: the SAref is now a proper extension. I did this
so that our SA create message was identical to stock PFKEYv2.
This also makes it easier to #include the KAME/NETKEY
liunx/pfkeyv2.h.
It turns out that NETKEY has skb->sp, which is:
struct sec_path
{
atomic_t refcnt;
int len;
struct sec_decap_state x[XFRM_MAX_DEPTH];
};
to this, I have added:
typedef unsigned int xfrm_sec_unique_t;
xfrm_sec_unique_t ref; /*reference to high-level policy*/
I don't know what the sec_decap_state{} structure does yet, but it
involves the xfrm code's policy check. We have a much easier "ref" which
is just an integer.
After checking the nfmark, we then check the skb->sp->ref value, and
if non-null use that instead for the SA lookup.
The other two modes for mastX are (for the record):
b) punt all traffic into a well-defined SA# ("PPP" mode).
This would be useful for building a virtual-leased line,
and would eliminate the GRE layer that is currently required to
make BGP-over-IPsec work.
c) a mode where it has a link-layer value, and for each tunnel
that is up, it populates the neighbour cache with something
is meaningful to it.
2) in ipsec_rcv(), we set up skb->sp to contain a sec_path that
has a ref set up correctly.
3) for UDP sockets, we set up a new IP-level option that permits the
skb->sp to be mapped to a new "ancilliary" data message. This message
is currently pretty primitive, consisting of two "ref"s.
These are called generally "ref" (or refme) and "refhim".
The first is refme, and is the SA on which the packet arrived.
The second is refhim, and is the ref of an SA on which a reply can be
sent.
For TCP sockets, the TCP layer will be taking care of this stuff.
(for SCTP, it will have to be a mix!)
For UDP, the application needs to worry about it.
Although, I guess connected UDP sockets could not care about it, but
that's really not that common a use.
4) In the case of xl2tpd, we use the "refhim" as an additional index into
the call/tunnel list, letting us distinguish two hosts that appear to
have the same outer IP.
Actually, we could probably use *ONLY* refhim.
We don't use "ref", although we do record it the first time, since it
currently changes during rekeys. I am going to be fixing that now
that I've figured exactly how.
The problem is that we have to keep the other (older) SAs around when
we are doing rekeys, since there may still be packets in flight. In
fact, the client could even, say, load balance among the currently
valid SAs... might occur for some HA system...
I'm going to be linking the "older" SAs to the newest SA on a singly
linked list.
Oh... didn't mention that SAs are now all properly reference counted.
(I HOPE!)
5) in pluto, we have to be a bit crafty to get things setup right.
KLIPS can assign SArefs if the passed SAref is IPSEC_SAREF_NULL,
and this is fine for new SAs. But, we need to know the outgoing SA
when we create the incoming SA, so that it can reference it.
Normally, pluto creates the incoming during QUICK_R1 (when it
receives I1, and sends R1). It then creates the outgoing SA when
it gets I2.
Now, if pluto doesn't know the outgoing SA's "refhim", it creates
the outgoing SA immediately. (It doesn't "eroute" it yet)
It then creates the incoming SA.
Then at I2, if it hasn't created the outgoing SA, it creates it.
Why might that happen? well, at a rekey, we already know the refhim
that we want to use, so we can just it normally.
For transport-mode with the MAST kernel driver, we actually don't
eroute ANYTHING.
We can find out the previous refhim by looking at the state
referenced by:
st->st_connection->newest_ipsec_sa
If the gateway is already EXPIRED the state, then we have a
problem. I don't have a solution for this problem yet. I think that
we will have to have some kind of mapping:
{public-key, SA-tuple} => refhim
6) for tunnel mode SAs, we can now use iptables.
We do:
iptables -I PREROUTING 1 -j IPSEC -t mangle
iptables -I IPSEC 1 -s 192.0.1.0/24 -d 192.0.2.0/24 -j MARK --set-mark 0x80120000
Except that it turns out iptabels doesn't grok 0x, and thinks of
set-mark value as being signed... so _updown.mast uses /bin/printf to do
the right thing
We then do:
ip rule add fwmark 0x80000000 fwmarkmask 0x80000000 table 50
ip route add 0.0.0.0/0 dev $PLUTO_INTERFACE table 50
fwmarkmask is a new option. It won't stay that way yet. It has kernel
side code too, which was posted, but needs changes to be accepted.
7) _updown is now a wrapper that basically does:
exec @IPSEC_LIBDIR@/_updown.${PLUTO_STACK} $*
where PLUTO_STACK={mast,klips,netkey}
8) for transport mode UDP sends, when the IPSEC_REFINFO option is
attached using sendmsg(), then basically we fill in the outgoing flow
information. We figure out which mastX device to force things to by
looking up the ref value given using ipsec_sa_getbyid(), and finding
the attached mastX device.
(Oh, yeah, each SA can now indicate which device it should go in or
out of, but only mast0 is supported by pluto)
9) tunnel and OE
we should be able to do the same trick with iptables with a new module:
iptables -I IPSEC 1 -s 192.0.1.0/24 -d 192.0.2.0/24 -j IPSEC --saref 0x012
just have to write this module. A trap will have to become
iptables -I IPSEC 1 -s 192.0.1.0/24 -d 192.0.2.0/24 -j IPSECTRAP --???
and once pluto sees that, it will have to change it to -j IPSEC.
the challenge is going to be programatic interface to iptables...
10) OE
iptables -I IPSEC 1 -s 192.0.1.0/24 -d 192.0.2.0/24 -j IPSECOE
This is different, because it has to use conntrack to look up the
particulars of the stream, and if there is no conntrack, then it
has to generate an ACQUIRE.
I may use the existing eroute code. Or not. I'm not sure yet.
IRC notes:
16:27:39) mcr: no rekeys yet.
(16:27:56) mcr: you want an explanation.... okay.
(16:28:16) mcr: so, the unique value that the l2tpd uses for the remote client is the "refhim" --- i.e. the outgoing SA.
(16:28:17) ***pat braces himself.
(16:28:45) mcr: l2tpd actually doesn't check the incoming SA anymore. This concerns me a bit, but we need to have a way to ask pluto, "is this SA the same as the previous one?"
(16:29:21) mcr: I was originally going to make the kernel keep the refme the same too. I didn't like my method. I've thought of a better method, and I'll be doing that soon.
(16:29:40) mcr: (we'll need it for TCP, and also for NAT-T friendly OE and NAT-T friendly BTNS...)
(16:30:00) mcr: so, the trick is that pluto tries to keep the refhim the same. It's okay to have only one outgoing SA --- because we get to pick which one.
(16:30:10) mcr: But, we need to keep all incoming SAs until the remote machine deletes them.
(16:30:37) mcr: one of the problems is if the gateway expires the IPsec SA soon --- then we have no record of what the SA was (no refhim=) so, we can't maintain it.
(16:30:44) mcr: okay, so fixed that, and then got a different problem.
(16:31:03) mcr: the client would rekey, and we'd try to keep the SA -- but we screwed up in logic.
(16:31:45) mcr: we have to install the outgoing SA before the incoming SA if we don't know what the outgoing SA is going to be (this is a change compared to what we did before. we would still install the "eroute" after we are sure everything is sane).
(16:31:59) mcr: so, we say, "if refhim==NULL" => install outgoing SA.
(16:32:15) mcr: then, when it is time to install outgoing stuff, we do again, "if refhim==NULL" => install outgoing SA.
(16:32:30) mcr: except that during a rekey, refhim is intentionally NOT NULL. (we keep the value).
(16:32:54) mcr: Since we know the value, we can install the outgoing SA whenever we liike (we already know what to reference in the incoming SA), but because the test was the same, we never
(16:33:19) mcr: installed the new outgoing SA, so we just kept using the old outgoing SA. Well, during a *REAL* rekey (due to expire of SA), the old SA on the client is gone.
(16:33:25) mcr: (vs when you simulate a rekey with --up!)
(16:33:29) mcr: which is what my tests did.
(16:33:50) paul: ahh. bad testcase :)
(16:41:41) pat: operator error :-)
