We have a ring of routers(5 of them) running unnumbered pppoe links between them. Adding OSPF authentication (auth trailer) works OK. However, removing OSPFv3 on ONE interface causes big problem in some cases, we loose auth in the whole ring and routing is then kaputt for all routers. We have noted that some PDUs(like LS update) are sent without auth trailer so the receiving nodes complains with "ospfv3_2: Authentication failed for nbr 0.0.139.1 on p1-1-3-1-4 - missing authentication trailer (0)" OSPFv3 Hello is OK though, it still has its auth trailer. Any ideas? Jocke
On Thu, Mar 11, 2021 at 12:35:40PM +0000, Joakim Tjernlund wrote:
We have a ring of routers(5 of them) running unnumbered pppoe links between them. Adding OSPF authentication (auth trailer) works OK. However, removing OSPFv3 on ONE interface causes big problem in some cases, we loose auth in the whole ring and routing is then kaputt for all routers.
We have noted that some PDUs(like LS update) are sent without auth trailer so the receiving nodes complains with "ospfv3_2: Authentication failed for nbr 0.0.139.1 on p1-1-3-1-4 - missing authentication trailer (0)"
OSPFv3 Hello is OK though, it still has its auth trailer.
Any ideas?
Hello Which version, 2.0.7 or master? -- Elen sila lumenn' omentielvo Ondrej 'Santiago' Zajicek (email: santiago@crfreenet.org) OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net) "To err is human -- to blame it on a computer is even more so."
On Thu, 2021-03-11 at 14:26 +0100, Ondrej Zajicek wrote:
On Thu, Mar 11, 2021 at 12:35:40PM +0000, Joakim Tjernlund wrote:
We have a ring of routers(5 of them) running unnumbered pppoe links between them. Adding OSPF authentication (auth trailer) works OK. However, removing OSPFv3 on ONE interface causes big problem in some cases, we loose auth in the whole ring and routing is then kaputt for all routers.
We have noted that some PDUs(like LS update) are sent without auth trailer so the receiving nodes complains with "ospfv3_2: Authentication failed for nbr 0.0.139.1 on p1-1-3-1-4 - missing authentication trailer (0)"
OSPFv3 Hello is OK though, it still has its auth trailer.
Any ideas?
Hello
Which version, 2.0.7 or master?
almost master, we have 2.0.7 + most ospf patches from master
On Thu, Mar 11, 2021 at 12:35:40PM +0000, Joakim Tjernlund wrote:
We have a ring of routers(5 of them) running unnumbered pppoe links between them. Adding OSPF authentication (auth trailer) works OK. However, removing OSPFv3 on ONE interface causes big problem in some cases, we loose auth in the whole ring and routing is then kaputt for all routers.
We have noted that some PDUs(like LS update) are sent without auth trailer so the receiving nodes complains with "ospfv3_2: Authentication failed for nbr 0.0.139.1 on p1-1-3-1-4 - missing authentication trailer (0)"
OSPFv3 Hello is OK though, it still has its auth trailer.
Hmm, that is strange. There is difference between Hello and LSUpd in OSPFv3 Auth receiver code, but not in sending code. That is essentially: if (ifa->autype != OSPF_AUTH_CRYPT) return; ... attach auth trailer ...
Any ideas?
LSUpd packets do not contain explicit flag whether they use OSPFv3 auth trailer. Such information is stored in neighbor structure based on received DBDes packet. So 'missing authentication trailer' for LSUpd really means that such neighbor structure says 'no auth'. But that should not happen as that DBDes packet would be rejected. One idea is that DBDes packets intended for one iface (non-authenticated) were sent to a different iface (authenticated) and they poisoned neighbor structure with 'no auth' info. You say 'loose auth in the whole ring', i can imagine that it will break adjacent links, but it will really break even non-adjacent links? -- Elen sila lumenn' omentielvo Ondrej 'Santiago' Zajicek (email: santiago@crfreenet.org) OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net) "To err is human -- to blame it on a computer is even more so."
On Thu, 2021-03-11 at 15:50 +0100, Ondrej Zajicek wrote:
On Thu, Mar 11, 2021 at 12:35:40PM +0000, Joakim Tjernlund wrote:
We have a ring of routers(5 of them) running unnumbered pppoe links between them. Adding OSPF authentication (auth trailer) works OK. However, removing OSPFv3 on ONE interface causes big problem in some cases, we loose auth in the whole ring and routing is then kaputt for all routers.
We have noted that some PDUs(like LS update) are sent without auth trailer so the receiving nodes complains with "ospfv3_2: Authentication failed for nbr 0.0.139.1 on p1-1-3-1-4 - missing authentication trailer (0)"
OSPFv3 Hello is OK though, it still has its auth trailer.
Hmm, that is strange. There is difference between Hello and LSUpd in OSPFv3 Auth receiver code, but not in sending code. That is essentially:
if (ifa->autype != OSPF_AUTH_CRYPT) return;
... attach auth trailer ...
Any ideas?
LSUpd packets do not contain explicit flag whether they use OSPFv3 auth trailer. Such information is stored in neighbor structure based on received DBDes packet. So 'missing authentication trailer' for LSUpd really means that such neighbor structure says 'no auth'. But that should not happen as that DBDes packet would be rejected.
One idea is that DBDes packets intended for one iface (non-authenticated) were sent to a different iface (authenticated) and they poisoned neighbor structure with 'no auth' info.
You say 'loose auth in the whole ring', i can imagine that it will break adjacent links, but it will really break even non-adjacent links?
Yes, it takes some time(about 30 min) though before the whole ring is gone. All nodes are seeing "missing authentication trailer" msg.
On Thu, 2021-03-11 at 15:06 +0000, Joakim Tjernlund wrote:
On Thu, 2021-03-11 at 15:50 +0100, Ondrej Zajicek wrote:
On Thu, Mar 11, 2021 at 12:35:40PM +0000, Joakim Tjernlund wrote:
We have a ring of routers(5 of them) running unnumbered pppoe links between them. Adding OSPF authentication (auth trailer) works OK. However, removing OSPFv3 on ONE interface causes big problem in some cases, we loose auth in the whole ring and routing is then kaputt for all routers.
We have noted that some PDUs(like LS update) are sent without auth trailer so the receiving nodes complains with "ospfv3_2: Authentication failed for nbr 0.0.139.1 on p1-1-3-1-4 - missing authentication trailer (0)"
OSPFv3 Hello is OK though, it still has its auth trailer.
Hmm, that is strange. There is difference between Hello and LSUpd in OSPFv3 Auth receiver code, but not in sending code. That is essentially:
if (ifa->autype != OSPF_AUTH_CRYPT) return;
... attach auth trailer ...
Any ideas?
LSUpd packets do not contain explicit flag whether they use OSPFv3 auth trailer. Such information is stored in neighbor structure based on received DBDes packet. So 'missing authentication trailer' for LSUpd really means that such neighbor structure says 'no auth'. But that should not happen as that DBDes packet would be rejected.
One idea is that DBDes packets intended for one iface (non-authenticated) were sent to a different iface (authenticated) and they poisoned neighbor structure with 'no auth' info.
You say 'loose auth in the whole ring', i can imagine that it will break adjacent links, but it will really break even non-adjacent links?
Yes, it takes some time(about 30 min) though before the whole ring is gone. All nodes are seeing "missing authentication trailer" msg.
I should mention that a birdc restart/configure(on each node) will correct the problem.
We can see auth trailer on the hellos, the neighborship is established, but still the trace is appearing. Does that mean that the options on the neighbor object are incorrect? bird> show ospf neighbors ospfv3_2 ospfv3_2: Router ID Pri State DTime Interface Router IP 0.0.139.2 0 Full/PtP 32.725 p1-1-5-1-6 fe80::206:9cff:fe0c:a533 bird> 2021-03-12 16:32:36.564 <AUTH> ospfv3_2: Authentication failed for nbr 0.0.139.2 on p1-1-5-1-6 - missing authentication trailer (0) /k -----Original Message----- From: Bird-users <bird-users-bounces@network.cz> On Behalf Of Joakim Tjernlund Sent: den 11 mars 2021 16:50 To: santiago@crfreenet.org Cc: bird-users@network.cz Subject: Re: OSPFv3 auth problem CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. On Thu, 2021-03-11 at 15:06 +0000, Joakim Tjernlund wrote:
On Thu, 2021-03-11 at 15:50 +0100, Ondrej Zajicek wrote:
On Thu, Mar 11, 2021 at 12:35:40PM +0000, Joakim Tjernlund wrote:
We have a ring of routers(5 of them) running unnumbered pppoe links between them. Adding OSPF authentication (auth trailer) works OK. However, removing OSPFv3 on ONE interface causes big problem in some cases, we loose auth in the whole ring and routing is then kaputt for all routers.
We have noted that some PDUs(like LS update) are sent without auth trailer so the receiving nodes complains with "ospfv3_2: Authentication failed for nbr 0.0.139.1 on p1-1-3-1-4 - missing authentication trailer (0)"
OSPFv3 Hello is OK though, it still has its auth trailer.
Hmm, that is strange. There is difference between Hello and LSUpd in OSPFv3 Auth receiver code, but not in sending code. That is essentially:
if (ifa->autype != OSPF_AUTH_CRYPT) return;
... attach auth trailer ...
Any ideas?
LSUpd packets do not contain explicit flag whether they use OSPFv3 auth trailer. Such information is stored in neighbor structure based on received DBDes packet. So 'missing authentication trailer' for LSUpd really means that such neighbor structure says 'no auth'. But that should not happen as that DBDes packet would be rejected.
One idea is that DBDes packets intended for one iface (non-authenticated) were sent to a different iface (authenticated) and they poisoned neighbor structure with 'no auth' info.
You say 'loose auth in the whole ring', i can imagine that it will break adjacent links, but it will really break even non-adjacent links?
Yes, it takes some time(about 30 min) though before the whole ring is gone. All nodes are seeing "missing authentication trailer" msg.
I should mention that a birdc restart/configure(on each node) will correct the problem.
I believe the "options" of the neighbor object is only updated as a result of NEIGHBOR_EXSTART. If the authentication is enabled after unauthenticated neighbor adjacency, then this does not result in a new NEIGHBOR_EXSTART. Thus the options of the neighbor object is stuck 0x113, and not the updated to 0x513 (i.e. OPT_AT) resulting in that OSPF packets are becoming dropped in procedure ospf_pkt_checkauth3 since auth_present is zero. If I manually kick the FSM when authentication is changed by doing ISM_DOWN-> ISM_UP then this works as expected (see pseudo code below). I believe there is some logic missing when changing the autype? Can this be done in some more fine granular way? /* AUTHENTICATION */ if (ifa->autype != new->autype) { OSPF_TRACE(D_EVENTS, "Changing authentication type of %s", ifname); ifa->autype = new->autype; ... ospf_iface_sm(ifa, ISM_DOWN); ospf_iface_sm(ifa, ISM_UP); } Thanks, Kenth -----Original Message----- From: Bird-users <bird-users-bounces@network.cz> On Behalf Of Joakim Tjernlund Sent: den 11 mars 2021 16:50 To: santiago@crfreenet.org Cc: bird-users@network.cz Subject: Re: OSPFv3 auth problem CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. On Thu, 2021-03-11 at 15:06 +0000, Joakim Tjernlund wrote:
On Thu, 2021-03-11 at 15:50 +0100, Ondrej Zajicek wrote:
On Thu, Mar 11, 2021 at 12:35:40PM +0000, Joakim Tjernlund wrote:
We have a ring of routers(5 of them) running unnumbered pppoe links between them. Adding OSPF authentication (auth trailer) works OK. However, removing OSPFv3 on ONE interface causes big problem in some cases, we loose auth in the whole ring and routing is then kaputt for all routers.
We have noted that some PDUs(like LS update) are sent without auth trailer so the receiving nodes complains with "ospfv3_2: Authentication failed for nbr 0.0.139.1 on p1-1-3-1-4 - missing authentication trailer (0)"
OSPFv3 Hello is OK though, it still has its auth trailer.
Hmm, that is strange. There is difference between Hello and LSUpd in OSPFv3 Auth receiver code, but not in sending code. That is essentially:
if (ifa->autype != OSPF_AUTH_CRYPT) return;
... attach auth trailer ...
Any ideas?
LSUpd packets do not contain explicit flag whether they use OSPFv3 auth trailer. Such information is stored in neighbor structure based on received DBDes packet. So 'missing authentication trailer' for LSUpd really means that such neighbor structure says 'no auth'. But that should not happen as that DBDes packet would be rejected.
One idea is that DBDes packets intended for one iface (non-authenticated) were sent to a different iface (authenticated) and they poisoned neighbor structure with 'no auth' info.
You say 'loose auth in the whole ring', i can imagine that it will break adjacent links, but it will really break even non-adjacent links?
Yes, it takes some time(about 30 min) though before the whole ring is gone. All nodes are seeing "missing authentication trailer" msg.
I should mention that a birdc restart/configure(on each node) will correct the problem.
On Mon, Mar 15, 2021 at 02:25:18PM +0000, Kenth Eriksson wrote:
I believe the "options" of the neighbor object is only updated as a result of NEIGHBOR_EXSTART. If the authentication is enabled after unauthenticated neighbor adjacency, then this does not result in a new NEIGHBOR_EXSTART. Thus the options of the neighbor object is stuck 0x113, and not the updated to 0x513 (i.e. OPT_AT) resulting in that OSPF packets are becoming dropped in procedure ospf_pkt_checkauth3 since auth_present is zero.
Yes, you are right. If iface is configured to not use auth, adjacency is established (n->options are set to not include OPT_AT) and then reconfigured to use auth, adjacency is kept (because Hello packets are authenticated) but LSUpd and others are ignored (because there is no OPT_AT flag in n->options). RFC 7166 is vague about updating OPT_AT flag, and basic OSPFv2/v3 RFC just describing setting n->options from DBDes during NEIGHBOR_EXSTART.
If I manually kick the FSM when authentication is changed by doing ISM_DOWN-> ISM_UP then this works as expected (see pseudo code below). I believe there is some logic missing when changing the autype? Can this be done in some more fine granular way?
I think that the best solution would be to update OPT_AT flag in n->options based on received Hello packets. Will make a patch. -- Elen sila lumenn' omentielvo Ondrej 'Santiago' Zajicek (email: santiago@crfreenet.org) OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net) "To err is human -- to blame it on a computer is even more so."
On Mon, 2021-03-15 at 15:59 +0100, Ondrej Zajicek wrote:
On Mon, Mar 15, 2021 at 02:25:18PM +0000, Kenth Eriksson wrote:
I believe the "options" of the neighbor object is only updated as a result of NEIGHBOR_EXSTART. If the authentication is enabled after unauthenticated neighbor adjacency, then this does not result in a new NEIGHBOR_EXSTART. Thus the options of the neighbor object is stuck 0x113, and not the updated to 0x513 (i.e. OPT_AT) resulting in that OSPF packets are becoming dropped in procedure ospf_pkt_checkauth3 since auth_present is zero.
Yes, you are right. If iface is configured to not use auth, adjacency is established (n->options are set to not include OPT_AT) and then reconfigured to use auth, adjacency is kept (because Hello packets are authenticated) but LSUpd and others are ignored (because there is no OPT_AT flag in n->options).
RFC 7166 is vague about updating OPT_AT flag, and basic OSPFv2/v3 RFC just describing setting n->options from DBDes during NEIGHBOR_EXSTART.
If I manually kick the FSM when authentication is changed by doing ISM_DOWN-> ISM_UP then this works as expected (see pseudo code below). I believe there is some logic missing when changing the autype? Can this be done in some more fine granular way?
I think that the best solution would be to update OPT_AT flag in n->options based on received Hello packets. Will make a patch.
What about IPv4? Does it also need some tweaking is this case? Jocke
On Mon, Mar 15, 2021 at 03:54:59PM +0000, Joakim Tjernlund wrote:
I think that the best solution would be to update OPT_AT flag in n->options based on received Hello packets. Will make a patch.
What about IPv4? Does it also need some tweaking is this case?
No, IPv4 behavior is different, used authentication mode is writte in auth_type field in each packet. -- Elen sila lumenn' omentielvo Ondrej 'Santiago' Zajicek (email: santiago@crfreenet.org) OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net) "To err is human -- to blame it on a computer is even more so."
On Mon, 2021-03-15 at 15:59 +0100, Ondrej Zajicek wrote:
On Mon, Mar 15, 2021 at 02:25:18PM +0000, Kenth Eriksson wrote:
I believe the "options" of the neighbor object is only updated as a result of NEIGHBOR_EXSTART. If the authentication is enabled after unauthenticated neighbor adjacency, then this does not result in a new NEIGHBOR_EXSTART. Thus the options of the neighbor object is stuck 0x113, and not the updated to 0x513 (i.e. OPT_AT) resulting in that OSPF packets are becoming dropped in procedure ospf_pkt_checkauth3 since auth_present is zero.
Yes, you are right. If iface is configured to not use auth, adjacency is established (n->options are set to not include OPT_AT) and then reconfigured to use auth, adjacency is kept (because Hello packets are authenticated) but LSUpd and others are ignored (because there is no OPT_AT flag in n->options).
RFC 7166 is vague about updating OPT_AT flag, and basic OSPFv2/v3 RFC just describing setting n->options from DBDes during NEIGHBOR_EXSTART.
If I manually kick the FSM when authentication is changed by doing ISM_DOWN-> ISM_UP then this works as expected (see pseudo code below). I believe there is some logic missing when changing the autype? Can this be done in some more fine granular way?
I think that the best solution would be to update OPT_AT flag in n->options based on received Hello packets. Will make a patch.
Are you thinking something like this? diff --git a/proto/ospf/hello.c b/proto/ospf/hello.c index d094f934..dc3bc912 100644 --- a/proto/ospf/hello.c +++ b/proto/ospf/hello.c @@ -253,6 +253,10 @@ ospf_receive_hello(struct ospf_packet *pkt, struct ospf_iface *ifa, neighbors = ps->neighbors; neigh_count = (plen - sizeof(struct ospf_hello3_packet)) / sizeof(u32); + if (rcv_options & OPT_AT) + n->options |= OPT_AT; + else + n->options &= ~OPT_AT; } if (rcv_helloint != ifa->helloint)
On Mon, 2021-03-15 at 17:26 +0100, Joakim Tjernlund wrote:
On Mon, 2021-03-15 at 15:59 +0100, Ondrej Zajicek wrote:
On Mon, Mar 15, 2021 at 02:25:18PM +0000, Kenth Eriksson wrote:
I believe the "options" of the neighbor object is only updated as a result of NEIGHBOR_EXSTART. If the authentication is enabled after unauthenticated neighbor adjacency, then this does not result in a new NEIGHBOR_EXSTART. Thus the options of the neighbor object is stuck 0x113, and not the updated to 0x513 (i.e. OPT_AT) resulting in that OSPF packets are becoming dropped in procedure ospf_pkt_checkauth3 since auth_present is zero.
Yes, you are right. If iface is configured to not use auth, adjacency is established (n->options are set to not include OPT_AT) and then reconfigured to use auth, adjacency is kept (because Hello packets are authenticated) but LSUpd and others are ignored (because there is no OPT_AT flag in n->options).
RFC 7166 is vague about updating OPT_AT flag, and basic OSPFv2/v3 RFC just describing setting n->options from DBDes during NEIGHBOR_EXSTART.
If I manually kick the FSM when authentication is changed by doing ISM_DOWN-> ISM_UP then this works as expected (see pseudo code below). I believe there is some logic missing when changing the autype? Can this be done in some more fine granular way?
I think that the best solution would be to update OPT_AT flag in n->options based on received Hello packets. Will make a patch.
Are you thinking something like this?
diff --git a/proto/ospf/hello.c b/proto/ospf/hello.c index d094f934..dc3bc912 100644 --- a/proto/ospf/hello.c +++ b/proto/ospf/hello.c @@ -253,6 +253,10 @@ ospf_receive_hello(struct ospf_packet *pkt, struct ospf_iface *ifa,
neighbors = ps->neighbors; neigh_count = (plen - sizeof(struct ospf_hello3_packet)) / sizeof(u32); + if (rcv_options & OPT_AT) + n->options |= OPT_AT; + else + n->options &= ~OPT_AT; }
That did a SEGV on a NULL nbr ... Now running this: diff --git a/proto/ospf/hello.c b/proto/ospf/hello.c index d094f934..e103d2fd 100644 --- a/proto/ospf/hello.c +++ b/proto/ospf/hello.c @@ -341,6 +341,11 @@ ospf_receive_hello(struct ospf_packet *pkt, struct ospf_ifa ce *ifa, n->bdr = rcv_bdr; n->priority = rcv_priority; n->iface_id = rcv_iface_id; + if (ospf_is_v3(p)) + if (rcv_options & OPT_AT) + n->options |= OPT_AT; + else + n->options &= ~OPT_AT; /* Update inactivity timer */ ospf_neigh_sm(n, INM_HELLOREC); --
On Mon, 2021-03-15 at 20:02 +0100, Joakim Tjernlund wrote:
On Mon, 2021-03-15 at 17:26 +0100, Joakim Tjernlund wrote:
On Mon, 2021-03-15 at 15:59 +0100, Ondrej Zajicek wrote:
On Mon, Mar 15, 2021 at 02:25:18PM +0000, Kenth Eriksson wrote:
I believe the "options" of the neighbor object is only updated as a result of NEIGHBOR_EXSTART. If the authentication is enabled after unauthenticated neighbor adjacency, then this does not result in a new NEIGHBOR_EXSTART. Thus the options of the neighbor object is stuck 0x113, and not the updated to 0x513 (i.e. OPT_AT) resulting in that OSPF packets are becoming dropped in procedure ospf_pkt_checkauth3 since auth_present is zero.
Yes, you are right. If iface is configured to not use auth, adjacency is established (n->options are set to not include OPT_AT) and then reconfigured to use auth, adjacency is kept (because Hello packets are authenticated) but LSUpd and others are ignored (because there is no OPT_AT flag in n->options).
RFC 7166 is vague about updating OPT_AT flag, and basic OSPFv2/v3 RFC just describing setting n->options from DBDes during NEIGHBOR_EXSTART.
If I manually kick the FSM when authentication is changed by doing ISM_DOWN-> ISM_UP then this works as expected (see pseudo code below). I believe there is some logic missing when changing the autype? Can this be done in some more fine granular way?
I think that the best solution would be to update OPT_AT flag in n->options based on received Hello packets. Will make a patch.
Are you thinking something like this?
diff --git a/proto/ospf/hello.c b/proto/ospf/hello.c index d094f934..dc3bc912 100644 --- a/proto/ospf/hello.c +++ b/proto/ospf/hello.c @@ -253,6 +253,10 @@ ospf_receive_hello(struct ospf_packet *pkt, struct ospf_iface *ifa,
neighbors = ps->neighbors; neigh_count = (plen - sizeof(struct ospf_hello3_packet)) / sizeof(u32); + if (rcv_options & OPT_AT) + n->options |= OPT_AT; + else + n->options &= ~OPT_AT; }
That did a SEGV on a NULL nbr ... Now running this:
diff --git a/proto/ospf/hello.c b/proto/ospf/hello.c index d094f934..e103d2fd 100644 --- a/proto/ospf/hello.c +++ b/proto/ospf/hello.c @@ -341,6 +341,11 @@ ospf_receive_hello(struct ospf_packet *pkt, struct ospf_ifa ce *ifa, n->bdr = rcv_bdr; n->priority = rcv_priority; n->iface_id = rcv_iface_id; + if (ospf_is_v3(p)) + if (rcv_options & OPT_AT) + n->options |= OPT_AT; + else + n->options &= ~OPT_AT;
/* Update inactivity timer */ ospf_neigh_sm(n, INM_HELLOREC);
Sent a proper patch, let us know what you think. Jocke
On Tue, Mar 16, 2021 at 08:35:20AM +0000, Joakim Tjernlund wrote:
Sent a proper patch, let us know what you think.
Hi Your patch is ok, i just wanted something more verbose (it is IMHO a notable change to log) and there is already a section in ospf_receive_hello() to handle such changes: https://gitlab.nic.cz/labs/bird/-/commit/ae9ae864d3fec20a74ce2567536e186ce17... -- Elen sila lumenn' omentielvo Ondrej 'Santiago' Zajicek (email: santiago@crfreenet.org) OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net) "To err is human -- to blame it on a computer is even more so."
participants (3)
-
Joakim Tjernlund -
Kenth Eriksson -
Ondrej Zajicek