This fixes an internal table cleanup bug introduced
in ff397df7.

This also fixes memory leaks from import/export tables being never
cleaned up and freed.

If there are roa_check() calls in channel filters, then the channel
subscribes to ROA table notifications, which are sent when ROA tables
are updated (subject to settle time) and trigger channel reload or
refeed.

James Lu authored 4 years ago and Ondřej Zajíček committed 4 years ago

(Minor syntactic changes by committer)

The patch add support for per-channel debug flags, currently just
'states', 'routes', and 'filters'. Flag 'states' is used for channel
state changes, remaining two for routes passed through the channel.
The per-protocol debug flags 'routes'/'filters' still enable reporting
of routes for all channels, to keep existing behavior.

The patch causes minor changes in some log messages.

Logging as a result of triggered receive limit in import table code
accesset rte->net, which was not filed yet.

Thanks to Pier Carlo Chiodi for the bugreport.

Put new non-best routes to the end of list instead of the second
position. Put updated routes to their old position. Position is changed
just by best route selection.

Use separate IPv4 and IPv6 implementation of prefix sets. Just this
change makes IPv4 prefix sets 60% smaller and 50% faster.

The old code stored route verdicts and temporary routes directly in
rtable. The new code do not store received routes (it immediately
compares them with exported routes and resolves conflicts) and uses
internal bitmap to keep track of which routes were received and which
needs to be reinstalled.

By not putting 'invalid' temporary routes to rtable, we keep rtable
in consistent state, therefore scan no longer needs to be atomic
operation and could be splitted to multiple events.

Use a hierarchical bitmap in a routing table to assign ids to routes, and
then use bitmaps (indexed by route id) in channels to keep track whether
routes were exported. This avoids unreliable and inefficient re-evaluation
of filters for old routes in order to determine whether they were exported.

For regular channels do not compare src in export table, as we want to
keep here only the best (exported) route per network.

When non-MPLS recursive route resolves to MPLS underlying route,
then it should get MPLS labels from the the underlying route.

Underlying (IGP) route may lead to PtP link, in this case it does not
need gateway. Which is different than direct route without gateway.

When recursive (BGP) route uses PtP route, it should not use recursive
next hop as immediate next hop, while for direct routes it should.

Missing cleanup can lead to dangling pointer to old next hops.

Exported route may be in modified state, we need to get cached one for
rte_same() and rta_clone() to work properly.

The patch implements optional internal export table to a channel and
hooks it to BGP so it can be used as Adj-RIB-Out. When enabled, all
exported (post-filtered) routes are stored there. An export table can be
examined using e.g. 'show route export table bgp1.ipv4'.

Thanks to Vincent Bernat for reporting this.

The temporary atttributes are no longer removed by ea_do_prune(), but
they are undefined by store_tmp_attrs() protocol hooks. This fixes
several bugs where temporary attributes were removed when they should
not or not removed when they should be. The flag EAF_TEMP is no longer
needed and was removed.

Update all protocol make_tmp_attrs() / store_tmp_attrs() hooks to use
helper functions and to handle unset attributes properly.

Also fix some related bugs like improper handling of empty eattr list.

Keep track of whether OSPF tmpattrs are actually defined for given route
(using flags in rte->pflags). That makes them behave more like real
eattrs so a protocol can define just a subset of them or they can be
undefined by filters.

Do not set ospf_metric2 for other than type 2 external OSPF routes and do
not set ospf_tag for non-external OSPF routes. That also fixes a bug
where internal/inter-area route propagated from one OSPF instance to
another is initiated with infinity ospf_metric2.

Thanks to Yaroslav Dronskii for the bugreport.

Route flags are mosty internal state of rtable, they are not significant
to whether a route has changed. With the old code, all routes received as
a part of enhanced route refresh are always re-announced to other peers
due to change in REF_STALE.

... and consted some declarations.

This should be revised, there are still ugly things in the filter API.

Update for one of previous patches, handles the the issue for
first-accepted mode of route propagation.

One of previous workarounds for phantom route avoidance breaks export
counters by expanding sending of spurious withdraws, which are send when
we are not sure whether we have advertised that routes in the past.
If not, then export counter is decreased, but it was not increased
before, so it overflows under zero.

The patch fixes that by sendung spurious withdraws, but not counting them
on export counter. That may lead to error in the other direction, but that
happens only as a race condition (i.e., in normal operation filters
return proper values about old route export state).

Using dummy net instead. This should help with performance on rejected
routes.

The earlier fix loosen conditions for not running filters on old
route when deciding about route propagation to a protocol to avoid
issues with ghost routes in some race conditions.

Unfortunately, the fix also caused back-propagation of withdraws. For
regular updates, back-propagation is prevented in import_control hooks,
but these are not called on withdraws. For them, import_control hooks
are called on old routes instead, changing (old, NULL) notification
to (NULL, NULL), which is ignored. By not calling export processing
in some cases, the withdraw is not ignored and is back-propagated.

This patch fixes that by contract conditions so the earlier fix is not
applied to back-propagated updates.

The patch implements optional internal import table to a channel and
hooks it to BGP so it can be used as Adj-RIB-In. When enabled, all
received (pre-filtered) routes are stored there and import filters can
be re-evaluated without explicit route refresh. An import table can be
examined using e.g. 'show route import table bgp1.ipv4'.

Once upon a time, far far away, there were the old Bird developers
discussing what direction of route flow shall be called import and
export. They decided to say "import to protocol" and "export to table"
when speaking about a protocol. When speaking about a table, they
spoke about "importing to table" and "exporting to protocol".

The latter terminology was adopted in configuration, then also the
bird CLI in commit ea2ae6dd started to use it (in year 2009). Now
it's 2018 and the terminology is the latter. Import is from protocol to
table, export is from table to protocol. Anyway, there was still an
import_control hook which executed right before route export.

One thing is funny. There are two commits in April 1999 with just two
minutes between them. The older announces the final settlement
on config terminology, the newer uses the other definition. Let's see
their commit messages as the git-log tool shows them (the newer first):

    commit 9e0e485e
    Author: Martin Mares <mj@ucw.cz>
    Date:   Mon Apr 5 20:17:59 1999 +0000

	Added some new protocol hooks (look at the comments for better explanation):

		make_tmp_attrs          Convert inline attributes to ea_list
		store_tmp_attrs         Convert ea_list to inline attributes
		import_control          Pre-import decisions

    commit 5056c559
    Author: Martin Mares <mj@ucw.cz>
    Date:   Mon Apr 5 20:15:31 1999 +0000

	Changed syntax of attaching filters to protocols to hopefully the final
	version:

		EXPORT <filter-spec>    for outbound routes (i.e., those announced
					by BIRD to the rest of the world).
		IMPORT <filter-spec>    for inbound routes (i.e., those imported
					by BIRD from the rest of the world).

	where <filter-spec> is one of:

		ALL                     pass all routes
		NONE                    drop all routes
		FILTER <name>           use named filter
		FILTER { <filter> }     use explicitly defined filter

	For all protocols, the default is IMPORT ALL, EXPORT NONE. This includes
	the kernel protocol, so that you need to add EXPORT ALL to get the previous
	configuration of kernel syncer (as usually, see doc/bird.conf.example for
	a bird.conf example :)).

Let's say RIP to this almost 19-years-old inconsistency. For now, if you
import a route, it is always from protocol to table. If you export a
route, it is always from table to protocol.

And they lived happily ever after.

The new MRT protocol is responsible for periodic RIB table dumps in the
MRT format (RFC 6396). Also the existing code for BGP4MP MRT dumps is
refactored and splitted between BGP to MRT protocols, will be more
integrated into MRT in the future.

Example:

protocol mrt {
        table "*";
        filename "%N_%F_%T.mrt";
        period 60;
}

It is partially based on the old MRT code from Pavel Tvrdik.

The patch implements long-lived graceful restart for BGP, namely
draft-uttaro-idr-bgp-persistence-03.