Then we can drop the other index, as it is not used.

As it locks up and the provided backtrace is useless anyway.

• Update the scripts to be more up to date.
• Import the remaining script to the repository.
• Not reviewed ‒ single-purpose scripts run manually.

User inputs (ssh commands, names, passwords) may contain invalid UTF-8
data, therefore store them in BYTEA columns. The main database already
uses BYTEA. Make sure the data are fed to the DB in the correct format,
with a trick with bind_param.

That way we are sure we have the current state still in the table. Also,
when deleting several days old events, we still have recent history.
This is to prevent the last event of too long active client from
dropping out.

In addition to keeping the current state in active_plugins, log the
history of changes into plugin_history. These tables are related in a
way that the last event for given client and plugin shall correspond to
the current state in active_clients.

Create the table for the plugin history storage. Update the clean up
script, so the history is cleaned up accordingly.

These fields are byte arrays and we have strings, python needs to be
told explicitly to convert/use different escaping.

This way we may have an idea how long each action takes and why some of
them take half a day.

The set of columns is little bit different.

Make sure the client string is in lower-case before looking that up in a
dictionary.

With SELECT *, it is impossible to drop columns (needed during migration
to new data types).

Don't rollback the database when we don't use transactions (they are
useless here, with single select being done to refresh again and again).

Also, don't recycle the cursor, clean it up (it is reportedly better for
the DB).

So the main thread can still answer queries and not get blocked.

Refresh the data from DB every 15 minutes. If it fails, try again next
time.

So we don't access the DB all the time.

The ssh honeypot now provides local IP addresses. Use them in the
export.

Some data come from attacker. And not all attackers know that passwords
should be in UTF8, so accept everything simply as raw data.

Allow changing the size of an IPset. The infrastructure should be able
to handle it now.

Don't include the addresses excluded from analysis into the export.
Also, reuse some code from the builders of address lists.

• Simplify the rules for inclusion in the blacklist. Count score for
  each client, leave out the low-score clients (hardcoded for 100 now,
  adjusting the scores for events to match that) and sum them together
  across each attacker IP. These are compared to limits.
• Split the computation of this into several views, to improve
  readability and understandability (this way it looks more procedural,
  as the views can be understood to be done one by one).
• Include the ssh honeypot as one of the sources.

Ensure the inet→text conversion in build_fwup_sets.pl doesn't produce
/32 at the end, as ucollect master doesn't handle that.

Don't try to create the IPset if it already exists. It may be of wrong
size (which will be fixed on the refill). Also, recreate IPsets if they
are missing on refill (may be caused by a queue breakage during setup)
and handle leftover temporary sets.