Cymru-whois bot returning registry 'OTHER' for 6to4 IPv6 address

[haam3r]

So I'm seeing things like this:

intelmq.lib.exceptions.InvalidValue: invalid value 'OTHER' (<class 'str'>) for key 'source.registry' is_valid returned False.
2017-06-06 03:43:20,871 - cymru-whois-expert - INFO - Current Message(event): {'source.network': '2002::/16', 'classification.type': 'b
lacklist', 'source.asn': 6939, 'feed.url': 'https://lists.blocklist.de/lists/all.txt', 'feed.accuracy': 100.0, 'time.observation': '201
7-06-05T18:52:38+00:00', 'source.ip': '2002:d06e:4282::d06e:4282', 'feed.provider': 'blocklist.de', 'feed.name': 'blocklist', 'raw': 'M
jAwMjpkMDZlOjQyODI6MDAwMDowMDAwOjAwMDA6ZDA2ZTo0Mjgy', 'source.as_name': 'HURRICANE - Hurricane Electric, Inc., US', 'classification.tax
onomy': 'other'}.

What should be the correct way to handle this?

[ghost]

IMHO we should (explicitly) ignore that

[haam3r]

I'm currently using a very dirty workaround of adding 'OTHER' as a valid registry in lib.harmonization at:
https://github.com/certtools/intelmq/blob/master/intelmq/lib/harmonization.py#L6893

But, yes I agree ingoring that sounds better

[ghost]

Upstream said that this is intention: As it's an 6to4 address, there's no registry. The same will happen for RFC1918 and private addresses too.

For 6to4 addresses we can do the registry lookup based on the extracted IPv4 address.

[ghost]

Also note that the network owners for IPv6 and IPv4 are different:

> dig +short 2.8.2.4.e.6.0.d.0.0.0.0.2.8.2.4.e.6.0.d.2.0.0.2.origin6.asn.cymru.com. TXT
"6939 | 2002::/16 |  | other |"
> dig +short 130.66.110.208.origin.asn.cymru.com. TXT
"32097 | 208.110.64.0/19 | US | arin | 2006-03-17"

We could duplicate such events to have both the v4 and the v6 data.

[navtej]

I ran into this today. Pushing a fix.