Implement global & extensible interfaces

[gpotter2]

A pretty major refactor based on a very old PR, but pretty cool. This:

• unifies how interfaces work on all platforms using a scapy/interfaces.py file and introducing conf.ifaces
• makes the switch to libpcap work in all cases
• drops the last places where "dnet" was used to make it clear we stopped supporting it (env var, "pcapdnet" file)
• merges get_if_list on all platforms
• improves how interfaces are detected on OSX
• gives extra detail about the status of the interfaces using interfaces flags and lists all interfaces in a human friendly way
• allows multiple interfaces providers to cohabitate, in a wireshark extcap style, and for each one to provide it's own sockets. This is very useful for USB sockets for instance, or if the user wants to add its own interfaces/providers to scapy.

It brings the "interfaces" objects to other platforms. Those are cool because they give you the content of the flags and avoid having to re-read the /etc/ files each time you need them.

Screenshots

e.g.: Linux

e.g.: Usbpcap AND libpcap

This is fully backward compatible: the interfaces objects implement the same functions as strings: conf.iface + "test" == "eth0test" so that it doesn't break anything, and all functions use a network_name function to get the network name from the object or a string (instead of the current if WINDOWS: .... checks)

Tested

• Windows & Linux (manual + unit tests)
• OS x (manual + unit tests)
• *BSD (manual)

How to test this

Mainteners, please test the changes (there are unit tests, but I'd like you all to give some feedback on the features): try playing around with conf.ifaces and conf.iface, then with conf.use_pcap to understand what it does.

[codecov]

Codecov Report

Merging #2707 into master will decrease coverage by 0.08%.
The diff coverage is 90.33%.

@@            Coverage Diff             @@
##           master    #2707      +/-   ##
==========================================
- Coverage   88.26%   88.18%   -0.09%     
==========================================
  Files         252      254       +2     
  Lines       53534    54004     +470     
==========================================
+ Hits        47252    47622     +370     
- Misses       6282     6382     +100     

Impacted Files	Coverage Δ
scapy/themes.py	80.91% <25.00%> (ø)
scapy/layers/usb.py	75.60% <57.14%> (-5.45%)	⬇️
scapy/supersocket.py	75.39% <75.00%> (+1.33%)	⬆️
scapy/arch/windows/__init__.py	68.43% <81.91%> (-2.82%)	⬇️
scapy/arch/bpf/core.py	87.59% <84.84%> (+3.43%)	⬆️
scapy/arch/libpcap.py	82.03% <88.67%> (ø)
scapy/utils.py	81.66% <90.47%> (+0.85%)	⬆️
scapy/interfaces.py	96.90% <96.90%> (ø)
scapy/all.py	100.00% <100.00%> (ø)
scapy/arch/__init__.py	92.50% <100.00%> (-0.69%)	⬇️
... and 30 more

[guedou]

The display looks way better now. Thanks! I did not notice it during my first test, but on macOS inactive interfaces are also listed. I think that is not necessary as they Scapy won't be able to use them. Another point, the "WARNING: No IPv4 address found" messages are not necessary too.

[guedou]

On a fresh Ubuntu 18.04 installation, the following creating a dummy network interface with sudo ip link add name scapy0 type dummy gives the following output:

>>> conf.ifaces
Source  Index  Name    MAC                IPv4          IPv6                               
sys     3      scapy0  ce:d1:6c:1f:ac:32  0.0.0.0                                          
sys     2      eth0    00:1c:42:69:bc:31  10.211.55.11  fdb2:2c26:f4e4:0:21c:42ff:fe69:bc31
                                                        fe80::21c:42ff:fe69:bc31           
sys     1      lo      00:00:00:00:00:00  127.0.0.1     ::1 

I see several issues here:

• scapy0 is not up and should not be displayed
• the IPv4 should be empty

[guedou]

04b8d1eaef3b7d1228b3673b3a45059c8746a275 works fine on FreeBSD 12.1 (^^)/

[gpotter2]

I'm pretty sure get_if_list() used to display all interfaces, even the ones not running. On Windows the interfaces that are not running were marked with a flag invalid=True. This flag tagged it as not suitable for default interface but let it appear elsewhere. The flag was replaced by a function specific to the provider, but still has the same effects.

Should we remove interfaces that are not up from get_if_list() (everywhere in scapy: routes, conf.ifaces, ...), or only from the output of conf.ifaces display?

Also sorry but I lost track of what 04b8d1eaef3b7d1228b3673b3a45059c8746a275 was ^^

[guedou]

Let's keep the current get_if_list() behavior for now. I think that conf.ifaces should only list interfaces that Scapy could use.

By the way, do you think that conf.ifaces could be ordered by the interface index? Index could be the first column. An interesting use case could be conf.iface = conf.ifaces[5].

[gpotter2]

I've settled on a middle ground:

>>> conf.ifaces
Source  Index  Name  MAC                IPv4       IPv6
sys     2      eth0  Microsof:aa:aa:aa  10.0.0.5   XXXXX
sys     1      lo    00:00:00:00:00:00  127.0.0.1  ::1

>>> conf.ifaces.show(hidden=True)
Source  Index  Name    MAC                IPv4       IPv6
sys     4      scapy0  9a:4c:10:71:24:07
sys     2      eth0    Microsof:aa:aa:aa  10.0.0.5   XXXXX
sys     1      lo      00:00:00:00:00:00  127.0.0.1  ::1

I also changed the sorting

[guedou]

@p-l- what's your call?

[guedou]

Can I merge this or do you want me to sqash the commits?

[guedou]

I still think that the current function name needs to be changed to something more meaningful =)