I've been working with my partner project on developing various sklearn ids models that take some pcap traffic data as input and get the classification traffic in the output (benign or attack name). Now we want to deploy our models in production so the input will now take real time traffic. What tools do we have to consider in order to get this work in a network real time interface?
Thanks!
Put some colleages computers on hub instead of switch and use tcpdump or wireshark? :D.
I doubt you will get "attack" traffic. but you will at least get realtime traffic.
Franco Vazquez ,
I would first analyse whether to deploy as inline or passive configuration. Depending on the demands of the model and actions afterwards (active filtering, etc.) you may need to just do passive monitoring. Depending on the OS you can use the tools mentioned by Oskars Lusis noting that for real time I would tend to prefer TCPDump.
Regards
I agree. IMHO as far as I understand from all this, - passive runs will be better in initial phases. Because "noise" in production will be massive. And you will need to return again and again to tune performance of algorithms and whole processing cycle. More processing and less effective algorithms, means more power consumption, higher temperatures, more expensive analysis itself etc.
Capture is one thing, but full cycle and algorithms will need serious performance tunings many times, before adapting it to real world.
I use tcpdump myself, because most of my work was pretty low level programming, often over ssh in terminal with compilations on machines with no fancy interfaces.
Tcpdump configuration at start may be weird and daunting. But you can run it on just regular almost naked linux/unix kernel, very close to machine. Very little resource consumption.
Another thing of course if you need to capture traffic from windows. Then you need proper GUI and better take wireshark, it is more friendly at start and can help you to filter out streams.
Wait you can even setup chain - -> ->
And use same TcpDump output in Wireshark so you can see nice GUI, filter out streams and enjoy.
https://www.wireshark.org/docs/wsug_html_chunked/AppToolstcpdump.html
Following up on the above suggestions, you can configure a port mirror on the network segment switch and put the machine that will receive the packet capture on the port where the mirror configuration is.
You can use TCPDump to do file rotation and you will need to do some programming to read the file based on the filename rotation convention to load the information to your other programming.
I would also configure TCPDdump to do some filtering before saving to file (e.g. filter a sub segment or by protocol). This will save a lot of space and for example avoid ARP messages and other broadcasts
- Badges
- Science topic