If i understand your question correctly. You have designed an IDS Algorithm and want to verify whether it is working properly on real traffic, isn't it ?

As you have said you need to extract the features from live traffic by deploying the IDS Sensor inline to the traffic.  For example check the documentation if Snort Modes to get understanding 

http://manual.snort.org/node11.html

If your IDS Algorithm is signature based then identifying abnormal traffic is trivial, if the rules are properly configured and matches with the traffic the alerts are generated.

If your IDS algorithm is some machine learning based then you need to profile the real traffic for some time in order to identify what is normal and abnormal. 

Your IDS is supposed to check the intrusions of a real-world data set, just recently released or just launch it by yourself. When we are doing real-time detection then we are supposed to launch the attacks or whatever you are interested in. After that run your algorithm and if it results well than you can surely use it for real time evaluation. KDD is no more. You can use Snort or other alternatives of Snort for doing so. 

Thank you Mr.Sashank Dara and Mr. Adnan Akhunzada

I have applied machine learning for anomaly detection using KDD CUP, and the system is evaluated using KDD CUP. Suppose i know the way for the extarcting the features and other requirement from on line real traffic, how can i evaluate the result of system on real traffic. For example, in KDD CUP because the LABEL {norml/abnormal} is presented, so after evaluation we can conclude for example 99% correct detction. How can i calculate the detection rate at my on line real traffic. In other word, the IDS will produce a result for each traffic, so how can i evaluate this result? OR how can i manually make a LABEL for online real traffic ?

Hi

You see the following url to get solution of your question

http://hackersonlineclub.com/intrusion-detection-system-ids/

You can use one of the packet builder  software to create attacks samples then you can check your system. Don't forget that the ids which is built on kdd usually off-line type and the dealing with online traffic is very complicated.

If you aren't already running network IDS, you should be. There are two types of Network IDS: Signature detection and Anomaly Detection.

In a signature-based IDS, there are rules or patterns of known malicious traffic that it is looking for. Once a match to a signature is found it generates an alert. These alerts can turn up issues such as malware, scanning activity, attacks against servers and much more.

With anomaly-based IDS, the payload of the traffic is far less important than the activity that generated it. An anomaly-based IDS tool relies on baselines rather than signatures. It will look for unusual activity that deviates from statistical averages of previous activities or activity that has been previously unseen. Perhaps a server is sending out more HTTP activity than usual or a new host has been seen inside your DMZ.

If you aren't already running network IDS, you should be. There are two types of Network IDS: Signature detection and Anomaly Detection.

In a signature-based IDS, there are rules or patterns of known malicious traffic that it is looking for. Once a match to a signature is found it generates an alert. These alerts can turn up issues such as malware, scanning activity, attacks against servers and much more.

With anomaly-based IDS, the payload of the traffic is far less important than the activity that generated it. An anomaly-based IDS tool relies on baselines rather than signatures. It will look for unusual activity that deviates from statistical averages of previous activities or activity that has been previously unseen. Perhaps a server is sending out more HTTP activity than usual or a new host has been seen inside your DMZ.

@Bambang Do you have any recommendations for open source anomaly based IDS ?

@Bambang Did you get my question?! 

You need to be clear in you mind which ID technique you are using Anomaly or Knowledge base detection. In anomaly based you need to do the training which will result in training profile which will reflect normal traffic/ events / behaviour. Then you will be able to test your IDS against real traffic. However, for knowledge base the process will be different.

       I hope this will help. 

Dr Adnan 

As far as running tests on your proposed system you might want to look at https://www.concise-courses.com/linux-distros/    especially Kali. Keep in mind you may want to make sure you either own the network or have permission to use any of the tools listed therein.

The advantage to these distros is the availability of the source code.

Simple you can detect the attacks using the IDS(Intrusion Detection System).

either using HIDS or NIDS or Hybrid IDS 

or There are lots of tools available for detecting various kind of attacks or traffic controlling...

check it out this paper for the different tools of IDS.. May be useful to you...

http://rsisinternational.org/2ICMRP2015/290-296.pdf

If I understand correctly: You have a model learned on some labeled data (e.g., KDD 99). Now you want to evaluate its accuracy on real-data for which you have no ground truth available (i.e., no labels). Possible ways: (1) Run your evaluation on real-traffic for two days. Manually analyze the traffic flagged by ur IDS to see if it is malicious (or abnormal as u put). (2) Employ SNORT (or any other signature based IDS) and your IDS in parallel on real-traffic. See how many true alerts SNORT has generated that were missed by your IDS. Finally, I don't like the idea of generating synthetic traffic to evaluate your model. Perhaps, its best to test on real traffic and compare it with the already proposed solutions and try to get insights on the results through manual, and obviously expert, inspection.

Hi

Ur question is pretty good.

I have been in forensic field especially playing with PCAP file to trace any attacks and find out malicious contents.

All my forensic processes are offline. 

Now I am also concentrating on Online Forensic activity.

Where we design a high speed packet capturing module using a hardware and collect the packets.

Now U can go for network packet log to see what are the packets arrived at your NIC.

U can do network packet pattern analysis to detect any attack.

Network packet log can be traced with help of Wireshark or Packetizer. software.

It has the facility in that.

For more details U can refer my papers.

Any help. Ping me

http://link.springer.com/chapter/10.1007/978-3-642-24037-9_38#page-1

http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=7351935&abstractAccess=no&userType=inst

Hi

Ur question is pretty good.

I have been in forensic field especially playing with PCAP file to trace any attacks and find out malicious contents.

All my forensic processes are offline. 

Now I am also concentrating on Online Forensic activity.

Where we design a high speed packet capturing module using a hardware and collect the packets.

Now U can go for network packet log to see what are the packets arrived at your NIC.

U can do network packet pattern analysis to detect any attack.

Network packet log can be traced with help of Wireshark or Packetizer. software.

It has the facility in that.

For more details U can refer my papers.

Any help. Ping me

http://link.springer.com/chapter/10.1007/978-3-642-24037-9_38#page-1

http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=7351935&abstractAccess=no&userType=inst

It just dawned on me to ask...

Where is the IDS positioned? Is it standalone on the network, between the network and the internet provider, software ran inside a server, or at the internet provider?  Is it a combination of these?

In any case, the question should NOT be knowledge OR Anomaly. It should be how best to have both working in tandem.

Dear William Shawn Wilkerson, 

first of all thank u for the reply, and second about your question , There is a router --or switch-- I want to use a computer to connect to this router--or switch-- and get the packets via PCAP. The current dataset such as KDDCUP have extracted features, i am going to extract same features -- i think the features are extracted from flow not packets--- however, suppose I have collected the packet -- or flows---  and apply the IDS to detect anomaly and suppose IDS shows some case is Anomaly. How  can conclude the detected aNOMALLY is correct? does it need to make some attack by my self?

sinkhole attack code in ns3