The method depends on the accuracy and speed you need, and how general it must be.

In THIS case, THIS frame, the green parts of the tree can easily be discerned from the background. The background is more or less black&white, e.g. the three color channels have more or less the same value. The stem however looks like background...

In the background I see a couple of wind turbines (?), and you mentioned a video sequence. If the tree (including stem) is the only thing that moves in the scene, then subtraction of each frame from the previous one, will give you the (outline of) moving parts. Hopefully the stem moves a bit.

I have some doubts about the feasibility: while the leaves are easily identified (anything of the "green hue", some cables seem to have the same color as stem/branches - thus hard to discern automatically from these. If stem and branches do not move (at least not significantly), it might be feasible to extract these once manually, using this extract as the base of the new sequence - only copying the leaves ("green) pixels) frame by frame from input to output.

Color thresholding may be useful!

If this is a video sequence then in case the little plant was not in the original scene you can use the first frame as your background training frame and subtract any new additions. This possible can work as well if you don't have any previous empty frames as the tree usually is the one that might move a bit and you can remove pixels with similar to the leafs color distribution. Look also on bootstrap algorithm and maybe entropy to extract salient points (especially with green leafs the entropy over time will be changing significantly).

I would agree that the task should be better specify what it the input and required output. From classical video perspective the background subtraction is assumed removing constant part in the stream and keeping inly parts of the scene which are changing. Otherwise it turns to be just image segmentation task that you can run in supervised or unsupervised manner... also you can think about application of deep learning if you have enough training data. The speed up can be also achieved by using super-pixels.

Some interesting links:

http://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_grabcut/py_grabcut.html

https://github.com/Borda/pyImSegm

https://github.com/matterport/Mask_RCNN

Hi all,

Thank you. Yes this is a snap shot from a recorded video of wind test on a tree. unfortunately, the stem does not move in the video, but the leaves and branches move. I need to capture the details since the area of the tree is important for me and I need it for my further analyses. I tried to use color thresholding but it did not work since the colors for the stem and branches are very similar to the background.

unfortunately, I do not have the frame without tree at the beginning of the test.

when I was looking at the internet I found the attached paper, which has used the texture analysis and I do not know how to make it in MATLAB.

Thanks,

Mojtaba

I see two options: (a) find a specific methods which would segment only tree, but I am afraid that it will be very sensitive on particular scene, or (b) use trained segmentation where in the first frame you approximately specify the area of interest - GrabCut

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.92.2154&rep=rep1&type=pdf

Just convert to HSV, set thresholds, generate mask, and extract foreground.

I quickly tried writing a simple program in OpenCV and got the attached results. I have also attached a python program for your reference. You can run the program and use trackbars to adjust HSV values. The values set in the program seems to be optimal to me. Note that the attached program only extracts green. You need to also extract brown color for stem in similar way. Unfortunately, the background is close to stem color. You can remove noise (jitters here and there) in "extracted2.jpg" which also shows stem, by using erosion and dilation operation. If your camera does not move then you can simply crop the area as shown in "tooldemo.png". Other than this straightforward approach, there are many deep learning based solutions for segmentation which have been proved to work well.

For your next experiments, I would suggest that you use an RGBD sensor (ex. Kinect or Xtion Pro or Stereo Cameras) to save data. With depth information, this segmentation would be easy and accurate.

Hi all,

Thank you so much. I'll go through your suggestions and recommendations, hope to do it in the best possible way.

Thanks,

Mojtaba

Is there movement ou illumination changes in the frames?

Hi Eanes,

The upper branches and leaves are moving but the stem and some lower barnacles don't. and I need to capture the whole projected area of the tree( including all branches, leaves and stem).

Thanks,

Mojtaba

Perhaps, you can start by applying background subtraction to

get a gross approximation of the interest area. Afterwards, you could apply some texture approach (LBP for example) combined with color recognition (histograms for example) to get the precise area.

Recently, we employed bayesian histogram color learning for skin detection, you could use some variation of this to you problem: Conference Paper Improving Face Detection Performance by Skin Detection Post-Processing

Link for background subtraction with opencv:

https://docs.opencv.org/3.3.0/db/d5c/tutorial_py_bg_subtraction.html

Background subtractor in OpenCV is the class you need. If you want to have the foreground object it will extract for you very smoothly and is very efficient also. Here's the link, you can view the code it's very simple and easy to understand https://docs.opencv.org/3.0-beta/doc/py_tutorials/py_video/py_bg_subtraction/py_bg_subtraction.html

I once found Water shade segmentation works very well in such case.