For each image segment you should have vector with histogram features. This will be a object. Object must have label [for example integers numbers]. So your matrix (learning set) should be [m x n], where m - numbers of objects, n - lenght of features vector plus class label
How can one come to know which feature from feature vector extracted from histogram, belongs to which segmented category on the basis of which one can construct label matrix for svm?