Hi,
I'm confused with the condition of the sum eeror for a wek learner.
Sometimes i find a condition that err
.
boosting won't make miracles with learners performing worse than chance !
therefore, you should have a learner performing better than chance (err < 1/2)
now a "weak" learner is a learner which performs barely better than chance (err = 1/2-eps with eps small) ; it is the kind of learner which benefits most from boosting
.
thank you for your response,
another point with adaboost,is in the algorith "-.thit is not clear
for my case i'm using knn,so i train with knn,then i compute the error if
.
basic boosting does not work well with knn because knn is a low variance model
there are however many variants of boosted knn and i do not know which you are experimenting
the two papers below give a good overview of this topic ; you might find the answer to some of your questions there :
http://www.stat.yale.edu/~lc436/papers/KNN/Garcia-Pedrajas_2009.pdf
https://pdfs.semanticscholar.org/97aa/0d3adec3981afa894bde8132d63fe1eb15b3.pdf
.
Thank you Fabrice.
In my case i have a database with 192 features and labels in {-1,1}.For the number of instanace i can create the number i want.
I'm looking for a ML algorithm witch allow me to have a high accuracy in test higherthan a classic function which give me ~75 of good classification.
I tried knn alone with different k but i didn't have more.
SVM linear is not suitable here because the hig number of features.
I read about adaboost,i want to try with knn.
In conclousion i'm looking for a ML algorithm with give me better a classic function.
.
hard to give advice without a hint of the problem ...
anyway,why not give a try to decision stumps (or short decision trees) as weak learners ?
for stumps, you can experiment with R
https://www.r-bloggers.com/improving-adaboosting-with-decision-stumps-in-r/
.
Hello,
I want to implement the algorithm of boosted -KNN given in "https://pdfs.semanticscholar.org/97aa/0d3adec3981afa894bde8132d63fe1eb15b3.pdf".
I begin by T=1.I try to modify weights of instance given the result of knnsearch.
But I can't implement this distance.In fact matlab allow custumer function with knnsearch.
My source code is attached.can anyone help please?
Sorry for my english.
Regards
%%%%%%%%%%%%%%%%%%%%
Y = tableau(:,193); % label
X = tableau(:,1:192); % data
% Keep 70% of the data for the training dataset
ratioTrainSet=0.7;
[Xapp,Yapp,Xtest,Ytest] = split(X,Y,ratioTrainSet);
W = zeros(size(Yapp,1));
for i=1:size(Xapp,1)
idx_q=i;query=Xapp(i,:);
idx_t=setdiff(1:size(Xapp,1),idx_q);train=Xapp(idx_t,:);
%the model is trainig with train
h=fitcknn(train,Y(idx_t,:),'NumNeighbors',11,'Distance','euclidean','DistanceWeight','inverse');
%we search in the train neighbors of query respecting to the distance D
%given by D(q,i)=sigmoid(wi)*norm(q,i) ;i is the ith instance of train,q is the query
%Using knnsearch with D
sigmoid=@(z)(1./(1+exp(-z)));
diste=@(q,i)(1./(sqrt(sum(q-i).^2)));
weuc = @(XI,XJ,Wg)(diste(XI,XJ).*sigmoid(Wg'));
Wgts=W(idx_t,1);vect=ones(1,192);res=prod(Wgts,vect);
aa=[];
for i=1:192
aa=[aa Wgts];
end
[idx2,dis]=knnsearch(train,query,'Distance',weuc(query,train,aa));
%given idx and dis we update W
end
- Badges
- Science topic
- Similar topics
- Computer Science and Engineering
- Artificial Intelligence