.

boosting won't make miracles with learners performing worse than chance !

therefore, you should have a learner performing better than chance (err < 1/2)

now a "weak" learner is a learner which performs barely better than chance (err = 1/2-eps with eps small) ; it is the kind of learner which benefits most from boosting

.

thank you for your response,

another point with adaboost,is in the algorith "-.thit is not clear 

for my case i'm using knn,so i train with knn,then i compute the error if

.

basic boosting does not work well with knn because knn is a low variance model

there are however many variants of boosted knn and i do not know which you are experimenting

the two papers below give a good overview of this topic ; you might find the answer to some of your questions there :

http://www.stat.yale.edu/~lc436/papers/KNN/Garcia-Pedrajas_2009.pdf

https://pdfs.semanticscholar.org/97aa/0d3adec3981afa894bde8132d63fe1eb15b3.pdf

.

Thank you Fabrice.

In my case i have a database with 192 features and labels in {-1,1}.For the number of instanace i can create the number i want.

I'm looking for a ML algorithm witch allow me to have a high accuracy in test higherthan a classic function which give me ~75 of good classification.

I tried knn alone with different k but i didn't have more.

SVM linear is not suitable here because the hig number of features.

I read about adaboost,i want to try with knn.

In conclousion i'm looking for a ML algorithm with give me better a classic function.

.

hard to give advice without a hint of the problem ...

anyway,why not give a try to decision stumps (or short decision trees) as weak learners ?

for stumps, you can experiment with R

https://www.r-bloggers.com/improving-adaboosting-with-decision-stumps-in-r/

.

thank you very much,

Hello,

I want to implement the algorithm of boosted -KNN given in "https://pdfs.semanticscholar.org/97aa/0d3adec3981afa894bde8132d63fe1eb15b3.pdf".

I begin by T=1.I try to modify weights of instance given the result of knnsearch.

But I can't implement this distance.In fact matlab allow custumer function with knnsearch.

My source code is attached.can anyone help please?

Sorry for my english.

Regards

%%%%%%%%%%%%%%%%%%%%

Y = tableau(:,193); % label

X = tableau(:,1:192); % data

% Keep 70% of the data for the training dataset

ratioTrainSet=0.7;

[Xapp,Yapp,Xtest,Ytest] = split(X,Y,ratioTrainSet);

W = zeros(size(Yapp,1));

for i=1:size(Xapp,1)

idx_q=i;query=Xapp(i,:);

idx_t=setdiff(1:size(Xapp,1),idx_q);train=Xapp(idx_t,:);

%the model is trainig with train

h=fitcknn(train,Y(idx_t,:),'NumNeighbors',11,'Distance','euclidean','DistanceWeight','inverse');

%we search in the train neighbors of query respecting to the distance D

%given by D(q,i)=sigmoid(wi)*norm(q,i) ;i is the ith instance of train,q is the query

%Using knnsearch with D

sigmoid=@(z)(1./(1+exp(-z)));

diste=@(q,i)(1./(sqrt(sum(q-i).^2)));

weuc = @(XI,XJ,Wg)(diste(XI,XJ).*sigmoid(Wg'));

Wgts=W(idx_t,1);vect=ones(1,192);res=prod(Wgts,vect);

aa=[];

for i=1:192

aa=[aa Wgts];

end

[idx2,dis]=knnsearch(train,query,'Distance',weuc(query,train,aa));

%given idx and dis we update W

end