How to tackle the scalability problem?

Abdullah Albarrak @Abdullah_Albarrak3

12 December 2015 1 8K Report

Say I have M dimensions, and each dimension has N unique values, then:

There are N^M different combinations, if M is increased by 1, then the number of combinations increases exponentially!

An algorithm A tasked to check all those combinations will be inefficient!

It is possible to overcome this problem without a decrease in effectiveness?

Bogdán Zaválnij

One possibility is to use a Monte Carlo algorithm. As the error of a MC algorithm -- O(1/sqrt(N)) -- is independent of the dimension, it is a commonly proposed method for large dimensional problems.

Badges
Science topic

Similar topics
Social Psychology
Values

Sample dilution for quantitative analysis using ELISA?

If the detection range is in ng/ml but the reference range is in ug/ml for a molecule or protein in serum or plasma .how to dilute and what is the initial volume to be taken for quantitative analysis

02 March 2021 7,670 3 View

What is/are the best ML classifier(s) for Sentinel2 images?

Hi I am trying to segment a sentinel2 image. At this stage, I want to run a binary classifier that assigns each pixel to either farm or non-farm pixel. For this purpose, I have 4 10m bands...

15 February 2021 3,596 9 View

Concentration of DNA?

Hello everyone! Is a normal value 295.8 ug/ul for cDNA? If not what does it mean? Where is the mistake? The measurement was done with nano drop. Thank you!

09 February 2021 6,202 1 View

How to choose the Hidden Layers Number and RBM Size in a DBN network?

Hello, When using a Deep Belief network (DBN) for images classification, how could we choose the number of hidden layers (number of Restricted Boltzmann Machines) and nodes (RBM size) in a hidden...

09 February 2021 4,618 5 View

What is the reference value of THD in BLDC MOTOR speed control ?

i used to control EHO+ANFIS to MRPID to Landsman converter for bldc drive i got 7% but some of them are telling IEEE standards are < 5% is it correct for Drives ?

07 February 2021 5,892 1 View

Is there any similarity index between two time series?

Hi We have two time series dataset. For example mean daily temperature from two stations. Each of them have 30 data for a month. T1={ t1 , t2 ,... , t30 } T2={ t1 , t2 ,... , t30 } Now we want to...

06 February 2021 6,434 7 View

Matrix based scientific data analysis: extrapolating to matrices of sets?

Data sets, when structured, can be put in vector form (v(1)...v(n)), adding time dependency it's v(i, t) for i=1...n and t=1...T. Then we have a matrix of terms v(i, j)... Matrices are important,...

30 January 2021 9,295 6 View

When is the right time to stop training a classification algorithm?

In this example giving 400 samples looks like there is an accuracy of 91.5% while training beyond that point, accuracy is getting decreased. Are 400 samples enough to keep for this example? If...

27 January 2021 5,657 3 View

How to know the threshold limit value of a sar image...??

i want to do thresholding by segregation of water areas from non water areas for that i am using model maker in ERDAS but i am unable to get the threshold limit value of my image.

26 January 2021 8,277 1 View

How can I make unsupervised clustering of trees species using algorithms like HAC, Kmeans... in R?

I am working on the development of multi-species volume allometric equation from diameter (DBH). I would like to form groups of species with the largest possible volume-diameter correlations by...

19 January 2021 5,752 5 View