Ms. Forgue,

The advice you have received so far is all good. Here are a few other points.

(1) You do need to decide what your budget is. This depends on how you are funding this project--a grant with a fixed amount , your own time only, you + other people, etc. You could measure this in hours per building and factor in how much time collection on a building requires, how many days you are able to spend collecting data, whether you need to spread the collection across weeks to catch different weather conditions, etc.

(2) There is an R package called PracTools (https://cran.r-project.org/web/packages/PracTools/index.html) that will calculate sample sizes for a variety of situations, including ones like those covered by the STRUTS notes.  The book Practical Tools for Designing and Weighting Sample Surveys (chapter 3) covers the things you need. (I should disclose that I'm one of the authors--not that I'm trying to sell you a book.)

The sample size formulas do account for how big n is relative to the population size N.  Unless n/N is more than about 0.05, the sampling fraction will not have much of an effect on SEs.

(3) Since you know very little about the population of buildings, your choices for sample design are limited.  One thing that would be useful would be to stratify the buildings by area of the city. Business district buildings probably have a larger surface area than detached residential. Large apartment buildings will have larger surface areas than small commercial or residential buildings. Stratifying by area of the city would insure that your sample is spread over the city. If building types are similar within area, then this spreads the sample over building type.

Deciding whether you just want full population estimates or whether you also need estimates for subgroups is a critical part of designing a useful sample.

(4) For inferences (like confidence intervals) you should be fairly safe in treating estimates as normally distributed (even though the underlying data like W/M2/hour may be skewed).

(5) You need to specify what you want to estimate.  It sounds as if you might be interested in things like:

- average W per square meter per hour for all buildings 

- average W per square meter per hour for all buildings of certain types (office, apartment, mixed use, etc.)

- proportion of buildings that have some characteristic (like have solar panels, have exterior brick walls, have flat roof, etc.) for the whole population and by building type.

Your sample buildings will each have a weight of w = N/n if you select an equal probability sample. If you stratify by area and select a different proportion in each stratum, the weight in stratum h will be wh = Nh/nh.  Both the averages and proportions will be estimated as

sum(w * y) / sum(w) 

where w is the weight for a building. y is whatever you record for a building. y could be 0 or 1 for a characteristic or a quantitative variable like W/m2/hour.

The estimated pop total is sum(w * y) if you want that.

(6) If you took measurements on the sample buildings on different days, you could have a time series of estimates to show the fluctuation across months and seasons. Of course, building up a time series is expensive and time consuming and you may not have the resources for that.

(7) There are two US surveys that have published data and downloadable microdata. One is called the Commercial Buildings Energy Consumption Survey, https://www.eia.gov/consumption/commercial/data/2012/index.php?view=characteristics.  There is a similar one for residential buildings.

You may be able to use these data to estimate the unit (population) variances and coefficients of variation needed for some of the sample size formulas. (Of course, buildings in France may be different from those in the US, but you always have to work with whatever data are available when designing a new sample.)

Hope this helps a little,

rv

If you don't plan to sample a significant proportion of the 170000 buildings, you can pretend having an infinite population. If you do a random sampling, the probability to select a building twice is negligible (even for a sample size of 10000, the probability to select at least one building more than once is lower than 0.0001).

How large your sample should be depends on who precise your estimates should be, and that precision is also influenced by the variance of the data. You can look up here to get an idea:

http://www.nrcse.washington.edu/research/struts/chapter2.pdf

Thank you Jochen for your answer.

My problem exactly concerns the precision of the estimates. I don't get on what this is calculated. What I understand is that I should be able to tell the variance of the variables associated to the buildings. Therefor, it means that we can't choose a sample size without already knowing informations on thoses variables ?

It's kind of tricky because I want to create a sample in order to avoid calculating the variables for too many buildings ...

Another thing is that those variables concern the solar irradiation perceived by the buildings, I will have 12 variables per building... On which one should I calculate the variance ?

For example I found this formula in the litterature : 

N= t² x * (p(1-p)/e²)

with N the number of individuals I should select to create the sampling, t = 1.96 for a 95% confidence interval (student low if I'm right). p stands for the standard variance of the sampling and e the precision wanted.

Here I don't understand on what should I base the calculation of p and same for e...

Is it on the variables associated to the buidlings ? But I will calcute them only once I've done the sampling...

I really don't get this part of sampling ... Tank you for your help. :) I'm sure I'll figure it out soon!

* My mistake : The formula is rather : N = t²*p*(1-p)/e²

And I know that each building has a 1/170 000 chance to be choosen (as they are selected at the same time) and every building amongst the 170 000 is ok to be selected...

Another question that came to my mind while looking at this formula, is that N doesn't actually depend on the population size ... Isn't that a little bit strange ?

You can only plan if you known on what to build your plan... so you must have some idea about the distribution and the variance of the values you will measure. This does not need to be exact. You just need some "educated guess".

An example (no idea if the values make sense! - just an example):

measured variable: some average incident solar radiation in MBTU/day/building

expected distribution (assumption): normal

expected mean (guess): 1.5 MBTU/day/building

expected standard deviation between buildings (guess): s = 0.4 MBTU/day/building

Task: determine the sample size needed to estimate the mean with a precision specified by the width of the 95% confidence interval, which should be w = 0.1 MBTU/day/building.

According to STRUTS (cited in my previos post), chapter 2.3:

n = 16 * s² / w² = 16 * 0.4² / 0.1² = 256

If the distribution can be assumed log-normal, then the entire analysis should be done in the same way but using logarithms of the values instead.

If the response is a binary variable (yes/no like), then the aim is usually to determine the proportion of buildings that take one of the values (e.g. variable = "building has a flat roof", values: "yes" or "no", interesting parameter: proportion of buildings that have a flat roof). If w is the desired precision as the witdth of the 95% confidence interval of the proportion, the sample size can be determined by

n = ( (2*0.5) / (2*w) )² - 4 = (0.5/w)² - 4

Say you want w = 0.04 (the 95% conficence interval on the proportion should have a width of less than 0.04 or 4%), then the required sample size will be

n = (0.5/0.04)² - 4 = 153.

(see for instance: http://www.stat.wisc.edu/~st571-1/10-power-2.pdf)

.

a first useful information would be : what do you consider "sampling too much buildings", in terms of order of magnitude ?

if, say, 100 is your limit, go ahead, sample 100 buildings at random ; there is nothing much you can save here in terms of resources !

if, say, you could consider sampling 10k buildings but you would prefer not waste too much ressources in the sampling, then, you could consider a plan in two phases

- first build a "cheap" random sample (say a few hundreds, it's "cheap" because you can afford 10k - my assumption above, possibly not your reality ! ) and from this sample you can get estimates of the variabilities of your quantities of interest

- from those estimates, and your accuracy target(s), as explained by Jochen above, you can calculate the orders of magnitude of the required sample size : if this size is more than 10k ... too bad, you'll have to lower your accuracy targets and live with a 10k sample ; it it is less than 10k, good news, you can avoid wasting resources

.

 Thank you for your answers, they help. Nevertheless, I'm working on buildings with multiple variables associated ( 8760 = 24 hours * 365 days)... How am I supposed to calculate the mean, evaluate the distribution etc.

I see how it would work if I was only studying one variable per building.

Here should I calculate the mean value per building and then use it to calculate the mean value considering the entire "mini"-sample .... would this result still have a meaning ?

For example, building 1 receives 200 W/m²/hour on average, building 2 receives 300 W/m²/hour on average etc. Therefor I'd have only one variable per building and would calculate the mean based on those values ...  ?

I am wondering if I'm complicated it or if the situation is actually complexe ... As I have to justificate it anyway, I'd be really sure about what I'm doing ...

Again, Thank you all very much for your help so far!

.

if the quantity of interest is the energy flux across 1m² during 1 hour, averaged on all buildings and all time periods, yes each buiding contributes its own average along time and your quantity of interest is the average of these averages (averaging across buildings)

the small sample allows you to have an idea of the variability of the measurements (measurement = average value along time for one building) and allows you to set the minimum size of your large sample so as to reach a predefined accuracy for the final estimation

.

Ms. Forgue,

The advice you have received so far is all good. Here are a few other points.

(1) You do need to decide what your budget is. This depends on how you are funding this project--a grant with a fixed amount , your own time only, you + other people, etc. You could measure this in hours per building and factor in how much time collection on a building requires, how many days you are able to spend collecting data, whether you need to spread the collection across weeks to catch different weather conditions, etc.

(2) There is an R package called PracTools (https://cran.r-project.org/web/packages/PracTools/index.html) that will calculate sample sizes for a variety of situations, including ones like those covered by the STRUTS notes.  The book Practical Tools for Designing and Weighting Sample Surveys (chapter 3) covers the things you need. (I should disclose that I'm one of the authors--not that I'm trying to sell you a book.)

The sample size formulas do account for how big n is relative to the population size N.  Unless n/N is more than about 0.05, the sampling fraction will not have much of an effect on SEs.

(3) Since you know very little about the population of buildings, your choices for sample design are limited.  One thing that would be useful would be to stratify the buildings by area of the city. Business district buildings probably have a larger surface area than detached residential. Large apartment buildings will have larger surface areas than small commercial or residential buildings. Stratifying by area of the city would insure that your sample is spread over the city. If building types are similar within area, then this spreads the sample over building type.

Deciding whether you just want full population estimates or whether you also need estimates for subgroups is a critical part of designing a useful sample.

(4) For inferences (like confidence intervals) you should be fairly safe in treating estimates as normally distributed (even though the underlying data like W/M2/hour may be skewed).

(5) You need to specify what you want to estimate.  It sounds as if you might be interested in things like:

- average W per square meter per hour for all buildings 

- average W per square meter per hour for all buildings of certain types (office, apartment, mixed use, etc.)

- proportion of buildings that have some characteristic (like have solar panels, have exterior brick walls, have flat roof, etc.) for the whole population and by building type.

Your sample buildings will each have a weight of w = N/n if you select an equal probability sample. If you stratify by area and select a different proportion in each stratum, the weight in stratum h will be wh = Nh/nh.  Both the averages and proportions will be estimated as

sum(w * y) / sum(w) 

where w is the weight for a building. y is whatever you record for a building. y could be 0 or 1 for a characteristic or a quantitative variable like W/m2/hour.

The estimated pop total is sum(w * y) if you want that.

(6) If you took measurements on the sample buildings on different days, you could have a time series of estimates to show the fluctuation across months and seasons. Of course, building up a time series is expensive and time consuming and you may not have the resources for that.

(7) There are two US surveys that have published data and downloadable microdata. One is called the Commercial Buildings Energy Consumption Survey, https://www.eia.gov/consumption/commercial/data/2012/index.php?view=characteristics.  There is a similar one for residential buildings.

You may be able to use these data to estimate the unit (population) variances and coefficients of variation needed for some of the sample size formulas. (Of course, buildings in France may be different from those in the US, but you always have to work with whatever data are available when designing a new sample.)

Hope this helps a little,

rv

Thank you for your answer Mr Valliant. I'm reading the chapter 3 today.

I do have some questions though:

4) what allows me to make this statement (that I can treat the data as if it was normally distributed)

6) Actually, I'm working on time series, and that's why I have more than just one variable per buildings and why I was wondering how to calculate mean and other informations needed to evaluate the sample size.

I might not have been clear enough about what I'm doing, I assumed my question was not that much linked to the data I've got and what I'd do with it...

First of all, the information I've got on buildings are only their geographical position and their height. If I start choosing districts as you wisely suggest, I'm afraid the sample will be biased. What I see is that the criterion with witch I'd choose the buildings would be comming from the assumption that those criterions are good enough to distinguish differents buildings and the way they will receive solar energy... 

And it is actually what I want to proove.

After sampling, the idea is to create clusters, based on the solar energy received by the buildings and see if each cluster has common criterion that we could use to determine the solar energy received by other buildings in the city.

I may not have emphasis enough the fact that I'm working with time series and I see now that it is the main problematic point. I mean, if the variable I was studying was the energy perceived per year and then I'd have only one variable, I could easily refer to all the informations everyone gave me here.

I'm also going to have a look at studies specifically concerning time series and sampling. I might find answers there.

Any solution I find I'll report it here so it can be useful for others looking for the same kind of answer.

Mr Clerot, Thanks for your answer as well.

Actually, no this is none of my interests. I want to create this sample so I can see if it exists clusters based on the solar energy perceived by buildings (in W/m²/hour/building, so 8760 variables per building as there are 24*365 = 8760 hours in a year).

I don't want to synthetise this as the information concerning the variability through the year should be kept.

Once I've worked on the sampling, I hope the results found thanks to it can be extended to the entire population. That's why I am so concerned about its representativity etc.

.

so you're out for the big game, "time series clustering" ...

quite different from your initial description : "Each building is first given an ID number, so you can tell the difference between them. That is all the caracteristics they've got before I do the sampling. Then I calculate the amount of solar energy perceived by the buildings of the sampling, so I've got 12 different variable per building that characterize them. "

i have not much advice for this regarding the size of the sample ; the bigger the better does not lead very far, i have to admit ...

.

what kind of cluster do you have in mind ?

very local effects such as, say, groups of small houses in the shadow of a large building ? this will be very hard to find with a sample since (for a reasonably small sampling rate), the probability of having a few members of this small group of houses in the sample is very low

larger scale effects, due to, say, the topography of the town ?

effects due to differences in the buildings themselves (height, orientation) ?

(my examples may be pretty silly, i am pretty far from my own territory here !)

.

maybe another question : why do you need to sample ?

- is it costly to gather all the time series for one building ?

- you do not want to have too much data to process ?

another approach might be to sample on a time basis ; for instance, received solar energy at night should be pretty low and pretty uniform across all buildings and i would not bother much about night periods (which change across the year but this can be known beforehand, knowing where the city is on the globe) ; this might save some volume

in the same kind of idea, i would assume that two buildings receiving the same flux at hour h will likely receive the same amount at hour h+1, so that i might not need all the daily hours to characterize the different clusters but maybe a sample of them only

again in the same direction, why would i need all the days of the year for this clustering ? buildings with the same exposure behaviour on monday will tend to have the same behaviour on tuesday (even if the values themselves may change)

.

if the data volume is a problem, maybe you would be better off sampling your variable space and gathering all the buildings on this reduced space

(but, once again, all this may well sound pretty silly)

.

Ok, so I see that I definitively didn't give enough information at first, and thank you so much for trying to understand where I'm going here ... :)

Here is the main idea of my research : finding a way to know how much W/m²/h from beam solar irradiation each building of a city receives on its façades. The amount of solar energy depends on the characteristics of the buildings itself but also on the way buildings around it are organised.

My idea of solving this was the following :

As the calculation time won't allow us to simply make the math for the entire city ( approximately 170 000 buildings), I have to find another way to make it simplier to evaluate.

Following the hypothesis that it exists differents types of curves describing the amount of energy received by a building during the year (the 8760 values in W/m²/h/building) if I can tell that those types of curves correspond to different types of environnements (discribed with variables like distance of the closest building, mean hight etc.) then I wish I could simply use the environment types to deduce the curve type to which a building correspond. In other words the kind of clusters I have in mind correspond to "way of receiving beam solar energy". I hope this formulation is clear enough ?

In order to test this hypothesis, as I can't make the calculation for the entire city because of computing time, I wanted to do this on a sample and then apply it to the other buildings of the city.

So, yes I want to sample because it's time consuming to gather the information for the entire city. (the information I'm looking to at the end is the amount of W/m²/hour of beam solar radiation each building of the city receives).

Other point you raised : If two buildings receive the same amount of energy at h, they could possibly not receive the same amount at h+1, it depends on the building's organisation around them - there might be shade on one and not on the other at h+1;

Your idea about sampling on time basis is interesting, do you mean that I could then calculate the beam solar irradiation on each building of the city and extrapolate the results on the entire year ? Thinking about this, we have to keep in mind that the sun doesn't follow the same path in winter and in summer, which means that the effects of the environment (shades) might not be the same from one season to another. What I mean is that it isn't that easy to deduce the yearly curve based on what's happening on a day.

And finally, your last idea might be true, but from one month to another, the typical day would change, according to the evolution of the sun path. But a week or a month might be a good time laps to simplify the data while keeping main informations.

Again, those are my goals, and reasons to sample. I don't know in what extend they can help to decide how to justificate the sample size... And this is the part I'm working on at the moment.

NB : I'm also gonna change the first description of the problem as I see it wasn't relevant and detailed enough,  sorry for that.

A few more comments but first one question: Do you have electronic or mechanical devices that will collect this data? Or, does a human have to be at a building to do it?

Whatever the answer is will help determine how big a sample you can afford.

(1) Using a stratified sample with strata being districts of the city will not bias results. You would divide the city into some districts that (ideally) differentiate building type and solar exposure. Then, select a sample from each of the strata.  (If you selected only a sample of the districts, this would be a cluster sample. Although a cluster sample will give unbiased estimates, it is more complicated to analyze.)

(2) Treating estimates as normally distributed is OK when the sample is large because of the central limit theorem.  Computing confidence intervals as if an estimate has a normal distribution is standard procedure in survey sampling (so, you would be doing what everyone else does).  What "large" means depends on the input data.  Highly skewed values will require bigger sample sizes than variables that have nice symmetric distributions.  Usually, 50 to 100 is enough to make the normality assumption reasonable, but some analysts would say that 30 is "large".

(3) Worrying about how to account for variability over time sounds like something that may be too hard. Apparently, you do not have any advance data that allows this anyway.  So, you should just compute a sample size as if you make observations at a single time period. 

(4) You could proceed this way ...

- Using whatever data you can find that is relevant to your problem (like the US energy survey data or data from the literature in your field), calculate population standard deviations (SD) and means (M) for some variables. The ratio SD/M is called a population coefficient of variation (CV). For quantitative variables, these often range from about 1 to 5 (but can be higher).  The pop CVs are nice for calculating sample sizes since it easier to understand their scales than the scales SDs. 

- Calculate a total sample size required to obtain a specified coefficient of variation of an estimated mean (like W/m2/hour) .  CVs of sample estimates of, say, 0.10 are usually considered good.  This would say that the standard error of an estimate was 10% of the estimate itself. Do this for a range of population CVs to see what sizes of sample would be required.  

- Stratify the city by district. An alternative would be to stratify by height of building if that is more related to those 12 variables you will measure.  

- Select what is called a "proportionally allocated" sample. This means that you select the same proportion of buildings from the pop in each stratum. The total sample size will be whatever you decide based on the calculation above. The weight for every building will be N / n, which will simplify analysis.

rv

.

sampling on the time axis is a way to reduce the computation burden for the clustering step while taking into account the very strong correlations inside one time series

it could be done by computing a typical day on a weekly or monthly basis ; it could be don by randomly sampling a subset of the 24*365 hourly measures (of course, the same subset for all buildings !)

the typical day approach may sound more interpretable and may fit your approach better

using a typical day per week (average or median ; i would favor median because there are not so many days in a week, but this is just a guess) will only reduce your volume by a factor 7 ; a typical day per month will give you a reduction factor by a factor 30 which may be better (moreover, the "typicity" will be better since you have 30 measures per hour instead of 7)

note that clustering a population does not require all the minute details of the data : clustering indeed is working on large robust behaviours, so starting with an agregated view of the data (12 typical days, one per month) may not be such a great loss in **useful** information

with a typical day per month description, i think you can work on your full population for the clustering

.

then i suppose you will train a supervised model to estimate the cluster label (or cluster label probabilities) from the side information you have on the buildings

if successful, you could take any building, gather the side information, compute the probabilities of belonging to a cluster and estimate the time series for this building as a mixture of the times series representative of each cluster

.

now, you'll object that there is no time series representative of each cluster as the clustering was bulit on typical days ...

well, when you have a clustering, you can easilly compute the average time series in each cluster : this is not computationally intensive

.

i guess i would split randomly the population in three equal parts and use 1/3 of the buildings to build the clustering, 1/3 to train and evaluate the model and 1/3 to evaluate the complete methodology (what's the difference between the estimated time series and the true one ?)

.

The data is created by a model on a computer and it takes around 22 hours to calculate the solar irradiation on 1000 buildings. That's why I don't have physical limits or budget limits.

What I understand is that I have to find a variable that describes my population, and this variable should be something I'd use to interprete the results of the beam solar irradiation calculated on the buildings of the sample ...

I'll try a comparison with social survey :

I wanted to evaluate the influence of socio-professional class (SPC) on the opinion of individus, then I'd like my sample to have the same proportion of each class as the entire population has... Therefore their opinion would be the thing I evaluate and don't know at first... but their SPC should be known and for example I'd want to create a sample with the same proportion of each as in the population... So once I've created the sample, I could say that the conclusion found on the opinion on a topic, linked to the SPC of people can be extended to the entire population ?

So in my context, I should find a variable that describes the buildings, one I would know for the entire population (for exemple, height) and once I know what is the distribution of building's size, the mean, the variance etc. I can evaluate the sample size and then calculate the solar irradiation on the buildings of the sample and finally conclude on the solar irradiation regarding building's size ... is that it ?

I guess I first have to find which variables I want to work on to describe the buildings. The limit I saw on this point is that I actually want to find out which caracteristics of buildings or of city shape are linked to the way buildings receive energy. So I thought that designing the sample on thoses variables would be like a vicious circle. (or vertuously biased ^^ )

The variables that go into the model would be the ones to use for sampling because they are predictors of solar irradiation. But, if you have only location and height in advance of sampling, those are the two variables you can use for sample design.  (Does this mean that you collect the other predictors in the model after you select the sample of buildings?)

If you can model solar irradiation in advance as y = f(location, height) and the predictions from that model are fairly good (or, at least better than assuming all buildings are the same), then you could do this:

- compute advance predictions \hat{y}

- stratify buildings based on size of \hat{y}

- select a proportionally allocated sample of 1000 (or whatever size you're willing to process)

This would spread your sample across the range of solar irradiation in the building pop, assuming that the \hat{y}'s and the real y's are well related.

Mr Clerot, I saw your answer after posting my last one.

When you say :

"note that clustering a population does not require all the minute details of the data : clustering indeed is working on large robust behaviours, so starting with an agregated view of the data (12 typical days, one per month) may not be such a great loss in **useful** information

with a typical day per month description, i think you can work on your full population for the clustering"  --> work on the full population ... do you mean on the 170 000 buildings ? If so it would take too much time to calculate the typical day per month for each building of the population, that's the reason why I want to work on a sample.

Otherwise, I understood while searching information on "sampling / time series" what it actually meant (shorten the time serie to work on less but representative information) I would have put it in the descriptive statistical, is it ? (Still interesting to use when it comes to the point of creating clusters and find common and differences between the time series, in this way I also thought about tools such as skewness and kurtosis, mean per month, typical day as you said etc.)

The last point you raise sound like a solution but it would take 51 days to create the 1/3 sample for clustering (which might be a little too long and I wish I could work on a smaller sample with great results, that's what I want to justify here)...

Maybe the justification will come from the reproductibility of the finding from one sample to another : if I find conclusion on sample 1 and use a random sample 2 to see if the clusters I've created work on this second sample (you are speaking about a 1/3 of the population, but it might be less... ?) and compare the results to the information obtained by calculation, I could validate the results (in some extend).

Mr Valliant,

(Does this mean that you collect the other predictors in the model after you select the sample of buildings?) Yes, indeed. I could possibly calculate them for the entire population but I'd rather test their reliability to solar irradiation before (thanks to the sample and with clustering or regression analysis).

What I see is that the current sample I've created (1000 buildings, selected randomly) is at least representative of the population regarding the height of the buildings : same mean (7,7 m and 7,6 m), median (6 m), but different spread : 3 to 35 m for the sample while the population' s height goes from 3 to 60.

Knowing this, could I say that the sample is representative of the population regarding the size of the buildings?

I understand, from all this discussion, that the representativity of a sample has no meaning if we don't say regarding what it is representative.

If I knew for sure that one parameter is strongly related to solar irradiation it would mean that I have the answer to my question... in some way.

But maybe am I overthinking it as there have been studies and conclusion lead on smaller samples or even on created environments and buildings.

Maybe 1000 buildings for a sample is already a lot and far enough to apply my method ... I might evaluate the representativity after the calculation by reporting the results on another sample and justify the size chosen by the calculation time associated (22h for 1000 building to obtain the 8760 variables concerning beam solar radiation).