Farm structure survey - measurement errors

This article presents characteristics with significant measurement errors identified by countries for the 2010 and 2013 farm structure survey (FSS). Moreover, it indicates FSS characteristics which by design cause comparability problems, since countries use different ways to measure them. The article is part of an online publication on methodological articles dealing with the farm structure survey. The analysis and findings are based on the national methodological reports (NMRs).

Full article

Characteristics with significant measurement errors in the FSS 2010 and 2013

A measurement error can be defined as the difference between the collected value and the true (individual) value. These errors may result from the respondent, the interviewer, the survey instrument (questionnaire), the mode of data collection (face-to-face interviewing, telephone interviewing, self-administered mail survey, on-line survey, use of register information, etc.), or the information system (respondent’s report-keeping system). Such errors may be random, i.e. caused by any factors that randomly affect measurement of the variable across the sample. If they are not random, they may result from a systematic error caused by any factors that systematically affect measurement of the variable across the sample, sometimes called bias.

All countries provided in their NMRs information on the causes of measurement errors, their prevention, detection and actions taken to remove them from the final dataset.

However, most countries limited the statement, mentioning only that some kind of prevention existed to avoid large measurement errors (e.g. training of enumerators, logical and arithmetical control rules in the questionnaire, data cross-check, etc.), and/or that if any measurement errors occurred they were detected during data checks and removed before the final dataset was transferred to Eurostat.

By contrast, only a few countries indicated specific characteristics for which significant measurement errors occurred (see Table 1). The reason for this could be that the other countries have not actually been able to identify specific characteristics that cause systematic measurement errors, or that such characteristics exist and were recognised but have not been reported. Moreover, it seems that it was rather difficult for countries to distinguish between various types of errors and thus the coverage errors, non-response errors, or processing errors were often reported as measurement errors.

Table 1: Characteristics with significant measurement errors

In general, the following (sets of) FSS characteristics were identified as having the largest measurement errors:

type of tenure and farming system (sensitive variables for respondents),
destination of the holding’s production, in particular whether the household consumes more than 50% of the value of the final production of the holding,
distinction between temporary grassland and ordinary permanent grassland as well as between permanent grassland no longer used for production purposes and rough grazings,
identification of the short rotation coppices,
classification of the crops grown for energy purposes,
distinction between production of renewable energy for farm and for holding's own use,
labour force:

number of working hours in the last 12 months carried out by holder, holder's spouse and other family members,
number of working hours for part-time farmers,
number of working days carried out by contractors,
working time on other gainful activities (OGA), especially for hired workers:

separation of farm work from forestry work,
distinction between farm or horticultural production and further processing.

importance of the OGA directly related to the holding,
tractors and harvesters, especially for machines used by several holdings,
rural development support schemes,
irrigation, in particular volume of water used for irrigation, surveyed in 2010,
agricultural production methods, surveyed in 2010.

In most cases, the questions/sections in the questionnaire were difficult for respondents either due to the complexity of the surveyed characteristics (i.e. the relatively large number of follow-up enquiries or retrospective calculations provided by farmers), a lack of comprehensibility of the requested characteristics (i.e. difficult or unclear definitions), or even the sensitivity of the information asked from farmers.

Characteristics which are differently measured by countries in the FSS 2010 and 2013

Countries were requested to comment on the comparability of the definitions, classifications, measurement and reference periods of characteristics in the FSS. This should allow evaluation of comparability across countries and over time.

Most of the countries reported mainly changes of definitions and sometimes also of reference periods. Moreover, most of them replied that no (significant) changes had occurred.

Based on the available information, the impact of possible changes in the way characteristics were measured on comparability could not be assessed.

What could be mentioned is the different way to measure and report labour force working time and areas of mushrooms.

Labour force working time: Annual Work Unit (AWU)

Commission Regulation (EC) No 1200/2009 defines the Annual Work Unit that is used to express the amount of work spent by the farm labour force for the holding.

According to the information provided in the NMRs, fourteen countries (Belgium, the Czech Republic, Germany, Estonia, Ireland, Croatia, Hungary, Malta, Portugal, Slovenia, Slovakia, Finland, Sweden and the United Kingdom) use exactly the same number of annual hours in the FSS (1 800) as provided in Commission Regulation (EC) No 1200/2009. For the remaining countries this number varies, with the longest working hours used in Greece and Luxembourg (2 200 hours, i.e. 275 working days of eight hours per day) and the shortest in France (1 575 hours, i.e. 225 working days of seven hours per day). Those countries that provided this information for FSS 2010 (Croatia, Lithuania, Luxembourg, Hungary, Malta, Austria, Poland, Romania, Finland, Norway) have used exactly the same definition as in the FSS 2013. The national definitions of AWU applied by countries are provided in Table 2.

Table 2: Definition of annual work unit (AWU) used by countries

Luxembourg

Besides the different number of working hours used to define the full-time equivalent employment as compared to other countries, Luxembourg makes a distinction between the amount of work by young and old farmers. More specifically, persons aged between 15 and 18 years, and persons aged at least 65 years, who were declared as full-time working in the survey, have been transformed into part-time working using fixed coefficients. Persons under 15 years or over 80 years are excluded from the calculation of working time. This approach is taken for reasons of consistency with the economic agricultural accounts in Luxembourg. The same definition has been used in 2010 and 2013.

Areas of mushrooms A specific approach is taken by France when it comes to measuring and reporting the areas of mushrooms.

France

The mushrooms characteristic is collected in terms of production and not in terms of area. One hectare is considered to produce 180 tonnes of mushrooms per year. The data is reported to Eurostat in terms of area. However, using the average production per hectare to convert the data might introduce some errors, especially for population breakdowns.

Conclusions

When comparing data across countries, users should pay close attention to the characteristics with significant measurement errors listed in the first section above, as well as to the labour force working time, and, concerning France only, to the areas of mushrooms .

Source data for tables and graphs

Farm structure survey - measurement errors: tables