Demarcating the Study

  1. Delimiting the Target Population
  2. Delimiting the Class of Informants
  3. Delimiting the Class of Evaluators
  4. If the Population is Very Large

En Español  In Finnish   Contents

If you want study more than one event or object - in other words, you are not doing a case study - one of the first things to decide is: what are the limits of the target population of the individuals or cases that you are interested in? It must be admitted that especially in exploratory studies where you initially know very little about the object you often cannot help revising the delimitation of your population after some data have been collected, but generally you should define the limits of your target population as early as possible and certainly before collecting any major amount of empirical material. Otherwise, you will run the risk of gathering with great expense unnecessary and even misleading data instead of the right ones.

An unambiguous demarcation of the population will give a secure basis for planning your study, reporting its findings and assessing their reliability. Besides, it will be useful for anybody who is considering to apply your findings to his or her own context because it permits comparing the intended new population of study to the one that was investigated earlier. Only when the two populations match sufficiently, it is possible to assume that the findings could be valid in the new context, too, though you have to remember that this assumption will always remain a speculation only.

Beside the class of objects of a study, a researcher often has to deal with a few other populations, too, whose delimitation may be necessary. These can include:

Delimiting the Target Population

When the study concerns several empirical objects or events, these usually belong to an already existing class. Physical objects can be either people, animals or lifeless objects like, for example, 'Americans' or 'cars'. Events are temporal changes or movements of physical objects, such as 'car rides', 'collisions' or 'telephone calls'. The first and most important delimitation of the target population of a study is thus normally based on the essence of the objects or events.

DelimitationsDelimitations that are based on essence lead often to very large, perhaps infinite populations which are difficult to enumerate and manage in research. 'Cars' without any other delimitation means all cars in the universe. The remedy is to apply simultaneously two or more delimitations, perhaps on geographical and/or temporal basis like in the diagram on the right.

The strategies available for delimiting the target population are a little different depending whether your study is

Delimiting the population of descriptive study. It is always commendable to use already well established classifications, or at least to define them in objective terms so that it is not the researcher who shall deem whether a given case belongs to the population or not. Otherwise, when selecting the cases to be studied the researcher's prejudices (which can be biased) influence too much the results of the inquiry, because the researcher can inadvertently select mostly such cases which corroborate his preconceptions or hypotheses.

The difficulty is that in the common usage of language few classes of objects or events are clearly defined, or there are several alternative definitions that are being used in different contexts.

DelimitationsLet us look at an example. In her study of photograph artists in Finland at the turn of the 1980s and 1990s, Sari Karttunen dealt with the definition of an artist and the question of how the set of artists should be demarcated. Photograph artists in Finland were fewer than 200 at that time, which made it possible for the researcher to include all of them in her study. The problem was that this set had not been listed anywhere.
Karttunen (p.45) thought of several different criteria with which people had been classified as artists in earlier studies. According to published studies, an artist is a person

All the above mentioned alternatives were considered in the study by Karttunen. She ended up choosing the last mentioned method. It was accomplished by composing a "jury" of 14 people who had authority in organizations related to art. These 14 persons were then asked to suggest candidates for the group to be researched. The artists that were mentioned by most members of the jury constituted the "core group", and those mentioned less frequently became the "marginal group". These were dealt with as separate groups in the study.

The problem with a jury is that renovators of art which later become highly appreciated are not always highly ranked in their own time. Besides, when setting up the jury you would already need the same kind of criteria which you already found difficult to define in the selection of artists.

In any case, all of the methods listed by Karttunen fulfil the requirement of being reasonably objective, that is, independent of the judgement of the researcher. Objectivity contributes to the credibility of the results.

Delimiting the population of normative study. Normative study aims at improving something, and normally this target already defines which is the class of objects in the study. It might be a single problematic case or a group of cases, which selfsame cases then shall receive the later improvements.

An alternative target could be that the improvements shall be directed, not to now existing objects, but to future objects belonging to the same class. In practice, these future objects quite often are new products of the company that is financing the study. In this case, you normally would select as study objects existing products that resemble the ideas that the company has in mind for the new product. They might be the company's now existing products or competing products in the market.

In normative purpose it is customary to study meritorious cases which can serve as exemplars for later products, but it is also possible to study "pathological" cases: educational examples which are hoped to illuminate earlier errors that can be avoided in later works. You could, for example, study existing products which suffer from a certain defect. Or you could study companies that were first doing well but then made a fatal move.

Delimiting the Class of Informants

Sometimes the researcher can collect all the facts that he needs directly from empiria and from literature, but quite often it turns out that some facts can be obtained only by asking people who have first hand knowledge about the matter. It is not always self-evident how this class of informants should be defined and where these people are to be found.

When contemplating which people might serve as reliable informants it can be helpful to think about possible sources of false information. Among these, the following two are common:

  1. Informants which do not really know the facts about the topic.
  2. Informants which do not want or do not dare to speak truth.

The latter circumstance is common in countries under dictatorship and it can be found elsewhere. It can be difficult to discern in a survey, because an unwilling informant may also dislike giving his grounds for the denial.

Case #1, the respondent's ignorance of facts, is a common nuisance for a researcher who wants to gather facts, not rumors or fantasies about things which the person has no experience on. Consider the following questions and answers that could be presented in a survey:

The examples above show that there is not much point in putting questions to people if you do not first ascertain that they know something about the matter. For selecting informants who have and who can give true and pertinent information of the matter, while keeping out the less reliable informants or filtering away their false speculations, you might consider, among others, the following strategies:

Note finally that when you filter out "unreliable" informants and information that you consider erroneous or irrelevant, you should always keep in mind the possibility that the anomalous data which do not harmonize with your expectations are perhaps not faulty at all, and in fact it is your hypothesis or your expectations that are at fault. This risk is particularly grave in descriptive research which includes no final practical evaluation of its proposals.

As a contrast, in normative research and development, a more lenient selection of information and informants can be acceptable in the early stages of the project, provided that you can submit the final proposals in their time to a more rigorous appraisal by more representative samples from the concerned populations.

Delimiting the Class of Evaluators

In normative study there is always the risk of subjectivity, that is, the personal evaluations of the researcher receive more weight than the views of other people. To avoid it, the researcher should deliberately recognize his personal preferences about the object of study. He should keep in mind that others may have other views on the matter, and explicitly define the populations of people, outside of the team of researchers, whose evaluations shall guide the normative study.

Least problematic is the selection of evaluators if the development concerns only one, known group of people. Such is the case often in Action Research (see page Developing an Activity) where the selfsame group of people who is carrying out the problematic activity can also evaluate the proposals.

More common is that several distinct classes of people are affected by the project and each of them has a different opinion on the future development. Such groups are enumerated on the page Normative Analysis under the title Point of View. They include typically the expected customers or users of a product that shall be developed, or those people who are now suffering from problems that could be alleviated with the help of research and development.

Another important population are those people whose assistance will be needed in the development phase. In product development, these normally include people in the manufacturing and marketing divisions of the company. Suitable persons to the research team are normally nominated by the managers of the enterprise.

Beside the above parties who voluntarily participate in the development project there can be people who agaist their will have to encounter some side effects of the project, for example the pollution from a new product or its fabrication. It will not always be possible to enumerate these "stakeholders" in a development project or to have direct contact with them. In such a case, you might consider inviting representatives from environmental or consumer associations to participate in evaluation.

If the Population is Very Large

Once you have delimited the target population, another question arises: should you study the entire target population, or do you prefer selecting just a part of it? There are alternatives:

En Español  In Finnish   Contents

August 3, 2007.
Comments to the author:

Original location: http://www2.uiah.fi/projects/metodi