Planning an Empirical Study

Planning an Empirical Study

En Español   In Finnish   Contents

The Goal of Research

When the object of study belongs to empiria, the tangible world of people, objects and events, the study is called "empirical" or "factual" as a contrast to formal sciences like mathematics and logic, which have no association to empiria. These latter deal with theory only, and they aim at clarifying its structures, i.e. the forms of thinking, such as the processes of logical or mathematical analysis. They will not be discussed on this site.

If we now want to get a general view on the usual approaches and methods in the research of professions and artefacts, it is worthwhile first to observe that the conventional dichotomy between qualitative and quantitative approaches (the "two cultures of research") is here not fruitful. When the problem to be studied comes from practice, it will seldom consist of qualities or quantities only, but instead it will contain both, or more exactly it will contain aspects that the researcher can choose to register as he pleases, either as qualities or quantities. In academic study it seldom does any harm if you define your problem so that you can use your favorite methods of measurement, but in practical studies you will have better prospects for success if you can use research tools of both types.

Besides, in the study of activities or industrial products qualitative and quantitative presentations are not the only possible ones - sometimes a picture can say more than a thousand words or measurements can. The approaches of research that are presented in the following, allow mixing several modes of presentation in the research project, though it cannot be denied that the mode of registering facts restricts the choice of method in their analysis.

Instead of the above mentioned researcher-centered classifications, it is instructive to categorize the methods in the empirical study of human activities and artefacts on the basis of the expected results from the study:

Another, less important dividing line between research methods is based on the expected degree of universality of the results of the study. This decision has to be taken into account when determining the extent of the study, i.e. how much material has to be collected, and this in turn influences the selection of analysis method. Two principal alternatives in this respect are:

When combining the two categorizations we get the following table which contains four approaches or styles of study with distinctly different methods. Note that these approaches, while being usual in sciences, are by no means exclusive to the scientific world - they are based on the same four types of reasoning that are very often used in your daily life. Some mundane counterparts of these four scientific approaches are written in green color.

  Descriptive styles of study: Normative styles of study:
Intensive study of one or a few cases: Case study. Study of the history of art or of design where objects are seen as individual entities. In daily life: inspecting an object new to you. See below. Evaluative case study. Critique of works of art. Testing products. Removing a problem. Augmenting or enriching an object. Developing a new industrial product. In daily life: planning an improvement to something. See below.
Extensive study which concerns the entire class of cases: Describing or explaining invariances, "laws", common to all the cases in the class. In daily life: elementary education. See below. Creating general theory of practice, e.g. procedures, algorithms, regulations or standards for an activity or for design. In daily life: teaching or learning a profession or developing it further. See below.

All the approaches of research enumerated above can be used to assist any professional or industrial activity, but each approach does it in a particular manner that differs from the others, as can be seen in the diagram below.

The four above mentioned approaches of research will be explained below in more detail. You can often select one of them as a starting point when planning your own project as a logical chain of operations which starts from the available inputs of theory and data and finally produces the desired descriptive or normative output.

An alternative point of departure could sometimes be adopting and modifying the approach of an earlier investigation, if a suitable one is at hand.

The Descriptive Approach

Descriptive research aims at gathering knowledge about the objects of study but it tries to avoid bringing about any changes in the objects. This knowledge consists mainly of describing the objects. There can also be explanations why the objects are as they are. Moreover, the researcher may sometimes want to collect opinions of people about the pleasing or unpleasant aspects of the objects, but a descriptive study never plans or proposes improvements to the objects.

Intensive Descriptive Approach

When the objects consist of one or a few cases only, in other words the study is ideographic, its process needs not much differ from a situation in normal daily life when you want to get acquainted with an object that is new to you. Because you are studying an object that you do not know well, it will be impossible to plan all the phases of the investigation exactly. It can even be difficult to decide which facts are to be collected, and this becomes clear first after some data have been analyzed. You must be prepared to change your plans as soon as the investigation deepens your understanding of the issue. This type of method is often called iterative.

ViewpointsThe process normally starts at studying the object from several different viewpoints, either from the angles of various established sciences (like in the diagram on the right) or just from miscellaneous practical points of view. Repeating the different vistas helps you to understand better the object, because the initial inspections can serve as a basis for later examinations. The process thus resembles a spiral which gets gradually closer the goal.

Sooner or later during the inspection you will be able to specify the most revealing points of view for your study and explain how you "understand" the object. Thereafter you will need to gather only such empirical data that are related to the problem; that will enable you to minimize the material you will have to analyse.

The iterative process is repeated as many times as necessary to reach a satisfactory result, or until the resources are exhausted.

Typical iterative processes are explained in detail on the pages Exploratory Research and Case Study. The method can also be used when you study a number of cases which are essentially similar; a suitable method for this is often Comparative Study.

Extensive Descriptive Approach

When you are studying an extensive number of cases you would end up with an immense amount of data, if you did not in advance restrict your interest into only a few types of data. To be able to specify meaningfully which data are to be collected, you need to have already at the outset of the project a clear idea about which data you need gather and how you want to analyze them. This in turn makes possible to plan in advance the entire process where each operation is done only once and thus the work becomes speedy and effective.

What then is an effective process of research? At its best it is a logical series of operations which starts from the target or problem of the project, exploits existing knowledge when available, obtains more information when necessary, and finally by analysing these produces the desired result, be it descriptive or normative. Each of these phases will be based on the results of preceding phases, and the quickest method would therefore be to carry them out as a series. Such a linear process is often given in textbooks of methodology as an ideal process of scientific research. It is, indeed, common in technological research which deal with unequivocally measurable physical things. The process is simply a sequence of distinct tasks, typically the following:

Research project

  1. Defining the problem, perhaps with the help of a study of literature, and selecting pivotal concept definitions. Formulating the hypotheses (if any)
  2. Planning the empirical study. Defining the population to be studied and the methods of sampling and measurement
  3. Gathering data.
  4. Analysing the data. It can consist, for example, of expressing the data as a model, or of verifying a hypothesis with them, or of predicting the future of the object of study.
  5. Assessing the validity and reliability of the results
  6. Reporting

The Normative Approach

The target of normative research is to improve the object of study or to create a new, better state of things. As was indicated in the table above, the approach will be slightly different depending on the extent of the study, i.e. how many objects that shall be improved.

Intensive normative approach

In intensive normative study, in other words, when attempting to improve an object or a state of things, it is often possible that some of those people participate in the project whose opinions or interests shall guide the preparation of the normative proposals. This is an option that may or may not be used, but in all cases the decision will have a deep effect on the methods of the study. In this respect, two distinct alternatives are (though intermediate approaches are possible, too):

Participatory normative study. A reliable though often arduous method of preparing proposals for improving a state of things is the participation of the people whose lives will be affected by the proposals when carried out. Interest groups that might be relevant in such a project are enumerated on the page Normative Point of View. However, quite often it would be difficult or impossible to arrange in practice the participation of all these people.

Normative AnalysisIn the case that at least the majority of pertinent groups of interest can be represented in the meetings of the project, there are good chances of finding an alternative acceptable for all, and in the best case it can be done quickly and cheaply.

When the problem to be corrected is simple and there is no disagreement about goals, often a single meeting of all parties is enough to agree about both the problem and its solution. As a point of departure can often be taken either the existing disadvantage or an ideal state of things which perhaps is in itself unattainable, and on the basis of one or both of these the meeting can agree about the proposal. In the best case further studies will not be needed at all.

If the first meeting, however, fails to reach unanimity, the normal option then is to agree on the topics to be investigated until the next meeting and on the principles that a renewed proposal should conform to. Normally the meeting also authorizes a workgroup or researcher for these tasks.

Because participation usually brings with it contrasting opinions, it is quite normal that disagreement compels redoing a part of the work and returning to an earlier stage of the process. If there are many such backward returns the process begins to resemble more a circle than a linear succession of decisions. Indeed, a spiral like the one on the right is a very typical model of a development project.

Normative spiralNormal phases in the iterative "spiral of development" are as follows.

  1. evaluative description of the initial state (perhaps including its earlier development) and defining the need for improvements
  2. analysis of relationships and possibilities to change things
  3. synthesis: proposal for improvement (and its testing, in a project of development)
  4. evaluation of the proposal or of the test.

By repeating the sequence from 2 to 4, and by gradually improving the proposal, an acceptable result is usually found.

Participatory normative approach is explained in more detail on the pages Normative Point of View, Recording Normative Data, Participating Normative Analysis, Evaluating Normative Proposals and Normative Reporting. Examples of typical normative research and development processes with this approach are described on the pages about Action Research, of Developing an Industrial Product and of Scientific development of a work of art.

Professional normative approach. When the preferences of all the pertinent interest groups are self-evident or the researcher is able to find them out with survey methods, or when there are practical reasons which prevent participation of these people, the entire process of normative research can be carried out by the professional researcher(s) with no participation of the people which will be affected by the project. The process might then consist of a linear series of simple decisions, for example as follows:

  1. Defining the target. It could be e.g. removing an existing inconvenience or creating a new product. An essential component of the target is also declaring the point of view that shall be used when making the normative proposals.
  2. Defining which factors in the context can be modified and which not. You might think, for example, that the quickest way to achieve the target could be to change the political system. However, the project cannot do it and therefore the present state of political power must be taken as "given".
  3. Planning how to reach the target, preferably as a few alternatives.
  4. Selecting the best alternative (which is either the one that fulfils best the target, or the one that gives a satisfactory result with least expenses).
  5. Making a detailed plan of action.
  6. Submitting the practical proposals to the people that can decide on them (e.g. the management of the company or a governmental agency) which may require redoing any of the preceding stages.
  7. The operations in practice (in a project of development).

The professional normative approach is explained in more detail on the pages Normative study of literature, Normative Point of View, Recording Normative Data, Professional Normative Analysis, Evaluating Normative Proposals and Normative Reporting. Examples of this approach are industrial new product development and the development of an existing activity with methods engineering.

Extensive normative approach

Studies which aim at developing a great number of objects, have a special character which influences their methods, too. Typical traits in these studies are:

You can often plan the method for an intended extensive normative project as a linear process, such as:

  1. Defining the target, which usually is to remove a widespread problem in present activity or in present production and/or to correct an outdated passage in existing theory. Defining the general principles and goals that have to be observed in the work, for instance the targets of safety or economy. An essential component of the target is also declaring the point of view that shall be used when making the normative proposals.
  2. Stating which facts in the context have to be taken as "given" facts which cannot be modified.
  3. Planning how the fulfil the target. This is done preferably as several alternatives, including one where the present state of things continues as such.
  4. Selecting the alternative that is best. This can either be the one which fulfils best the targets, or the cheapest of the acceptable alternatives.
  5. Asking opinions or statements from interested parties.
  6. Presenting the proposals to the direction of the commissioning organisation, which then either accepts the work or demands new alternatives to be made.
  7. Once the proposals have been accepted, it can be necessary to arrange a campaign of publicity or professional training, to disseminate the new instructions to all those who can make use of them.

The process of extensive normative study is further explained on the pages Normative study of literature, Normative Point of View, Recording Normative Data, Professional Normative Analysis, Evaluating Normative Proposals and Normative Reporting. Examples of applying it into a few particular fields are given under the titles How to Create Theory of Design and How to Create Theory for an Activity.

Examples of theories created with this approach are several Theories of Production, Theories of Design of various products such as architecture or furniture, and theories about goals of product design, in topics such as usability, beauty, message, ecology, economy and safety of products.

Modifying the Method of Another Investigation

The general models of process given above are not the only possible starting point in selecting the method of investigation for your particular problem. A working set of methods can often be adopted from an earlier published research project, thus saving much time othervise spent in planning and testing a fresh tailor-made method.

Note that when duplicating the methods of an earlier project you have to take care of not copying those procedures that are unsuited to your special problem of study. When imitating the method of an earlier project, you cannot avoid of tacitly accepting many components of its paradigm - approach, models, definitions of concepts, and even tacit evaluations. This may be advantageous because it promotes "normal science" i.e. the steady growth of the field of study where the scientists base their work on the results achieved earlier.

One potential disadvantage of relying on an existing paradigm is that it tends to restrict the area of new studies. A strong paradigm invites the neophyte scientist to study such problems that are firmly related to the existing theory and which already have been studied to some extent, and for which there are well-tried methods. This is the reason behind the fact that many research institutions today are specializing in either "qualitative" or "quantitative" studies (the Two Cultures of Research), which often unnecessarily restricts their work.

Imitating earlier methods may be convenient when you study problems that have emerged inside the scientific community. This is the case often if your goal is just to make research, for example a thesis. However, when the problem originates from the practical world, from the recent evolution in society and in industry, your chances of finding a solution to it may improve if you use the more tortuous approach that is outlined below.

Planning the Use of Resources

The target of the project - what you are expected to accomplish - has been discussed above, and when well defined it will then give the basis for scheduling your work and planning the resources necessary for the work. Targets can make the work easier and faster: it is easier to proceed when you know what you are aiming at.

Beside the goals for what shall be achieved, it may be useful to plan those resources which are critical or scarce, like e.g.

Scarcity of resources can compel you to revise the project plan, as there are great differences in the costs of methods. Savings can be attained by e.g. the following strategies:

Of course, such savings often result in a lower level in the reliability, validity and in the practical usefulness of the outcome.

Time Schedule

Timing of the research project is often governed by outside requirements and restrictions, like:

Gantt diagram If you can divide your project into separate tasks with their individual targets, it also becomes possible to plan each task in advance.
If you present each task as a bar on a calendar based grid, the result is a Gantt diagram, sometimes called "road map", an example of which can be seen in the upper figure on the right. Such a diagram may help defining the most effective sequence for the jobs and allocating resources to all the various tasks. It also helps in budgeting your incomes and expenses, or your own weekly hours and those of your research assistants.

Follow-up The calendar based Gantt diagram is also an effective tool in the follow-up of the progress of the project. If you weekly mark in red the real progress of each task, like in the lower figure, you will get a good overview of the general situation of your project. In the example, the interviews are well ahead of their timetable, while reporting lags behind and might require assistance.

PERT diagram The tasks in a project are often linked in such a way that it is possible to start a task only when some other task is completed. In a project diagram, you can indicate such a link with an arrow between the tasks, and thus create a PERT graph (abbreviation from Program Evaluating and Review Technique). An example is on the left. You can also include such "tasks" (thin black arrows in the graph) which are just logical dependences between the stages of work and involve no working time. The graph can be drawn on a calendar grid, or just on plain paper. It is usually made in a horizontal position.

If you refine the PERT graph by adding the estimated duration of each task, it becomes possible to specify the critical path of your project. It means the sequence of those tasks which dictate the shortest possible duration of your project (assuming that you have enough resources at your disposal). In the PERT graph on the left, this succession of tasks (the red arrows) contains 2+1+2+2+3+2 = 12 working days.
In reality, few research projects have unlimited man-power resources; nevertheless a PERT graph may prove useful in clarifying the logical chain of tasks.

Other types of models suitable for planning a project, such as the Unified Modeling Language, are enumerated on the page Models.

For managing a large project, a computer with a project planning program is often used. If you feed the durations and the logical relationships of the various tasks into the computer, it then prints out the general plan of the project as a Gantt or PERT graph, whichever you choose. The project planning program can also help you in the follow-up of the project.

En Español  In Finnish   Contents

August 3, 2007.
Comments to the author:

Original location: http://www2.uiah.fi/projects/metodi