Social testing

Published in Social Alternatives, Vol. 25, No. 4, Fourth Quarter 2006, pp. 39-42
pdf of published article

Brian Martin

Go to

Brian Martin's publications

Brian Martin's website


Social alternatives need to be tested. That means experiments, including ones on a large scale. The main challenges in social testing are predictable. The biggest obstacle to social innovation is resistance by vested interests. Social testing could bring about dynamism in social change analogous to what occurs with science and technology.


Science and technology are undoubtedly dynamic: there are significant discoveries and new products every year. By comparison, social processes are static, with relatively little innovation. Could some of the dynamism of science and technology be imported to the social realm, to help create a better society?

So intimately are science and technology linked that they are sometimes called technoscience. What makes them so dynamic today? One key factor is open testing: theories are scrutinised and objects are probed to see whether they will explain the world or accomplish some task. Scientists and engineers look at the evidence: they don't automatically accept the word of an authority that the current way is the only way. They develop new ideas and objects and see whether they are better. This at least is the ideal, from which there are many deviations, as I'll discuss later.

The process of innovation can be divided into components such as research, development, demonstration and implementation (which in a market can mean commercialisation). These components do not necessarily operate in a sequence. Sometimes research leads to ideas for products, but often the demand for products leads to ad hoc solutions, stimulating research to study why things work. In any case, a second key factor in the dynamism of technoscience is a receptive audience - sometimes a profitable market - for outputs. Computational mathematics is a thriving research field not just for intellectual reasons but because there are so many practical applications.

Social innovation, by comparison, is held in a straitjacket. There is relatively little money for open testing. The reason is that many powerful groups don't want results showing the value of an innovation they oppose.

I use the expression social testing to refer to the introduction of major social alternatives for the purposes of research, demonstration and social learning. There are lots of possibilities.

* Setting up cantons with entirely different economic and political systems (Kendall and Louw 1987).

* Establishing defence systems based on nonviolent civil resistance.

* Evaluating levels of creativity and innovation with and without intellectual property.

* Trying out different ways of designing housing (e.g., Turner 1977).

* Using random selection of decision makers as an alternative to electoral democracy.

* Evaluating participatory processes for running workplaces.

* Using gross national happiness as an alternative to gross national product (Layard 2005).

* Trying out free distribution of goods and services as an alternative to the market.

* Testing alternatives to the prison system.

* Implementing transport systems not involving cars.

* Using local currencies and LETS (Local Employment and Trading System).

* Evaluating guaranteed minimum income systems.

* Assessing the long-term effects of early childhood interventions.

The first response of many researchers to such a list is to say, "We've done this already!" And they're right. There has been a large amount of testing of social alternatives. For example, studies of industrial democracy go back decades (Melman, 1958).

But this is like saying, in the year 1600, "There has been a lot of observation of the natural world." That was also correct. But the scientific revolution, coupled with developments in instruments, led to a far more systematic process of building on previous knowledge. The same applies to social testing. Research so far has been on a relatively small scale. Where are the social experiments on a scale equivalent to a giant accelerator or the human genome project, costing billions of dollars? Where are the social innovations with backing equivalent to a new model of car or a major software release?

Consider the jury system, the subject of public debates about compulsory jury service, the size of juries, and unanimous versus majority verdicts. There has been a fair bit of research on juries but, considering their importance compared to breakfast cereals, not really that much. Imagine an experiment in which several different jurisdictions used different methods of selecting and training jurors and running trials. All standard social science techniques could be used: for example, experimental conditions could be assigned to jurisdictions randomly, and results evaluated by an independent panel. Conditions could be switched after a period, so each jurisdiction experienced different processes, to reduce the effect of demographic factors.

Such an experiment, if done well, would have credibility for making policy. A single experiment would not be the end of the story: others might challenge the result and perform their own experiments, just as in scientific research.

There have been lots of small experiments using randomly selected citizens to make decisions about contentious policy issues (Carson and Martin, 1999). Those were difficult enough to fund and carry out. But imagine an experiment with an entire community using random selection as an alternative to local government, attempting to see whether approaches used in ancient Athens could be adapted to contemporary society. The research budget might run to tens or hundreds of millions of dollars. But if a system for generating better decisions could be developed, the benefits would be far greater than the costs.

Experiments could be lengthy and complex as well as expensive. Developing a social alternative is bound to require quite a bit of trial and error, just like developing a new technology. Early electoral systems needed refinement to overcome problems such as ballot stuffing and gerrymandering. Alternatives to electoral politics would need a similar process of learning and adaptation.

Today, social science is seen as a cheap, low-status cousin of natural science. In part this is due to social science being held back by the resistance of those opposed to social innovation that threatens their position and worldview, analogous to the way science was treated, centuries ago, by the church.


The point of social testing is beneficial social innovation, not publications, research empires or support for a particular interest group. There is definitely a risk that the interests of experimenters, or their funders, could bias the experiments and the interpretation of results (Lindblom 1990). One way to counter this risk is to involve participants in the experimental design and evaluation, as co-experimenters rather than passive subjects. This is in the spirit of action research, namely research designed to increase knowledge through promotion of social change.

Scientific research can be difficult at the best of times, and likewise there are many challenges in undertaking social testing. Vested interests are probably the biggest obstacle, but also important are the role of humans as experimental subjects, ethical and legal issues, and entrenchment.

One big difference between social testing and an experiment in the lab, using microwaves or fruit flies, is that people are involved as subjects, and they might take actions wrecking the validity of the study. Imagine a study of two enterprises, one with equal wages and the other with highly unequal wages, to determine which wage system leads to greater productivity. Some participants might have personal preferences and try especially hard to make their enterprise successful, or perhaps to undermine it.

This problem is not new. Psychologists have a great deal of experience dealing with knowing subjects. One approach is to use deception: subjects are told a study is about visual perception when actually it is about cooperation. So in the two enterprises, the participants might think the study is about productivity but actually it is about job satisfaction. But with larger-scale experiments, participants can talk with each other and have more time to assess what is happening, possibly affecting the results.

When medical researchers test treatments for pain and illness, they often use placebos, substances or interventions with no active ingredient. But placebos often lead to genuine improvements in health, complicating research. The marketing of new products has its own version of this effect: some people will swear a new consumer product is better even though the only difference is labelling, packaging and marketing.

The response of subjects to being part of experiments is certainly a complication, but there are various ways to take it into account. For example, if there are lots of experiments that go on for years, the impact of being a special participant is likely to wane. On the other hand, if participants continue to respond favourably, this can be a positive, just like the placebo effect.

Another complication is the ethical status of social testing. Having a different policy on terrorism, drugs or imprisonment might lead to disastrous effects. Is it ethical to subject a community to the risk? The issues here are not new, just on a grander scale. Obtaining permission for social testing could be arduous. One way of addressing individual concerns would be to allow people to opt out or opt in, for example by changing jobs or moving to a different neighbourhood. However, opt-in and opt-out opportunities might reduce the validity of results.

Then there are legal issues. Could participants sue because of the 'social side effects' of experimentation?

The ethical and legal dimensions of social testing need to be put in context with current practices concerning new products and policy making. Companies routinely introduce new products. These could be likened to experiments, but seldom is ethical approval required. A product like the mobile phone has enormous social impacts, but the responsibility of manufacturers is highly circumscribed. Similarly, governments routinely introduce policies on crime, drugs and terrorism. But few expect to be able to sue the government over the adverse effects of these policies.

Should social testing be held to completely different standards than today's product or policy innovations? To do so might reflect a resistance to change rather than a genuine concern about the effects of experimentation on society.

Technoscience is undoubtedly dynamic, but not always for human betterment. For example, there is a vast amount of research and development into new weapons, but very little into technologies for nonviolent struggle. The implication is to orient social innovation to social improvement. This is similar to medicine, which is oriented to improving health rather than harming it. There is no point in social testing on how to make people less friendly or less skilled.

Another potential problem with technology is entrenchment: large, expensive and hazardous technologies, with long lead times, are hard to change once introduced. The archetypal example is nuclear power (Collingridge, 1983). In contrast, energy efficiency and small-scale renewable technologies are flexible: low investment and low risk make changes easier and errors less damaging.

There is also a risk of entrenchment in social innovation. For example, US urban renewal in the 1960s involved building huge apartment blocks that turned out to be social disasters. Today's huge US prison system is becoming entrenched due to the vast financial and organisational investment in what is sometimes called the prison-industrial complex (Christie, 1994). Actually, both these examples suggest the potential value of social testing. Instead of demolishing vast areas of slum housing and building high-rises, trials could have been carried out in a few cities, with several options compared.


Entrenched ideas are central to resistance to social innovation. Powerful groups, such as politicians and corporate leaders, have a strong stake in current social arrangements and seek to keep questioning within narrow limits. Significant change is dismissed as utopian or dangerous. Few members of powerful groups want social testing that might demonstrate that change is feasible and desirable. In other words, social testing is a threat to powerful groups today just as scientific experimentation was a threat to church leaders in the 1600s.

Powerful groups are often quite happy with change, just so long as their power is maintained. They engage in systematic social testing for particular purposes, namely control and manipulation. Firms do a vast amount of market testing and governments investigate how to win support for their preferred policies. But most such investigation is not in the public domain. Corporate and government leaders are not keen on testing when the results aren't under their control and might lead to pressure for change they don't want.

In the 1990s, there was a proposal to test the medical prescription of heroin to addicts in Canberra. The research team went to extraordinary lengths to consult stakeholders, reduce risks and create the conditions for a valid study (Bammer, 1997). Despite this careful preparation, the federal government torpedoed the study: key politicians were worried about the response of foreign governments and vocal domestic groups that did not want a demonstration contrary to current drug policy.

Different countries have different electoral systems, with significant variations. Some countries have voluntary voting, others 'compulsory voting' (compulsory attendance at polling booths). Some countries have single-member electorates, others have proportional representation. What is surprising is how seldom countries change from one system to another. Even rarer is research into alternatives to electoral democracy. One obvious reason is that if the system is changed, some current politicians might lose their jobs. In Australia, compulsory voting has been supported or opposed mainly on pragmatic grounds, namely whether it would help key parties. It would be a relatively simple matter to try out voluntary voting in one or two states, or in selected electorates, and evaluate the results. But research of this sort is absent from the agenda - so researchers have to be content with comparing how different electoral systems operate in different countries.

In corporations, managers do not want research into how workers could do the work without managers. It might show that workers' control is viable, and that is a threat. Even in universities, all sorts of changes in policy and structure are made without any effort to rigorously evaluate effectiveness.

Vested interests, linked to dominant sets of ideas, constitute the most powerful source of opposition to open social testing and popular innovation. But there is another source of resistance: some critics of the status quo and promoters of utopias. Testing might show that their ideas don't stand up to scrutiny. Supporters of socialism, for example, might have reservations about a test of two communities, one run on socialist principles and the other on capitalist principles. The test might be set up to make socialism look bad. But it might be set up fairly and still make socialism look bad. That is a risk.

I suspect most advocates of alternatives would welcome a fair test. Critics of television would probably welcome a study of a TV-free community. Critics of the nuclear family would probably welcome a study of communal alternatives.

Social testing is bound to be contentious. A lot is at stake. But even when experiments are run, there is a further obstacle: implementing the findings. After all, there are plenty of findings available today, based on evaluation of naturally occurring experiments, that are almost totally ignored. For example, a study of democratic transitions over the past several decades shows that 'far more often than is generally understood, the change agent is broad-based, nonviolent civic resistance - which employs tactics such as boycotts, mass protests, blockades, strikes, and civil disobedience to de-legitimate authoritarian rulers and erode their sources of support' (Karatnycky and Ackerman, 2005). But this has not led defence departments to pour vast resources into nonviolent civic resistance, nor even much support for further independent research into this alternative.

So, perhaps more than anything else, what is needed is research into how findings can be used to stimulate implementation of innovations, not forgetting that these innovations in turn need to be subject to further social testing.

Learning is part of everyone's life, in trying to figure out what is worth doing and how to go about it. Some learning is formal but most is informal, including a lot of trial and error. Social testing should be added to this collective learning process.


I thank Gabriele Bammer, Lyn Carson, David Hess, Don Eldridge, Don Fletcher, Truda Gray and Ned Woodhouse for valuable discussions and comments on drafts.


Bammer, G. 1997. 'The ACT heroin trial: intellectual, practical and political challenges. The 1996 Leonard Ball Oration.' Drug and Alcohol Review 16: 287-296.

Carson, L. & Martin, B. 1999. Random Selection in Politics. Westport, CT: Praeger.

Christie, N. 1994. Crime Control as Industry: Towards Gulags, Western Style, 2nd ed. London: Routledge.

Collingridge, D. 1983. Technology in the Policy Process - Controlling Nuclear Power. London: Frances Pinter.

Karatnycky, A. & Ackerman, P. 2005. How Freedom is Won: From Civic Resistance to Durable Democracy. New York: Freedom House.

Kendall, F. & Louw, L. 1987. After Apartheid: The Solution for South Africa. San Francisco: ICS Press.

Layard, R. 2005. Happiness: Lessons from a New Science. London: Penguin.

Lindblom, C. E. 1990. Inquiry and Change: The Troubled Attempt to Understand and Shape Society. New Haven, CT: Yale University Press.

Melman, S. 1958. Decision-making and Productivity. Oxford: Basil Blackwell.

Turner, J. F. C. 1977. Housing by People: Towards Autonomy in Building Environments. New York: Pantheon.