HumansMatter_logoNoMvt

Acting Together

Acting Together

Idea

Can We Measure Our Thoughts?

Health, Learn, Work

Contact

Stéphanie de Chalvron

Contact

Stéphanie de Chalvron

Contact

To:
- Get more information
- Share your challenges
- Get a product demo

The context

As soon as one wishes to measure something other than a distance, time, weight, height, etc. it becomes complicated to find and then use the right measuring instrument. The first scientific rule to measure an emotion, a behavior, a social representation, a state of mind, the presence or absence of a pathology, etc. is to use an existing and above all… valid tool. Valid? What does that mean? This means that the tool has been built according to a scientific method (the “Churchill” method), that it has been verified that it measures what it is supposed to measure and only what it is supposed to measure; and that it gives a threshold (from which the person is or is not at risk with respect to a given criterion; e.g., burnout) and/or norms (a mean and standard deviations according to Gaussian distribution). To develop this type of tool, it generally takes between 3 and 9 years of research, which is no small feat! The problem is that most of these valid tests are either subject to copyright, or accessible via scientific articles that you have to know how to decipher… Understanding the results of a factorial analysis is not necessarily obvious.

Beware the “homemade” tools

The result for companies (at least in many cases) is that in order to measure the environmental stress in a department, for example, they will build “homemade” tools, without suspecting in the least that metrology (sociometry, psychometry) is a science in its own right. They may also use tests that have been presented to them as being reliable; however, without a scientific validation article reviewed by experts, a test may be reliable, or it may not be, there is no way to be sure, which is quite a risk! In both cases, it is almost like trying to measure the length of a table with a thermometer, the example may seem a bit over the top, but it actually is not that far off the mark… Why?

The steps of Churchill’s method

The first two steps of Churchill’s method consist of interviewing several populations concerned by the criterion to be evaluated (social representations, behaviors, opinions, motivations, values, etc.). First and foremost, a population of experts on the subject, but then also people likely to be concerned in some way – the aforementioned average Joe and Jane. These interviews allow for a “rupture”; in other words, they allow researchers to break with their prejudices. It is a human flaw (a cognitive bias) to believe that we know how the majority of people think, we categorize… and without having collected as many opinions as possible, we can miss what is most important, or get our priorities wrong. These interviews also allow us to identify the vocabulary used by these average Joes and Janes. You will agree that the statement “at the moment, anhedonia characterizes me” means the same thing as “at the moment, I don’t feel like anything”, but the first formulation may not be understood by everyone; therefore, it may lead to erroneous answers (“Oh, I thought it meant […]”) or to an avoidance of the question entirely due to not understanding it. Once the verbatim (faithful transcription of the interviews) has been compiled, a content analysis is used to generate what is called the “item pool”; i.e., a list of assertions which is then proposed in the form of a questionnaire (after checking the context effects for the order of the items and managing potential biases in the responses as best as possible).

The following stages of Churchill’s method allow the tool to be refined, to eliminate complex items for example; i.e., those which were not always interpreted in the same way by all the respondents, or those which in the end do not provide much information necessary for understanding the problem. We also check the validities (there are several) and the reliability of the tool.

It is only after all these steps that we begin to identify thresholds or norms… This is why it takes so long to build a valid tool, but this is also why it is a valid tool, one could also say “a tool that minimizes error.” The less valid a tool is, the more likely it is that the scores it produces will be false (from a slight tendency to a clearly false result); and therefore the more likely it is that decisions will be made on a totally random basis… In fact, in this context, using the scores collected for an important decision is about the same as flipping a coin; except that anyone who does not know the first thing about measure theory will have had the impression that they were making a considered decision.

Well then, what is the right way to do it? How to develop a reliable tool, adapted to each case when you do not have 9 years to spare?

The idea is simple and effective: ask the future respondents to design the questionnaire themselves. Thus, on one or more themes, defined by a word or an expression (e.g., to know what individuals are interested in, the words will be “death”, “love”, “the environment”, etc.), each person gives one or more statements and justifies them. People can add their own theme(s), the goal being to collect as many opinions as possible and as many verbatims (vocabulary) as possible while leaving as much freedom of expression as possible.

The file is then analyzed using Q methodology[5], which allows us to identify the main themes and a first factorial analysis allows us to categorize them. Then, a work of experts will recover the information, thus the sentences and expressions of the individuals to gather them in the form of questionnaire(s).

All that remains is to choose the scale (Likert, VAS, etc. depending on the need for precision), to check the order of the items, and to submit the whole to a sample of people (remember the rule: 10 people minimum per item; thus for a questionnaire of 20 items, a sample of at least 200 people will be needed); then, once the questionnaire has been refined, resubmit it to a larger sample in order to study the validities and reliability, and then to identify the norms and/or thresholds.

For the example of our survey on the subjects of attention, a study which could have been satisfied with being a simple survey intended to know what the attention of French people was focused on in the middle of an unprecedented health crisis, we obtained 640 responses for the first stage and 4725 for the second.

If we had chosen the polling method, our results would have been necessarily biased; indeed, we would have very certainly inserted questions about health. This is a striking example, because although the theme of “health” was proposed, it did not emerge in the analyses, and in the end very few people chose it… Were they in denial, or fed up? It is difficult to know, but what we did learn, thanks to our method, is that this subject was no longer part of the French people’s attention at that time. In addition to a theme proposed by the respondents and which we had not thought of – social networks and the transmission of information – the themes concerning the values of love, altruism, generosity, learning, etc. were the most chosen. These choices seem to contradict certain press articles – for example those mentioning the saturation of emergency numbers by denunciation calls (Kauffmann, Le Monde, April 10, 2020; Vidalie, L’Express, April 4, 2020; France Info, April 14, 2020; etc.). Those articles seem to suggest a health crisis leading to or accentuating a crisis of values. While there is perhaps a link, it does not in any case mean the press’ assumption is entirely right. In any case, reading these articles as well as many others could have encouraged us to carry out a survey on the basis of so-called “sensational” information, and we wanted to be totally free of any bias, which is what this method allows.

Cultivating openness to support your employees

This is also why this method is so interesting: it can be used for other purposes. Let us take another example: in the age of telecommuting, how can we best support and help employees who are likely to have a hard time coping with the current situation? Of course, there are a plethora of diagnostic tools, but are these tools adapted when a company is not looking to “cure” its employees but to support them? It can thus start by enabling free, anonymous speech, by letting people express themselves on what they are experiencing, on the negative aspects of the situation but also on the positive ones. Once all the discourse has been gathered – anonymously of course – group work will allow us to create not only measurement tools, but also support tools, ideas, and applications. This work will also give meaning to an unprecedented situation, team cohesion, and acceptance of the tools thus created.
This method, which is based on two rigorously scientific methods (Churchill and Q methodology), allows the discourse to emerge freely, to be quantified, to be made “measurable”, but also to release the different thought structures in a fluid manner and without preconceived ideas, thus recreating meaning.

This method and its various applications clearly confirms that letting people speak and keeping the discussion open with them is much richer in solid information than any survey.

Churchill 2.0 has been tested on a large scale in the framework of the “Attention!” survey in order to measure and represent the attentional structures of the French. The results are currently being published.

Humans Matter now wishes to develop new use cases for this method by associating with actors in studies, trends, and consumer/user research in general.

Contact

Stéphanie de Chalvron

Contact

Stéphanie de Chalvron

Contact

To:
- Get more information
- Share your challenges
- Get a product demo

Other topics we may have in common

Article

#Work

From Digital to Societal

The new transformation of companies

Read more

Project

#Health

Digital Therapies for Brain Injuries

Partnership with the Paris Brain Institute

Read more

Case Study

#Work

Total Carbon Quest

How to raise awareness of the climate issues in an oil company, even in remote areas?

Read more

Case Study

#Work

AXA Partners

Can cognitive engineering help develop the leadership of managers of managers?

Read more

Article

#Learn

Assessing Mathematical Cognition

Examath 5-8 in the spotlight in the “speech therapy” press

Read more

Project

#Health

Mental Health in the Workplace

Caring for an essential resource

Read more

Article

#Work

Towards New Cooperations

How to reconnect individuals to their environments?

Read more

Project

#Health

BP Passport

An innovative way to deliver healthcare, dedicated to people with bipolar disorder

Read more

Article

#Work

When Training Evolves

to take care of employees’ mental health

Read more

Article

#Health

Caring for Mental Health

Through social skills training

Read more

Receive our articles and invitations

Recherchez dans tout le site

Be the first to receive our invitations and articles
Sign up