Methodology

Truthmeter is an information service and public resource that analyzes the following areas:

  • the veracity of the statements made to the public by elected and appointed officials, as well as other public figures who influence the creation of public policies and public discourse; and
  • the pre-election programs of the political parties, to check their realization within the given deadline or until the end of their term.

Through these analyses, the citizens gain the opportunity to get information about the dominant topics and attitudes in the public debate on democratic policies, as well as about the activities and results achieved by the local and central government.

 

(FACT-CHECKING) Statements of elected and appointed officials 

Truthmeter.mk’s team of experts in relevant areas analyzes the statements of politicians, officials and other policy creators and public opinion makers and presents them in an easy-to-read, aggregated form.

The main purpose of Truthmeter.mk is to increase the knowledge of citizens, media, nongovernmental organizations as well as political parties and members thereof, and it offers the possibility for familiarizing with series of aspects of the Macedonian political process.

Assessments

The statements of politicians, officials or other policy creators and public opinion makers made in the media or at public events are analyzed by using the same criteria and within the deadline implicated in the statement itself, which are then assessed as follows:

  • Untrue: the statement is untrue
  • Half-true: the statement is correct, but it lacks important details or it is out of the context
  • True: the statement is true
  • Inconsistent: the statement is illogical or it cannot be checked
    1. the statement is contrary to the previous one on the same question

    a.) without additional arguments,

    b.) without the emergence of new relevant information that would lead to a change in attitudes or

    c.) without other justified reasons;

    1. the statement is illogical; and
    2. the statement cannot be verified

Criteria for selection and fact-check

The basic criteria that are taken into account when selecting a statement for a fact-check are the following:

  • Does the fact-check have the capacity to spark off a public debate on the topic in question;
  • What’s the position and influence of the politician whose statement is fact-checked;
  • What’s the importance of the statement as regards its influence and whether it affects a large number of citizens;
  • How often the specific official’s statements are checked;
  • What’s the type and size of coverage in the media (whether it is covered by a larger number of media, whether it had been previously analyzed elsewhere and whether it had reached wider audience or not);
  • How much can Truthmeter contribute with new information and arguments when doing the fact-check;
  • Had the statement in question been presented to the public by multiple media, had it triggered interest and is it a topic with the capacity to reach a wider audience; 

Originality of statements

Truthmeter always strives to analyze original statements, i.e. ones available in the form of video, while the second place is reserved for the ones in audio format. This contributes to the analyzed statement’s accuracy and helps reduce any possible transcription errors. If there are no video or audio files of a statement, we then resort to transcribed statements. These statements are then fact-checked in as many sources as possible, so the most accurate transcription could be used. From time to time, Truthmeter analyzes statements of political parties given in their press releases. In such case, the said press releases are taken directly from the political party’s website.

Citing sources

All sources used in the fact-check are listed at the end of every analysis. Hence the readers can quickly see the cited sources and can use them to replicate our fact-checks if they wish to do so. Apart from citing the sources, the date when each of the sources had been accessed is also included, thus readers know when the sources had been accessed.

Besides listing the sources at the end of each article, we also insert links in the article itself, so the readers can open them while reading if needed.

All quotes are marked differently in the text. They are bolded, therefore easily noticeable. This goes for the statements as well as quoted documents. These are crucial segments of our fact-check, so we want to make them easily noticeable for the readers.

In cases when sources want to remain anonymous, we do everything we can to describe their background as much as possible. But generally, we rarely use anonymous sources and such sources are never the only ones in the analysis. There must always be clearly named sources in the analysis besides them.

Conflict of interest

Each case of conflict of interest, if any, is clearly stated on the spot.

Manner of publishing the statements

The analyzed statements, downloaded from written sources in Truthmeter’s analyses, are published in their original form, without intervening in the language, grammar and sense. As for the remaining text in the article, it is, of course, proofread. The statements are published in their original form, together with any mistakes and characteristics so the readers can get the real image about their look, sound, manner of publication and original content.

(FACT-CHECKING) Promises

The promises given in the Election Programmes are considered as separate entities on truthmeter.mk. To gain such status, the promise from the programme must include:

  • specific goals – results that the party intends to achieve after coming to/staying in power, as well as
  • measures – activities necessary to achieve the goal.

The promises with specific goals clearly state what the desired result is and enable the voter to clearly recognize when that result is achieved. It means that the expected result should be described in a clear, precise and unambiguous manner, with clearly determined parameters i.e. indicators that are the grounds for measuring the fulfillment of the promise. Precise parameters are those elements of the promise which determine the essential properties (quantity, quality and comprising elements) of the set goal, which have to be satisfied to declare the goal fulfilled. Such parameters depend on the nature of the promise and can comprise:

  • Starting and ending magnitudes: if growth is promised in a certain area – precise parameters for the desired growth rates and development about the current situation; the parameters comprise specific sums/percentages when it comes to promises from the financial scopes.
  • If qualitative improvements are promised in a certain area, the parameters must elaborate on the promised progress about the current situation. If the promised changes are not empirically measurable (i.e. by numbers (wider policies and strategies, social climate, culture, systems of value, international cooperation), then the parameters must include detailed descriptions of the expected changes with specific short term and long term measures.
  • Institutional parameters – if the promises are related to the legal frameworks, they should particularize the legislative acts, which will help the voter see what changes are being made, to what extent and through which acts. In the case of establishing new institutions, the parameters must detail their level and competences.
  • Pinpointing the mechanisms for monitoring and controlling the indicators of the implementation of the promises, unless that is already covered with the existing institutional mechanisms.
  • Parameters for determining the timeframe for the fulfillment of the promises.

The specific measures represent activities carried out by the promise giver in order to fulfill the given promise, via specific parameters showing that the set goal is to be met. Regarding the goal itself, the activities can include:

  • A precise legal framework that will regulate the given area or problem
  • Announcement of a study/research, strategies, plans that will provide the promised result
  • Origin and amount of the resources that would secure the fulfillment of the goal
  • Deadlines for completion of the announced measures
  • Identifying the existing or announcing the establishment of new competent institutions which would implement the stated activities
  • Pinpointing the mechanisms for monitoring the implementation
  • Other activities that should be undertaken in order to fulfill the promise.

The promises which are illogical or given in a way that cannot be checked are marked as inconsistent.

Every identified promise is assessed by its level of fulfillment: fulfilled, partially fulfilled or unfulfilled:

  • Unfulfilled: If there is no activity regarding the given promise, i.e. there is nothing implemented within the deadline
  • Partially fulfilled: several activities are taken to fulfill the promise
  • Fulfilled: the promise is completely fulfilled
  • In progress: the work on the fulfillment of the promise has been started
  • Inconsistent: illogical promise or one that cannot be checked

The analyses and assessments of the promises are published on the Internet, and if possible, are also delivered to the parties. The promise givers, as well as the entire public, have to opportunity to read and to directly respond to all analyses, to file denials, to give remarks, suggestions and additional information and arguments that can lead to modification of the assessment of the promise, either by commenting under the article or by other forms of direct communication with the analyst team.

 

COUNTERSPIN

Counterspin (subtitled “disclosing all spins”) is a subset of the “Analysis” section. Statements, groups of statements, but also parts or whole speeches of politicians and public figures are analyzed more broadly, which cannot be clearly assessed as true or false through the methodology for assessing the veracity of statements, but require a broader explanation and elaboration to cover the context and to “untangle” what the politician is saying in instances and situations like the following (and many others):

  • persistent and constant use of untrue data and their indirect, hardly noticeable abuse;
  • complex context manipulations;
  • the problem with the inability to verify the statement and break down its speculative nature;
  • complex and vague words that are connected in such a way that they mean nothing but sounds as if they are saying something very important;
  • the internal contradiction in the statement itself which is not visible or is difficult to understand at first sight or in the absence of additional information;
  • complete speeches that must be analyzed paragraph by paragraph, i.e. separately part by part to understand the content, which, in turn, differs from the imposed tone;
  • promoting extreme views using “high” (often vague) rhetoric to show off, to give the impression that they are legitimate;
  • promoting side and unimportant views and their “pedestal setting” as essential in order to obscure another much more important problem;
  • abuse of the semantics (meaning) of the words, which are given another invented meaning, but also abuse of the lexicon of the Macedonian language by inventing words;
  • statements and speeches in foreign languages, but mispronounced or misinterpreted/mistranslated;
  • statements that in a very covert or seemingly “normal” and “common sense” language, spread discrimination, insults or even indirect hate speech;
  • and most importantly: statements and speeches that contain, are based on, or promote spin or several intertwined spins that are difficult to recognize as such;
  • and more.

These articles, given their complexity, are exclusively prepared by professional journalists with years of analytical experience and proven ability to recognize spins and “untangle” complex manipulations produced by party-political spin-doctors used by politicians. Such an approach leads to the creation of serious analyzes, which treat politicians’ statements from different angles in order to expose them more comprehensively. Their goal is to clear up all the “fog”, ambiguity, manipulation, but also the goals and intentions of the statements and claims of politicians and other important public figures who are part of the creation of policies and public discourse.

Given the fact that the section itself is called Counterspin, these texts do not have special assessments. The elaboration of a given statement, several statements, part of a speech, or an entire speech within this section automatically implies that the Truthmeter editorial board considers that the analysed text requires at least a serious clarification or is simply a spin, i.e. contains elements that imply the existence of spin.

Given the looks of the final product, its obvious analytical journalistic background and form, these counterspins are popular with the media, who regularly republish them, and thus reach the citizens in the widest possible way. In addition, given the numerous citations and links included, NGOs use such articles in their research and projects as an argument in support of their activities.

 

(FACT-CHECKING) Partnership with the social network Facebook

 

Starting in August 2020, the portal Truthmeter and the Metamorphosis Foundation, as its publisher, established cooperation with the social network Facebook in the field of fact-checking the contents of this social network. Additional information on this partnership can be found here.

To implement this collaboration, Truthmeter relies on three basic pillars:

The first pillar is the use of its professional, comprehensive and standardized methodology for fact-checking in the media, which is used within the entire Metamorphosis Foundation for fact-checking, and which was previously used in the Media Fact-Checking Service (2013-17), as well as in the project for checking facts from the media and media literacy CriThink – Critical Thinking for Mediawise Citizens (2018-2021). This methodology was developed in 2012-13 and is updated as needed. It is a document that explains and systematizes the way of collecting information, their systematization and selection, use of sources and all other journalistic techniques for finding and managing information but placed in the context of fact-checking of previously published content by various actors in public discourse. The rules, practices and techniques set out in this methodology are comprehensive and can apply to all types of media, but special attention is paid to the specifics of fact-checking in content published online. Our comprehensive fact-checking methodology can be found here.

The second pillar is the Code of Principles of the International Fact-Checking Network, which refers to non-partisan and fair behavior, respect for standards and transparency of sources, respect for transparency of sources of funding and organization, respect for standards and transparency of methodology, adherence to an open and honest correction policy. Truthmeter has been a member of this organization since 2019 along with 90 other fact-checking organizations around the world.

The third pillar is the agreed obligations, requests and instructions of the partner – Facebook, which refers to different segments of the fact-checking and can be found in the “Help” section of the social network. The most important thing to know is that the Truthmeter editorial office has no option to delete or in any way remove content from Facebook and the only obligation agreed with the partner is to check the facts in the content and assign one of the five assessments defined by Facebook: disinformation, partial disinformation, lack of context, satire and altered/falsified content. More comprehensive explanations of these ratings can be found in the documents in the Facebook help section. It is also very important to know that the assessments assigned by Truthmeter are changeable and can be withdrawn if the assessed content is corrected.

 

Јазици MK SQ EN