Coding and Documenting with Qualcoder
Practicing with the Case Example
Now that we’re more familiar with QualCoder’s interface and the basics of our project, let’s walk Sarah through the coding analysis of the transcripts using the software’s features and some best practices. QualCoder offers a versatile platform for coding not just text, but also images (including PDFs), video, and audio. While Sarah’s project focuses on textual data, feel free to explore these features for your own work later on!
To make the most of our time, we’ll focus our coding examples and activities on the interviewees’ responses to Question 4:
What do you see as the key ethical responsibilities of content creators and digital influencers? How should these ethics apply to endorsements and recommendations? Can you share any personal examples?
We’ll be using a combination of top-down and bottom-up approaches to demonstrate some important coding features in QualCoder. For now, don’t worry about the various icons on the top bar—we’ll explore those shortly. Instead, let’s concentrate on the lower right-hand window.
Creating Categories & Codes
After we presented the distinction between inductive and deductive coding approaches, Sarah mentioned that her preliminary literature review on the topic revealed some broad topics that could guide her initial coding framework.
She found that accountability, authenticity and transparency are ethical responsibilities frequently associated with the the work of content creators and social media influencers. She’s eager to explore these broad categories while keeping them flexible, allowing for refinement, including branching and or combination of existing codes, and the incorporation of new codes as they emerge. Therefore, we will first create these initial three categories to get her started with some coding using the tool.
It’s important to keep in mind that this process typically requires multiple iterations, which we might not be able to afford in a short episode. While we want to demonstrate the overall process to Sarah, she will likely need to keep adjusting the codes until she comes up with a more solid list of unambiguous ones.
So, we will start by representing these three main categories in QualCoder. To do that, let’s do a ctrl+click on the panel. This will prompt a window with the most important actions for you to engage with the development and organization of your codebook.
Click add a new category, and enter “Accountability”. Complete the same process for the other two: authenticity and transparency. You may also add a memo with some explanation about their meanings.
With QualCoder we can create as many levels we want (as long as they are not ambiguous) and also drag categories and codes to reorganize them. We’ll see more about it later.
We now have three initial categories to work with. Assuming we’re familiar with the transcripts, we’ll explore excerpts related to these categories and consider the possibility of developing new codes. It’s important to remember that while we can’t directly assign categories to excerpts, they serve as a useful guide for exploring the data. This approach will assist us in identifying relevant themes later on, as we’ve seen in previous episodes.
Ok, let’s open the Interviewee_01 transcripts. look at the answer to Q4, let’s read the first paragraph. In these few sentences we can identify a few relevant keywords: trust, reliability, respect, genuine/genuinely. There could be multiple ways to go about it, but let’s say we want to code this section as “Genuine Added-Value” and relate this to the “Authenticity” category.
We can then drag the new code and arrange it under Authenticity. You will notice that a color will be automatically assigned to the code. You may change colors, rename, add memos, move or edit codes using the menu options.
Note that the code now shows that there is one excerpt associated with the code. If you continue assigning more excerpts to a giving code, they will be reflected in that number, but at the file level in this view. You may check all excerpts assigned to a code across files by selecting Show coded files.
Your turn! Code the second paragraph in relation to the category “Transparency” and code it as “Honest Reviews”.
Alright, we have two new codes assigned to this interview, let’s move to other interviews and see if we can find excerpts related to this existing codes. I trust you have found some right away. After assigning honest reviews to another interview excerpt, let’s check the list of coded files.
More Coding Options
Great! Now that we understand how to create categories, assign and organize codes, and view excerpts related to those codes, let’s move our attention now to the top bar we mentioned earlier to understand other coding options we have to assist our coding tasks.
You may use the search box to perform queries using Regex functions and locate specific terms within and across multiple documents. This can be extremely handy to speed up the process of getting to specific passages of the data. Once you perform a search you, unfortunately you won’t see the total of matching results. QualCoder will highlight those and you may use the back and forward arrows to navigate results.
Another useful feature is the ability to create file annotations for selected excerpts and coded sections, all conveniently displayed in one window. This allows you to highlight points that need clarification and passages you may want to revisit, especially if they have the potential to inspire new codes. You can use this tool to streamline your workflow and enhance your analysis. Note that the passage will be highlighted in bold so you can more easily spot what needs your attention. Similar approach can be followed with already coded passages, but typically those memos will reflect decisions behind the coding of that passage.
Alright, here are a few more features to support Sarah’s coding efforts. It’s clear that this approach to coding is much more effective than using post-its or word processors, wouldn’t you agree?
Automatic Coding
QualCoder also offers auto-coding text features represented in the icons below:
You can from left to right:
• auto-code exact text
• auto-code sentences based on a text fragment in the current file
• auto-code sentences based on a text fragment for all files
• auto-code the current file using start and end text marks. You can use ‘\n’ for a line ending character
Note that automatic coding is case-sensitive and requires a defined end for each sentence, with a default setting being a period followed by a space.
To auto-code exact text matches, you can assign multiple text sections using the pipe ‘|’ symbol. For instance, both “politics” and “politicians” can be assigned to the same code simultaneously (only for exact text matches).
Additionally, there is an undo option to revert recent auto-coding actions; however, this option will be lost if the project is closed and then reopened.
The auto-code exact text button includes a right-click menu with additional options. By default, it auto-codes all text, but you can also choose to auto-code only the first instance or specific matching text. When using the auto-code exact text feature, you can select which files to apply it to, making it a useful option if you want to identify a single instance across multiple files.
For example, let’s say Sarah Identified another code under the “Authenticity” category named “Genuine Engagement” to represent content/creators connections with their community of followers. In this case, after we create the code and select it, by choosing the first wand, we can auto-code all mentions of “genuine connection” as “Genuine Engagement”.
We can select as many files we would like to apply the auto-code to the chosen piece of text.
Note that the text is highlighted and coded accordingly.
First, it may lead to a loss of context and limit the scope of analysis, potentially overlooking important themes. Second, the subjective nature of code selection can result in inconsistencies, and complex ideas may become oversimplified. Third, this approach can complicate comparisons across different data sets, since it is unlikely terms and expressions derived from various sources will match.
Auto-coding features, which can be also referred as in vivo, verbatim coding, literal coding use exact phrases or terms from participants’ responses, enabling researchers to highlight key themes and concepts directly from the data. While this method can be effective for preliminary and first cycle coding, researchers should be aware of its potential drawbacks as the primary method. This approach can result in a loss of insight, as context may be overlooked. Additionally, auto-coding often oversimplifies the data, focusing on specific terms rather than capturing broader themes.
Additional features in the Coding Panel
You may also edit text using the blue pencil (proceed with care as this are irreversible) and export your coded files with the list of codes in html, odt or txt. Caution in clicking the red X, that will erase all your coding efforts for that particular file.
Reporting & Graphing
After reviewing the basics with Sarah, she gained confidence in exploring the data using the tool independently. She initiated a new project over the weekend, concentrating on Q4, where she experimented with the tool and developed categories and codes. When she returned, she had some preliminary coding ready, allowing us to guide her through the reporting and graphing features in QualCoder. Although she hasn’t finished the analysis yet, Sarah feels like she is in a good place to navigate the reporting options within the tool with us.
Before we proceed, please open the folder influencers_q4.qda from the project example sarah-project you might have already downloaded and unzipped and open it in QualCoder.
QualCoder offers a few options for users to explore, query, visualize the data and export those reports and outputs. It can provide code counting summaries in a variety of formats, word clouds, plotting in bars and pie charts, heat and tree maps. It also supports intercoder reliability testing for datasets coded, independently, by multiple people. Let’s see how Sarah some of the options she might want to consider for her analysis.
Let’s click reports to see options in QualCoder. We will start with coding reports:
Once we select coding reports, we will be presented with a few additional filtering options. Since Sarah was the solo coder, we will only have one default coder for this project. Let’s leave it like that for now, and we will return to it a little later. If we click {x}, we will open the attribute selection parameters, where we can refine our report to include only responses from male influencers, for example. However, beforehand, we need to select at least one code from our list. Let’s choose the codes under the “authenticity and transparency” category.
Once completed, click the play button to see the results. You may also export the results in your chosen format after naming the file.
We can create a matrix by selecting files and desired codes. Let’s select all files and all codes within the authenticity and transparency category. Select the top categories in the drop-down menu and click the play button. Great! We have a matrix, but if we check the transposed matrix, we will have a better view of the relevant excerpts within the selected category and across documents.
The `code summary` option allows the inspection of files and a detailed word summary for each code. Check it yourself!
Well, there are more reports to experiment and not all of them will be relevant for this project. So let’s now focus on the visualization options within the tool.
The view graph option will allow you to create visual based on existing selected codes by clicking on the bule plus sign. You may move things around and apply additional customization, but we want to keep it simple. Assuming we are happy with it, we can save the graph by clicking the downward arrow, and adding a name and description to it.
Let’s move on to the charts section, where you’ll encounter several options. We typically recommend researchers avoid pie charts, as they are not effective for communicating information with three or more slices; human eyes struggle to distinguish curves. Instead, let’s explore more effective alternatives.
Take time to explore the options available. Which ones would make more sense for you visualize Sarah’s coding?
Multiple Coders and Coding Comparison
QualCoder was designed for individual use, allowing one person to work on a project at a time. However, a second coder can access the same project on the same computer, or the project folder can be easily transferred to another computer. To add a new coder, navigate to the project settings, enter the coder’s name, and click “Apply.”
Add yourself as Sarah’s helper. To do this, return to the coding text feature and code one or two excerpts using the existing codes, ensuring that your name is listed as the coder.
For a quick refresher on the steps: go to the coding section, then select “text,” and choose one of the interviews or files. Once you’ve completed the coding, head back to the reporting tab. Select the coders and click the play button to view the results. This will generate a table similar to the one shown below:
To interpret the table, you can click the question mark icon for a detailed explanation of what each column represents:
Agree %: Calculated across all text files as the (total dual coded plus the total uncoded)/total characters
A and B %: Calculated as the total dual coded characters/total characters
Not A Not B %: The characters not coded by either coder/total characters
Disagree %: Is 100% minus the total agreement percent.
Agree coded only %: Is the dual coded characters divided by the dual coded and single coded characters
Kappa: The statistic or coefficient also know as Cohen’s kappa (κ, lowercase Greek kappa) used to measure inter-rater reliability as described by McHugh (2012). This coefficient essentially assesses the degree of agreement and how well two independent coders classified/coded the same content into thematic groups.
You might be wondering how this can be useful to you, right? Well, let’s imagine you plan to publish your study. Several journals and peer reviewers might require that information, specially for disciplines that are more traditionally quantitative. Indicating that an independent coder agreed with your classification helps to support reliability, consistency and credibility of coding decisions that guided data analysis. But what are the thresholds?
While there are some variations in the literature, below is a typical rule of thumb:
Kappa = 0.00 should be taken as representing no agreement
0.00 ± 0.20 as “slight” agreement
0.21 ± 0.40 as “fair” agreement
0.41 ± 0.60 as “moderate” agreement
0.61 ± 0.80 as “substantial” agreement
0.81 ± 0.99 as “almost perfect” agreement.
A kappa coefficient of 1 represents perfect agreement.
Below are a few examples of papers reporting the kappa coefficient for interview data:
Coates, A. E., Hardman, C. A., Halford, J. C. G., Christiansen, P., & Boyland, E. J. (2019). Food and beverage cues featured in YouTube videos of social media influencers popular with children: An exploratory study. Frontiers in Psychology, 10, Article 2142. https://doi.org/10.3389/fpsyg.2019.02142
Dekoninck, H., Van Houtven, E., & Schmuck, D. (2023). Inspiring G(re)en Z: Unraveling (Para)social Bonds with Influencers and Perceptions of Their Environmental Content. Environmental Communication, 17(7), 701–719. https://doi.org/10.1080/17524032.2023.2237693
Kopp, B., & Dinkel, C. (2024). Adolescents’ values and values education through social media. EDULEARN24 Proceedings, 5257–5260. 10.21125/edulearn.2024.1289
Documenting Things
When conducting qualitative research, it’s crucial to maintain thorough documentation to ensure transparency and ethical compliance. Documentation should include both project and file level documentation. QualCoder allows you to export a few important files to support interpretation and reusability. Here’s what we should recommend Sarah to save:
Codebook: the coding framework used, including definitions of codes and categories.
All relevant materials generated within the tool, such as memos, notes, network diagrams, and classifications.
REFI-QDA CodebookAs seen, a codebook in qualitative or mixed-methods research is a structured list of defined keywords or phrases that represent key themes, concepts, or topics of interest to the researcher. These keywords or phrases are commonly referred to as “codes.”
In qualitative data analysis (QDA) software, codes can be linked to specific segments of data where they appear. While the codebook serves as a comprehensive listing of codes, many QDA programs also allow for the organization of these codes in various ways, depending on the features of the software used.
The REFI-QDA Standard enables interoperability between Qualitative Data Analysis Software (QDAS or CAQDAS) programmes. It ensures that a codebook developed in one of the compatible software programs can be exported and imported into any other listed programs. QualCoder is currently experimenting with this format, and allows .
It’s important to note that because each program has unique functionalities regarding codes and their organization, users may encounter some modifications when importing a codebook into a different software environment.
Check which QDA software are REFI-QDA compatible: https://www.qdasoftware.org/refi-qda-codebook
You may export your project and codebook using the REFI-QDA standard under the Export > Project option.
Attributes are also exportable in csv file, as indicated below:
Save these files using a consistent naming convention and ensure they are included in your data archiving package along with other important files, which will be discussed in more detail in the next episode.
Recommended/Cited Sources:
McHugh, M. L. Interrater reliability: the kappa statistic. Biochem Med, 2012;22(3):276-82. https://pmc.ncbi.nlm.nih.gov/articles/PMC3900052