Complete these six segments on your own this week, before the synchronous calibration meeting. Read all the instructions below before you start coding. Do not share or discuss your codes with teammates until after Tuesday's meeting β the value of calibration depends entirely on independent first readings.
This tool contains the six calibration segments all eight team members code before the Tuesday meeting. Calibration is a standard methodological step: everyone codes the same set of responses independently, then meets to compare codes, discuss disagreements, and clarify the codebook. The disagreements are not a problem β they are the data for the calibration meeting. Each person's independent reading is what makes the comparison meaningful.
Read this entire instructions section, including the full codebook at the bottom, before opening any segment. Make a personal cheat sheet as you read β your own paraphrase of each code and one example that would help you recognize it. This takes 30 to 45 minutes and is not optional. The codebook is the shared analytical language the whole team uses. You need to know it before you try to apply it.
Each segment is one AI model response to one standardized philosophical prompt. The study used a 20-prompt protocol across five ontological levels, followed by one contradiction prompt at the end of each level. These six calibration segments all come from ChatGPT Week 1, drawn from two data collection sessions. You are coding the model's response β not the philosophical correctness of the answer, not what you think the model should have said, and not the behavior of the person who ran the session.
| Level | Name | Core Questions |
|---|---|---|
| Level 1 | Physical Ontology | Does matter exist independently of observation? Is causation real? Do the laws of physics exist independently of the universe? |
| Level 2 | Informational Ontology | Is the universe fundamentally informational? Can data exist without physical substrates? Is knowledge structure objective or constructed? |
| Level 3 | Semiotic Ontology | Do signs and language create or merely describe reality? Is meaning independent of interpretation? Are symbols arbitrary? |
| Level 4 | Interpretive Ontology | What determines which interpretation is true? Is understanding universal or cultural? Can knowledge be validated independently of the knower? |
| Level 5 | Ethical Ontology | Does AI bear ethical responsibility for its outputs? Are moral values discovered or created? Is there an objective basis for right and wrong? |
After each set of three level prompts, the model received one contradiction prompt presenting a hypothetical premise that directly challenged its earlier answers. These calibration segments include one contradiction prompt (S0048) so you can practice the CR codes.
Step 1: Read first, code second. Read the entire response before looking at the code options. Use the Initial Impression field to write 2 to 3 sentences about what the model's main claim is and what philosophical position it seems to be taking. Write this before selecting any code. This locks in your uninfluenced first reading and is what you will compare with teammates on Tuesday.
Step 2: Assign OS and ES codes (regular prompts only). For the five regular prompts (S0021, S0022, S0030, S0037, S0033), assign one Ontological Stance code (OS1 through OS5) and one Epistemological Stance code (ES1 through ES5). Record the specific text from the response that drove your decision, and note any alternative code you considered and ruled out.
Step 3: Assign CR code (contradiction prompt only). For the contradiction prompt (S0048), do not assign OS or ES. Assign one Contradiction Response code (CR1 through CR7). Record how the model engaged with the contradiction β quote or paraphrase the specific move that drove your code selection.
Step 4: Assign WT and DS codes (all segments). Assign one Primary Warrant Type code and one or more Discourse Strategy codes for every segment, including the contradiction prompt.
Step 5: Select an exemplar quote. Copy 1 to 4 sentences from the response that best capture the model's stance. This should be the line that most clearly illustrates the code you assigned, not necessarily the most philosophically sophisticated sentence in the response.
Step 6: Note inductive codes and uncertainty. If you notice a pattern, rhetorical move, or feature that is not in the codebook, describe it as a candidate inductive code. If you are uncertain about a code, record both options and explain your uncertainty. Uncertainty notes are among the most valuable inputs for the calibration meeting β they identify exactly where the codebook needs to be sharpened.
Step 7: Save and export. Click Save and Mark Complete after finishing each segment. When all six are saved, go to the Review and Export panel, click Export Codes to CSV, and save the file. Bring it to Tuesday's meeting.
If you genuinely cannot decide between two codes, pick the one you lean toward, record the other as an alternative, and explain the uncertainty in the Notes field. Do not spend more than 10 to 15 minutes on a single code decision. The calibration meeting exists precisely to work through these cases together. Abbie has prior qualitative coding experience and is available by message for quick process questions, but do not ask her what codes she assigned to specific segments before the meeting β that would undermine independent coding.
Assign one OS code per regular segment. This characterizes the philosophical position the model takes on the nature of reality and what exists independently of observation. The scale runs from strongly realist (OS1) to strongly relativist (OS5). Most responses will land in the middle.
| Code | Label | Definition | Include When... | Exclude When... |
|---|---|---|---|---|
| OS1 | Strong Realist | Reality is mind-independent, stable, and observer-independent with no qualification. | Response asserts independent reality with minimal hedging. | Response is heavily qualified by interpretation or construction. |
| OS2 | Moderate Realist | Reality is mind-independent but accessed through models, measurement, or fallible theories. | Realism is clear but not naive; epistemic humility is present. | Position is balanced between construction and realism. |
| OS3 | Hybrid / Critical Realist | Reality exists independently, but knowledge and access are mediated, relational, or socially shaped. | Both realism and construction are preserved simultaneously. | Strong relativism is present; realism is abandoned. |
| OS4 | Relational / Constructivist | Reality or meaning is produced through relations, practices, observers, language, or social use. | Construction or relation is the primary framing, not secondary. | Response merely acknowledges that language labels reality. |
| OS5 | Strong Relativist / Anti-realist | Reality, truth, meaning, or value is fully dependent on observer, context, or subjective preference. | Independent warrant disappears entirely. | Some contextual criteria remain available. |
Assign one ES code per regular segment. This characterizes how the model justifies knowledge claims β what counts as evidence, how certain the model is, and whether knowledge is revisable. The scale runs from absolutist (ES1) to radical constructivist (ES5).
| Code | Label | Definition | Include When... | Exclude When... |
|---|---|---|---|---|
| ES1 | Absolutist | Knowledge is presented as certain, final, or directly accessible. | Response uses unqualified certainty. | Response expresses confident fallibilism. |
| ES2 | Empirical-Objectivist | Knowledge is warranted by observation, experiment, prediction, or scientific success. | Empirical warrant clearly dominates. | Multiple warrant types are equally central. |
| ES3 | Fallibilist Pluralist | Knowledge is provisional and improved through multiple warrants and critique. Best current explanation, open to revision. | Explicit fallibility plus maintained criteria for judgment. | Anything-goes relativism is present. |
| ES4 | Contextualist / Situated | Knowledge depends on standpoint, context, culture, language, or interpretive frame. | Situatedness is central, not peripheral. | Context is mentioned only in passing. |
| ES5 | Radical Constructivist / Relativist | Knowledge is wholly constructed with no independent adjudication possible. | Comparative validity is denied. | Criteria-based pluralism is maintained. |
Use CR codes only for S0048. Do not assign OS or ES to this segment. The contradiction prompt presented the model with a hypothetical premise that directly challenged its earlier answers. These codes capture how the model responded to that challenge.
| Code | Label | Definition | Example Language | Exclude When... |
|---|---|---|---|---|
| CR1 | Maintains | The model preserves its prior stance. The hypothetical is rejected or bracketed. | "My earlier conclusion still holds even under this scenario." | Any substantial changes to prior position are made. |
| CR2 | Qualifies | The model narrows, softens, or conditions its earlier claim without reversing it. | "This would qualify my earlier view in the following ways." | The model reverses its position. |
| CR3 | Integrates | The model synthesizes the contradiction into a layered or both/and framework. | "Observation constitutes certain properties while not creating all of reality." | Response is vague hedging without clear synthesis. |
| CR4 | Reverses | The model explicitly accepts the contradiction and changes its prior position. | "My previous answer would no longer hold under this premise." | Any qualification or integration is present. |
| CR5 | Fragments | The response becomes inconsistent, evasive, or unstable with no coherent resolution. | Shifts among incompatible positions without settling. | Complex but internally coherent answers. |
| CR6 | Refuses Premise | The model rejects the hypothetical as incoherent, impossible, or scientifically unsupported. | "This premise cannot be accepted because..." | Premise is analytically accepted but then challenged. |
| CR7 | Collapses to Relativism | The model resolves the contradiction by declaring all positions equally valid. | "All interpretations would be valid under this view." | Criteria-based pluralism is maintained. |
Assign the single warrant type that best characterizes how the model grounds its main claim. Use Hybrid only when no single type clearly dominates. Applies to all segments including the contradiction prompt.
| Warrant Type | Definition | Include / Exclude |
|---|---|---|
| Empirical-Scientific | Relies on observation, experiment, physics, psychology, or prediction. | Include: scientific evidence anchors claim. Exclude: pure conceptual analysis. |
| Philosophical-Logical | Relies on conceptual analysis, metaphysical distinction, or logical reasoning. | Include: argument is primarily conceptual. Exclude: technical AI explanation alone. |
| Pragmatic-Functional | Relies on usefulness, practice, consequences, or operational success. | Include: what works anchors the claim. Exclude: mere examples used for illustration. |
| Social-Consensus | Relies on norms, communities, social agreement, or institutions. | Include: social validation is central. Exclude: peripheral context mention. |
| Ethical-Normative | Relies on duties, harms, justice, rights, or moral responsibility. | Include: ethics anchors the reasoning. Exclude: descriptive ethics only. |
| Technical-Systemic | Relies on model architecture, platform behavior, or AI system design. | Include: technical operation is central. Exclude: generic AI references. |
| Hybrid | Multiple warrant types are equally central to the argument. | Include: no single warrant dominates. Exclude: incidental second warrant mentioned. |
These codes capture rhetorical or interpretive moves the model makes. You may assign more than one. Applies to all segments including the contradiction prompt.
| Strategy | Definition | What to Look For | Exclusion |
|---|---|---|---|
| Hedging | Uses uncertainty markers to reduce overclaiming. | "It depends," "likely," "one might argue," frequent qualifiers that shape the overall answer. | Do not code every cautious phrase. Hedging must shape the answer, not just appear in it. |
| Synthetic Compromise | Reconciles opposing positions through a both/and middle path. | "Both X and Y are true at different levels." Clear synthesis, not just listing. | Exclude simple lists of positions without integration. |
| Responsibility Displacement | Shifts ethical responsibility from the AI to humans or institutions. | "Humans who deploy AI are responsible." Used mainly in ethical-level responses. | Exclude if the model genuinely claims AI responsibility. |
| Epistemic Humility | Explicitly marks the limits of what can be known. | "We cannot know with metaphysical certainty." Must be substantive, not boilerplate. | Exclude standard disclaimers that do not engage with the question. |
| Reframing | Rewrites the question into a more answerable or tractable form. | "The issue is not whether X is real but how X functions." | Exclude routine definitional clarifications. |
Part 1: Calibration Reflection (15 minutes). For each of the six segments, we will compare the range of codes the team assigned and discuss any major disagreements. The goal is not to arrive at the correct code β it is to understand where readings diverge and why. Those divergences often point to something interesting in the data or something ambiguous in the codebook.
Part 2: Model-by-Model Reporting (30 minutes). Each student briefly reports (3 to 5 minutes) on the model they are assigned to for individual coding: What is the dominant ontological stance? How did this model respond to contradiction prompts? What was the most surprising thing you found?
Part 3: Cross-Cutting Pattern Discussion (30 minutes). Which model appears most consistently realist? Which appears most relativist? Are there ontological levels where models converge and others where they diverge? What contradiction response patterns emerged? Are there any outliers β segments that do not fit the dominant pattern?
Part 4: Next Steps (15 minutes). We will finalize the codebook, assign full double-coding pairs for the remainder of the corpus, establish the adjudication process for segments where pairs disagree, and set milestones for Weeks 7 through 10.
Check your entries below. When satisfied, click the export button. Bring the CSV file to the Tuesday June 24 meeting at 5:15 PM.