What is a PTIP?

The Post-Test Information Package, or PTIP, is a critical element in the assessment program. Every Skill-Task Assessment (STA) is summarized in a PTIP. The PTIP describes the test design and group outcomes as well as the individual (anonymized) results. Instructions are included in each PTIP for interpretation of the test.

Background

Forensic handwriting examination can be regarded as a skill. The skill is applied to cases which vary according to the amount and complexity of both the questioned and known material. No one test will determine the validity of the skill. However, over time and given sufficient trials, a picture of the skill for individuals should emerge. This will allow us to determine whether or not skills claimed by examiners are valid and reliable and also allow us to estimate the potential error (misleading) rate for different types of examinations/comparisons.

Structure of a PTIP

A PTIP may vary in content depending on the nature of the trial. However, in general, it will provide the following information:

  • An introduction to ST2AR and the trial process,
  • Overview of the signature trial
  • Group results
  • Individual results
  • Individual misleading opinion analysis
  • Other information

The introduction section outlines the ‘ground truth’ for the trial. It includes a description of sample preparation (both specimen and questioned materials), general information about the trial participants (including demographics), the instructions given to the participants, the answer book and coding for the answers, as well as terminology used in the PTIP.

Group results are provided. The results are intended to give a picture of how the group performed in the overall sense.  The PTIP looks at the number/percentage of calls made (vs. inconclusive) and the number/percentage of correct vs. misleading calls. Aside from overall group performance it is helpful to consider inter-examiner variability. High inter-examiner variations means that overall group scores will not be a good measure of individual performance in this trial. The extent of inter-examiner variation should be assessed by inspecting the total number of correct, misleading and inconclusive opinions expressed by each of the participants, and for each of the various writing types (e.g., genuine, disguise, and simulation). Very often the PTIP will include a breakdown of peer-reviewed responses vs. non-PR responses, by questioned signature type.

Individual examiner results are provided, but only in anonymized form. Booklet codes are shown but only the participants have their own codes. Results are presented by questioned signature type.

An analysis is individual misleading calls is usually done. This breakdown is particularly helpful as feedback for the examiner. It is done to let an examiner assess whether apparent skill in determining the authorship of one questioned signature type (as may be evidenced by low misleading and low inconclusive scores) is real or not. The following image is an example of the graphical feedback form — examiners are on the bottom axis with % inconclusive (yellow), % correct (green) and % misleading opinions (red) provided for each signature type.

Example of PTIP individual outcomes

Detailed instructions are provided as to how this information can and should be interpreted.

Some PTIPs include other information depending upon the outcomes in the trial. For example, opinion strength may be discussed in terms of correct vs. misleading calls.