Using Predictive Coding

This topic provides instruction for completing the following tasks associated with using predictive coding:

Creating a new predictive coding project

When you set up a predictive coding project, Nebula automatically creates score fields. The new scoring fields display the project name, coding option, and choice can be viewed on the Document List using Customize Fields.

To create a new predictive coding project

  1. In the Nebula menu bar, click Review > Admin > Predictive Coding.
  2. In the Predictive Coding heading, click the Select Project Action icon and select New Project.
  3. In the New Project dialog box, enter the project Name.
  4. Enter the text you want to Use for Training Flag.
  5. View the Index Fields, view the default metadata from the documents the predictive coding service will take into account.

    6. Note: Index Fields cannot be edited after a training session has been run.

  6. Click Create.
  7. View the new project on the Predictive Coding page. The name of the project appears in the Select Project drop list.

  8. To view the index fields, click the Information icon in the Indexed Documents header.

  9. In the Classifiers header, click the Action icon and select Add Classifier.
  10. On the Add Classifier dialog box, select the coding fields you want included in the predictive coding profile and click Select.

    11. Note: Entries and edits on the Predictive Coding page are automatically saved.

To delete a predictive coding project

Note: Deleting a predictive coding project also permanently deletes all associated classifiers and fields, and cannot be undone. A project cannot be deleted during indexing or training and classifying, nor while currently used in a Workflow stage.

  1. In the Nebula menu bar, click Review > Admin > Predictive Coding.
  2. In the Predictive Coding menu, choose the project you want to delete with using the project drop list.
  3. Click the Predictive Coding Action icon and select Delete.
  4. On the Delete Predictive Coding Project dialog box, click Delete.

To add classifiers

  1. On the Predictive Coding page Classifiers header, click the Action icon and select Add Classifier.
  2. In the Add Classifiers dialog box, select the coding fields you want included in the predictive coding profile and click Select.

To delete classifiers

  1. On the Predictive Coding page, in the Classifiers section, locate the classifier you want to delete and click its Delete icon .
  2. On the Delete Classifier dialog box, click Delete.

To set the object level security for predictive coding

  1. In the Predictive Coding menu, choose the project you want to work with using the project drop list.
  2. Click its Action icon and select Object Security.
  3. Complete the Object Security dialog box.

Adding documents to a predictive coding project

Documents are added into a predictive coding project through the Reviewer Document List.

To add documents to a coding profile

  1. In Review, choose the Saved Search or document set from Review Explore menu (Review > Explore) with emails and attachments you want to include in the predictive coding profile. (This is the document collection predictive coding will learn from and provide a score or ranking to.)
  2. On the Document List, click the Action icon and select Predictive Coding.
  3. On the Predictive Coding dialog box, select Add to Project,
  4. In the Project Name drop list, select the predictive coding project you created.
  5. Click Proceed.
  6. On the Predictive Coding page (Review > Admin > Predictive Coding), choose the predictive coding set from the menu bar drop list.

  7. View the Indexing Status in beneath the Index Fields section.

  8. View the document set selected in step 1 in the Index Documents section. When indexing completes, Train & Classify is available to run a training session.
    To view the documents in the current index population in the Review Document List, click the Indexed Documents header Action icon and select View Documents. (Available when the Index Status is Complete and the current index contains at least one document.)

To remove documents from a coding profile

  1. In Review, choose the Saved Search or document set from Review Explore menu (Review > Explore) with emails and attachments you want to remove from the predictive coding profile.
  2. On the Document List, click the Action icon and select Predictive Coding.
  3. On the Predictive Coding dialog box, select Remove From Project,
  4. In the Project Name drop list, select the predictive coding project you want the documents removed from.
  5. Click Proceed.
  6. On the Predictive Coding page (Review > Admin > Predictive Coding), choose the predictive coding set from the menu bar drop list.

  7. View the removed documents in the Indexed Documents list and the updated document count in the Index Status.

Running a training session

Before running a training session, create a seed set of documents (a collection of reviewed documents in the document set in which the training fields have been selected when coding the documents).

To run a training session

  1. On your Review matter, click Review > Admin > Predictive Coding.
  2. In the Predictive Coding menu, choose the project you want to work with using the project drop list.

  3. Click Train & Classify.
  4. View the Training Status.

  5. When the training session completes, view its Training Status of Complete.

    6. Note: Should a training session fail, click Failed in the Last Training Session to view the details of the problem on the Tasks (Review) page.

Rebuilding an index

If new documents have not been added to an existing predictive coding project, the project's index must be built prior to the next training session.

To rebuild an index

  1. On your Review matter, click Manage > Review > Predictive Coding.
  2. In the Predictive Coding menu, choose the project you want to work with using the project drop list.
  3. In the Indexed Documents header, click the Action icon and select Rebuild Index.
  4. On the Rebuild Index dialog box, click Proceed.

Using Control Sets

Control Sets enable you to check the model quality and view the results of the predictive coding as a graph in the Performance section.

The Performance section graph displays the following metrics:

  • Recall: The proportion of True Positive documents (positive basing on PC score per given threshold AND manual review) to all documents marked as positive by manual review (actual positives).
  • Precision: A proportion of True Positive documents to all documents that are positive per given threshold basing on PC service scores.
  • F1: The harmonic mean of Precision and Recall.

To add a control set

  1. Enable the Control Set by doing one of the following on the Predictive Coding page:
    • In the Classifiers header, click the Action icon and select Control Set.
    • In the Performance section, if the Control Set is disabled, click here.

  2. On the Control Set dialog box, toggle Enable Control Set.
  3. Select the Confidence Level and the Target Error Margin.
  4. Click Save.
  5. On the Predictive Coding page, click Train & Classify.
  6. In the Performance section, view the Score Thresholds.

To adjust the options of the Control Set, such as the Primary Classifier , Priority Metric (Prevalence, Recall, or Precision), Target-Cut-off value, Confidence Level, and Target Error Margin, open the Control Set dialog box (Classifiers header > Action icon > Control Set) and update the levels and margins as necessary.

Control Set customization requires a minimum of 270 documents reviewed. When less than this amount of documents are present, the Performance section displays the following message:

Viewing the training history

You can view the details all training sessions that have been conducted for the project in the Training History.

To view project's predictive coding training history

  1. In the Nebula menu bar, click Review > Admin > Predictive Coding.
  2. In the Predictive Coding menu, choose the project you want to work with using the project drop list.
  3. Click its Action icon and select Training History.
  4. On the Training History dialog box, view the list of all training sessions that have been conducted for the project.

  5. Click a training session's Status to view its task details on the Tasks (Review) page.
  6. If a training session resulted in a Failed status with a single document error, click the Download Report icon in the session's Exceptions column to download an Exceptions Report.

Viewing predictive coding scores

You can configure the Reviewer Document List to display predictive coding scores using Customize Fields.

To view predictive coding scores

  1. In Review, choose the Saved Search (Review > Searches) or document set from Review Explore menu (Review > Explore) with predictive coding scores you want to view.
  2. On the Document List, click the Action icon and select Customize Fields.
  3. On the Doc List Views dialog box, click the Action icon to the right of Name and choose Add.
  4. Enter a Name of the predictive coding listing view.
  5. Select the Scope for the predictive coding listing view.

    6. Note: The scope must be at the matter or user level.

  6. In the left table, select the fields to include in your view, making sure to include the predictive coding scoring fields (those with a format of [profile name]/[coding option]/[choice].

  7. Click Save.
  8. Click Set Profile.
  9. View the profile scores on the Document List.

Note: To avoid biased coding, avoid allowing reviewers to see the predictive coding scores on their listing view profiles (document list views).