TRULEO's Large Language Models detect dozens of events automatically within a call transcript
An event label is the representation of a pre-defined event within a BWC transcript
There are three types of event labels.-
- Auto-generated events
- Pending events that require human verification
- Human-verified events that clarify further auto-generated event labels
In the descriptions below, an example of each label is shown like this:
Here's an example of an event
Events are categorized into Call Events, Officer Language, and Community Member Language. Officer Language only refers to the language of the officer wearing the camera.
AUTO-GENERATED EVENTS
Auto-generated events are automatically generated by TRULEO and appear instantly in the transcript. An explanation of how our models were trained to identify auto-generated events is available below.
Call Events
Call events are detected from speech of any speaker on a call.
Person In Crisis
A person whose mental health symptoms or level of distress have exceeded
the person’s internal ability to manage their behavior or emotion
You're telling me you want to hurt yourself?
Domestic Violence
Abusive behavior in any relationship to gain power or control
Did your husband give you that bloody lip?
Terry Stop
A stop conducted by an officer where the person is not free to leave
You are not free to leave
Traffic Stop
A temporary detention of the driver of a vehicle and its occupants to investigate a possible crime or minor violation
Can I see your license and registration
Search
The act of an officer searching a person or their belongings
I am going to search your bag
Pursuit
Officer(s) chase a person(s) in car or on foot in an effort to detain or arrest
He’s turning right in the alley on 47th
Arrest
The act of taking a person into custody
You are being placed under arrest
Personal Data
Redaction of a person’s personal identifiable information
My address is [******]
Muted
Segments of video that are muted
Translated Spanish
TRULEO automatically detects Spanish language and translates to English
Puede mostrarme su identificacion?
Can you show me your ID?
Officer Language
Officer Language is detected in transcription that comes from the officer wearing the camera via TRULEO's Officer ID models
Introduction
An officer introducing himself / herself
Hi, my name is Officer Joe from the Springfield Police Department
Explanation
An officer offering reasoning for actions before taking them
The reason we were called here tonight is because your neighbor reported
Traffic Stop Reason
An officer providing a reason for a traffic stop
The reason I’m pulling you over is...
Business Card
An officer offers a business card to a person
Here’s my card with my badge number and contact information
De-Escalation Attempt
Commands or persuasive language given by an officer in an attempt to de-escalate a non-compliant person
Sir, take your hands out of your pockets
You're a big dude. I don’t want to fight you but I do need you to listen to me
Offering Services
An officer offers information about support services or help to a person in crisis or domestic violence victim
I'm going to connect you with some support services to help you
Community Member Language
Community Member Language is detected in transcription that comes from any speaker that is not the officer wearing the camera
Upset Person
A person who directs profanity or insults at an officer
F*** you, you're a *****”
Noncompliant Person
A person refusing to comply with officer commands
No I will not do that
Community Gratitude
Expressions of gratitude from a person towards an officer
Thank you so much officer, I really appreciate it
Potential Complaint
Expressions by a community member that they may file a complaint
I want to speak to your supervisor
PENDING EVENTS
Pending events are detected by TRULEO but only shown to administrators and reviewers until they are human verified. These events deal with sensitive scenarios that may require context outside of the transcript to verify what they are. About 1% - 3% of all calls analyzed contain these pending events.
Calls that are eligible for High Professionalism or High Composure but contain these levels will not be awarded until these labels are verified.
Discussion of Force
Discussion of physical effort to compel compliance with a non-compliant person
I’m tasing you right now
Impolite Language
Directed profanity and insults (racial slurs) directed at a person or about a person
F*** you, you're a *****”
HUMAN-VERIFIED EVENTS
Human-verified events are verified through TRULEO's pending workflow by an administrator or supervisor / reviewer. Reviewers typically spend a few minutes per week verifying pending events and producing these labels.
Once these labels are verified, calls eligible for High Professional or High Composure that contain pending labels can be awarded if they do not contain Use of Force or Below Standard Language.
Use of Force
A Use of Force label is created when a Discussion of Force label is verified by a human reviewer after they determine the officer wearing the camera was involved in a Use of Force. Alternatively, they can verify that the event is only a Discussion of Force and not a Use of Force.
Below Standard Language
A Below Standard Language label is created when an Impolite Language label is verified by a human reviewer after they determine:
- The officer wearing the camera was the one speaking
- The use of the impolite language was "Below Standard" as determined by the reviewer
How TRULEO Trains Models
TRULEO trains its NLP models using data where people tag parts of conversations with important events and labels. A team of experts, including former NYPD sergeants and community members, does the tagging. The goal is to ensure that more than 90% of the tags match among different taggers. Our labeling team to date has tagged over 4 million data points.
The models identify events by looking at pieces of text spoken by one person, anywhere from 1 to 50 words long, and giving them a label. These events are anything that shows something happens during a police interaction, whether spoken by the officer or the community member.
The models are tested to ensure they are accurate more than 90% of the time. After testing, humans review the labels to confirm accuracy, aiming again for over 90% agreement among them. Calls are rated as professional if officers explain themselves clearly and avoid unprofessional language.
To measure the accuracy of these models, we focus on precision, which looks at how many times the model is right when it labels something as an event or type of language. Precision is chosen because it gives reliable results even if not all events are captured on body-worn camera videos.
TRULEO's models also constantly learn from user inputs as users validate or remove labels on transcripts within the platform.