Dice Ambient Scribe STREAM 1.0 documentation

Dice Ambient Scribe STREAM 1.0

The STREAM API uses websockets to perform real-time transcription and clinical notes generation from media capture devices.

API Flow

General specifications

The STREAM endpoint requires that all messages are sent/received as JSON text frames.
No messages and data are stored from Dice, so we recommend collation/storage of information on your end.
The server or the user should pass their bearer authentication token as an Authorization header when initiating the websocket. Example: url: 'wss://api.dice.com/v1/scribe/copilot/server/listen', protocol: 'copilot-listen-protocol', extra_headers: { 'Authorization': 'Bearer ' }.

Example operation:

EMR App opens a websocket connection with the auth token.
EMR App sends first message specifying the stream and transcription configuration.
EMR App sends small audio chunks for each stream continuously
Dice Ambient Scribe sends transcript items continuously
EMR App stops media stream and/or sends a message { "object": "stop"}.
Dice Ambient Scribe evaluates and sends the clinical note.
Dice Ambient Scribe closes the websocket.

Servers

wss://api.dice.health/v1/scribe/serverwssserver
Server API called from backend application
wss://api.dice.health/v1/scribe/userwssuser
User API called from frontend application

Operations

PUB /
Send messages to the API

Operation IDsendMessage
Accepts one of the following messages:
- #0stream_config
  Stream and response config to setup API.
  (Sent from the app to the Scribe API) This is the first message sent to the websocket API to configure the stream as well as the transcription/note generation settings.
  
  object
  uid: stream_request
  object
  required
  string
  Allowed values:
  "stream_request"
  required
  anyOf
  Output to be returned from the API (one or both of transcript/note)
  
  Can adhere to:
  string
  Allowed values:
  "transcript"
  "note"
  array<any>
  Items:
  any
  required
  array<object>
  Description of audio streams. Use if different channels are being used to record separate doctor/patient audio. If unsure, use single stream with unknown speaker flag and the Scribe API will diarize it for you.
  
  id
  required
  string
  An identifier associated with the stream
  
  speaker
  required
  string
  Who is speaking in this stream
  
  Allowed values:
  "provider"
  "patient"
  "unknown"
  sample_rate
  integer
  Audio sampling rate
  
  language
  required
  string
  Language of consultation
  
  Allowed values:
  "en-US"
  "fr-CA"
  note_template
  required
  string
  Clinical note template to use (e.g. soap, consult)
  
  Allowed values:
  "soap"
  "consult"
  style
  required
  string
  Formatting style to use for each section in the clinical note
  
  Allowed values:
  "paragraph"
  "bullet"
  "auto"
  transcribe_gen_mode
  required
  string
  Quality of transcription
  
  Allowed values:
  "fast"
  "medium"
  "best"
  note_gen_mode
  required
  string
  Quality of note generation
  
  Allowed values:
  "fast"
  "medium"
  "best"
  Additional properties are allowed.
  Examples
  { "object": "stream_request", "output_objects": "transcript", "streams": [ { "id": "string", "speaker": "provider" } ], "sample_rate": 0, "language": "en-US", "note_template": "soap", "style": "paragraph", "transcribe_gen_mode": "fast", "note_gen_mode": "fast" }
  
  This example has been generated automatically.
- #1audio_chunk
  Small audio segment for transcription/note-generation.
  (Sent from the app to the Scribe API) Short audio chunk payload of an audio track from the consultation. Max duration 1s.
  
  object
  uid: audio_chunk
  object
  required
  string
  Object Name
  
  Allowed values:
  "audio_chunk"
  payload
  required
  string
  Raw audio chunk in base64 string
  
  stream_id
  required
  string
  Name of the stream as defined in the streams parameter of the stream configuration.
  
  Additional properties are allowed.
  Examples
  #1 Example
  { "object": "audio_chunk", "payload": "wac=aasdZawvewad234141", "stream_id": "stream1" }
- #2stop
  To notify stream stoppage.
  (Sent from the app to the Scribe API) End audio stream and ask Scribe to return the clinical note and close the websocket.
  
  object
  uid: stop
  object
  required
  string
  Stop Message
  
  Allowed values:
  "stop"
  Additional properties are allowed.
  Examples
  { "object": "stop" }
  
  This example has been generated automatically.
SUB /
Messages that you receive from the API

Operation IDreceiveMessage
Accepts one of the following messages:
- #0transcript_item
  A segment of the ongoing transcript (sent piece-wise)
  (Sent from the Scribe API to the app) A segment of the ongoing transcript. Usually, it's the sentence currently spoken, derived from the most recent audio segments sent. This sentence might be partial because transcription continues as we get audio segments. Update the earlier transcript entry with the identical ID until "is_final" is set to true.
  
  object
  uid: transcript_item
  object
  required
  string
  Object Name
  
  Allowed values:
  "transcript_item"
  id
  required
  string
  Unique ID of transcript_item. Collate all transcript items with the same ID together.
  
  text
  required
  string
  Transcription of audio.
  
  speaker
  required
  string
  Speaker of transcribed audio.
  
  Allowed values:
  "provider"
  "patient"
  "unknown"
  start_offset_ms
  required
  integer
  Start time of trancript_item with respect to the websocket opening time (in milliseconds).
  
  end_offset_ms
  required
  integer
  End time of trancript_item with respect to the websocket opening time (in milliseconds).
  
  is_final
  required
  boolean
  True when it is the final version of the transcript item.
  
  Additional properties are allowed.
  Examples
  { "object": "transcript_item", "id": "string", "text": "string", "speaker": "provider", "start_offset_ms": 0, "end_offset_ms": 0, "is_final": true }
  
  This example has been generated automatically.
- #1note
  Clinical note response
  (Sent from the Scribe API to the app) Clinical note generated from the consultation
  
  object
  uid: note
  object
  required
  string
  Object Name
  
  Allowed values:
  "transcript_item"
  id
  required
  string
  Unique ID of transcript_item. Collate all transcript items with the same ID together.
  
  is_final
  required
  boolean
  True when it is the final version of the transcript item.
  
  required
  array<object>
  Clinical Note structured as array of sections.
  
  sec_title
  required
  string
  Key to identify section name. Possible values depend on note template used (soap, consult) etc.
  
  Allowed values:
  "SOAP_SUBJECTIVE"
  "SOAP_OBJECTIVE"
  "SOAP_ASSESSMENT"
  "SOAP_PLAN"
  "REASON_FOR_REFERRAL"
  "HISTORY_OF_PRESENT_ILLNESS"
  "PAST_MEDICAL_HISTORY"
  "MEDICATIONS"
  "ALLERGIES"
  "SOCIAL_HISTORY"
  "FAMILY_HISTORY"
  "ASSESSMENT"
  "PLAN"
  sec_heading
  required
  string
  Text heading of the section (human-readable)
  
  sec_text
  required
  string
  Text content of the section in the format specified in the stream config
  
  Additional properties are allowed.
  Examples
  { "object": "transcript_item", "id": "string", "is_final": true, "sections": [ { "sec_title": "SOAP_SUBJECTIVE", "sec_heading": "string", "sec_text": "string" } ] }
  
  This example has been generated automatically.
- #2duration_limit
  Time left in seconds until the stream limit (1 hour) is reached.
  (Sent from the Scribe API to the app) Time left in seconds until the stream limit (1 hour) is reached. Countdown starts at 60s.
  
  object
  uid: duration_limit
  object
  required
  string
  Allowed values:
  "duration_limit"
  remaining_sec
  required
  integer
  Seconds left until stream websocket closes (countdown starts at 60s).
  
  Additional properties are allowed.
  Examples
  { "object": "duration_limit", "remaining_sec": 0 }
  
  This example has been generated automatically.
- #3error_msg
  Error Message
  (Sent from the Scribe API to the app) Error Message received from the API
  
  object
  uid: error_msg
  object
  required
  string
  Allowed values:
  "error_msg"
  message
  required
  string
  Error Message
  
  Additional properties are allowed.
  Examples
  { "object": "error_msg", "message": "string" }
  
  This example has been generated automatically.

Messages

#1stream_config
Stream and response config to setup API.
(Sent from the app to the Scribe API) This is the first message sent to the websocket API to configure the stream as well as the transcription/note generation settings.
object
uid: stream_request
object
required
string
Allowed values:
"stream_request"
required
anyOf
Output to be returned from the API (one or both of transcript/note)

Can adhere to:
string
Allowed values:
"transcript"
"note"
array<any>
Items:
any
required
array<object>
Description of audio streams. Use if different channels are being used to record separate doctor/patient audio. If unsure, use single stream with unknown speaker flag and the Scribe API will diarize it for you.

id
required
string
An identifier associated with the stream

speaker
required
string
Who is speaking in this stream

Allowed values:
"provider"
"patient"
"unknown"
sample_rate
integer
Audio sampling rate

language
required
string
Language of consultation

Allowed values:
"en-US"
"fr-CA"
note_template
required
string
Clinical note template to use (e.g. soap, consult)

Allowed values:
"soap"
"consult"
style
required
string
Formatting style to use for each section in the clinical note

Allowed values:
"paragraph"
"bullet"
"auto"
transcribe_gen_mode
required
string
Quality of transcription

Allowed values:
"fast"
"medium"
"best"
note_gen_mode
required
string
Quality of note generation

Allowed values:
"fast"
"medium"
"best"
Additional properties are allowed.
#2audio_chunk
Small audio segment for transcription/note-generation.
(Sent from the app to the Scribe API) Short audio chunk payload of an audio track from the consultation. Max duration 1s.
object
uid: audio_chunk
object
required
string
Object Name

Allowed values:
"audio_chunk"
payload
required
string
Raw audio chunk in base64 string

stream_id
required
string
Name of the stream as defined in the streams parameter of the stream configuration.

Additional properties are allowed.
#3stop
To notify stream stoppage.
(Sent from the app to the Scribe API) End audio stream and ask Scribe to return the clinical note and close the websocket.
object
uid: stop
object
required
string
Stop Message

Allowed values:
"stop"
Additional properties are allowed.
#4transcript_item
A segment of the ongoing transcript (sent piece-wise)
(Sent from the Scribe API to the app) A segment of the ongoing transcript. Usually, it's the sentence currently spoken, derived from the most recent audio segments sent. This sentence might be partial because transcription continues as we get audio segments. Update the earlier transcript entry with the identical ID until "is_final" is set to true.
object
uid: transcript_item
object
required
string
Object Name

Allowed values:
"transcript_item"
id
required
string
Unique ID of transcript_item. Collate all transcript items with the same ID together.

text
required
string
Transcription of audio.

speaker
required
string
Speaker of transcribed audio.

Allowed values:
"provider"
"patient"
"unknown"
start_offset_ms
required
integer
Start time of trancript_item with respect to the websocket opening time (in milliseconds).

end_offset_ms
required
integer
End time of trancript_item with respect to the websocket opening time (in milliseconds).

is_final
required
boolean
True when it is the final version of the transcript item.

Additional properties are allowed.
#5note
Clinical note response
(Sent from the Scribe API to the app) Clinical note generated from the consultation
object
uid: note
object
required
string
Object Name

Allowed values:
"transcript_item"
id
required
string
Unique ID of transcript_item. Collate all transcript items with the same ID together.

is_final
required
boolean
True when it is the final version of the transcript item.

required
array<object>
Clinical Note structured as array of sections.

sec_title
required
string
Key to identify section name. Possible values depend on note template used (soap, consult) etc.

Allowed values:
"SOAP_SUBJECTIVE"
"SOAP_OBJECTIVE"
"SOAP_ASSESSMENT"
"SOAP_PLAN"
"REASON_FOR_REFERRAL"
"HISTORY_OF_PRESENT_ILLNESS"
"PAST_MEDICAL_HISTORY"
"MEDICATIONS"
"ALLERGIES"
"SOCIAL_HISTORY"
"FAMILY_HISTORY"
"ASSESSMENT"
"PLAN"
sec_heading
required
string
Text heading of the section (human-readable)

sec_text
required
string
Text content of the section in the format specified in the stream config

Additional properties are allowed.
#6duration_limit
Time left in seconds until the stream limit (1 hour) is reached.
(Sent from the Scribe API to the app) Time left in seconds until the stream limit (1 hour) is reached. Countdown starts at 60s.
object
uid: duration_limit
object
required
string
Allowed values:
"duration_limit"
remaining_sec
required
integer
Seconds left until stream websocket closes (countdown starts at 60s).

Additional properties are allowed.
#7error_msg
Error Message
(Sent from the Scribe API to the app) Error Message received from the API
object
uid: error_msg
object
required
string
Allowed values:
"error_msg"
message
required
string
Error Message

Additional properties are allowed.

Schemas

object
uid: stream_id
id
required
string
An identifier associated with the stream

speaker
required
string
Who is speaking in this stream

Allowed values:
"provider"
"patient"
"unknown"
Additional properties are allowed.
object
uid: stream_request
object
required
string
Allowed values:
"stream_request"
required
anyOf
Output to be returned from the API (one or both of transcript/note)

Can adhere to:
string
Allowed values:
"transcript"
"note"
array<any>
Items:
any
required
array<object>
Description of audio streams. Use if different channels are being used to record separate doctor/patient audio. If unsure, use single stream with unknown speaker flag and the Scribe API will diarize it for you.

id
required
string
An identifier associated with the stream

speaker
required
string
Who is speaking in this stream

Allowed values:
"provider"
"patient"
"unknown"
sample_rate
integer
Audio sampling rate

language
required
string
Language of consultation

Allowed values:
"en-US"
"fr-CA"
note_template
required
string
Clinical note template to use (e.g. soap, consult)

Allowed values:
"soap"
"consult"
style
required
string
Formatting style to use for each section in the clinical note

Allowed values:
"paragraph"
"bullet"
"auto"
transcribe_gen_mode
required
string
Quality of transcription

Allowed values:
"fast"
"medium"
"best"
note_gen_mode
required
string
Quality of note generation

Allowed values:
"fast"
"medium"
"best"
Additional properties are allowed.
object
uid: transcript_item
object
required
string
Object Name

Allowed values:
"transcript_item"
id
required
string
Unique ID of transcript_item. Collate all transcript items with the same ID together.

text
required
string
Transcription of audio.

speaker
required
string
Speaker of transcribed audio.

Allowed values:
"provider"
"patient"
"unknown"
start_offset_ms
required
integer
Start time of trancript_item with respect to the websocket opening time (in milliseconds).

end_offset_ms
required
integer
End time of trancript_item with respect to the websocket opening time (in milliseconds).

is_final
required
boolean
True when it is the final version of the transcript item.

Additional properties are allowed.
object
uid: note_section
sec_title
required
string
Key to identify section name. Possible values depend on note template used (soap, consult) etc.

Allowed values:
"SOAP_SUBJECTIVE"
"SOAP_OBJECTIVE"
"SOAP_ASSESSMENT"
"SOAP_PLAN"
"REASON_FOR_REFERRAL"
"HISTORY_OF_PRESENT_ILLNESS"
"PAST_MEDICAL_HISTORY"
"MEDICATIONS"
"ALLERGIES"
"SOCIAL_HISTORY"
"FAMILY_HISTORY"
"ASSESSMENT"
"PLAN"
sec_heading
required
string
Text heading of the section (human-readable)

sec_text
required
string
Text content of the section in the format specified in the stream config

Additional properties are allowed.
object
uid: audio_chunk
object
required
string
Object Name

Allowed values:
"audio_chunk"
payload
required
string
Raw audio chunk in base64 string

stream_id
required
string
Name of the stream as defined in the streams parameter of the stream configuration.

Additional properties are allowed.
object
uid: note
object
required
string
Object Name

Allowed values:
"transcript_item"
id
required
string
Unique ID of transcript_item. Collate all transcript items with the same ID together.

is_final
required
boolean
True when it is the final version of the transcript item.

required
array<object>
Clinical Note structured as array of sections.

sec_title
required
string
Key to identify section name. Possible values depend on note template used (soap, consult) etc.

Allowed values:
"SOAP_SUBJECTIVE"
"SOAP_OBJECTIVE"
"SOAP_ASSESSMENT"
"SOAP_PLAN"
"REASON_FOR_REFERRAL"
"HISTORY_OF_PRESENT_ILLNESS"
"PAST_MEDICAL_HISTORY"
"MEDICATIONS"
"ALLERGIES"
"SOCIAL_HISTORY"
"FAMILY_HISTORY"
"ASSESSMENT"
"PLAN"
sec_heading
required
string
Text heading of the section (human-readable)

sec_text
required
string
Text content of the section in the format specified in the stream config

Additional properties are allowed.
object
uid: stop
object
required
string
Stop Message

Allowed values:
"stop"
Additional properties are allowed.
object
uid: duration_limit
object
required
string
Allowed values:
"duration_limit"
remaining_sec
required
integer
Seconds left until stream websocket closes (countdown starts at 60s).

Additional properties are allowed.
object
uid: error_msg
object
required
string
Allowed values:
"error_msg"
message
required
string
Error Message

Additional properties are allowed.

Dice Ambient Scribe STREAM 1.0

API Flow

Servers

Operations

PUB /

Examples

This example has been generated automatically.

Examples

#1 Example

Examples

This example has been generated automatically.

SUB /

Examples

This example has been generated automatically.

Examples

This example has been generated automatically.

Examples

This example has been generated automatically.

Examples

This example has been generated automatically.

Messages

Schemas