Google AI Hackathon Submission

Andrew Cook

2 May 202402:42

Summary

TLDRThe presenter introduces a cognitive remediation device designed to assist individuals experiencing transient auditory hallucinations. The device, powered by a Raspberry Pi 02W and an IQ audio codec zero board, features a microphone, a tactile button, and two LED lights. It operates by continuously recording sound and, upon pressing the tactile button, clips the last two minutes of audio. The Gemini API is then queried to detect any significant deviation in sound from the first minute and 45 seconds to the final 15 seconds of the recording. A green LED indicates a detected sound, suggesting a hallucination, while a red LED signifies no novel sound, implying no hallucination. The device also supports debugging with a simultaneous red and green flash for new recordings. The presenter demonstrates the device's functionality during the presentation, showing a correctly labeled negative output with a red LED flash when no significant sound change is detected. Enhancements could include reducing response latency and adding a web app for data visualization.

Takeaways

🎓 The presenter has developed a cognitive remediation device for individuals experiencing transient auditory hallucinations.
📡 The device uses a Raspberry Pi 02w connected to an IQ audio codec zero board with a microphone, a tactile button, and two LED lights.
🔊 The device is programmed to run a Python script at startup, which continuously records sound.
🕒 Upon pressing the tactile button, the device clips the last 2 minutes of audio and queries the Gemini API.
🔍 The Gemini API checks for a significant deviation in sound in the final 15 seconds compared to the previous 1 minute and 45 seconds.
🟢 A green LED lights up if no novel sound signal is detected, suggesting no hallucination.
🟠 A red LED lights up if a significant deviation is found, indicating a possible hallucination.
💬 The device has been recording the presenter's speech during the presentation.
🔴 A red LED flash indicates a correctly labeled negative output, meaning no hallucination was detected in the speech.
🟢🔴 A simultaneous flash of both LEDs is used for debugging, signaling a new recording is taking place.
🕒 The presenter suggests feature enhancements such as reducing response latency and adding a web app for data visualization.
🙏 The presenter thanks the audience for their time and concludes the presentation.

Q & A

What is the primary purpose of the cognitive remediation device presented?
-The cognitive remediation device is designed to assist individuals who experience transient auditory hallucinations by recording and analyzing sound to help identify the presence or absence of such hallucinations.
What components are used to create this device?
-The device is made using a Raspberry Pi 02W, an IQ audio codec zero board with a microphone, a tactile button, and two LED lights.
How does the device record sound?
-The device is programmed to run a Python script at startup that constantly records sound.
What happens when the tactile button is pressed?
-Upon pressing the tactile button, the device clips the last 2 minutes of audio and queries the Gemini API to check for any significant deviation in sound from the initial recording.
How does the Gemini API contribute to the device's functionality?
-The Gemini API is used to analyze the sound profile of the recording. It returns a response indicating if there is a significant deviation in the sound during the final 15 seconds compared to the prior minute and 45 seconds.
What does a green LED light signify?
-A green LED light signifies that a significant deviation from the sound profile was detected, suggesting the presence of a hallucination.
What does a red LED light signify?
-A red LED light signifies that no significant deviation from the sound profile was detected, indicating that the user is not experiencing a hallucination.
What does a simultaneous flash of red and green LEDs indicate?
-A simultaneous flash of red and green LEDs is used for debugging purposes to indicate that a new recording is taking place.
How does the device help in the presentation scenario?
-In the presentation scenario, the device correctly identifies the lack of a hallucination by the user, as the speaker has been speaking consistently, resulting in a red LED light flash.
What feature enhancements could be made to the device?
-Possible enhancements include reducing response latency, which is the time between the tactile button being pressed and the LED light flashing, and adding a web app for visualizing user data.
How does the device differentiate between normal speech and a hallucinatory sound?
-The device differentiates by comparing the sound profile of the final 15 seconds of the recording to the first minute and 45 seconds. If there is a significant deviation, it suggests a hallucination; otherwise, it indicates normal speech.
What is the significance of the device's ability to record and analyze sound?
-The significance lies in its potential to provide real-time feedback to users experiencing transient auditory hallucinations, helping them understand their condition better and possibly aiding in the management of their symptoms.

Outlines

00:00

🎧 Cognitive Remediation Device Presentation

The presenter introduces a cognitive remediation device designed for individuals experiencing transient auditory hallucinations. The device, built with a Raspberry Pi 02w and an IQ audio codec zero board, features a microphone, a tactile button, and two LED lights. It operates a Python script that continuously records sound. Upon pressing the tactile button, the device clips the last two minutes of audio and uses the Gemini API to check for any significant deviation in sound from the first minute and 45 seconds. If a deviation is detected, indicating a possible hallucination, a green LED lights up; otherwise, a red LED indicates no novel sound signal. The device has been recording the presenter's speech, and a red LED flash signifies no hallucinatory sound detected. The presenter also demonstrates that a whistle blown during the presentation would trigger a green LED, indicating a detected signal. The script concludes with potential enhancements, such as reducing response latency and adding a web app for data visualization.

Mindmap

Keywords

💡Cognitive Remediation Device

A cognitive remediation device is a tool designed to help improve cognitive functions, particularly in individuals with conditions that affect their cognitive abilities. In the context of the video, this device is specifically tailored for individuals experiencing transient auditory hallucinations. It uses sound recording and analysis to provide feedback on the nature of the sounds heard by the user, potentially assisting in differentiating between real and hallucinated auditory experiences.

💡Transient Auditory Hallucinations

Transient auditory hallucinations refer to brief episodes where a person perceives sounds that are not actually present in their environment. These are not constant occurrences but happen intermittently. In the video, the device is intended to assist users in managing these experiences by providing an objective analysis of the sounds they hear.

💡Raspberry Pi 02w

The Raspberry Pi 02w is a small, low-cost, single-board computer that is widely used for various electronic projects. In the video, it serves as the central processing unit for the cognitive remediation device, handling tasks such as recording sound and running the Python script that analyzes the audio.

💡IQ Audio Codec Zero Board

The IQ Audio Codec Zero Board is an integrated circuit board that processes audio signals. It is equipped with a microphone, making it suitable for recording sounds. In the context of the video, this board is attached to the Raspberry Pi and is used to capture the user's auditory environment for later analysis.

💡Python Script

A Python script is a sequence of instructions written in the Python programming language. In the video, the script is used to automate the device's functionality. It constantly records sound and, upon the user's command, clips the audio for analysis. The script's role is crucial as it enables the device to function as intended without continuous manual input from the user.

💡Gemini API

An API, or Application Programming Interface, is a set of rules and protocols that allows different software applications to communicate with each other. The Gemini API, as mentioned in the video, is queried by the device to analyze the recorded audio and determine if there is a significant deviation in sound patterns, which could indicate a hallucination.

💡LED Lights

LED stands for Light Emitting Diode, a type of semiconductor light source that is used in the device for visual feedback. In the video, two LED lights are used to indicate the analysis results: a green light for a significant deviation (suggestive of a hallucination) and a red light for no deviation (not suggestive of a hallucination). The simultaneous flash of both LEDs is used for debugging, signaling the start of a new recording.

💡Sound Profile

A sound profile refers to the characteristic features of a sound, including its frequency, amplitude, and other qualities. In the context of the video, the device compares the sound profile of the final 15 seconds of the recording to the first minute and 45 seconds to detect any significant changes that might indicate a hallucination.

💡Tactile Button

A tactile button is a physical push-button that provides a distinct tactile feedback when pressed, allowing users to interact with the device. In the video, pressing the tactile button triggers the device to clip the last 2 minutes of audio and initiate the analysis process via the Gemini API.

💡Response Latency

Response latency refers to the delay between the initiation of an action and the system's response to it. In the video, reducing response latency is suggested as a feature enhancement, which would improve the user experience by minimizing the time between pressing the tactile button and receiving the analysis result.

💡Web App

A web app is an application that runs in a web browser over the internet. It is mentioned in the video as a potential enhancement to the device, allowing users to visualize their data. This could provide a more comprehensive understanding of their auditory experiences and the device's analysis over time.

Highlights

Presenting a cognitive remediation device for individuals experiencing transient auditory hallucinations

Device uses a Raspberry Pi 02W and IQ audio codec zero board with a microphone, tactile button, and two LED lights

Python script runs at startup, constantly recording sound

Pressing the tactile button clips the last 2 minutes of audio

Queries the Gemini API to check for a significant sound deviation in the final 15 seconds compared to the prior 1 minute and 45 seconds

Green LED lights up if no novel sound signal is detected, indicating not suggestive of a hallucination

Red LED lights up if a significant sound deviation is detected, supportive of the existence of a hallucination

Board has been recording the presenter's speech during the presentation

Pressing the tactile button should result in a red LED flash if the final 15 seconds of recording matches the profile of the first minute and 45 seconds

Correctly labeled negative output demonstrated with red LED flash

Simultaneous red and green LED flash used for debugging to indicate a new recording

Blowing a whistle and pressing the button should result in a green LED if a signal is detected

Potential feature enhancements include reducing response latency and adding a web app for data visualization

Response latency is the time from when the tactile button is pushed to when the LED flashes

Web app would allow users to visualize their data

Device aims to help individuals with transient auditory hallucinations by detecting and providing feedback on novel sound signals

The device's effectiveness in identifying the lack of a signal was demonstrated during the presentation

Thank you for your time and attention to this innovative device