Google AI Hackathon Submission
Summary
TLDRThe presenter introduces a cognitive remediation device designed to assist individuals experiencing transient auditory hallucinations. The device, powered by a Raspberry Pi 02W and an IQ audio codec zero board, features a microphone, a tactile button, and two LED lights. It operates by continuously recording sound and, upon pressing the tactile button, clips the last two minutes of audio. The Gemini API is then queried to detect any significant deviation in sound from the first minute and 45 seconds to the final 15 seconds of the recording. A green LED indicates a detected sound, suggesting a hallucination, while a red LED signifies no novel sound, implying no hallucination. The device also supports debugging with a simultaneous red and green flash for new recordings. The presenter demonstrates the device's functionality during the presentation, showing a correctly labeled negative output with a red LED flash when no significant sound change is detected. Enhancements could include reducing response latency and adding a web app for data visualization.
Takeaways
- 🎓 The presenter has developed a cognitive remediation device for individuals experiencing transient auditory hallucinations.
- 📡 The device uses a Raspberry Pi 02w connected to an IQ audio codec zero board with a microphone, a tactile button, and two LED lights.
- 🔊 The device is programmed to run a Python script at startup, which continuously records sound.
- 🕒 Upon pressing the tactile button, the device clips the last 2 minutes of audio and queries the Gemini API.
- 🔍 The Gemini API checks for a significant deviation in sound in the final 15 seconds compared to the previous 1 minute and 45 seconds.
- 🟢 A green LED lights up if no novel sound signal is detected, suggesting no hallucination.
- 🟠 A red LED lights up if a significant deviation is found, indicating a possible hallucination.
- 💬 The device has been recording the presenter's speech during the presentation.
- 🔴 A red LED flash indicates a correctly labeled negative output, meaning no hallucination was detected in the speech.
- 🟢🔴 A simultaneous flash of both LEDs is used for debugging, signaling a new recording is taking place.
- 🕒 The presenter suggests feature enhancements such as reducing response latency and adding a web app for data visualization.
- 🙏 The presenter thanks the audience for their time and concludes the presentation.
Q & A
What is the primary purpose of the cognitive remediation device presented?
-The cognitive remediation device is designed to assist individuals who experience transient auditory hallucinations by recording and analyzing sound to help identify the presence or absence of such hallucinations.
What components are used to create this device?
-The device is made using a Raspberry Pi 02W, an IQ audio codec zero board with a microphone, a tactile button, and two LED lights.
How does the device record sound?
-The device is programmed to run a Python script at startup that constantly records sound.
What happens when the tactile button is pressed?
-Upon pressing the tactile button, the device clips the last 2 minutes of audio and queries the Gemini API to check for any significant deviation in sound from the initial recording.
How does the Gemini API contribute to the device's functionality?
-The Gemini API is used to analyze the sound profile of the recording. It returns a response indicating if there is a significant deviation in the sound during the final 15 seconds compared to the prior minute and 45 seconds.
What does a green LED light signify?
-A green LED light signifies that a significant deviation from the sound profile was detected, suggesting the presence of a hallucination.
What does a red LED light signify?
-A red LED light signifies that no significant deviation from the sound profile was detected, indicating that the user is not experiencing a hallucination.
What does a simultaneous flash of red and green LEDs indicate?
-A simultaneous flash of red and green LEDs is used for debugging purposes to indicate that a new recording is taking place.
How does the device help in the presentation scenario?
-In the presentation scenario, the device correctly identifies the lack of a hallucination by the user, as the speaker has been speaking consistently, resulting in a red LED light flash.
What feature enhancements could be made to the device?
-Possible enhancements include reducing response latency, which is the time between the tactile button being pressed and the LED light flashing, and adding a web app for visualizing user data.
How does the device differentiate between normal speech and a hallucinatory sound?
-The device differentiates by comparing the sound profile of the final 15 seconds of the recording to the first minute and 45 seconds. If there is a significant deviation, it suggests a hallucination; otherwise, it indicates normal speech.
What is the significance of the device's ability to record and analyze sound?
-The significance lies in its potential to provide real-time feedback to users experiencing transient auditory hallucinations, helping them understand their condition better and possibly aiding in the management of their symptoms.
Outlines
🎧 Cognitive Remediation Device Presentation
The presenter introduces a cognitive remediation device designed for individuals experiencing transient auditory hallucinations. The device, built with a Raspberry Pi 02w and an IQ audio codec zero board, features a microphone, a tactile button, and two LED lights. It operates a Python script that continuously records sound. Upon pressing the tactile button, the device clips the last two minutes of audio and uses the Gemini API to check for any significant deviation in sound from the first minute and 45 seconds. If a deviation is detected, indicating a possible hallucination, a green LED lights up; otherwise, a red LED indicates no novel sound signal. The device has been recording the presenter's speech, and a red LED flash signifies no hallucinatory sound detected. The presenter also demonstrates that a whistle blown during the presentation would trigger a green LED, indicating a detected signal. The script concludes with potential enhancements, such as reducing response latency and adding a web app for data visualization.
Mindmap
Keywords
💡Cognitive Remediation Device
💡Transient Auditory Hallucinations
💡Raspberry Pi 02w
💡IQ Audio Codec Zero Board
💡Python Script
💡Gemini API
💡LED Lights
💡Sound Profile
💡Tactile Button
💡Response Latency
💡Web App
Highlights
Presenting a cognitive remediation device for individuals experiencing transient auditory hallucinations
Device uses a Raspberry Pi 02W and IQ audio codec zero board with a microphone, tactile button, and two LED lights
Python script runs at startup, constantly recording sound
Pressing the tactile button clips the last 2 minutes of audio
Queries the Gemini API to check for a significant sound deviation in the final 15 seconds compared to the prior 1 minute and 45 seconds
Green LED lights up if no novel sound signal is detected, indicating not suggestive of a hallucination
Red LED lights up if a significant sound deviation is detected, supportive of the existence of a hallucination
Board has been recording the presenter's speech during the presentation
Pressing the tactile button should result in a red LED flash if the final 15 seconds of recording matches the profile of the first minute and 45 seconds
Correctly labeled negative output demonstrated with red LED flash
Simultaneous red and green LED flash used for debugging to indicate a new recording
Blowing a whistle and pressing the button should result in a green LED if a signal is detected
Potential feature enhancements include reducing response latency and adding a web app for data visualization
Response latency is the time from when the tactile button is pushed to when the LED flashes
Web app would allow users to visualize their data
Device aims to help individuals with transient auditory hallucinations by detecting and providing feedback on novel sound signals
The device's effectiveness in identifying the lack of a signal was demonstrated during the presentation
Thank you for your time and attention to this innovative device
Transcripts
hello everyone I'm excited to present
what I've been working on for the past
few weeks a cognitive remediation device
to be used by individuals who experience
transient that is not constantly
occurring auditory
hallucinations you see here is a
Raspberry Pi 02w attached to an IQ audio
codec zero board featuring a a
microphone a tactile button and two LED
lights
this board has been programmed to run a
python script at startup that constantly
records
sound and upon the Press of the tactile
button Clips the last 2 minutes of audio
and queries the Gemini API asking if a
sound is present in the final 15 seconds
of the recording compared to the prior
minute and 45
seconds if there is a sound that
significantly deviates from the sound
profile of the beginning portion of the
file the model should return a response
indicating as such and a green LED will
light up if no novel sound signal is
detected then a red LED light will light
up these outputs can be respectively
interpreted as one not suggestive of a
hallucination or two supportive of the
existence of a of a hallucination by the
user so all this time this board has
been recording me give this presentation
and when I click the tactile
button uh it should after some time
respond with a flash of the red LED
light because I've been speaking
constantly throughout the final 15
seconds oh sorry I've been speaking
constantly throughout and the final 15
seconds of the recording matches the
profile of the first minute and 45
seconds so that was a correctly labeled
negative output and the red and green
simultaneous flash is just being used
for debugging purposes to indicate that
a new recording is taking
place so we expected to correctly
identify the lack of a signal but if if
I were to blow a whistle as
such and press a
button I should receive a response from
the Gemini API indicating a signal was
indeed detected and a green light will
be displayed to the
user very good feature enhancements
could include reducing response latency
that is the time from when the tactile
button is pushed to when the light
flashes and adding a web app to
visualize user data thank you for your
time
5.0 / 5 (0 votes)