Google AI Hackathon Submission

Summary

The presenter introduces a cognitive remediation device designed to assist individuals experiencing transient auditory hallucinations. The device, powered by a Raspberry Pi 02W and an IQ audio codec zero board, features a microphone, a tactile button, and two LED lights. It operates by continuously recording sound and, upon pressing the tactile button, clips the last two minutes of audio. The Gemini API is then queried to detect any significant deviation in sound from the first minute and 45 seconds to the final 15 seconds of the recording. A green LED indicates a detected sound, suggesting a hallucination, while a red LED signifies no novel sound, implying no hallucination. The device also supports debugging with a simultaneous red and green flash for new recordings. The presenter demonstrates the device's functionality during the presentation, showing a correctly labeled negative output with a red LED flash when no significant sound change is detected. Enhancements could include reducing response latency and adding a web app for data visualization.

Takeaways

• 🎓 The presenter has developed a cognitive remediation device for individuals experiencing transient auditory hallucinations.
• 📡 The device uses a Raspberry Pi 02w connected to an IQ audio codec zero board with a microphone, a tactile button, and two LED lights.
• 🔊 The device is programmed to run a Python script at startup, which continuously records sound.
• 🕒 Upon pressing the tactile button, the device clips the last 2 minutes of audio and queries the Gemini API.
• 🔍 The Gemini API checks for a significant deviation in sound in the final 15 seconds compared to the previous 1 minute and 45 seconds.
• 🟢 A green LED lights up if no novel sound signal is detected, suggesting no hallucination.
• 🟠 A red LED lights up if a significant deviation is found, indicating a possible hallucination.
• 💬 The device has been recording the presenter's speech during the presentation.
• 🔴 A red LED flash indicates a correctly labeled negative output, meaning no hallucination was detected in the speech.
• 🟢🔴 A simultaneous flash of both LEDs is used for debugging, signaling a new recording is taking place.
• 🕒 The presenter suggests feature enhancements such as reducing response latency and adding a web app for data visualization.
• 🙏 The presenter thanks the audience for their time and concludes the presentation.

Q & A

What is the primary purpose of the cognitive remediation device presented?

The cognitive remediation device is designed to assist individuals who experience transient auditory hallucinations by recording and analyzing sound to help identify the presence or absence of such hallucinations.

What components are used to create this device?

The device is made using a Raspberry Pi 02W, an IQ audio codec zero board with a microphone, a tactile button, and two LED lights.

How does the device record sound?

The device is programmed to run a Python script at startup that constantly records sound.

What happens when the tactile button is pressed?

Upon pressing the tactile button, the device clips the last 2 minutes of audio and queries the Gemini API to check for any significant deviation in sound from the initial recording.

How does the Gemini API contribute to the device's functionality?

The Gemini API is used to analyze the sound profile of the recording. It returns a response indicating if there is a significant deviation in the sound during the final 15 seconds compared to the prior minute and 45 seconds.

What does a green LED light signify?

A green LED light signifies that a significant deviation from the sound profile was detected, suggesting the presence of a hallucination.

What does a red LED light signify?

A red LED light signifies that no significant deviation from the sound profile was detected, indicating that the user is not experiencing a hallucination.

What does a simultaneous flash of red and green LEDs indicate?

A simultaneous flash of red and green LEDs is used for debugging purposes to indicate that a new recording is taking place.

How does the device help in the presentation scenario?

In the presentation scenario, the device correctly identifies the lack of a hallucination by the user, as the speaker has been speaking consistently, resulting in a red LED light flash.

What feature enhancements could be made to the device?

Possible enhancements include reducing response latency, which is the time between the tactile button being pressed and the LED light flashing, and adding a web app for visualizing user data.

How does the device differentiate between normal speech and a hallucinatory sound?

The device differentiates by comparing the sound profile of the final 15 seconds of the recording to the first minute and 45 seconds. If there is a significant deviation, it suggests a hallucination; otherwise, it indicates normal speech.

What is the significance of the device's ability to record and analyze sound?

The significance lies in its potential to provide real-time feedback to users experiencing transient auditory hallucinations, helping them understand their condition better and possibly aiding in the management of their symptoms.