Updates | Envision Glasses Beta

• Fixed issues related to the Korean language.
• Added Indonesian as a device language, as it was missing from the list.
• Added Kannada as a device language.

‍

• The string issue has been fixed.
• Default mode is now Walkie-Talkie mode.
• Ability to open the context menu inside a call: Unfortunately, we can’t restore a call, and a new call will be initiated instead.
• Updated audio for mute and unmute.

‍

• Fixes the issue with making Aira calls.

‍

Dear Envision Community,

We are thrilled to announce the release of Envision Glasses 2.3.2-beta4, a beta update that brings significant enhancements to your experience. This update focuses on refining our scene description capabilities and improving the multi-lingual voice command system.

1. Enhanced Describe Scene Feature:

• Quick Alt-Text Descriptions: Upon capturing an image, the glasses will now provide a concise, "alt text" length description, giving you an immediate understanding of your surroundings.
• Detailed Descriptions On Demand: For a more in-depth perspective, simply prompt with "describe in as much detail as possible." This will unlock a richer, more verbose narrative of the scene.
• Focused Descriptions: Should you seek specifics, commands like "describe the man in the image in more detail" or "what items are on the table" will yield targeted information about particular elements in your environment.

2. Multi-Lingual Voice Commands Revamped:

• Enhanced Command Recognition: We've extensively debugged the system to ensure a seamless experience. The issues leading to the activation of incorrect features have been resolved.
• Simplified Feature Access: Activate the features you need effortlessly. Just speak the name of the feature, and the glasses will comply, recognizing commands in multiple languages.

Your Participation Matters:As beta testers, your feedback is invaluable. Please explore these new capabilities and share your experiences. Your insights are instrumental in shaping the future of Envision Glasses.

To update, simply follow the standard beta update process. Should you encounter any challenges or have queries, our support team is ready to assist.

Thank you for your continued support and for being a part of this journey towards greater independence and empowerment.

Warm regards,

The Envision Team

We're excited to roll out a fresh update to our Envision Glasses Beta! Your feedback has been invaluable in driving the evolution of our product, and we're eager to introduce some enhancements we believe you'll love.

What's New:

1. Enhanced Voice Commands: We have made significant strides in our voice recognition algorithms. You should now experience more responsive and accurate voice commands across all features of the Envision Glasses.
2. Improved "Describe Scene" Functionality: Our "Describe Scene" feature is now even more intuitive! We've integrated the "Ask Envision" capability directly into it. Simply ask Envision about any scene, and it will provide a more detailed and context-aware description.

We Need Your Feedback!

To our dedicated beta testers: we encourage you to put these new features to the test. Your feedback has been instrumental in shaping the Envision experience, and we rely on your insights to make further improvements.

• Test the updated features and use them in different scenarios.
• Report any bugs, inconsistencies, or areas of improvement.
• Share your overall experience and any suggestions for future updates.

Thank you for being an integral part of the Envision journey. We look forward to hearing from you!

Multi-lingual voice command for "Ask Envision" within Scan text.

Hi everyone, we have a new beta update (2.3.0-beta2). In this update, we are introducing multi-lingual voice commands. Currently, this is only optimised for the "Ask Envision" feature within Scan Text. Based on feedback and performance, we will work on optimising it for other voice commands as well.

Disclaimer: Only test this feature, if you know a language that's available on Envision Glasses, apart from English. The last thing we would want is for you to be stuck using the interface in a language you don't understand and not being able to figure out how to switch back.

If you want to test, you may follow these steps:

1. Switch the device language to any language you speak apart from English. You can do this by going to Setting and then Languages.

2. Do a Scan Text and once you're on the reader, just ask it the question in that language. For example, if you changed the device language to French, you can now ask your question in French.

If all is working well, Envision should both understand your question and be able to offer you the answer back in the same language. Also, play around with asking it to translate the text to different languages. Do report your feedback and observations on the WhatsApp group.

Introducing “Ask Envision about image”
‍
We are introducing a new feature where you can take a picture and you will be given a very accurate and extensive description of what is in front of you. The great thing is, afterwards you can ask it questions about the image. There are lots of examples of questions you can ask, and examples of what kind of photos to take, but I will let you figure that out on your own.


What to test?

After doing the update, you will find a new feature called ‘Ask Envision’.
Please test this feature as much as possible with taking different photos and asking it different questions.

Keep in mind this feature is still in beta. Additionally, this is still a very experimental feature so I would also want to put a disclaimer to the output of the AI, that it might not be accurate. So, even if Ask Envision says “I’m 100% sure it’s a cat in front of you”, that percentage of 100 is definitely not the case.

It does pretty okay with some bottles/OCR’ing text, but in general the Instant Text and Scan Text is miles better. So please give it a try and let us know what you’ve been testing it with and the kind of questions that helped for you. All of this will help us in developing this feature, both from the output of the questions, but also the implementation and the final user experience.

As always: there are no wrong questions and there is no wrong feedback.

‍

Note:

Here's a bit more explanation and context behind Ask Envision. The version we have pushed now and you're currently testing is a super early beta version. It's in its very raw form and we put it out so we can capture feedback and mould it accordingly. There's a long way to go before this gets ready for prime time.

The way the technology behind this feature works is very similar to ChatGPT. To oversimplify: it has been trained on billions of described images scraped off the internet and it combines that knowledge to describe a new image that you show it. How exactly it thinks and interprets things in an image is a bit of a mystery (similar to ChatGPT). This is by design, as the idea is to mimic how human brains think and interpret things (which is also mysterious).

The way this gets better is by using better prompts and controlling the parameters of the output. Currently, it is almost on full blast so we're asking the AI to keep on giving as much description of something as possible. A bit like squeezing the lemon dry. That's why after the first few paragraphs which are good, it often starts to give opinions of the atmosphere and things like that. We're doing this to test the limits of this and will play around with these parameters until we find an acceptable range. That's why your feedback on this is essential.

The other thing that happens when you push the AI to describe things too hard (squeeze the lemon dry), it starts to hallucinate. Hallucination is the term used when an AI provides a confident response that does not seem to be justified by its training data. That's why it imagines a chair or cup when there might not be one. This is a common problem that is plaguing even ChatGPT, where if you ask it for the best smart glasses in the market, it will respond with Apple Glasses (which doesn't exist). Hallucinations are trickier to debug and something we will be closely monitoring to check how their occurrence can be reduced.

Just so you know, we will be collecting and reviewing the images that are captured with Ask Envision feature during the beta testing phase, so we can learn from it and improve our models. So do avoid capturing any sensitive images during this phase.

We're very excited for the interactive AI future this unlocks and we're glad to have you beta testers as pioneers of this future!

Offline Scan Text and Batch Scan, Improved Find Object and Explore and an Improved Describe Scene. Read below for more details.

• Offline Scan Text and Batch Scan: you can can now use the feature Scan Text and Batch Scan offline. Enable Offline Mode in the context menu of these features. Once the offline mode is enabled, keep in mind that you can not use Layout Detection. Additionally, it’s also good to know that Language Detection doesn’t work as good as the Online Scan Text yet. Just like Instant Text, Offline Scan Text and Batch Scan are much much faster than the online one, but it also means it’s a little less accurate. You are still able to use ‘Ask Envision’ afterwards in the Reader but you do need to be connected to the internet for that.
• Improved Find Object: in the new Find Object feature you will now be able to find new and more objects. Additionally besides giving an alert when the object is detected, the location of the object will be given as well. So e.g. “Bottle 11 o’clock”. Much requested objects such as door, door handle and stairs are added.
• Improved Explore: the Explore feature has been improved significantly where a lot more objects are recognised by the AI and just like in Find Objects, a clock-notation location of the object will be given.
• Improved Describe Scene: we made some changes in the back-end and the Describe Scene should be a little faster, but fore mostly provide much more detail.

As always, this version comes with all-over minor improvements and bug fixes.

Finally, we added localisation for the Korean language.

‍

How to test:

• Use the Scan Text and Batch Scan feature with Offline Mode enabled and disabled and report about handy use cases when to use Offline Scan Text and when you find it easier to switch to Online Scan Text.
• Try out the new Find Object and Explore feature and report about the ease of finding an object.
• Improved Describe Scene: report back about the difference with the old Describe Scene and whether this new Describe Scene has new use cases for you.
• As always, any feedback is welcome and we'd love to hear both your positive and negative feedback about all added changes.

‍

‍

This beta update has an improved version of Describe Scene. This mostly was a backend change where we are now using a completely different model to process the images taken and provide a description. In our internal testing, we have found the description to be significantly more accurate and more descriptive than the previous version. It can also sometime recognise signs, product labels and large text.

‍

How to test:

• Update to 2.2.0 (this is a backend change, so technically updating is not as necessary, but we would still recommend doing it).
• Go to Describe Scene feature and use it with different situations. Try taking picture of various surroundings or various products.
• Report your findings and comparisons to the old describe scene features.

‍

This updated fixes a small bug that had the reader content being spoken out when the prompt was still being processed.

This updates brings the following changes to the 'Ask Envision' feature witin Scan Text:

• To ask follow-up prompts after the first one, you must now press and hold the hinge button, instead of double tap.
• The reponses now also come with language code, meaning they will be spoken out with the right TTS. That means, when you ask Envision to translate a text to another language like German, it will speak out with a German voice.
• We have fixed the errors where it will sometime attempt to complete a question by responding with a question mark or finishing sentences.
• We have increased the wait time for speaking out the "still processing" message to 15 and 35 seconds.

What to test?

• Just use the Ask Envision feature and report on any odd behaviours or bugs.
• Try to ask it to translate any text to different languages and have it speak it out.