Meta unveils a huge breakthrough for Artificial Intelligence

 Meta AI labs unveil ImageBind, a huge advance for AI. It allows artificial intelligence to simultaneously understand multiple sources: audio, image, text and even heat.

Generative artificial intelligence can quickly create text, with ChatGPT, or images, with Midjourney. It is sometimes possible, especially since GPT-4, to provide it with an image to understand, but the process of the AI will pass by a creation of text from this image to be able to make treatment. So we always come back to text as a method to communicate with the AI.

With ImageBind, Meta unveils a new method that could revolutionize artificial intelligence in its current form. The company wants to go much further and foresees a method that allows the AI to interpret up to 5 completely different sources simultaneously.

Approaching the human

As Meta reminds us in its announcement, humans are used to using multiple senses to create information, for example, seeing a traffic jammed street and at the same time hearing the noise of car engines.

Meta was inspired by this idea to develop ImageBind, a new artificial intelligence model that the firm wants to make open source. It is the first model capable of combining information from 6 different types of sources: text, image, audio, depth (3D), thermal (via infrared) and velocity.

With all this, Meta announces that its AI is able to understand from a photo what objects will sound like, their shapes in 3D and whether they will be hot or cold to the touch. For another application, Meta says that an AI based on its model can now imagine images from a sound track, for example a forest from the sound of rain on the trees.

The first tests conducted by Meta show that ImageBind is much more efficient than models specialized on a single modality as source.

The researchers now want to go further and integrate other possible sources for their model, such as smell and touch, in order to get even closer to human capabilities.