Microsoft is gearing up for the next phase of Bing’s expansion, and artificial intelligence (AI) will be a central component.
In a preview event held in New York City, Microsoft executives, including Yusuf Mehdi, the company’s Chief Marketing Officer, and Corporate Vice President, showcased the range of features set to be added to Bing over the next few months.
Bing will become more visual, offering image and graphic-centric responses in Bing Chat. The search engine will also be more personalized, allowing users to export their Bing Chat history and integrate content from third-party plugins.
In addition, Bing Chat will be able to answer questions in the context of images, marking a shift towards multimodality. Mehdi believes this marks the start of a new generation of the “search mission.”
The new version of Bing is now available without a waitlist. Users can try it out by signing in with their Microsoft account.
At a preview event, a spokesperson demonstrated Bing Chat’s new feature by asking the chatbot if the saguaro cactus grows flowers, and it responded with a paragraph-long answer and a corresponding image of the cactus.
It is unclear which types of content will trigger image results, but Microsoft has put in place filtering mechanisms to ensure that explicit images do not appear.
Microsoft’s Head of Responsible AI, Sarah Bird, explained that Bing Chat uses toxicity classifiers and blocklists to keep conversations clean. However, when first launched in February, these measures did not prevent the chatbot from spreading misinformation and hate speech. Reports revealed instances of the chatbot spouting vaccine misinformation, making threats, and even shaming users for admonishing it.
Despite these early hiccups, Microsoft is confident that Bing Chat will provide its users with a safe and informative experience. By leveraging AI models to detect harmful prompts and offensive language, the chatbot is better equipped to deliver accurate and helpful responses while keeping the conversation respectful and appropriate.
Microsoft aims to transform search with the next generation of Bing. The new Bing is designed to be more visual, with image-centric answers and more personalized features. Bing Chat users will have the option to export their chat histories and incorporate third-party plugins. The updated Bing Chat will also be equipped to answer queries related to images in their proper context.
Bing Chat will soon respond to questions with relevant images, much like the standard Bing search flow, but condensed into a card-like interface. The chatbot will benefit from filtering and moderation already in place with Bing search, with a team of human moderators in place to watch for abuse.
Productivity Enhancer
In addition, Bing Chat will offer citations to identify the source of the information when summarizing or discussing the contents of a document. Moreover, Bing Chat will provide search results visually appealingly and display them in the chart format preferred by the user.
In February 2021, Bing Chat rolled out in preview, but it did not prevent the chatbot from going off the rails, spouting misinformation, and writing a hateful screed. However, meaningful progress has been made, with a team of human moderators in place to watch for abuse.
Bing appears to be moving closer to a complete productivity platform, especially with the forthcoming enhanced text-to-image capabilities.
Bing Image Creator, Microsoft’s tool that generates images from text prompts using DALL-E 2, will soon support over 100 languages besides English, allowing users to refine their generated images with follow-up prompts.
When asked to Sarah Bird whether Microsoft is looking for ways to compensate creators whose work has been included in training data, despite the company’s official position that it is a fair use matter. Many generative AI tool platforms, such as Shutterstock, have established creators’ funds, while others, such as Spawning, have developed mechanisms for artists to opt out of AI model training.
Bird suggested that these issues would eventually need to be addressed and that content creators deserve compensation, but she was not willing to commit to anything concrete at this time.
Bing Chat is expanding its capabilities to include image recognition, allowing users to upload images and search for related content. For example, a user can upload a picture of a crocheted octopus and ask Bing Chat, “How do I make that?” to receive step-by-step instructions. The Edge app for mobile also features a new page context function that allows users to ask questions in Bing Chat related to the page they are viewing.
Multimodal Abilities
The new multimodal abilities in Bing Chat and the Edge app for mobile are likely powered by GPT-4, which can understand both images and text. Although OpenAI has not made the model’s image understanding capabilities widely available, Microsoft, being a major investor and collaborator with OpenAI, may have privileged access.
To prevent the abuse of image upload tools, Microsoft is using automated filtering and hashing to block illicit uploads. However, the effectiveness of these measures is unclear, as the testing of image uploads was not made available to reporters.
New Chat Features
One of these features is the ability for users to save their chat histories, allowing them to resume conversations or review past chats easily. This feature is similar to what OpenAI recently introduced in ChatGPT, which shows a list of chats and the bot’s responses.
The details of the chat history feature are still being worked out, such as how long chats will be stored. However, Microsoft has confirmed that users will be able to delete their history at any time, addressing concerns raised by some European Union governments about ChatGPT.
Bing Chat will also allow users to export and share conversations, including graphs and images created within the chat. Microsoft is working on a more robust copy-and-paste system for these features, although it is not yet available for preview.
The most exciting addition to Bing Chat is the introduction of plugins. Partners like OpenTable and Wolfram Alpha will offer plugins that greatly expand the capabilities of Bing Chat. These plugins can be toggled on or off from the Bing Chat web interface. They will allow users to do things like book reservations or get answers to complex scientific and mathematical questions.
Although the plugins feature is still in the early stages of development and there is no plugins marketplace yet, Dena Saunders from Microsoft’s web experiences team hinted that the Bing Chat plugins scheme might be associated with or identical to OpenAI’s recently-introduced plugins for ChatGPT.
Bing Chat With Updated Edge
Bing Chat can be accessed via the Edge browser and the web. The Edge browser has recently undergone an overhaul, including rounded corners to align with the Windows 11 design philosophy. Additionally, the browser elements are more containerized, with subtle tweaks throughout, such as the Microsoft Account image moving left-of-center.
In Compose, Edge’s Bing Chat-powered tool, users can write emails and other messages using a basic prompt. Users can now customize the generated text by adjusting its length, phrasing, and tone using a new feature. Bing Chat filters prevent problematic tones like “hateful” or “racist.”
Actions in Edge are even more exciting than in Compose. These features translate specific Bing Chat prompts into automation that carries out specific tasks.
For example, “bring my passwords from another browser” in Bing Chat opens Edge’s browsing data settings page, while “play ‘The Devil Wears Prada’” pulls up a list of streaming options like Vudu and the Microsoft Store. Actions also include an automatic tab organizer that color-coordinates browsing tabs.
Actions are still in the early stages of development, but it’s clear that Microsoft is headed in a specific direction with these features. Actions could extend beyond Edge to other Microsoft products like Office 365 or even the entire Windows desktop.
While Dena Saunders, a partner GM in Microsoft’s web experiences team, wouldn’t confirm or deny this, she did suggest that more information may be revealed at the upcoming Microsoft Build developer conference.