What do B2B marketers need to know about New Bing?

By John Woods | May 3, 2023

Generative AI and the future of search marketing

We’ve been hearing a lot recently about whether the new wave of AI – especially “generative AI” techniques like ChatGPT – will change the world.

I think there’s little doubt that these will be world-changing technologies. But that’s not a particularly helpful observation! If someone had said “electricity will change the world!” back in 1883, for sure they would have been right, but they weren’t being especially useful. It’s much more valuable to look at the emerging applications of a new wave of technology. Electric lights – just around the corner in 1883. Electric cars – took nearly 150 years to become mainstream.

So to try to offer B2B marketers something useful in the generative AI debate I’ve narrowed my focus and have been evaluating one particular application. I’m looking at “New Bing”, Microsoft’s ChatGPT-powered search engine, from the perspective of B2B search marketing.

New Bing – is it relevant to B2B?

Like most internet technologies, New Bing has been developed primarily with mass market B2C applications in mind. That makes sense – those mass B2C audiences are the easiest places to make money.

So we shouldn’t assume New Bing has automatic relevance for B2B marketers. Is New Bing going to turn the world of B2B search marketing on its head, so that we B2B marketers have to throw away everything we’ve learned about SEO and PPC and start again?

That’s a complex question, but to my mind there’s a key starting point: is anyone actually going to use New Bing for B2B research? In other words, will users change their behaviour and switch to New Bing instead of conventional search engines for their B2B buying journeys?

Changing user behaviour is hard. We’ve all built up habits and expertise about how to use conventional search engines as “magic question-answering machines” and we’re committed to those habits because we know the search engines meet our question-answering needs. User experience experts will tell you that people won’t change their behaviour away from existing successful habits unless there’s a substantial reason to do so. So a switch in user behaviour is only going to happen if New Bing offers a significantly better user experience.

So this is the question I’ve set myself to answer: does New Bing offer a sufficiently compelling user experience for B2B research that we might expect people to switch away from conventional search engines?

How I evaluated New Bing

I wanted to make sure my evaluation was grounded in the reality of B2B research. So before I started, I wrote down a list of real-world B2B research questions. I tried out New Bing with each question and evaluated the results on three criteria:

Ease of use – how easy, or difficult, was it to get a meaningful output from New Bing – if it’s too difficult to use, it doesn’t matter how great the results are, no one is going to switch.
Value of the results – did New Bing answer my question, and was it better or worse than a conventional search engine for the same question? Again, no one is going to switch away from established search habits unless there’s something of at least equal value to be had.
The “delight factor” – did New Bing give me something surprising or remarkable – not necessarily directly related to my question – that might bring me back for more? (Even if the ease of use or value don’t quite measure up.)

One detail here – when looking at the relative value of the result, I compared New Bing with results from “old Bing”. I stuck to Bing because I didn’t want my evaluation to be distracted by differences between Bing and Google. And I used an equivalent query in the conventional search engine – in fact, I used the query that New Bing generated itself:

Here are the questions I used (exactly as I worded them for New Bing) and the equivalent search wording for a conventional search engine:

New Bing prompt

How can PV generation be integrated with industrial unit roofing?

How has COVID affected how people use office space?

What are the pros and cons of Edge Data Centres?

How should I split my budget between Google Search and LinkedIn Ads?

What’s the best coworking space in Oxford?

How do I choose a data centre supplier?

What’s the most comprehensive source of construction industry leads in the UK?

What are the leading industrial laundries in the south of England?

Is Sharp Ahead a good b2b marketing agency?

What are the best alternatives to WP Engine?

Is YPO a good value supplier of school equipment?

Is the Curious Lounge a high quality coworking space?

Equivalent web search (from New Bing)

PV generation industrial unit roofing integration

COVID office space usage

What are the pros and cons of Edge Data Centres?

How should I split my budget between Google Search and LinkedIn Ads?

Best coworking space Oxford

How to choose data center supplier

Construction industry leads UK

Industrial laundries south England

Is Sharp Ahead a good b2b marketing agency?

Best alternatives to WP Engine

YPO school equipment value supplier

Curious Lounge coworking space quality

The results

First of all here’s how New Bing scored on my ease-of-use criterion across the 12 questions:

Ease of use: % of questions

Poor – couldn’t obtain anything useful: 8%
OK – got a useful result without too much effort: 42%
Good – easy to get a useful result: 50%

Pretty good all told, especially considering this was my first serious attempt to use New Bing. New Bing mostly showed a good understanding of my queries and generated relevant output. Now my value criterion:

Value of results: % of questions

No value or badly misleading: 0%
Somewhat valuable but less so than conventional web search: 58%
Equivalent value to conventional web search: 25%
More valuable than conventional web search: 17%
Much more valuable than conventional web search: 0%

I have to admit this was a pleasant surprise. There were no cases across my 12 questions where New Bing completely missed the mark or came up with badly misleading answers. And there were a lot of cases where the New Bing results were of similar value, or greater value, than conventional search.

Finally the delight factor. Fully 75% of my queries included something that “delighted” me at least a little! A really pleasant surprise, and probably the thing that is most likely to bring me back to New Bing again.

In this example, New Bing found – and accurately explained – the highly specific industry jargon term “BIPV” which wasn’t present in my original query.

Here the follow-up questions that New Bing suggests are extremely relevant, pulling out price and opening hours which are very likely to be important factors in a decision-making process about a coworking space. This would save a lot of time in a real B2B research task.

Some details

A few detailed observations, in no particular order:

New Bing is opinionated, but can admit its limitations: New Bing doesn’t shy away from holding an opinion. So for example when I asked it “Is X a good value supplier?” it said yes. But it often caveats its opinions, or admits when it doesn’t know.

I liked that humility and found it made me more trusting of the remaining results.

New Bing’s shortlists are, erm, short: when I ask New Bing for a list, it comes up with a few alternatives – perhaps two, or three, or a handful. This is potentially a good thing – who really wants to wade through a list of 50 possible suppliers? – but is going to be a massive challenge for marketers if this becomes a standard feature of future search engines. In a conventional search engine, being position 5 or 10 or even 20 for a given search will still lead to some degree of success. With New Bing, if you’re not in the top few, you are literally nowhere.

Conversational approach adds value (sometimes): New Bing’s “chat”-style interface sometimes adds some value by suggesting follow-up questions that are more pertinent than the “people who asked that also asked this…” functionality of conventional search. For example in response to a question about coworking spaces in Oxford, New Bing offered to list comparative prices – really useful and highly relevant to a B2B researcher. (Admittedly with some other questions the chat options were less relevant, or irrelevant.)

Conversational memory is sometimes limited: sometimes New Bing seems a bit forgetful. For example, I asked New Bing to list coworking spaces in Oxford. It gave me a plausible shortlist and offered to list some more. I carried on the conversation and asked for the longer list, which turned out to include some repeats of the earlier shortlist. This didn’t undermine the value of the results, but it was a bit disappointing.

New Bing’s results reflect old Bing rankings: New Bing makes a lot of changes to the Bing search results (content, ranking and presentation) but it does clearly rely on them for its raw material. So if you’ve made a huge effort on SEO for Google but ignored Bing, you might find your content doesn’t feature on New Bing. If New Bing gains a lot of market share then we’re going to need to broaden our SEO focus.

Geographic nuances are lacking: New Bing doesn’t have as much understanding of geographic cues as I’d hoped for. For example, I asked for “alternatives to X” where X is a UK-based supplier of a geographically-based market. New Bing listed US-based alternatives.

There are ads (at the moment): Good news perhaps for PPC – the New Bing results do include some search ads. I can’t be sure but these seem to be normal Microsoft Ads listings triggered against keywords. It’s not surprising that Microsoft is exploring ways to monetize New Bing.

Reflections – is New Bing relevant for B2B search?

I came to this evaluation with an open mind, and if I’m honest with a degree of scepticism about the relevance of New Bing and ChatGPT for something as nuanced as B2B research.

Based on my evaluation, I’d definitely use New Bing as it stands today as an adjunct to a conventional search engine. I’d expect it to be particularly useful for certain types of question and not useful for others. So I’m not about to make New Bing my ONLY B2B search engine, but I’ll certainly use it for some types of research.

And it’s clear that New Bing (and its competitors like Google Bard) are only going to get better at these B2B research use cases. There are already some hints of really valuable user experiences where New Bing did “delight” me and exceed my expectations. They are incomplete and unreliable, but I’m certain they will improve rapidly as the technology matures. Here are a couple of examples of the sort of thing I mean:

New Bing has assembled a useful summary of pricing there. It’s incomplete, and not as directly comparable as I’d like, but it’s already a time-saver versus a conventional search engine.

In that search, New Bing has found a really plausible list of alternative suppliers and has made an attempt to build out a table of pros and cons of each. Unfortunately a lot of the table has no information in it! But I could imagine a future iteration of the technology could fill that out.

So I don’t think that New Bing as it stands today is going to replace conventional search engines, but I think it’s already good enough to deserve a place in the toolkit that we use for B2B research. And I’m certain that in the future, the evolution of New Bing and other generative-AI-enabled search engines WILL replace conventional search. That could happen in a lot of different ways – we have an arms race between Microsoft and Google, and potentially other players. We might see a new player emerge. Or we might see an incremental evolution of conventional search engines so that generative AI gradually becomes more prominent in search – that sort of thing has happened before with other evolutions of search engines. But one way or another, there’s going to be a big change in B2B search engines and B2B search marketers are going to have to take note.

What should B2B search marketers do today about New Bing?

There are two simple things that all B2B marketing teams can do right away to gain some experience with New Bing and to prepare to manage the impact of this wave of change.

First, get some experience with New Bing yourself. It’s now open to all (having previously been accessed only via a waitlist). Just go to https://www.bing.com/new to sign up. Once you have access, try some questions that are relevant for buyer research in your industry, such as:

Top of funnel research like “what are the best solutions for [some problem that your product addresses]?”
Opinions about your brand such as “who are the competitors for [your company]?” and “is [your product] good value for money?”

Secondly, make sure you have Bing Webmaster Tools set up. This is the Microsoft equivalent to Google Search Console and it’s the method that Microsoft will use to give you feedback on how your content appears in Bing search. If New Bing causes Bing to gain market share over Google, Bing Webmaster Tools will become more important. So get it in place now so that you can refer back to the data in future. Visit https://www.bing.com/webmasters/ about for details.

What’s next?

This wave of AI technology is certainly going to change the world of B2B search marketing, and that will have important implications for B2B search marketers. But exactly what will happen when is highly uncertain. Right now, most people are still using conventional search engines for their B2B research. So there’s no urgent need to change. We are all going to need to watch, wait and be prepared to adjust our strategies and tactics when the timing is right.

Given the significance of this technology change you can be sure we’re going to keep a close eye on developments with generative AI-powered search in the future. So sign up for our email newsletter and keep an eye on our blog to stay in the loop!