search engine indexing – Can I force Bing to reindex our site?

Our website is ranking quite well (first page) on Google for quite a lot of keywords.

However, our website is not on the first 10 pages of Bing for most of these keywords.

I notice our Google rankings have increased since I upgraded our website to WordPress, and submitted the new sitemap to Google. Google re-indexed our site within 24 hours of submitting a new sitemap.

I’ve submitted our sitemap to Bing, but I notice Bing’s index of our site still contains our old site’s pages, 5 days after submitting our new sitemap, even though Bing has crawled URLs in our sitemap.

Is there a way I can force Bing to re-index our site, or do I just have to be patient?

What is the normal frequency of the Bing bot revisiting a site?

What is the normal frequency of the Bing bot revisiting a site after deleting the old sitemap and submitting a new sitemap?

Thanks.

Connect SemRush with Bing webmaster

Advertising

y u no do it?

Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

Starts at just $1 per CPM or $0.10 per CPC.

Cannot index Bizdustry – Bing Console | Forum Promotion

Forgive me if I am butting in here, but… I think I know why. And this is not exclusive to Bing.

You have two things on your board. [Attached below.] You have adblockers, and the “three views and you’re locked out” plugin. I get what you’re trying to do here, but how is a spider or bot going to “slurp” your thread and index your threads? You did this across all of your forums. Even the smallest board.

Disable them and let the boards be indexed. When you have at least 1,000+ Visitors A WEEK. Go ahead and put it back on…

Ideally I’d never use both. Because Adsense and Googlebot is spidering, slurping, and indexing your site. Bing is less established, so that’s why I’m focused on Google.

Bing uses a similar technology as Yahoo, it was done for partnership purposes. That’s why it seems “broken.”

Edit: Also, I almost forgot one of google’s tenets is “how would a user interact with the site?” If I’m being honest, if I saw these prompts as a user. I’d leave the site. You’re losing users every day with those prompts.

How tell Google their instructions for changing search engine from “Bing” to “Google Search” in the EDGE browser are out of date and do not work?

The current (June 21 2021) official Google website instruction for changing search engine from Bing to Google Search within the Microsoft Edge browser are out of date and no longer valid. They do not work.

How do I tell the Google Search unit about this problem ? Google does not make it easy for non-business users to report problems. Trying to save money on staff.

P.S. MINOR ISSUE: I tried to insert a screenshot in this post of the out of date instructions but I could not figure out how to insert the screenshot. I pointed to the jpg image of the screenshot on my computer but it would not load to this post and produced confusing error messages.

apache2 – When Bing requests a sitemap or a robots.txt, why do I see Apache reject the 1st request with a 403, followed immediately by a 2nd successful request?

This is a small issue that has been ongoing for quite some time.

When I check my Apache server log for 403’s, all entries are related to Bing, and all from when it requests a robots.txt, sitemap.txt, or sitemap.xml.

The problem only happens with Bing and only when they request a sitemap or robots.txt. The problem does not happen with actual Bing crawling, and even if I do an URL inspection / live test on one of the robots.txt or sitemap files, the problem does NOT happen that way.

What I’m seeing is that Apache will reject the first request with a 403, and then there will immediately be a 2nd request that will be successful. So it’s pretty harmless except for junking up the logs.

Here are some examples of what I see in the logs:

(2021-06-20/15:11) (mydomain1).org 207.46.13.96 "GET /sitemap.txt HTTP/2.0" 403 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
(2021-06-20/15:11) (mydomain1).org 207.46.13.96 "GET /sitemap.txt HTTP/2.0" 200 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"

(2021-06-20/07:36) (mydomain2).net 207.46.13.45 "GET /robots.txt HTTP/2.0" 403 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
(2021-06-20/07:36) (mydomain2).net 207.46.13.45 "GET /robots.txt HTTP/2.0" 200 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"

(2021-06-20/15:13) (mydomain3).com 40.77.167.21 "GET /sitemap.xml HTTP/2.0" 403 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
(2021-06-20/15:13) (mydomain3).com 40.77.167.21 "GET /sitemap.xml HTTP/2.0" 200 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"

Interestingly if I route the traffic through Cloudflare the problem goes away (or maybe the initial bad request never reaches my server and I only see the 2nd good request).

Cloudflare examples:

(2021-06-20/15:11) (mydomain4).tk 2400:cb00:28:1024::6ca2:f654 "GET /sitemap.txt HTTP/1.1" 200 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"

(2021-06-20/15:12) (mydomain4).tk 2400:cb00:28:1024::6ca2:f6c8 "GET /sitemap.xml HTTP/1.1" 200 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"

I have mod_security running but it’s in DETECTION_ONLY mode and shouldn’t be blocking anything on its own, just logging.

When one of these 403’s happens, I get an entry in modsec_audit.log, because it logs all 403’s, but it’s not very helpful:

--9ea2ef5a-A--
(20/Jun/2021:15:11:29 --0500) YM@g8YvFIvJDLJuWINL3ewAAAAE 40.77.167.21 53442 (myIP) 443
--9ea2ef5a-B--
GET /sitemap.xml HTTP/2.0
Cache-Control: no-cache
Pragma: no-cache
Accept: */*
Accept-Encoding: gzip, deflate
From: bingbot(at)microsoft.com
User-Agent: Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
Host: (mydomain1).com
--9ea2ef5a-H--
Stopwatch: 1624219889292746 4001 (- - -)
Stopwatch2: 1624219889292746 4001; combined=1982, p1=1533, p2=0, p3=0, p4=0, p5=443, sr=492, sw=6, l=0, gc=0
Producer: ModSecurity for Apache/2.9.3 (http://www.modsecurity.org/); OWASP_CRS/3.2.0.
Server: Apache/2.4.41 (Ubuntu) OpenSSL/1.1.1f
Engine-Mode: "DETECTION_ONLY"
--9ea2ef5a-Z--

python – Inaccurate resulte from Bing Maps Geocode Dataflow

I am using a Python POST request to geocode the addresses of my company’s branches, but I’m getting wildly inaccurate results.

I looked at this answer, but the problem is that some results aren’t being processed. My problem is different in that all of my results are inaccurate, even ones with Confidence="High". And I do have an enterprise account.

Here’s the documentation that shows how to create a geocode Job and upload data:
https://docs.microsoft.com/en-us/bingmaps/spatial-data-services/geocode-dataflow-api/create-a-geocode-job-and-upload-data

here’s a basic version of my code to upload:

import requests    

SDS_Geocode_URL = "https://spatial.virtualearth.net/REST/v1/dataflows/geocode?"

geocode_post_vars = {
    "input": "xml",
    "output": "json",
    "content-type": "application/xml",
    "key": Bing_key,
}

geocode_Post_URL = (
    SDS_Geocode_URL
    + urllib.parse.urlencode(geocode_post_vars)
    + "&dataLocation="
    + "https://account-name.blob.core.windows.net/myDataFile.xml"
)

r = requests.post(geocode_Post_URL)

And once the data is geocoded, I use a GET request to get the geocoded XML

geocode_Results = requests.get(
    "https://spatial.virtualearth.net/REST/v1/dataflows/Geocode/"
    + jobID
    + "/output/succeeded?key=" + Bing_key
)

print(geocode_Results.text)

And I get results like this:

<GeocodeEntity Id="12040000">
    <GeocodeRequest Culture="en-US" IncludeNeighborhood="true" Query="Rush Truck Center - Los Angeles" IncludeQueryParse="true">
        <Address AddressLine="8830 Slauson Ave, Pico Rivera, CA 90660" AdminDistrict="California" CountryRegion="US" Locality="Pico Rivera" PostalCode="90660" />
    </GeocodeRequest>
    <GeocodeResponse Name="Los Angeles, CA" EntityType="PopulatedPlace" Confidence="High" MatchCodes="UpHierarchy">
        <Address AdminDistrict="CA" CountryRegion="United States" AdminDistrict2="Los Angeles County" FormattedAddress="Los Angeles, CA" Locality="Los Angeles" />
        <GeocodePoint CalculationMethod="Rooftop" Latitude="34.0522384643555" Longitude="-118.243347167969" Type="Point" UsageTypes="Display" />
        <QueryParseValue Property="Locality" Value="Los Angeles" />
        <BoundingBox SouthLatitude="33.6968193054199" WestLongitude="-118.683807373047" NorthLatitude="34.3506469726563" EastLongitude="-118.138854980469" />
        <Point Latitude="34.0522384643555" Longitude="-118.243347167969" />
    </GeocodeResponse>
    <StatusCode>Success</StatusCode>
    <TraceId>#######</TraceId>
</GeocodeEntity>

In my Geocode request, I have a specific address and company branch name (redacted for this question). Additionally, I have added city, state, country, and zipcode attributes exactly as stated in the documents.

Even with all that information, the geocoded response data returns is no more granular than general city of Los Angeles, and the Lat/Long coordinates for the city, not the address I provided (for the record, it’s ten miles away from where it should be).

Also, when I upload the XML file manually through the Bing Maps portal, it’s perfectly accurate. Am I formatting the request wrong, or is this a Bing problem?

seo – Site de-indexed by Bing, then block removed by the Bing team but rankings not returning

Last month Bing de-indexed my site (www.generatormix.com) – after a review by the Bing team the block was removed (they didn’t specify why it got de-indexed in the first place but it must have been a “false positive” in the algorithm somewhere.)

Since the block was lifted on my site on the 22nd of April there seems to be no improvement at all in my site presence in bing despite being advised by a Bing rep that it would take 2-3 weeks to recover from the block.

enter image description here

On April 22nd it seemed to be coming back with 99 impressions on that day:

enter image description here

Then the impressions went back to almost nothing and stayed that way. On May 4th for example it was 9 impressions:

enter image description here

There is only one phrase I can see my site ranking for “random movie generator” (no quotes) (pos 2) it doesn’t rank for any other significant phrases that it used to as of May 16th. Even if you type exactly “generatormix.com” into Bing (without quotes) my site doesn’t appear; just recommendations for competitors to use instead.

Is it suddenly going to come back like it used to before the block at some point in the near future, because one would think it would be coming back gradually over the course of 2-3 weeks and it has been longer than that now?

Has anyone on here has an experience similar to this and if so, what do I do now? The person from Bing was very nice and helpful but it appears they have determined the issue is resolved and haven’t replied to my last email.

Thanks, John

php – Schema data for Google & Bing bots crawling partial hydrated pages

We are using static html, combined with partial page hydration via javascript. Visitors download a full html page when they first arrive at the website, from there javascript intercepts any internal link clicks, fetching a json file with the content that changes and then hydrates the existing DOM with the new content.

However our schema data is included in the page as ld+json, so not contained within the content that gets hydrated.

I am wondering, when Googlebot and Bingbot visit a webpage such as ours, do they look at the html and simply fetch the url listed in links getting the full static version? Or do they simulate a link click in which case the js would preform the page hydration which they would sub-sequentially crawl?

I ask because we have not yet figured out how to extract the content within <script type="application/ld+json">.....</script> with PHP and then later reinsert it with javascript. If the bots simply go to the next url, then they will always get the correct schema data, but with page hydration we would need to update the schema data.

how can I parse the output of Google, Yandex bing? — 👉 GSA SEO and Marketing Forum 👈

how can I parse the output of Google, Yandex bing? If possible. Parse the links that will be issued on requests

Tagged:

  • scraping
  • parser

licensing – Microsoft Bing Image Search API License for Machine Learning Datasets

There are several online tutorials as well as fast.ai that use the Microsoft Bing Image Search API to build datasets for machine learning. However, Microsoft’s license seems to disallow such usage:

Use data received from the Search APIs as part of any machine learning or similar algorithmic activity. Do not use this data to train, evaluate, or improve new or existing services that you or third parties might offer.

Am I missing something (perhaps fewer restrictions for education / academia) or is this kind of usage really against Microsoft’s ToS?