Re: What Would Cause ISA to Block GoogleBots?

Here's an update and what I'm seeing in the log files. It probably sheds
more light then I could ever hope to :-)

This looks to me like Google coming on my site and are consecutive log
entries;

Google comes on with a Client IP of 66.249.66.204 and tries to reach
Destination IP of 70.66.128.238 (my published webserver) on Port 80 which
ISA denies.

Next, the log shows that Client IP 70.66.128.238 (my webserver) then tries
to connect to Google at 66.249.66.204 on Port 55334 and ISA denies it.

On some DNS requests the log file looks like this;

Client IP 192.168.1.50 (my webserver's internal IP) tries to get to a
Destination IP of 192.175.48.6 (which looks like a ROOT DNS Server) on Port
53 and ISA Denies the request.

I don't know a lot about this stuff but my spidey sense tells me this might
be the best indicator of the problem. For instance, on the DNS log entries
shouldn't my Client IP be my external IP address?

As far as the back & forth between Google and me - I have no idea why that
would happen.

Arghhhh...... :-)
Marvin :-)






"Marvin Miller" <sales@xxxxxxxx> wrote in message
news:%23h0R%239VQGHA.3728@xxxxxxxxxxxxxxxxxxxxxxx
Hi Virgil :-)

The good news is that the network has been performing flawlessly from the
inside. There's nothing like having perfect DNS and solving that cased a
number of small issues that when combined used to drive me nuts. From the
'inside' of the network everything appears perfect and is MUCH faster.
It's
properly laid out now and that's a key issue for me.

What I'm seeing though is about a 75% drop in traffic and my site is
always
consistent. In keeping with the lower then normal figures, I'm seeing some
complaints about the site and I'm thinking there's an issue somewhere.
Here's what was reported;

"Had some troubles this morning. Started with a page cannot be displayed,
hit refresh, got a message saying the page is not configured for a web
address. I hit refresh again and it loaded. When I tried getting into our
internal forums it repeated the whole process. All seems fine now though
and
I'm able to move from section to section fine."

"It took two attemps to post the previous response. Page cannot be
displayed
the first time I hit submit. I hit the back button and then submit and it
posted fine the second time.

Then I wanted to edit the post to add this info and each time I hit submit
I
got "No web site is configured at this address".

I'm also finding it takes 5 or 6 attempts to get any threads to open
correctly. I keep getting the No web site is configured at this address
message."

"Still having the same problem whenever I try to go on the site and go
into
any thread. Refresh the 2 or 3 times and then it loads.

"I haven't tried logging off and back on. I'm one of those people that
never
logs off the site when I leave so that when I return I'm automatically
logged on."

I made sure compression was turned off and also I believe I turned caching
off totally. It almost seems to me that ISA seems confused and is blocking
too much - I know, not very scientific :-)

The single biggest indicator though seems to be Google and their
Googlebots.
Because of their volume of traffic to the site it's more clear that there
are issues with people being denied access to the published site. Why it's
happening I'm not sure.

I guess I still need help :-(
Is it possible to un-install ISA and then re-install, apply service pack 1
and then restore the rules/settings and skip SP#2? I think we've certainly
got the network to the point where we _know_ ISA is at fault or that the
issue is in ISA.

Best & Thanks :-)
Marvin





"ZVR" <no_spam_ever@xxxxxx> wrote in message
news:440c456c$0$5658$9a6e19ea@xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Your site IS getting indexed, don't worry. Most likely ISA is rejecting
requests targeted directly to port 80 of your IP address instead of
requested via the proper host header (www.yoursite.com). When you create
a
web publishing rule in ISA you specify the host header for which
requests
will be accepted, and requests to your IP which don't include that
header
are rejected. This is by design.

Virgil



"Marvin Miller" <sales@xxxxxxxx> wrote in message
news:O2qI9IOQGHA.2992@xxxxxxxxxxxxxxxxxxxxxxx

"Marvin Miller" <sales@xxxxxxxx> wrote in message
news:OupboAOQGHA.3872@xxxxxxxxxxxxxxxxxxxxxxx
Hi Folks;

I'm running ISA 2004 and created a web publishing rule for my web
server.
All is fine and the site works well but when I click on the
Monitoring
Tab
and then set a filter for;

Action = Denied Connection

I end up seeing quite a few port 80 requests that are discarded.
Checking
the IP address for some of these discarded requests shows that many
of
them
are from Google trying to index my website.
I guess I don't have to tell you how important it is for Google to be
able
to index my site :-) I do see the GoogleBot on the site at times but
nevertheless there's a lot of attempts that seem to get denied by
ISA.

Does anyone know why?

Thanks;
Marvin



Just an update on this - I created filters to show that IP's activity
in
the
logs and it is making it through - it's just that some attempts get
denied
by ISA. What would cause that?

Thanks ~ Marvin








.