Re: Question about robots.txt

[" kseifried@redhat.com " < kseifried@redhat.com >]

On 13/05/17 01:55 AM, jericho wrote:
>
> On Thu, 11 May 2017, Coffin, Chris wrote:
>
> : > That said, after Kurt's mail in December of 2015... in the last ~
> 30 -
> : 60 days, I noticed that MITRE finally changed that. Google is now
> : indexing and caching the CVE pages.
> :
> : We made the change to allow indexing back in Feb of 2016, which was
> a
> : few months after Kurt had pointed out the issue. We apologize to
> all for
>
> Something I cannot prove, because I don't screenshot my daily
> "missing
> CVE" searches as far as Google results go. But I would swear this is
> not
> the case. We'll have to agree you have your official statement, and I
> have
> my 'anecdotal' evidence as someone who searches on new/missing CVE
> IDs
> every day.
>

Nope, archive.org has their robots.txt going back to 2001 with pretty
much daily records:

https://web.archive.org/web/*/cve.mitre.org/robots.txt

> : not replying to the original thread at that time. Dan also
> mentioned the
> : same in a response to you back in April of this year
> :
> ().
>
> You see, I don't have to read that. Kurt mailed in Dec 2015, you say
> Dan
> replied to me in April 2017. Use your numbers. My point stands about
> MITRE's promise of following up.

You can check archive.org to see the exact day it changed.



--

Kurt Seifried -- Red Hat -- Product Security -- Cloud
PGP A90B F995 7350 148F 66BF 7554 160D 4553 5E26 7993
Red Hat Product Security contact: secalert@redhat.com