A couple of months ago I needed to set the zoom level of a Google Map so that all markers in the map are visible. After hours of reading the API documentation, I did not find any solution on that issue, so I wrote a little Javascript function that does exactly that.
Kategorie-Archive:Web
Recrawl script for nutch
Note: The information contained in this post may be outdated! Here’s a small shell script for doing the recrawl process in nutch. You might have to change certain lines because I did some customizations, but it should work for you too 🙂
Blocking web crawlers on lighttpd
Note: The information contained in this post may be outdated! Nutch did ignore my robots.txt (for whatever reason, I was unable to figure out why), so I had to find another way to forbid those directories for the crawler. I finally came up with this neat piece of config for lighty:
|
1 2 3 4 5 |
$HTTP["useragent"] =~ "(Nutch|Google|FooBar)" { $HTTP["url"] =~ "^(/one/|/two/|/three/)" { url.access-deny = ( "" ) } } |
– throws an […]
Nutch – meta description in search results
In order to get our meta descriptions displayed in the results we need to write a plugin that extends 2 different extension points.
Nutch – prevent sections of a website from being indexed
I thought an ideal solution would be telling nutch to ignore specific sections. A good and common practice doing this kind of stuff is creating HTML comment tags.