Comments of N2H2, Inc.

In the matter of

Request for Comment on the Effectiveness of Internet Protection Measures and Safety Policies,

Docket No. 020514121-2121-01

Before the

National Telecommunications and Information Administration

David Burt, N2H2
Public Relations Manager
900 4th Avenue
Suite 3600
Seattle, WA 98164
206 892-1130

Table of Contents 2

Preliminary Statement 3

Evaluation of Available Technology Protection
Measures, Question 4: "Please explain how the
technology protection products block or filters
prohibited content."
N2H2 Comments 4

Evaluation of Available Technology Protection
Measures, Question 5: "Are there obstacles to
or difficulties in obtaining lists of blocked or
filtered sites or the specific criteria used?"
N2H2 Comments 6

Evaluation of Available Technology Protection
Measures, Question 6: "Do technology
companies readily add or delete specific web
sites from their blocked lists upon request?"
N2H2 Comments 9

Fostering the Development of Technology
Measures, Question 1: "Are current blocking
and filtering methods effectively protecting
children or limiting their access to prohibited
Internet activity?"
N2H2 Comments 10

Fostering the Development of Technology
Measures, Question 4: "Can currently available
filtering or blocking technology adjust to
accommodate all age groups?"
N2H2 Comments 22

List of Attachments 23

Footnotes 25
Preliminary Statement

These comments are made on behalf of N2H2, inc. They respond to the Request for Comment on the Effectiveness of Internet Protection Measures and Safety Policies, by the NTIA, Docket No. 020514121-2121-01.

N2H2, Inc. is a global Internet content filtering company whose software helps customers control, manage and understand their Internet use by filtering content, monitoring access and delivering concise user activity reports. N2H2's Bess and Sentian product lines are used by millions in businesses, schools, and libraries around the world. N2H2 is the leading provider of Internet filtering in the K-12 market. N2H2's Bess for Schools product provides filtering to over 25,000 U.S. schools.

As N2H2 is the market leader in educational filtering, we are uniquely qualified to address several of the questions raised by the NTIA. We are pleased to be able to provide responses to questions raised by the NTIA about how our filtering products work, how well they work, our responsiveness to user requests, and the customizability of our products. We will also provide more general information about filter effectiveness.

The remaining questions relating to the effectiveness of individual school policies and procedures are best answered by the schools themselves.

Evaluation of Available Technology Protection Measures, Question 4: "Please explain how the technology protection products block or filter prohibited content."

N2H2 does not rely on "word blocking" (the blocking of individual sites based on words in the text), image recognition, on-the-fly artificial intelligence, or ratings schemes such as RSACi or PICS. Instead, N2H2's entire of line Sentian and Bess filtering products rely on the same confidential and proprietary database of over 4,000,000 individual URLs that has been carefully developed by N2H2. This database is divided into 42 separate categories, any or all of which may be implemented by the end user. This database is continually updated, and a new version of the database is created daily.

The N2H2 database is populated using a four step process: 1) URLs on the Internet are flagged as potentially fitting one or more N2H2 categories; 2) URLs which have been flagged are matched against N2H2's existing database and prioritized for review; 3) URLs are reviewed by N2H2's review team and placed into one or more of 42 categories; 4) URLs in the existing database are continually reexamined.

1) URLs on the Internet are flagged as potentially fitting one or more N2H2 categories.

N2H2 and other filtering vendors have developed a number of techniques for identifying web sites to add to our lists. The most common technique is the use of “robots”: automated programs the search the web for web sites that contain certain words and phrases included in domain names, meta tags, or page text. N2H2 searches the web for candidate URLs. N2H2 also makes use of content already indexed in the various search engines to identify candidate URLs using “search parasites.”

N2H2 also makes use of a technique called “spidering”, where a “robot” program retrieves URLs linked to pornography sites, particularly “pornography search engines” such as Persian Kitty and Naughty.com. Another technique N2H2 uses is performing “whois” searches of domain name registries for new domain name registrations that contain words commonly association with pornography sites such as “xxx” or “adult”. Additionally, N2H2 monitors Usenet newsgroups and e-mail lists devoted to announcing new pornography sites.

2) URLs that have been flagged are matched against N2H2's existing database and prioritized for review.

This initial “catch” of candidate URLs is then matched against our existing database, and subjected to more complex proprietary AI algorithms. These automated processes continuously feed a list of sites to N2H2’s review department. A small percentage of websites are automatically added to the pornography category.

3) URLs are reviewed by N2H2's review team and placed into one or more of 42 categories.

N2H2 maintains a staff of professional reviewers who categorize our URLs. Each day, an average of 15,000 URLs are added to the database in one of 42 categories. Categorization is based on the content contained in the site, such as pornography, gambling, e-commerce, sports, or nudity. For a complete list of categories and the criteria used, visit N2H2 on the Web: http://www.n2h2.com/products/categories.php

Organizations can turn any of the 42 N2H2 categories on and off as they see fit, customizing filtering to match their unique Internet Use Policy. Because each organization is different, the ability to customize filtering is a powerful tool in managing Internet use.

4) URLs in the existing database are continually reexamined.

Recognizing that web content changes, N2H2 continually uses artificial intelligence technology to reexamine URLs in the N2H2 database, removing URLs that have expired, and resubmitting for review sites where the content may have changed significantly. Additionally, N2H2's users regularly submit sites for reconsideration to our database.

Evaluation of Available Technology Protection Measures, Question 5: "Are there obstacles to or difficulties in obtaining lists of blocked or filtered sites or the specific criteria used?"

N2H2 does not believe there are significant obstacles that impair the public or researchers from learning about or evaluating the content, comprehensiveness, or quality of our filtering database. To meet these needs, N2H2 provides extensive information to the public about our database, the criteria used for classifying content, and the contents of our database. N2H2 publishes its categories, detailed descriptions of the criteria used to populate categories, and an online tool where users may see how every URL is categorized.

N2h2's Categories and Criteria

N2H2 publishes the details, descriptions, and criteria of our 42 content categories on our website. This list is available at http://www.n2h2.com/products/categories.php, and is also supplied as Attachment #1.

Each category is accompanied by a description of the criteria used. As an example, here is the N2H2 "Drugs" category:

Drugs

Sites that promote or advocate recreational drug use. This category is not limited to controlled substances. Sites that promote or advocate recreational use of prescription drugs are also included. The Drugs category includes sites that contain information about topics such as growing, buying, or selling marijuana, glass pipes, or bongs; mixing a legal substance with alcohol, running methamphetamine labs, or inhaling various forms of fumes.

N2H2 has always prided itself on keeping our content classification criteria open, objective, and viewpoint-neutral. N2H2 openly solicits suggestions from our users and the public at large on ways to improve and refine our category definitions.

N2h2's URL Checker

On the N2H2 website, N2H2 provides a resource called the URL Checker. The URL Checker is a web-based tool that allows anyone with Internet access to view how any URL in our database has been categorized, providing a high level of user transparency.

In the example shown below, a user has entered the URL www.playboy.com, and the URL Checker has informed the user that www.playboy.com has been categorized by N2H2 as "Adults Only" and "Pornography." The user is given the opportunity to request a recategorization of the URL. Screenshots of the URL Checker are provided as attachments 2,3, and 4:

N2H2 believes there are no significant barriers to filter research

While N2H2 makes available a tool that allows researchers and the public at large to determine the categorization of every site in N2H2's database, N2H2 does not publish its complete confidential and proprietary list of over four million URLs in one place in its entirety. N2H2's reasons for this are three fold: the proprietary nature of our database and source code; the value that such a carefully created database represents; and the potential harm to children in publishing our database.

As is implied in the process of database creation described on pages 4 and 5, N2H2 has spent seven years and millions of dollars developing its database. Publishing this list in its entirety would seriously diminish the value of our database.

Successful evaluations of filters by professional testing facilities and researchers have been conducted since 1995 without access to entire filter databases. Nowhere in the text of the 26 independent laboratory tests of filters supplied in our attachments is the non-publication of entire filter databases cited as an impediment to filter research by the professional researchers.

On the contrary, the landmark 2002 National Research Council report on technology protection measures includes an extensive discussion on the merits of the two main methodologies for filter evaluation, and pointedly does not state that encrypted blacklists are a barrier to research:

A controversy over methodology was the subject of testimony to the committee. One approach is that the number of appropriate pages should be estimated on the basis of a random sampling of Web pages. A second approach is that the number should be estimated on the basis of actual usage, which weights certain popular Web pages more heavily than those not accessed as frequently.[1]

As the NRC report found, there are two basic methods used by researchers to evaluate filters. The first and most common method is to select a sample of web pages and test them against the filter. The second method is "based on actual use", usually by examining the Internet "log files" of actual Internet use. As discussed in the section on pages 10-21 addressing question 1, these methods are fully adequate to address the needs of researchers.

Evaluation of Available Technology Protection Measures, Question 6: "Do technology companies readily add or delete specific web sites from their blocked lists upon request?"

N2h2, through its URL Checker discussed in the previous section, actively solicits user feedback to improve the quality of our database. N2H2 considers user feedback a key source for the continued improvement and refinement of our database.

N2H2 makes every effort to promptly respond to requests of users to either add sites that should be categorized, and to delete sites that are improperly categorized from our database.

These requests are significant. Between January 1, 2002 and August 15, 2002 N2H2 received over 60,000 requests to either add or delete sites from our database. All of these requests were reviewed within 2 days of submission. Approximately 20% of these requests resulted in an addition, deletion, or change to our database.

Fostering the Development of Technology Measures, Question 1: "Are current blocking and filtering methods effectively protecting children or limiting their access to prohibited Internet activity?"

The topic of the effectiveness of Internet filtering software is often a controversial one. Fortunately, a large body of independent research conducted between 1995 and 2001 has addressed this topic.

Laboratory Tests of Filtering Software Effectiveness

Between 1995 and 2001, 26 independent laboratory test were conducted by ten professional software testing laboratories: ZD Net Labs, Consumer Reports Labs, Camden Associates, IW Labs, eWeek Labs, the PC World Test Center, the Info World Test Center, MacWorld Labs, Network World Test Alliance, and Real-World Labs. The 26 tests included 108 individual product tests, and the results were published in various technology and consumer publications.

Most of the 26 articles easily fit into one of three categories, "found filters effective", "found filters of mixed effectiveness" and "found filters ineffective." An overall finding of the test results was usually readily determinable by statements in the introductory or concluding paragraph. In the few cases where an overall finding was not readily apparent, an overall finding was determined by evaluating each comment about effectiveness, and these "borderline" articles were mostly placed in the "found filters of mixed effectiveness" category.

A total of 19 tests contained statements like "all of these products provide solid blocking capabilities," and "All the products lived up to filtering expectations, staying out of the way except when necessary to block access," and were placed into the "found filters effective" category.

A total of four tests where the overall verdict was clearly mixed, such as "While each of the products is sold for the explicit purpose of blocking objectionable material, only three are able to do that with reasonable certainty", or came to no conclusion and offered mixed evidence of effectiveness were placed in the "found filters of mixed effectiveness" category.

A total of three articles found filters overall to be ineffective. These articles contained summary comments such as "Most of the products we tested failed to block one objectionable site in five."

Tests Finding Filters Effective

PC Magazine Tests

PC magazine is probably the best known, and among the most highly regarded sources of software testing. Since 1982, PC Magazine has published thousands of software tests. PC Magazine's test laboratory, ZDNet Labs, is described as performing "Comprehensive performance and functionality testing. Our objective, precise, and repeatable testing methods--utilizing benchmarks accepted by the industry."[2]

PC Magazine has conducted more formal testing of filters than any other publication. The testing laboratories employed by PC Magazine conducted eight rounds of testing multiple filters, for a total of 47 product tests from 1995 to 2001. The first test conducted in 1995 gave filters a mixed review (see section "Tests finding filters of mixed effectiveness"), but the next seven rounds of testing were largely positive.

The second PC Magazine test of filtering software effectiveness was conducted in April of 1997. Seven filters for the home market -- Cyber Patrol, CyberSitter, CyberSnoop, Net Nanny, Rated PG, SurfWatch, and X-Stop were examined. ZDNet Labs " tested how well each product filters words and sites," and found that "all of the products performed well in their areas," concluding that "these products can be a valuable tool in the process of parental monitoring of a child's computer activity."[3].

One month later, in May 1997, PC Magazine tested five filters designed for the workplace, and found that "LittleBrother, SmartFilter, and SurfWatch all provide solid blocking capabilities, and ON Guard's real strength is monitoring; WebSense is the only product that provides full functionality in both areas " [4]

In March of 1998, PC Magazine for a fourth time had ZDNet Labs test filtering software blocking effectiveness. Ten products were tested: Cyber Patrol, Cyber Sentinel, Cyber Snoop, Cyber Sitter 97, Net Nanny, SurfWatch,Time's Up!, WatchDog, WebChaperone, and X-Stop. PC Magazine provided a summary:

Our tests involved trying to access extensive lists of URLs, words, and phrases while using each of the products. We tried to access well-known pornography sites as well as less obviously objectionable sites, some of which made no reference to sex…Our testing confirms that these packages principally block sites with pornography, obscenity, and sexually explicit content--and they do a pretty good job.[5]

In May 1999, PC Magazine tested filters for a fifth time, this time with an emphasis on business products, testing Cyber Patrol, Little Brother Pro, SmartFilter, and Websense. In this test, ZDNet Labs "created a list of 100 URLs in nine categories and then tried to browse them through these products," and concluded:

The software packages in this roundup have matured as the demand for them has increased--and in more ways than the addition of productivity categories… All in all, these products delivered as advertised, though some do so with more panache than others.[6]

A sixth test of filters, this time for home software, was conducted in the April 2000 issue of PC Magazine. BAIR, Cyber Sentinel, eyeguard, SOS KidProof, and X-Stop were tested. PC Magazine concluded:

Regardless of which you choose, once you install a parental filtering utility, your kids can explore the world of the Web without wandering into a virtual red-light district.[7]

The seventh PC Magazine test occurred in the September 2001 issue. This was the most extensive test to date, involving twelve filters: AOL Parental Control, CyberSitter, CyberSnoop, Internet Guard Dog, Net Nanny, Norton Internet Security, IM Web Inspector, Super Scout, Surfin Gate, 8e6, Iprism, and NetSpective. PC Magazine concluded:

In testing, most products blocked more than 85 percent of objectionable content—good enough to make a serious dent in inappropriate Internet usage.[8]

The eighth and most recent test was conducted by PC Magazine in November 2001, and involved a single product, WebSense 4.3. PC Magazine found that "We weren't able to fake out Websense filtering with a random sampling of sites." [9]

Info World Tests

Info World is one of the leading technology publications, and provides "in-depth technical analysis on key products, solutions, and technologies for sound buying decisions and business gain."[10] Like PC Magazine, Info World conducts regular software testing through a professional testing laboratory, the InfoWorld Test Center:

The InfoWorld Test Center differentiates itself by providing the most real-world approach to testing. Our tests, which are conducted by the most knowledgeable analysts in the industry, focus on products and solutions as they are used and exist in IT environments.[11]

From 1997 to 2000 the InfoWorld Test Center conducted four tests of filtering software blocking effectiveness. In the August 1997 issue, InfoWorld tested WebSense, and found that, "Every time I tried to access a blocked site, I was presented with my customized "access denied" message."[12]

In February 1998, InfoWorld tested Cyber Sentinel, and concluded, "Cyber Sentinel proved quite adept at flagging all of my attempts at accessing offensive material."[13] In November 1998, InfoWorld tested SOS Pro, and found that "offensive sites were blocked successfully."[14] In May 2000, InfoWorld tested WebWasher, and found the product "prevents offensive materials from being brought into the office via the company's Internet connections," and noted "WebWasher's effectiveness."[15]

PC World Tests

PC World is the world's largest computer magazine, with a readership of nearly 6.9 million. Like PC Magazine, PC World has conducted thousands of software tests through its testing laboratory, the PC World Test Center. PC World conducted two tests of filtering effectiveness in 1997 and in 2001. The 1997 test produced mixed results (see section "Tests finding filters of mixed effectiveness"), but a January 2001 test of Net Nanny found that "In testing, Net Nanny blocked unsuitable content fairly well and appropriately." [16]

MacWorld Tests

MacWorld has been testing software for 17 years in its MacWorld Labs facility. [17] MacWorld conducted two tests, a 1997 test that found filters effective, and a 2001 test that found them ineffective (see section "Tests finding filters ineffective").

MacWorld's November 1997 issue tested Cyber Patrol, SurfWatch, and X-Stop and found that "All the products lived up to filtering expectations, staying out of the way except when necessary to block access."[18]

Internet Magazine Tests

In December 1997, ZD Internet Magazine used the ZD Net testing labs to measure the effectiveness of eight filters: Bess, Cyber Patrol, CyberSitter,SafeSurf, SurfWatch, WebSense, X-Stop and Cyber Snoop. ZD Net Labs found the majority of them effective. Internet Magazine reported that SafeServer and CyberSnoop were less effective, but did find the majority of the products effective:

During our tests, Bess performed well, blocking all the pornographic and objectionable sites on our test list.

In our testing, Cyber Patrol performed fairly well, blocking access to most of the sites on our list. All the pornographic sites were blocked effectively.

During our testing, CYBERsitter 97 blocked access to most of the pornographic sites on our testing list.

SurfWatch was the best performer on our site-blocking test, blocking access to all the pornographic sites we tested, as well as adequately blocking attempts to search for obscene words with Yahoo! and other search engines.

In our tests, WebSENSE performed exceptionally well.

In our site blocking tests, X-Shadow performed quite well, preventing access to almost all the pornographic sites, as well as preventing searches on obscene words.[19]

Network World Tests

Another well-known technology publication, Network World, conducted a round of filter tests through its Network World Test Alliance network of testing labs. Network World frequently tests software, and is described as "the premier source of objective, authoritative reviews in the network market."[20] Network World tested seven filters: LittleBrother Pro, WebSense, WizGuard, SOS, and NNPro. Network World found that "All the products with predefined databases allow you to customize their lists, but we found that locating inappropriate sites the vendors didn't include was a challenge." [21]

Network Computing Tests

Network Computing is another leading technology publication that regularly tests software. As described on the company website, "Network Computing performs hands-on product reviews in our Real-World Labs co-located on the sites of two large universities, a Fortune 100 corporation, as well as bench-test facilities."

Real-World Labs tested SurfControl Super Scout, Elron Internet Manager, Little Brother, SmartFilter, Iprism, WebSense, and N2H2:

We installed and configured each product to monitor and block Web traffic on our production network. We then configured each product to block traffic to unproductive or "improper" sites while letting productive uses of Web, e-mail and FTP traffic go past…We visited a broad range of improper Web sites to evaluate each product's content policies and, if applicable, dynamic policy rules.

Our test results showed that network administrators can choose from many effective content-monitoring solutions capable of stifling the most adamant of browsers.[22]

Internet Week Tests

The now-defunct technology publication Internet Week tested a variety of security software applications in the April 2000 issue. The Camden Associates labs conducted the tests. Among the products tested was Cyber Patrol Proxy. Internet Week's test found that "Cyber Patrol Proxy for Microsoft's Proxy Server does an excellent job of blocking undesired sites."[23]

eWeek Tests

Another popular technology publication is eWeek, which regularly tests software through the eWeek Labs. In the February 2001 issue, eWeek Labs tested the effectiveness of SmartFilter, and concluded that, "We were impressed with the quick response from SmartFilter when we tried to access Web sites that were in the "Deny" ACL.[24]

Computer Shopper Tests

Computer Shopper is a widely distributed technology trade publication that has conducted thousands of software tests. In the November 1997 issue, CyberSitter was tested through ZD Net Labs:

Although installing Cybersitter is a smart way to keep your children safe on the Internet, keep in mind that nothing is foolproof. Although it took several hours, we were able to bring up three sites with inappropriate content ourselves…However, for those times when you need a quick way to tame the World "Wild" Web for young cyber surfers, Cybersitter 97 is a good start.[25]

Tests Finding Filters of Mixed Effectiveness

Internet World Tests

Since 1995, Internet World has been one of the leading Internet technology publications, and regularly tests Internet software in IW Labs. In September 1996, Internet World examined Intergo, Cyber Patrol, Net Nanny, Net Shepherd, Specs for Kids, CyberSitter, and Surfwatch:

To evaluate how well the current programs work, IW Labs rounded up every available commercial product and tested them under controlled laboratory conditions…While each of the products is sold for the explicit purpose of blocking objectionable material, only three (Cyber Patrol, InterGo, and Specs for Kids) are able to do that with reasonable certainty.

	Inter go	Cyber Patrol	Net Nanny	Net Shep	Specs for Kids	Cyber sitter	Surf watch
Drugs	Excellent	Excellent	Poor	Fair	Excellent	Fair	Good
Sex	Excellent	Excellent