SoDakLIVE! - Agency Support

This is your guide to SoDakLIVE. The guide covers what you need to know about the program and how your government organization can benefit from participation.

Guide to SoDakLIVE

What is SoDakLIVE?
What is the benefit to agencies that participate in SoDakLIVE?
Using SoDakLIVE to search for information
How SoDakLIVE Works
SoDakLIVE Process
What Tools and Assistance are Available to Agencies?
Description of Metadata
Metadata and Dublin Core
Getting Started with Indexing
Placing SoDakLive Attributes in HTML coding
SoDakLIVE Spidering
Keeping your pages out of SoDakLIVE!
Additional Services and Reports available to South Dakota State Agencies

What is SoDakLIVE?

South Dakota LIVE is:

A process to enhance the discovery of your Web-based information.
Procedures to identify, index, and describe your digital publications or Website.
A World Wide Web-based "one-stop" directory of state government information.

What is the benefit to agencies that participate in SoDakLIVE? Back to top

People will find your pages. Using SoDakLIVE indexing, your agency's Web information will be found and posted higher in the search results lists of SoDakLIVE.
Demonstrate your interest in public service. By assisting the public with locating your information, your agency shows its commitment to public service.
Prevent Information Overload. Without indexing, discovery of agency data will become hopeless. We have a chance, by starting now, to avoid the "ten thousand hits" problem and keep search and retrieval precise.
Provide custom searching of your Website. Put a SoDakLIVE Link on your Web pages for focused searching of your Website.
Provide access to all citizens. By following SoDakLIVE guidelines for site accessibility, all citizens have an opportunity to locate your site.

Using SoDakLIVE to search for information Back to top

The State Library provides the SoDakLIVE database and search engine 24 hours each day. The public accesses this tool using an Internet connection and a browser (e.g., Netscape or Internet Explorer). In addition to personal or business-owned computer access, many public libraries have computers with Internet connections for public use.

Once connected to the SoDakLIVE site, a searcher can locate both Web information and state documents that have been scanned and added to the database as PDF files. Searches allow general term queries ("keyword" searching) or more specific, fielded attribute searching.

Search results are displayed in lists in relevant order. Access to the source document is through a link to an agency's Web site or directly to the PDF document.

How SoDakLIVE Works Back to top

For locating printed government publications the State Library scans the documents and imports them into the SoDakLIVE database as PDF files.
For agency Web pages (including electronic documents, home pages, and databases), the agency uses the core SoDakLIVE attributes to create an index for each major item.
The agency's Webmaster or the State Library staff embeds the index in the hidden, <HEAD> section of the Web page. This information is not displayed on the screen when "browsed", but is found by the SoDakLIVE spider.
SoDakLIVE visits your Web site and creates a locator record for each Web page.
SoDakLIVE builds an index made up of words from the Web pages plus any "metatagged" SoDakLIVE attributes discovered.
The searcher links to SoDakLIVE and enters terms for searching.
The "Results List" describes your pages; the searcher follows the displayed link to your page to view it.

SoDakLIVE Process Back to top

All government agencies should:

Submit copies of their public printed publications to the South Dakota State library system.
For electronic available material: determine the major branching or "node" pages of their Web site.
Assign values (using the core SoDakLIVE attributes) for each Web node page and major document to be accessed by the public. Use SoDakLIVE METADATA Attributes.
Embed the SoDakLIVE index information into the <HEAD> section of the Web page or document.
Revise the "dateofLastModification" field when you change information on the indexed page.
Upload the indexed page to your Web server.
Notify SoDakLIVE staff when the site is ready to be visited by the SoDakLIVE "spider" to capture your index information.

What Tools and Assistance are Available to Agencies? Back to top

State and local agencies can turn to the State Library for assistance in the indexing process.
Knowledgeable, trained staff are available to:

Recommend subjects in popular demand
By tracking frequently asked questions (FAQs) statewide and talking to agency staff, SoDakLIVE specialists can suggest topics that should be included in your Web site. Also, your FAQs can be added to the SoDakLIVE Search engine.
Help with indexing
SoDakLIVE staff can provide assistance in completing the indexing process for Web pages and documents. This includes demonstrating the creation of "metatag" attributes in your Web pages and checking that your index is complete.
Provide detailed instructions on the SoDakLIVE Web site
All tasks required for indexing your Web site are described in greater detail at SoDakLIVE.

A Description of Metadata Back to top

Metadata is data that describes information or data associated with an object that describes it. As metadata is stored within the area of a Web page, it is not visible to Web browsers. This embedding in the page also allows the metadata to remain current when the document is created, moved or updated.

The ability to search and find information is enhanced by controlled vocabularies linked to the metadata elements. In addition, as metadata is combined with controlled subject indexes, it allows more precise searching and document management.

Why use metadata in the first place? Metadata gives focus to searching. Imagine a database where every record has only one field. You can find records, but records can't be grouped by unique attributes, because there aren't any. You can use the HTML meta tag to specify the summary text that will appear in a search results list and to control if and how your page is indexed by the search engine. The meta tags must be placed within the HEAD portion of your web page. Do not use any HTML tags within the meta tag itself. In the absence of metadata, the search engine will index all the words in a document except for comments, and will use the first few words as a summary to describe your page in the search results.

The SoDakLIVE online document index is like a fielded database, and the fields consist of the SoDakLIVE attribute set elements. Any given document has a number of attributes that make it unique and could potentially help others to find it. We have attempted to capture those unique elements.

Suppose your page contains:

The search engine will index this page so the that it can be found with any of the queries:

author:Secretary

author:"Secretary of State"

author:"secretary of state"

Metadata and the Dublin Core Elements Back to top

The SoDakLIVE Project has adopted a 13-element metadata set, which is modified Dublin Core for describing network-accessible materials. This core set of metadata elements is defined by the Dublin Core Working Group. Endorsed by the W3 Consortium in 1998, it has recently been approved as National Information Standards Institute (NISO) standard number Z39.85, and sent on for American National Standards Institute (ANSI) approval.

As the collection of electronic documents on South Dakota Agency Internets (and intranets) grows, metadata is emerging as a powerful tool to find useful information. Placed in Web pages, metadata will allow information to be found more easily and accurately. It also can be used for records management and archiving. Metadata helps search accuracy on commercial search engines, too. Finally, SoDakLIVE - DC allows standards to be developed that relate to electronic document cataloging, retrieval or archiving using Dublin Core Elements.

There are several reasons for adopting Dublin Core for SoDakLIVE:

Dublin Core is easy to create and provides uncomplicated descriptions.
Dublin Core is simple to index and use for describing a resource's location, form, etc.
Dublin Core offers controlled vocabularies that enable greater searching precision than full text searches.
Dublin Core is a standard agreed upon by the W3 Council.
Dublin Core offers extensibility and interoperability with other standards.
Dublin Core enhances the quality of resource management.

Getting Started with Indexing Back to top

Introduction: The SoDakLIVE metadata attribute set lists the "core elements" of content indexing adopted as an indexing standard by the South Dakota State Library. These thirteen elements are adopted as the minimum number of indexed metadata fields for the purpose of describing information content in government produced information sources. Six attributes are mandatory and seven are optional. The thirteen elements are listed with definitions and examples are provided. The field definitions may be used as reference when creating meta tagged fields in the header of a web page. When these meta tags are present in the header of a web page, the SoDakLIVE server will index the site and include links to the information in the SoDakLIVE database.

Placing SoDakLIVE Attributes in HTML coding Back to top

It is important to use the following conventions when placing your index information in the Web page or document.

Using a text editor, type (or paste from the template) the SoDakLIVE attributes (now called METATAGS) in the <HEAD> section of the Web page or document:

Using several SoDakLIVE Attributes for examples:

(Note: subject values are from the controlled terms of the SoDakLIVE Subject Tree)

SoDakLIVE Spidering Back to top

Once your agency has added the SoDakLIVE Attributes (METATAGS) to your major Web pages and documents, replace the pervious version in your server with the indexed version.
Notify the SoDakLIVE staff that you have information at your site for SoDakLIVE spiders to access.
SoDakLIVE spiders will visit your site, read the Web pages, and add your information to the SoDakLIVE database.
It is not important to index all information. The SoDakLIVE spider will create a simple locator record for non-indexed information. However, adding index data significantly improves the discovery of your pages and documents.
The SoDakLIVE spider will regularly visit your site. Presently, we are on a weekly spidering schedule. Therefore, your documents should be captured and assimilated in a matter of days.
The spider will inventory all of your primary links and pages. This information will be compared to locator records in the SoDakLIVE database. Expired links will be removed and new / changed pages will be added / replaced.
If after two weeks, your pages do not show up in search results, please contact the SoDakLIVE staff at (605) 773-3131 or by email at digitization@state.sd.us.

Keeping Your Pages Out of SoDakLive Back to top

There may be some pages that you don't want to be part of our system. For instance, if your files are in one of the major state servers like state.sd.us, chances are your files have been spidered and indexed into our system anyway. How do you keep your pages from being spidered?

First of all, to see which pages are spidered and which aren't, it's useful to understand how the SoDakLIVE robot collects pages. It begins by fetching a starting point page, typically the server's root page. It collects all the links on that page, then visits each of those pages. If the page meets our criteria, then that page is scanned for links, and those links are visited, and so on. This means that, for a page to be collected into our system, it must ultimately be navigable from the root, starting point page. If nothing links to a page, it won't be found by our spider -- nor by any other spider.

One way to keep spiders from your pages is by adding and configuring a robots.txt document in your server's root.

However, you can also add meta tags that will block robot access. To quote from the Netscape Compass Server documentation, here's an explanation.

The META tag that controls robot behavior uses the name ROBOTS. Its content tells a visiting robot whether it should include the document itself in its index and whether to follow hyperlinks found in the document to index the linked documents.

The general format for the ROBOTS tag is as follows:

The terms in the CONTENT portion can be any of the following, separated by commas:

Content String	Meaning
ALL	The robot is welcome to include this document in its index and to follow any links found in it. This is the default value. You can get the same result by leaving the CONTENT portion empty, by omitting the ROBOTS tag entirely, or by using the contents "INDEX, FOLLOW".
NONE	The robot should ignore the page. This is the equivalent of "NOINDEX, NOFOLLOW".
INDEX	The robot is welcome to include the document in its index for searching.
NOINDEX	The robot should not include the document in its index. The robot can still follow links, unless you also include the NOFOLLOW string.
FOLLOW	The robot is welcome to follow any hyperlinks in the document to locate other documents for its index.
NOFOLLOW	The robot should not follow any hyperlinks in the document to locate other documents. This enables you to index just the entry point of a complex document, for example, or to index the open access point to an otherwise restricted site.

Additional Services and Reports available to South Dakota State Agencies Back to top

All government agencies that participate in SoDakLIVE are entitled to additional helpful services and reports. The following benefits are available:

Site Traffic Report
Itemization of encounters between their URLs and public visits through Find-It! including detailed counts and terms searched. However, searcher identity is neither available nor provided.
Query Referral Service
If searchers can not find what they are looking for, a Mailto: e-mail box is provided to convey their search query to the SoDakLIVE staff. State Librarians assist in directing the searcher to the appropriate agency site or in forwarding the query on to the appropriate agency contact person.
Webmaster Support
Training is provided routinely to state government Webmasters and private Webmasters under government contract. The content includes starting SoDakLIVE indexing and Metatags.