Decision Tree Supervised Classification Example

The following SQL example steps through creating your document and classification tables, classifying the documents, and generating the rules. It then goes on to generate rules with CTX_CLS.TRAIN.

Rules are then indexed to create CTXRULE index and new documents are classified with MATCHES.

The CTX_CLS.TRAIN procedure requires an input training document set. A training set is a set of documents that have already been assigned a category.

After you generate the rules, you can test them by first indexing them and then using MATCHES to classify new documents.

To create and index the category rules:

Create and load a table of training documents.

This example uses a simple set of three fast food documents and three computer documents.

create table docs ( doc_id number primary key, doc_text clob); insert into docs values (1, 'MacTavishes is a fast-food chain specializing in burgers, fries and - shakes. Burgers are clearly their most important line.'); insert into docs values (2, 'Burger Prince are an up-market chain of burger shops, who sell burgers - and fries in competition with the likes of MacTavishes.'); insert into docs values (3, 'Shakes 2 Go are a new venture in the low-cost restaurant arena, specializing in semi-liquid frozen fruit-flavored vegetable oil products.'); insert into docs values (4, 'TCP/IP network engineers generally need to know about routers, firewalls, hosts, patch cables networking etc'); insert into docs values (5, 'Firewalls are used to protect a network from attack by remote hosts, generally across TCP/IP');

Create category tables, category descriptions and IDs.

----------------------------------------------------------------------------

-- Create category tables -- Note that "category_descriptions" isn't really needed for this demo - -- it just provides a descriptive name for the category numbers in -- doc_categories ---------------------------------------------------------------------------- create table category_descriptions ( cd_category number, cd_description varchar2(80)); create table doc_categories ( dc_category number, dc_doc_id number, primary key (dc_category, dc_doc_id)) organization index; -- descriptions for categories insert into category_descriptions values (1, 'fast food'); insert into category_descriptions values (2, 'computer networking');

Assign each document to a category.

In this case, the fast food documents all go into category 1, and the computer documents go into category 2.

insert into doc_categories values (1, 1); insert into doc_categories values (1, 2); insert into doc_categories values (1, 3); insert into doc_categories values (2, 4); insert into doc_categories values (2, 5);

Create a CONTEXT index to be used by CTX_CLS.TRAIN.

To experiment with the effects of turning themes on and off, create an Oracle Text preference for the index.

exec ctx_ddl.create_preference('my_lex', 'basic_lexer'); exec ctx_ddl.set_attribute ('my_lex', 'index_themes', 'no'); exec ctx_ddl.set_attribute ('my_lex', 'index_text', 'yes'); create index docsindex on docs(doc_text) indextype is ctxsys.context parameters ('lexer my_lex');

Create the rules table that will be populated by the generated rules.

create table rules( rule_cat_id number, rule_text varchar2(4000), rule_confidence number );

Generate category rules.

All arguments are the names of tables, columns, or indexes previously created in this example. The rules table now contains the rules, which you can view.

begin ctx_cls.train( index_name => 'docsindex', docid => 'doc_id', cattab => 'doc_categories', catdocid => 'dc_doc_id', catid => 'dc_category', restab => 'rules', rescatid => 'rule_cat_id', resquery => 'rule_text', resconfid => 'rule_confidence' ); end; /

Fetch the generated rules, viewed by category.

For convenience’s sake, the rules table is joined with category_descriptions so that you can see the category that each rule applies to.
```
select cd_description, rule_confidence, rule_text from rules, category_descriptions where cd_category = rule_cat_id;
```
Use the CREATE INDEX statement to create the CTXRULE index on the previously generated rules.
```
create index rules_idx on rules (rule_text) indextype is ctxsys.ctxrule;
```

Test an incoming document by using MATCHES.

set serveroutput on;

declare incoming_doc clob; begin incoming_doc := 'I have spent my entire life managing restaurants selling burgers'; for c in ( select distinct cd_description from rules, category_descriptions where cd_category = rule_cat_id and matches (rule_text, incoming_doc) > 0) loop dbms_output.put_line('CATEGORY: '||c.cd_description); end loop; end; /