{{TableOfContents}}

This page describes the TBWiki table architecture

Table data is managed in the TBWiki with the table module.

== Table types ==
The table code recognizes three distinct table types:
 * attribute databases
 * row-column databases
 * scraped databases

Attribute databases consist of records which have name-value
pairs.  These are usually read from multiple TBWiki page files.

Row-column databases consist of values which are associated
with each other by row and column (similar to a relational
database).  These are usually read from a single TBWiki page file
Currently, the data is formatted in MoinMoin table markup.

Writing code to handle these two different types of tables
is challenging.


=== commonalities and differences ===
Here are some commonalities and differences between the two 
database types:
 * attr dbs have sparse field values (not every record has every field)
 * attr dbs are parsed from multiple files
 * attr dbs have to keep track of which record came from which file
 * attr dbs have NO inherent record or field ordering

 * row-col dbs have no identifying record-id (this has to be synthesized
 from the row position)
 * row-col dbs have inherent record and field ordering (row and col position)

Another possible database is the web-scraped database.

This database has records and fields which are "scraped" from 
existing TBWiki pages or from external web sites.
 * scraped dbs have sparse field values
 * scraped dbs have non-editable fields

== Form creation ==
The current table code can create simple data entry forms for
a database that consists of fields on type "single-line text".

The current table code supports user-specified data entry forms.

Forms for displaying the database and for adding or editing records
are expressed as multi-line strings with python string variables
where the field values or field form elements are to be placed.
The rest of the form is expressed in HTML (not TBWiki markup).

For example, if a database 'People' has 'Name' and 'Phone' fields, a form
for outputting this could look like this:
{{{
"""<table>
<tr><td>Name</td><td>Phone</td></tr>
<tr><td>%%(Name)s</td><td>%%(Phone)s</td></tr>
</table>"""
[This example is not complete.]
}}}
How do you specify the start, record portion and end of a formspec??

----
A record data entry form might look like this:
{{{
Name: %%(NameField)s
Phone: %%(PhoneField)s
%%(SaveButton)s %%(CancelButton)s
}}}

Forms can be stored as TBWiki pages, with the names:
PeopleAddForm or PeopleEditForm.

It would be nice to auto-detect form elements for checkboxes, radio
buttons, selects, and multi-line text.

(Maybe use schema-by-example...???)

== Schema determination ==
Field names are autodetected from the table data itself.

Field attributes, such as type (which can influence the type of form element
used for inputing that field), possible values, and others, are expressed
in a configuration block.  These are referred to internally as the field_db.

Here's a sample field_db block:
{{{
field=Description
type=TextArea

field=Status
type=Select
possible_values="""not done
in progress
done"""
colors="""not done:red
in progress:yellow
done:green"""
}}}

=== Field attributes ===
Possible types are:
 * Text
 * TextArea
 * Select
 * Radio
 * Checkbox

Field attributes currently implemented are:
 * type
 * possible_values
 * colors

==== Field color ====
The syntax for colors is:
colors="""<cexpr>:<color>
<cexpr2>:<color2>
..."""

<cexpr> can be a straight value, or an boolean expression.
If it is a boolean expression, it must start with an open paren, '(',
and do some comparison on the variable 'value'.  A <cexpr> which is
a straight variable is converted into the expression:
"(value=="<variable>")"

The following are some examples:
{{{colors="""not done: red
done:green"""
}}}

{{{colors="""(int(value)<5):green
(int(value)>=6):red"""
}}}

Field attributes that I've thought about, but are not implemented yet are:
 * constraints - complex expression-oriented constraints, such as "one of", "in range", or "re.match"
 * display_attrs - more complex display attributes (besides just bgcolor)


== Current Code ==
table.py

functions:
 * def print_error(msg):
 * def generate_field_input(col_name, field_type, value):
 * class form_field_generator_class():
    * def __init__(self, record, col_map, field_type_hints=None):
    * def __getitem__(self, key):
 * def cell_trans(value):
 * def value_trans(value):
 * class table_class():
    * def __init__(self, source_spec, page_name="(no page name)"):
    * def add_or_edit_numbered_record(self, record, record_id=""):
    * def remove_numbered_record(self, record_id):
    * def set_col_output_list(self, order=None):
    * def set_row_output_list(self, order=None):
    * def set_row_filter(self, filter):
    * def filter_match(self, record):
    * def show(self):
    * def add_to_message(self, message):
    * def html_string(self):
    * def show_form(self, form_spec=None, record_id=None):
    * def show_record_form(self, form_spec=None, record_id=None):
    * def show_edit_table_form(self, form_spec=None):
    * def edit_link(self):
 * def get_table(req, args):
 * def get_form_spec(req, tb, form, source_spec, operation):
 * def add_row_form(req):
 * def edit_table(req):
 * def parse_moin_table(tb, data, source_spec):
 * def read_moin_table_file(tb, data_dir, source_spec):
 * def read_moin_table_from_page(tb, data_dir, source_spec):
 * def read_attrdb_file(file_path, source_spec):
 * def load_schema(tb):
 * def db_from_attrdb_files(tb, data_dir, source_spec):
 * def save_attrs_to_file(tb, record):
 * def save_attrdb(tb):
 * def remove_attrdb_file(tb, record_id):
 * def render_moin_table(tb):
 * def get_table_type(source_spec):
 * def get_table_data(req, data_dir, source_spec, page_name):
 * def save_table_data(data_dir, source_spec, tb):
 * def action(req):

== New Code (Refactor ideas) ==
Split code into:
 * db read
 * db present
 * db edit
 * db write

Allow different presentation modules and different editing modules:
 * present as a list of records
 * present as a table
 * add a single record
 * edit a single record
 * edit as a table

[Describe different interfaces here]

== Interface A - between table.py and db ==
 * db_read()
 * db_write()
 * db_get_record(record_id)
 * db_update_record(record, record_id)
 * db_add_record(record, record_id)
 * db_remove_record(record_id)
 * db_backup()??

== Interface B - between db and tbwiki_engine ==
 * read_page(page_name[, page_namespace??])
 * write_page(page_name[, page_namespace??])
 * make_backup(page_name[, page_namespace??])
 * read_page_item(page_name, item_name)
 * write_page_item(page_name, item_name, content)