DocProcessors 

TB Wiki

Login

DocProcessors

Introduction [edit section]

TBWiki supports two kinds of extension mechanisms: "macros" and "processors".

There are a few builtin macros, but add-on macros and processors can be created and are "run" when a page is parsed and sent by TBWiki.

The function of both of these extension mechanisms (or plugins) is to allow for dynamic creation of page content at the time a page is read by a user. Thus, these act like a kind of embedded CGI-script inside of the TBWiki framework.

In general, Macros are used for simple transformations of content or for page-related content creation. Processors are for doing more complicated processing of page or wiki data, possibly requiring configuration settings or multiple interactions with the user.

Name and Location [edit section]

By convention, the name of a processor is in CamelCase (words strung together with the first letter of each word capitalized).

Plugin processors are placed in the data/plugin directory. They are python files and must have a filename starting with the "Processor", then the processor name, and ending in the extension ".py". For example the "Foo" processor would have the path and filename

/data/plugin/ProcessorFoo.py

Declaration in a TBWiki Page [edit section]

A processor is declared on a tbwiki page with the syntax:
{{{#!Foo
configuration lines
and content lines
}} }

The processor name in the declaration must match the name part of the filename for the processor module in the plugin directory. The lines inside the processor block can consist of anything, and are parsed and processed by the processor itself. (That is, they are not interpreted by TBWiki engine, and are specific to the processor being invoked.)

Interface from TBWiki to the processor [edit section]

A processor must be an importable python module, with some specifically-named function definitions. The TBWiki calls functions in the processor to perform actions, usually resulting the transformation or creation of new content that will be returned to the user as part of the page where the processor is declared.

main() function [edit section]

A processor must define a function called "main()". As a page is rendered by TBWIKI, when the processor definition is encountered in the wiki page, TBWiki calls the 'main' function to invoke the processor functionality.

This "main" function takes two arguments, which are the request object and the content block string.

The processor operates on (i.e. processes) the content data, possibly using the information and functions provided in the request object, and returns a string of HTML data, for output as part of the returned page.

The block containing the processor declaration inside the page is replaced by TBWiki with the result of executing the processor.

help() function [edit section]

If a processor defines the special function "help()", then the system can provide online help for the processor, for a page displaying the SystemInfo. This function is accessed via a URL with the name: <page_url>?action=<processor_name>.help

other functions ("action" functions) [edit section]

In general, any of the functions in the processor can be accessed using a URL that includes the name of a page that declares the processor, the processor name, and the function name. So if a page "Some_Page" had a declaration of processor "Foo" on it, that defined a function "bar", then the URL for causing the invocation of that function would be:

   ../SomePage?action=Foo.bar

These are referred to as "action functions" (or just "actions"), in this documentation.

Action functions are used to implement state machines or complex processors, which require multiple interactions with the user, or to display multiple pages in sequence (such as a blog or an image gallery).

See the #Actions section below for more details.

Interface from the processor to TBWiki [edit section]

request object [edit section]

The request object has all the data available to tbwiki about the request for this page. This includes the page_name, the entire tbwiki config, and other stuff.

Here are some pertinent data fields that some processors use:

  • req.form = the CGI form data for the request (in the format provided by the python cgi module)

  • req.page_filename = filename (including full path) of the requested page
  • req.page_name = name of the requested page
  • req.config = configuration settings for the wiki
    • req.config.data_dir = location in the local file system where the data pages are located.
  • req.data = data values (and data value functions) for this wiki
    • req.data.version - version of the tbwiki engine
    • req.data.timestamp - timestamp string for the current time
    • and many more

req functions [edit section]

  • req.add_to_message() - a function to add to the status message for the page. This is often used for debugging purposes, since it appears separately from the page content
  • req.html_error() - show a string in red on the page
  • req.block_to_html() - convert a block of text from tbwiki markup to HTML
  • req.parse_conf() - used to parse configuration items from a block of text

tbwiki functions [edit section]

In some cases, it is necessary to load the tbwiki_engine module itself into the processor module, in order to access global module functions or data. Here are some tbwiki_engine functions or data that might be useful for a processor:
  • tbwiki_engine.make_url()
  • tbiki_engine.parse_state = state used for processing lines during page parsing
  • tbiki_engine.show_line() = function used to parse a line of TBWIKI-format text and convert it into HTML output (which is printed by show_line)

return value [edit section]

The processor's main() function should return a string consisting of HTML to be output as part of the returned page HTML.

If you want markup processed in the content string, you must process it yourself. Usually, this means you would do whatever processing on the content is appropriate for your processor, then call req.block_to_html(content), and return the result of that.

Example Processor - FooReplace [edit section]

Here is sample code for a very simple processor:

FooReplace processor code [edit section]

This would be in the file ProcessorFooReplace.py
import re

def main(req, content):
        result = re.sub("foo", "bar", content)
        return result

FooReplace processor invocation [edit section]

To use this processor, a user would place the following block on a page:

{{{#!FooReplace
This text has foo, but it (foo) should be replaced with 'bar'.
}}}

FooReplace processor result [edit section]

Here is what the resulting output would look like on the page:

This text has bar, but it (bar) should be replaced with 'bar'.

Actions [edit section]

You can cause the engine to invoke different functions in your processor. This is useful for handling a flow of operations requiring sequential displays or multiple user inputs.

To have the engine call a particular function in your processor, put the function name in the action portion of the the URL for some link, button or form element that is selected by the user for you system. The specific syntax is:

   base_url/?action=<processor_name>.<action>

That is, specify the value for the 'action' argument in the URL as the processor name, followed by a period, followed by the name of the action function inside that processor to call.

Action invocation [edit section]

Action functions for your processor are invoked differently from the processor's main() function.

When an action function is specified in a URL to the TBWiki engine, the page in the URL is not rendered. Instead, the processor is called, with the request object only. If the content of the processor block is needed, or some other arguments are needed (e.g. passed from some other processor function to this one), then those need to loaded or transferred using some other mechanism. This is usually done by passing session-specific arguments on the URL used to activate the action.

Reading the processor block content [edit section]

A action function does not receive the content block automatically, as one of its arguments. However, it is common for an action function to need to read the content of the processor block. For example, the processor content block is often where the configuration for the processor is stored.

One way to accomplish this is to pass the block_name of the processor block to the action function, as part of the action URL.

in main:

def main(req, content):
  block_name = req.state.block_name
  # add the block name to the URL, like so:
  link = ...?action=Foo.bar&block_name=" + block_name

in an action function:

def bar(req):
 block_name = req.form["block_name"].value
 content = req.get_page_item(block_name)
 # parse content as needed

Reading config values [edit section]

Some processors need to handle both configuration settings and content. This is true, for example, of the Table processor, which has table definition and display settings, as well as the actual content to be placed in the table.

One convention used to support this is to have configuration lines at the top of the processor block content, followed by a dashed line (a line consisting of only 3 dashes), like so:

---

Here is an example showing configuration content in the same processor block.

{{{#!ColoredTimer
color=red
duration=10m30s
interval=5s
----
Here is a timer:
 <<  <font color=red>&lt;missing data value for key "timer"&gt</font>  >>
It should be counting down.
} }}

Assuming you have the processor block content in the variable "content", you could use the following lines to parse this into two parts, including a configuration map (a python dictionary containing config items).

   config_str, content_str = content.split("---\n")
   config_map = req.parse_conf(config_str)
   # do something with content_str (based on settings in config_map)

Action output [edit section]

Unlike your main processor function, action functions are not called in "page context". That is, an action function does NOT return output to be placed on the page in place of the block, during page rendering.

Rather, when the function is entered, no html has been emitted yet -- no header information, nav bars, nor even the "Content-type:" line. Thus they have complete control to emit any HTML output they want.

This is usually done by calling req.show_header() with some custom title to reflect the action or operation being performed. Then the HTML for the page is printed directly by the action function.

Note that if your routine calls req.show_header() to output a standard wiki page header (with the current theme), then req.show_footer() is called after the processor action routine. However, if the action routine does not call req.show_header(), then req.show_footer() will not be called automatically, and it is up to the action routine to complete the HTML output for the request.

Action Sample - UserChoice [edit section]

Here is sample code for a simple processor that shows what choice of two items a user selected. The choices are on two lines in the block content.

UserChoice processor code [edit section]

This would be in the file ProcessorUserChoice.py
def UserChoice(req, content):
         lines = content.split("\n")
         if len(lines)<2:
                return "ERROR: not enough lines in UserChoice processor block"
         link1 = req.page_url + "?action=UserChoice.choice1"
         link2 = req.page_url + "?action=UserChoice.choice2"
         html = """Please choose one of the following:
                  <ul>
                     <li><a href="%s">%s</a>
                     <li><a href="%s">%s</a>
                  </ul>""" % (link1, lines[0], link2, lines[1])
         return result

def choice1(req):
         req.show_header("User Choice result")

         result = "You chose choice1 - this is inside the wiki page and theme"
         print result

def choice2(req):
         html = """Content-type: text/html

               <body bgcolor="yellow">
               <h1>User Choice result</h1>
               """

        html += "You chose choice2 - this is outside the wiki page and theme"
        html += "<br> and it is bright yellow!"
        html += "</body>"
        print html

UserChoice processor invocation [edit section]

To use this processor, a user would place the following block on a page:

Stuff before the processor block

{{{#!UserChoice
This is the first option: apples.
This is the second option: bananas.
} }}

Stuff after the processor block

UserChoice processor result [edit section]

Here is what the resulting output (from main) would look like on the page:

Stuff before the processor block

Please choose one of the following:

Stuff after the processor block

UserChoice processor action outcome [edit section]

Here is what would be displayed if the user selected the 'apples' option:

You chose choice 1 - this is inside the wiki page and theme

TBWiki engine 1.9.1 by Tim Bird