TBWikiToDo
Note that some test materials are on a separate wiki at: http://bird.org/cgi-bin/test.cgi/FrontPage
Before working on this wiki any more, re-read this article by a developer who has written 7 personal wikis:
- https://borretti.me/article/unbundling-tools-for-thought
- the basic idea is that there are better tools for thought than personal wikis
- but that's OK - tbwiki is a hobby project that is fun to work on
- the basic idea is that there are better tools for thought than personal wikis
Next major items to work on [edit section]
- support captions
- if caption="foo bar" is an attribute of an image, use figure, and figcaption to surround the image.
- see https://www.w3schools.com/TAGS/tryit.asp?filename=tryhtml_figcaption_css
- MacroCategory.py - show a list of pages in a particular category
- implement category tags, like this:
{{Category(Tasklist,2023)}} {{Category(New Feature)}}
- implement category reference, like this:
{{ShowCategory(Tasklist)}} or {{Category(show:Tasklist)}}
- implement category tags, like this:
- Blog processor
- support adding a blog entry
- (done) ExportSite processor
- Search macro:
{{Search(word or phrase)}}
- do performance enhancements, possibly using the Scalene profiler
- add a "blog" processor
- for ideas, see: https://jekyllrb.com/docs/posts/
- save blog pages as: blog-YYYY-MM-DD
- don't support comments
- show index of blog posts, current post
- research blog layouts:
- see Blog Layouts and Features for ideas
- for ideas, see: https://jekyllrb.com/docs/posts/
- figure out why I keep getting logged out (is there something wrong with the cookie expiration date?)
- don't add an extra line when saving.
- use splitlines instead of split to avoid adding an extra line on an edit??
- there appears to be no split here
- do some testing to see where the extra \n gets introduced:
- client side or server side?
- in form population (send from server) or in form response data (sent from client)?
- is there a way to detect a new newline, and remove it?
- grep split tbwiki_engine.py | grep [\\]n
- use splitlines instead of split to avoid adding an extra line on an edit??
- get css from page area of local wiki
- don't retrieve from /var/www/css
- Support wiki on mobile devices
- (done) fix up mobile theme
- change mobile.css and mobile.footer and mobile.header
- some pages still show very small fonts on my phone
- autodetect mobile phone browser, and use mobile theme automatically
- user UserAgent field
- (done) fix up mobile theme
- switch to wsgi - I considered flask, see FlaskResearch
- should switch to wsgi - I have experience with that
- enhance blog system (see TBWikiToDo#Blog)
- add page lock (to support multi-user)
- add help routine to all processors
- add SystemInfo processor to handle help requests
- with help action item, that calls help with a processor name
- processors only have to return a block of tbwiki markup
- add SystemInfo processor to handle help requests
- make tables robust
- eliminate save_table_data - placing support in appropriate routines
in the io_handler (tb.handler)
- one of write() or some other class method.
- fix bug 017. See BugList
- fix bug with edit-in-place on QuarterlyData table
- make automated test suite for testing page rendering
- fix all known bugs, with all known table types
- make a table test matrix
- eliminate save_table_data - placing support in appropriate routines
in the io_handler (tb.handler)
- table flow control
- for transcription program (ProcessorTranscriber.py)
- make a view form and edit form for the transcriber database
- read documentation on view and edit forms
- add processor_name to the transcriber database config
- make a view form and edit form for the transcriber database
- for transcription program (ProcessorTranscriber.py)
- add support for hybrid tables
- use case is to add a spreadsheet formula to the FIXTHIS pages, to show totals
- ability to have tables with records from different sources
- add io_handler per-record
- need permission checks (readable, writable) per-record
Things to do (random ideas, un-triaged) [edit section]
- don't allow caching a page edit page
- use http meta cache-control directive to prevent caching
- set caching relatively low anyway
- add page locking for edits - to prevent collisions
- lock a page for 15 minutes when an edit is started
- create a lock file in the admin directory
- put user, start time, and page hash in lock file
- start a timer to handle lock timeout
- create a lock file in the admin directory
- generate a hash of the page when the edit starts
- if someone else tries to edit, give them a warning that they have lower priority - indicate who has the lock
- when someone submits a page, compare the page hash and lock
- if page hash doesn't match, don't save the data
- remove the lock
- after 15 minutes, remove the lock
- lock a page for 15 minutes when an edit is started
- support arbitrary field addressability in json data:
- JsonPageFile:field.list_field[n].foo.bar
- fix bug: edit-in-place wipes out conf entries in inline block
- add 'tbclient list' feature
- modify tbclient to send 'list' request
- modify ProcessorDataHandler.py to respond to list request
- add TODAY() function to spreadsheet
- add websed over snapshotted values
- add websed over diff of snapshotted values
- add chart over snapshotted values
- use plot charts?
- add access-controlled content
- the purpose of this is to add a page with contact information for other test projects, on the fuego wiki
- see https://moinmo.in/HelpOnAccessControlLists
- at the block level, with ProcessorAccessControl
- at the page level, with accesscontrol directive
- put required permissions at top of block
- prevent reading or writing to block (or page)
- add link to page for file column in attrdb and parsed data files (to go to page)
- This is handy for FIXTHIS entries, and for editing
- _bug0013 - can't edit-in-place the buglist
Major Features that are not complete enough to use [edit section]
User accounts [edit section]
- Creating and administering user accounts
- need to re-test user accounts, with new features (next_num)
- Users can create accounts, but physical access to the server is required to change admin flags, or to remove an account
- may be able to use a special version of attrdb (raw page edit) for user accounts. It wouldn't be great but would get the job done for now.
- is the user edit form not working??
Table usability [edit section]
- Specific table usability features:
- other usability feature is deciding which links to use for a table (I never use query, and always have to decide between 'edit-in-place' or 'edit')
BugList, FIXTHIS, Regressions [edit section]
- See the BugList page for bugs that need to be worked on.
- that bug list is from a database. Here is one that just text on a page:
- See the CodeFixthisList page for a list of FIXTHIS items in the code
- See the PageFixthisList page for a list of FIXTHIS items in the pages
Table of combined issues [edit section]
Date ^ | bug_count ^ | FIXTHIS_count ^ | missing_FEATURES ^ | total ^ |
2017-03-08 | 15 | xx | xx | 15 + xx + xx |
2016-08-12 | 14 | 94 | 2 | 110 |
2016-08-02 | 13 | 92 | 3 | 108 |
2011-04-21 | 17 | 78 | 1 | 96 |
2008-10-06 | 17 | 78 | 1 | 96 |
2008-02-04 | 13 | 66 (55 cgi-bin+ 11 plugin) | 3 | 82 |
2007-07-25 | 9 | 62 | 3? | 74 |
2007-02-06 | 9 | 41 | 5 | 55 |
2006-11-15 | 8 | 35 | 6 | 49 |
2006-10-06 | 8 | 37 | 7 | 52 |
2006-08-25 | 6 | 35 | 5 | 46 |
Regression count [edit section]
Periodically, check the regression test page to see if things still work as expected.Here's a table, manually maintained, with regression testing history.
date ^ | version ^ | commit ^ | OK ^ | missing output ^ | bad ^ | bad page list ^ |
2016-08-12 | 1.6.2 | a45cce77 | 91 | 2 | 2 | TestMarkupVariables, TestMarkupVariablesDyamicTest |
2016-07-18 | 1.5.3 | 61eb2f1e | 39 | 31 | 4 | TestMarkupVariablesStatic, TestSandbox, tbwiki.footer, tbwiki.header |
Other items to work on, by category [edit section]
Style [edit section]
- enhance themes feature
- see DocThemes
- for an example of self-editability - see http://kernelnewbies.org/navigation_menu
- calculate theme from user account or other variable
- current use of page name is lame
- allow configurable navigation links
- allow configurable toolbox links
- read page dressing from nested pages??
- allow header and footer pages to be HTML or TBWIKI format
- if TBWIKI, allow for sub-page references that are loaded, parsed and formatted recursively
- allow header and footer pages to be HTML or TBWIKI format
- see DocThemes
- tweak mobile.css
- see w3c pages about automatic layout - http://www.w3schools.com/html/html_layout.asp
- switch to MediaWiki page syntax?? (not yet)
- see http://meta.wikimedia.org/wiki/Help:Wikitext
- need to switch bullets handling
- need to switch table parsing
- have css come from page area??? (would allow editing css using wiki)
General [edit section]
- convert upload to be a processor
- allows putting upload on any page
- control number of files via config
- can specify link(" action=upload&dest_filename=foo") for missing files - to prompt user if file is missing
- upload result should be shown in page message?
- add page rename
- can do create, move content, delete, but that's awkward
- generate tbwiki stats from log
- cache page content, for static content
- change all prints to output routines (to control routing of output)
- allow administrators to view and edit files in the 'admin' directory
using normal tbwiki access (just like regular pages)
- treat it like a MoinMoin overlay directory
- make a mobile viewer
- add ability to grab content at specific intervals
- kind of a combination of crontab and websed
- allows for charting something automatically over time
- make a crontab processor
- present a form for when to run, and action
- make a syntax for block names that are not included in block output
- we have anonymous blocks, hidden blocks, and named blocks
- named blocks show both the name and the content. It would be nice to hide the name
- possibly: #.BlockName ??
- we have anonymous blocks, hidden blocks, and named blocks
- make tbwiki more relocatable (for use on laptop)
- on bird.org:
- move *.cgi scripts from www/cgi-bin to ~/work/tbwiki/cgi-bin
- put link in www/cgi-bin
- add base_dir to *.cgi scripts
- write script for laptop that does:
- downloads latest tbwiki and tbwiki-data
- adjust base_dir in *.cgi files
- adjust base_url in *.cgi files
- adjust plugin symlinks in tbwiki-data dirs
- on bird.org:
- make unit test system - see Unit Test Notes
- NOTE - already have ProcessorRegressionCheck.py
- have a page specify a test
- have markup, and html and compare them
- here's one unit test:
record=bold 1 markup='''bold''' html=<b>bold</b>
- here's one unit test:
- make a ProcessorUnitTest.py processor
- read all "UnitTest.*" pages, convert markup to html, compare to html
- report results
- put this categorized todo list in a table?
- change main() in tbwiki_engine.py
- call main when tbwiki_engine.py is called from command line
- make a tbwiki.py command?
- program should have the ability to create a static page from the command line
- e.g. tbwiki.py FrontPage -o FrontPage.html
- program should have the ability to run unit tests?
- program should have the ability to create a static page from the command line
Security [edit section]
- check permissions on table write operations
- allow administrators to view and edit files in the 'admin' directory
using normal tbwiki access (just like regular pages)
- treat it like a MoinMoin overlay directory
- have rights affect the displayed table edit links
Processors [edit section]
- add help routines for all Processors
- add a processor for "select an item"
- used for choosing a group of words to practice
- used for choosing a story to read in scriptures
- present a list of items from a database, that are unselected
- allow a user to select one
- or, select an item randomly from a list of database records
- allow user to select it or keep choosing a different one
- have 3 categories: unselected, in-progress, finished
- database must "state" field?
- processor allows only legal state transitions
- specify legal state transitions (or is this too meta?)
- support mapping routines (e.g. to convert chapters to pages and back)
- ProcessorScriptureLookup.py - maybe should be a macro?
- take a reference, and replace it with the text for the versus
- this should be very handy for church talks, lessons, etc.
Blog [edit section]
- add right sidebar with:
- recent posts
- categories
- search bar
- tags
- recent posts
- use individual pages for each post
- create add-post form
- page schema has title, content, date, tags, id
- create add-post form
- navigation to previous and next posts
- response area
- show threaded responses with "reply" button on each one
- response form with "post comment button
- response area
- show date, tags, response count
- see example: https://lava-block.com/godot-engine-tutorial-making-a-2d-game-ui-mechanics-sound/
- support customizable background image
- ability to drag and drop images to upload to wiki
ProgressChart [edit section]
- add support for goal lines to the ProgressChart processor
- handle chart attributes
- make chart_attrs_class, to generate defaults when attrs are missing
- create flat namespace for chart attributes
- translate chart atttributes into call arguments during chart creation
- un-specialize pychartwrapper.py
- chart another data type using same wrapper??
- generic multi-series data chart
- scripture reading chart?
- weight chart
- running chart??
- chart another data type using same wrapper??
- support data entry and modification for TimsRunning data
- see ProcessorProgressChart.py
- use Add Form for table??
- support current date for date entry??
SlideShow [edit section]
- SlideShow
- make sure all nested parsing works
- add sidebar list of slides (section 1 headings), with links to pages
- support show attributes and per-slide attributes (bgcolor, bgimage, margins, etc.)
ImageGallery [edit section]
- ImageGallery
- support links=1 - to show individual images in full screen
- autodetect image size for full-screen (standalone) image gallery
- use Javascript to detect browser window size??
- support num_images - to show only some of the images
- gallery has prev and next links on sides (similar to blog navigation)
- support image borders in ProcessorImageGallery
- support link to fullsize image in ProcessorImageGallery
- support displaying the filenames in ProcessorImageGallery
- add javascript zoom functions
Tables [edit section]
- add sortby=select_order - to specify that text fields should be sorted
in the order that they appear inm the select possible_values list.
- this helps the user control the sort order for things like word-based priority: ex: very high, high, medium, low, very low
- add messages to table module
- allow for outputting messages about an individual table
- table.add_table_message(msg)
- this would be handy for table-specific error messages
- should have block-specific messages in general
- allow for configuration of table control links
- have user specify which links to show, and where
- use better filenames
- use unique part of filename for record_id
- allow user to pick filename when creating the record
- support edit-in-place for tbwikidb files
- use get_section and set_section
- modify to read/write definition lines
- use get_section and set_section
- eliminate difference between record, record_id
- use tbwikidb or attrdb for user database??
- this puts human readable filler on every page, but that's the wiki way
- this allows creating a multi-file database just by creating the template
- this is fragile - human editors will break the pages parseability
- add table query cheat sheet to query table form page
- make prototype family info database on bird.org
- automatically create a User account file on login???
- make MemberXX records for family members
- determine the schema (what fields)
- create a table view with contact information
- table queries and views FEATURE
- support <table>RecordView
- support read-only attribute on whole table
- Support time-sampled database
- schedule a websed operation that grabs data at crontab-like intervals
- or that creates a new record whenever the page is accessed (and schedule something to access the page automatically at periodic intervals)
- schedule a websed operation that grabs data at crontab-like intervals
- support automatic data snapshotting
- write a 'table creation wizard' processor
- ask for field names and types, and create the schema file and cover file (or template file, if its a tbwikidb)
- document table config concepts:
- base config (from original conf block)
- dynamic config (from request form variables)
- query config (from query conf block)
- saved queries
- specify query list in conf block
- each item in list specifies a query conf block
- make backup file when editing table files
- finish all routines for editing a table
- use {{{Table(arg1,arg2,arg3...) notation for inline tables
- support other args:
- formspec, query line (row filter, column filter, etc.)
- support other args:
- use {{{Table(arg1,arg2,arg3...) notation for inline tables
- document Table use with data_scan module
- DocTables has some information (one example, the FIXTHIS table)
- multi-file tables
- display multi-line values correctly
- display '<no-data>' values correctly
- auto-sense type of each field (text, select, textarea) and use for auto-form generation
- handle field types with enumerated values (lists and radio buttons)
- document table architecture and interfaces - see Table Architecture
- document table attributes
- document chart attributes
- make ProcessorTablePick.py
- See Table Pick Page
- present simplified list of "unpicked" items
- allow user to pick one item
- save picked item with attribute "state" changed to "in-progress"
- present detailed list of "in-progress" items
- allow user to mark an item as "done"
- save done item with attribute "state" changed to "done"
- whether user wants to have multiple items in "in-progress" state is their choice
- queries, query_lists
Forms editing and Templates [edit section]
- use form emitter for user accounts.
Users and attributes [edit section]
- Support online creation of new users
- make a "create user" form
- create a new user account file from a form
- provide help if user forgets password
- link to e-mail the password??
- password hint??
- create 'user.edit' action, which edits a user account file
- support different roles (admin vs. non-admin)
- allow user to edit their attribute values
- make an "edit user" form
- REFACTOR - use existing form code for user forms
- make user account management it's own processor module
- ProcessorUser.py - we now have sub-actions, so this should work.
- it needs to access table operations, but in a special way
- ProcessorUser.py - we now have sub-actions, so this should work.
Long term ideas [edit section]
- add 'save expected output and show again' to RegressionTest processor
- add link to backup page on history page
- look at other wikis for feature ideas
- in particular, ikwiki statically compiles the wiki on every page edit
- documentation
- table editing:
- should support "save and edit again" and "save and add another row" buttons for saving edit forms.
- Table Of Contents
- do auto-table of contents, like wikipedia??
- handle included files and text expanded from processors??!!
- check HTML and CSS on multiple browsers with BrowserCam or browsershots
- make a journal processor (based on progress chart)
- make a blog processor (with threaded comments)
- make a forum processor (with threaded and voted-on comments)
- make a page cache
- add dependencies to a makefile, and rebuild cache when dependencies change
- put config into admin page?
- reduce cgi-bin to just some directory references
Things Completed [edit section]
- 2021-03 Remove '?action=save' from url in edit form
- 2018-10 Add image rotate to ImageGallery app.
- 2017-04 add triggerd - ability to detect triggers that are due and execute them
- 2017-03 add ProcessorTrigger - ability to trigger a snapshot
- 2017-03 add ProcessorSnapshot - page/block/table snapshot function
- 2017-02 add tbclient - ability to get,put and edit items in the wiki
- 2017-01 add page trails
- 2017-01 add relative index to cell address (so you can do $.-1)
- 2016-12 ProcessorImageGallery - finish showing images in full size page
- 2106-11 add ability to customize table operation links (edit_links, sort_links)
- 2016-9 add ability to read a specific block, by number
- 2016-08 convert to per-table messages (for debugging and errors)
- 2016-08 support editing sections with nested sub-sections
- 2016-08 allow default_values with generated values (%(timestamp), %(next_int), %(next_page_num))
- 2016-08 add page delete
- 2016-07 add tbwikidb database type
- 2016-07 fix data_scan module to parse multi-line fields
- 2016-05 make ProcessorImageGallery
- 2016-05 make ProcessorBlog
- 2016-03 make better diff output
- 2016-03 put engine_version in data_class, and add to footer
- 2016-03 add "printable format" option for page
- 2016-03 made a test wiki at: http://bird.org/cgi-bin/test.cgi/FrontPage
- 2014-05 make talk pages optional - see config.has_talk_page global config
- 2010-10 support inline tables - FEATURE
- 2010-?? websed - add websed feature to scrape other web pages
- 2010-?? data-scan - add as a processor first, then make into a database grabber
- 2008-10 table - back up page on table edit
- 2008-09 chart - draw chart of table data
- 2008-08 processor - add a charting package (pychart)
- 2008-02 table - added support for sorting table rows
- 2008-02 table - support col_list and row_filter specifiers
- 2008-02 table - added ability to edit whole table at a time
- 2008-02 macro - create Include macro
- 2008-02 table - separated conf into separate class
- add field_db (parsing, handling)
- add field types, colors, possible values
- add routine to read block from page
- add field_db (parsing, handling)
- 2008-02 table data - separated IO handler into separate class
- move tbwiki with authentication to bird.org
- aug03 - add user accounts.
- put user account files in an 'admin' directory
- authenticate user from files in 'admin' directory in action user.login
- look up user by name to process login form (in action user.login)
- jul07 - add cookie code for user accounts
- feb07 - use pages for page dressing (see tbwiki.header and tbwiki.footer)
- jan07 - support file upload FEATURE
- 11/15 - add support for inline images
- change backup routine to save current version of page instead of old version of file
- save edit comment and show in history
- edit sections FEATURE
- parse and format:
- bullets
- headers
- horizontal rules
- paragraph breaks
- preformatting
- write edit form for page
- save edited data to file
- read page from data directory and emit as html
- write handlers for various actions
- create add_record form (using existing edit_form code)
- write mtable.save_form() routine
- re-write edit_form to present whole table with edit links
- support Table internal macro, with args
- modified backup time in file extension to be human-readable
- wrote mtable.modify_record() routine
- wrote mtable.remove_record() routine
- parse and display URLs as links
- use pages for page dressin
- make items in table of contents into links
- make an anchor target for each heading
- use href for anchor target in TableOfContents macro
- make nicer format for table of contents box
- add search form to header
- support multi-page table source_spec
- support e.g. "User*" as the spec
- read all pages starting with User
- finish parse_table_pages() routine
- read each page matching the source_spec
- parse out values using a page template
- start with simple name=value parser
- finish parse_table_pages() routine
- create page on add record
- write page on edit record
- remove page on remove record
- read all pages starting with User
- support e.g. "User*" as the spec
- support history action
- show history from backup times
- allow diff between two versions